Purdue University Graduate School
Browse
Thesis_dec23.pdf (5.1 MB)

Hardware Implementation of Learning-Based Camera ISP for Low-Light Applications

Download (5.1 MB)
thesis
posted on 2023-12-20, 16:57 authored by Preston Rashad RahimPreston Rashad Rahim

A camera's image signal processor (ISP) is responsible for taking the mosaiced and noisy image signal from the image sensor and processing it such a way that an end-result image is produced that is informative and accurately captures the scene. Real-time video capture in photon-limited environments remains a challenge for many ISP's today. In these conditions, the image signal is dominated by the photon shot noise. Deep learning methods show promise in extracting the underlying image signal from the noise, but modern AI-based ISPs are too computationally complex to be realized as a fast and efficient hardware ISP. An ISP algorithm, BLADE2 has been designed, which leverages AI in a computationally conservative manner to demosaic and denoise low-light images. The original implementation of this algorihtm is in Python/PyTorch. This Thesis explores taking BLADE2 and implementing it on a general purpose GPU via a suite of Nvidia optimization toolkits, as well as a low-level implementation in C/C++, bringing the algorithm closer to FPGA realization. The GPU implementation demonstrated significant throughput gains and the C/C++ implementation demonstrated the feasibility of further hardware development.

History

Degree Type

  • Master of Science in Electrical and Computer Engineering

Department

  • Electrical and Computer Engineering

Campus location

  • West Lafayette

Advisor/Supervisor/Committee Chair

Stanley Chan

Additional Committee Member 2

Qi Guo

Additional Committee Member 3

Anand Raghunathan

Additional Committee Member 4

Fengqing Zhu

Usage metrics

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC