Learning Lighting Models with Shader-Based Neural Networks

To correctly reproduce the appearance of different objects in computer graphics applications, numerous lighting models have been proposed over the past several decades. These models are among the most important components in the modern graphics pipeline since they decide the final pixel color shown in the generated images. More physically valid parameters and functions have been introduced into recent models. These parameters expanded the range of materials that can be represented and made virtual scenes more realistic, but they also made the lighting models more complex and dependent on measured data.

Artificial neural networks, or neural networks are famous for their ability to deal with complex data and to approximate arbitrary functions. They have been adopted by many data-driven approaches for computer graphics and proven to be effective. Furthermore, neural networks have also been used by the artists for creative works and proven to have the ability of supporting creation of visual effects, animation and computational arts. Therefore, it is reasonable to consider artificial neural networks as potential tools for representing lighting models. Since shaders are used for general-purpose computing, neural networks can be further combined with modern graphics pipeline using shader implementation.

In this research, the possibilities of shader-based neural networks to be used as an alternative to traditional lighting models are explored. Fully connected neural networks are implemented in fragment shader to reproduce lighting results in the graphics pipeline, and trained in compute shaders. Implemented networks are proved to be able to approximate mathematical lighting models. In this thesis, experiments are described to prove the ability of shader-based neural networks, to explore the proper network architecture and settings for different lighting models. Further explorations of possibilities of manually editing parameters are also described. Mean-square errors and runtime are taken as measurements of success to evaluate the experiments. Rendered images are also reported for visual comparison and evaluation.