Purdue University Graduate School
Singleton_William_Thesis_4_20_2020.pdf (2.13 MB)

Increasing CNN Representational Power Using Absolute Cosine Value Regularization

Download (2.13 MB)
posted on 2020-04-21, 21:19 authored by William Steven SingletonWilliam Steven Singleton
The Convolutional Neural Network (CNN) is a mathematical model designed to distill input information into a more useful representation. This distillation process removes information over time through a series of dimensionality reductions, which ultimately, grant the model the ability to resist noise, and generalize effectively. However, CNNs often contain elements that are ineffective at contributing towards useful representations. This Thesis aims at providing a remedy for this problem by introducing Absolute Cosine Value Regularization (ACVR). This is a regularization technique hypothesized to increase the representational power of CNNs by using a Gradient Descent Orthogonalization algorithm to force the vectors that constitute their filters at any given convolutional layer to occupy unique positions in Rn. This method should in theory, lead to a more effective balance between information loss and representational power, ultimately, increasing network performance. The following Thesis proposes and examines the mathematics and intuition behind ACVR, and goes on to propose Dynamic-ACVR (D-ACVR). This Thesis also proposes and examines the effects of ACVR on the filters of a low-dimensional CNN, as well as the effects of ACVR and D-ACVR on traditional Convolutional filters in VGG-19. Finally, this Thesis proposes and examines regularization of the Pointwise filters in MobileNetv1.


Degree Type

  • Master of Science in Electrical and Computer Engineering


  • Electrical and Computer Engineering

Campus location

  • Indianapolis

Advisor/Supervisor/Committee Chair

Dr. Mohamed A. El-Sharkawy

Additional Committee Member 2

Dr. Brian S. King

Additional Committee Member 3

Dr. Dongsoo S. Kim