A Principal Component Analysis Approach for Embedding Local Symmetries into Deep Learning Algorithms

Building robust-by-design Machine Learning algorithms is key for critical tasks such as safety or military applications. By leveraging on the ideas developed in the context of building invariant Support Vectors Machines, this paper introduces a convenient methodology for embedding local Lie groups symmetries into Deep Learning algorithms by performing a Principal Component Analysis on the corresponding Tangent Covariance Matrix. The projection of the input data onto the principal directions leads to a new data representation which allows singling out the components conveying the semantic information useful to the considered algorithmic task while reducing the dimension of the input manifold. Besides, our numerical testing emphasizes that, although less efficient than using Group-Convolutional Neural Networks as only dealing with local symmetries, our approach does improve accuracy and robustness without introducing significant computational overhead. Performance improvements up to 5% were obtained for low capacity algorithms, making this approach of particular interest for the engineering of safe embedded Artificial Intelligence systems.