Algorithmic Advances in Riemannian Geometry and Applications

For Machine Learning, Computer Vision, Statistics, and Optimization

herausgegeben von: Hà Quang Minh, Vittorio Murino

Verlag: Springer International Publishing

Buchreihe : Advances in Pattern Recognition

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten

Über dieses Buch

This book presents a selection of the most recent algorithmic advances in Riemannian geometry in the context of machine learning, statistics, optimization, computer vision, and related fields. The unifying theme of the different chapters in the book is the exploitation of the geometry of data using the mathematical machinery of Riemannian geometry. As demonstrated by all the chapters in the book, when the data is intrinsically non-Euclidean, the utilization of this geometrical information can lead to better algorithms that can capture more accurately the structures inherent in the data, leading ultimately to better empirical performance. This book is not intended to be an encyclopedic compilation of the applications of Riemannian geometry. Instead, it focuses on several important research directions that are currently actively pursued by researchers in the field. These include statistical modeling and analysis on manifolds,optimization on manifolds, Riemannian manifolds and kernel methods, and dictionary learning and sparse coding on manifolds. Examples of applications include novel algorithms for Monte Carlo sampling and Gaussian Mixture Model fitting, 3D brain image analysis,image classification, action recognition, and motion tracking.

Inhaltsverzeichnis

Frontmatter

Chapter 1. Bayesian Statistical Shape Analysis on the Manifold of Diffeomorphisms

Abstract

In this chapter, we present Bayesian models for diffeomorphic shape variability in populations of images. The first model is a probabilistic formulation of the image atlas construction problem, which seeks to compute an atlas image most representative of a set of input images. The second model adds diffeomorphic modes of shape variation, or principal geodesics. Both of these models represent shape variability as random variables on the manifold of diffeomorphic transformations. We define a Gaussian prior distribution for diffeomorphic transformations using the inner product in the tangent space to the diffeomorphism group. We develop a Monte Carlo Expectation Maximization (MCEM) algorithm for the Bayesian inference, due to the lack of closed-form solutions, where the expectation step is approximated via Hamiltonian Monte Carlo (HMC) sampling of diffeomorphisms. The resulting inference produces estimates of the image atlas, principal geodesic modes of variation, and model parameters. We show that the advantage of the Bayesian formulation is that it provides a principled way to estimate both the regularization parameter of the diffeomorphic transformations and the intrinsic dimensionality of the input data.

Miaomiao Zhang, P. Thomas Fletcher

Chapter 2. Sampling Constrained Probability Distributions Using Spherical Augmentation

Abstract

Statistical models with constrained probability distributions are abundant in machine learning. Some examples include regression models with norm constraints (e.g., Lasso), probit, many copula models, and latent Dirichlet allocation (LDA). Bayesian inference involving probability distributions confined to constrained domains could be quite challenging for commonly used sampling algorithms. In this work, we propose a novel augmentation technique that handles a wide range of constraints by mapping the constrained domain to a sphere in the augmented space. By moving freely on the surface of this sphere, sampling algorithms handle constraints implicitly and generate proposals that remain within boundaries when mapped back to the original space. Our proposed method, called Spherical Augmentation, provides a mathematically natural and computationally efficient framework for sampling from constrained probability distributions. We show the advantages of our method over state-of-the-art sampling algorithms, such as exact Hamiltonian Monte Carlo, using several examples including truncated Gaussian distributions, Bayesian Lasso, Bayesian bridge regression, reconstruction of quantized stationary Gaussian process, and LDA for topic modeling.

Shiwei Lan, Babak Shahbaba

Chapter 3. Geometric Optimization in Machine Learning

Abstract

Machine learning models often rely on sparsity, low-rank, orthogonality, correlation, or graphical structure. The structure of interest in this chapter is geometric, specifically the manifold of positive definite (PD) matrices. Though these matrices recur throughout the applied sciences, our focus is on more recent developments in machine learning and optimization. In particular, we study (i) models that might be nonconvex in the Euclidean sense but are convex along the PD manifold; and (ii) ones that are neither Euclidean nor geodesic convex but are nevertheless amenable to global optimization. We cover basic theory for (i) and (ii); subsequently, we present a scalable Riemannian limited-memory BFGS algorithm (that also applies to other manifolds). We highlight some applications from statistics and machine learning that benefit from the geometric structure studies.

Suvrit Sra, Reshad Hosseini

Chapter 4. Positive Definite Matrices: Symmetric positive definite (SPD) matrices Data RepresentationData representation and Applications to Computer Vision

Abstract

Numerous applications in computer vision and machine learning rely on representations of data that are compact, discriminative, and robust while satisfying several desirable invariances. One such recently successful representation is offered by symmetric positive definite (SPD) matrices. However, the modeling power of SPD matrices comes at a price: rather than a flat Euclidean view, SPD matrices are more naturally viewed through curved geometry (Riemannian or otherwise) which often complicates matters. We focus on models and algorithms that rely on the geometry of SPD matrices, and make our discussion concrete by casting it in terms of covariance descriptors for images. We summarize various commonly used distance metrics on SPD matrices, before highlighting formulations and algorithms for solving sparse coding and dictionary learning problems involving SPD data. Through empirical results, we showcase the benefits of mathematical models that exploit the curved geometry of SPD data across a diverse set of computer vision applications.

Anoop Cherian, Suvrit Sra

Chapter 5. From Covariance Matrices to Covariance Operators: Data Representation from Finite to Infinite-Dimensional Settings

Abstract

This chapter presents some of the recent developments in the generalization of the data representation framework using finite-dimensional covariance matrices to infinite-dimensional covariance operators in Reproducing Kernel Hilbert Spaces (RKHS). We show that the proper mathematical setting for covariance operators is the infinite-dimensional Riemannian manifold of positive definite Hilbert–Schmidt operators, which are the generalization of symmetric, positive definite (SPD) matrices. We then give the closed form formulas for the affine-invariant and Log-Hilbert–Schmidt distances between RKHS covariance operators on this manifold, which generalize the affine-invariant and Log-Euclidean distances, respectively, between SPD matrices. The Log-Hilbert–Schmidt distance in particular can be used to design a two-layer kernel machine, which can be applied directly to a practical application, such as image classification. Experimental results are provided to illustrate the power of this new paradigm for data representation.

Hà Quang Minh, Vittorio Murino

Chapter 6. Dictionary Learning on Grassmann Manifolds

Abstract

Sparse representations have recently led to notable results in various visual recognition tasks. In a separate line of research, Riemannian manifolds have been shown useful for dealing with features and models that do not lie in Euclidean spaces. With the aim of building a bridge between the two realms, we address the problem of sparse coding and dictionary learning in Grassmann manifolds, i.e, the space of linear subspaces. To this end, we introduce algorithms for sparse coding and dictionary learning by embedding Grassmann manifolds into the space of symmetric matrices. Furthermore, to handle nonlinearity in data, we propose positive definite kernels on Grassmann manifolds and make use of them to perform coding and dictionary learning.

Mehrtash Harandi, Richard Hartley, Mathieu Salzmann, Jochen Trumpf

Chapter 7. Regression on Lie Groups and Its Application to Affine Motion Tracking

Abstract

In this chapter, we present how to learn regression models on Lie groups and apply our formulation to visual object tracking tasks. Many transformations used in computer vision, for example orthogonal group and rotations, have matrix Lie group structure. Unlike conventional methods that proceed by directly linearizing these transformations, thus, making an implicit Euclidean space assumption, we formulate a regression model on the corresponding Lie algebra that minimizes a first order approximation to the geodesic error. We demonstrate our method on affine motions, however, it generalizes to any matrix Lie group transformations.

Fatih Porikli

Chapter 8. An Elastic Riemannian Framework for Shape AnalysisShape analysis of Curves and Tree-Like Structures

Abstract

Shape analysis of complex structures formed by Euclidean curves and trees are of interest in many scientific domains. The difficulty in this analysis comes from: (1) Manifold representations of curves and trees, and (2) need for the analysis to be invariant to certain shape-preserving transformations. Additionally, one is faced with the difficult task of registering points and parts across objects during shape comparisons. We present a Riemannian framework to solve this problem as follows: we select an elastic Riemannian metric that is invariant to the action of re-parameterization group and use a square-root velocity function to transform this metric into the \(\mathbb {L}^2\) norm. Re-parameterization of objects is considered to control registrations across objects, and an inherited distance on the quotient space of shape representations modulo shape-preserving transformations forms the shape metric. The resulting framework is used to compute geodesic paths and sample means, and to discover modes of variability in sample shapes. We demonstrate these ideas using some simple examples involving planar shapes and neuron morphology.

Adam Duncan, Zhengwu Zhang, Anuj Srivastava

Backmatter

Titel: Algorithmic Advances in Riemannian Geometry and Applications
herausgegeben von: Hà Quang Minh
Vittorio Murino
Verlag: Springer International Publishing
Electronic ISBN: 978-3-319-45026-1
Print ISBN: 978-3-319-45025-4
DOI: https://doi.org/10.1007/978-3-319-45026-1