Learning and Shape Analysis of Image Manifolds of Three-Dimensional Objects

Abstract.

Despite high-dimensionality of images, the sets of images of three-dimensional objects have long been hypothesized to form low-dimensional manifolds, a notion expressed as the so-called manifold hypothesis. What is the nature of such manifolds? How do their geometries vary across objects and object classes? Answering these questions can provide key insights into explaining and advancing the success of machine learning algorithms in computer vision. This paper investigates dual tasks—learning and analyzing shapes of image manifolds—by revisiting a classical problem of manifold learning but from a novel geometrical perspective. It uses geometry-preserving transformations to map the pose manifolds, sets of images formed by rotating three-dimensional objects, to low-dimensional latent spaces. The low-dimensionality of latent spaces facilitates estimation and analysis of these pose manifolds using training data. This paper compares shapes of these manifolds in latent spaces, or latent manifolds, of different three-dimensional objects, using Kendall’s shape analysis, i.e., modulo rigid motions and global scaling. Interestingly, latent manifolds for objects from the same class are frequently similar in geometries. We demonstrate the use of latent geometries to simplify several image processing tasks, such as image interpolations via geodesics, image denoising via manifold projections, and image classification using distances to manifolds.