Arti4D: Statistical Analysis and Modelling of the Spatio‐temporal Variability in Articulated 4D Shapes

Abstract

We propose a novel framework for the statistical modeling and analysis of the spatio‐temporal shape variability in articulated 4D (i.e., 3D + time) shapes such as human bodies and animals. We treat articulated 3D shapes, represented using parametric models such as SMPL or its variants, as elements of the product space of shape and pose parameters. 4D shapes can then be seen as trajectories in this space, which has a nonlinear Riemannian structure. Our key contribution is to treat these trajectories as elements of a Riemannian shape space and propose computational tools that ( 1 ) perform temporal alignment of such trajectories to account for variations in their execution rates, ( 2 ) compute geodesics between trajectories, and thus 4D shapes, even when they exhibit different execution rates, and ( 3 ) statistically model the spatio‐temporal variability of collections of 4D shapes, enabling us to compute statistical summaries such as means and principal modes of variation. We derive a simple, yet efficient, framework for characterizing populations of 4D shapes using statistical models, which in turn can be used as a generative model for synthesizing novel 4D shapes by sampling from these distributions. We demonstrate the effectiveness of the proposed framework using publicly available 4D human and animal datasets, and show that it outperforms the state‐of‐the‐art both in terms of accuracy and computational efficiency. Our code, dataset, and videos that illustrate the results are available at https://arti4d.github.io/Arti4D/ .