Product-Manifold Contrastive Learning for Characterizing Neural Representations of 3D Visual Transformations in the Avian Visual System
Xingtong Wang, Pingge Hu, Xiaoteng Zhang, Li ShiMost neural decoding methods prioritize predictive accuracy but provide limited means to test hypotheses about latent representational geometry. Guided by projection geometry, we recorded neural activity from the ENTO–MVL pathway of five pigeons performing matched 2D rotation, 2D stretch/compression, and 3D view-rotation tasks. Preliminary response analysis identified two groups with different 2D transformation preferences, both responsive during 3D view rotation, motivating a hypothesis of joint engagement. We introduce ProMaC, a product-manifold contrastive framework with specialist encoders and a prespecified S1 × ℝ coordinate prior, to test this hypothesis through cross-transformation generalization. Trained exclusively on 2D tasks, frozen branches recovered silhouette-derived projected rotation and log-scale descriptors from held-out 3D trials and aligned 2D and 3D centroids, outperforming matched controls (Holm-adjusted p < 0.001). Fixed-time-bin analyses supported correspondence beyond shared temporal progress, and leave-one-pigeon-out sensitivity analyses retained positive model advantages. Auxiliary analyses showed factor selectivity and greater projected-descriptor accuracy for direct ProMaC-Joint predictions than for angle-mediated predictions, without establishing information beyond physical angle. Persistent homology supported a stable loop but did not establish a complete product manifold. These findings support the generalization of factor-specific, 2D-derived coordinates along the tested one-parameter 3D view-rotation trajectories.