Embodied Temporal Structure and Grouping in Pianists’ Body Motion and Sound

This study investigated how temporal features and hierarchical groupings are embodied through body motion and performance sound in solo classical piano repertoire. Kinematic and audio data were collected from an extensive dataset of 240 performances. A semi-automatic alignment method using music information retrieval (MIR) techniques linked note-level timing with score-based temporal structures. Principal component analysis (PCA) was used to extract torso motion features. Along with sound features (spectral flux and amplitude envelope) these were analyzed in relation to formal temporal elements (meter and tempo), as well as hierarchical grouping layers (beat, downbeat, phrase, and cadence). Findings showed distinct roles of body motion and performance sound in representing different levels of musical time. Antero-posterior torso motion corresponded more closely with beat-level structures, while both medio-lateral and antero-posterior motion aligned with higher-level groupings. Differences in medio-lateral torso motion were observed across multiple hierarchical grouping levels. Auditory features emphasized metrical accents and decay at structural boundaries. The study contributes to the understanding of cross-modal embodiment in music performance by combining detailed motion and sound analysis within a clearly articulated theoretical framework.