Characterizing multidimensional facial synchrony for relationship quality detection

Relationship quality is typically assessed using self-report questionnaires, which are time-consuming and may not fully capture interpersonal processes during interaction. Automated facial expression analysis provides an opportunity to study nonverbal behavior during naturalistic interactions in a more objective and continuous manner. We characterize dyadic facial synchrony from video-based facial action unit time series and examine its association with marital relationship quality. Specifically, dyadic coordination is quantified using zero-lag cross-correlations between partners’ facial behaviors across multiple action units. Computed over successive temporal windows, these synchrony measures form a multidimensional synchrony profile, termed the S-matrix. We evaluate the proposed method using an iMotions dataset comprising video recordings of in-person conversations, frame-level facial expression estimates, and Quality of Marriage Index (QMI) survey responses from 27 married couples. Results from an unsupervised clustering analysis demonstrate that the proposed synchrony representation provides better separation between couples with high and low marital relationship quality than conventional statistical feature baselines.