DOI: 10.3390/info17111106 ISSN: 2078-2489

A Self-Supervised Hierarchical Motion Tokenizer for Low-Label Cross-Domain Ethnic Dance Retrieval and Digital Preservation: A Motion Capture Case Study of Mongolian Dance

Shiyang Zhang, Tao Lin

Motion capture archives preserve joint-level trajectories, but their reuse remains difficult when the target collection has few labeled examples. We study cross-domain transfer in a low-label regime in which 1%, 5%, 10%, or 25% of target-domain motif annotations are available, while the 100% condition is retained only as a fully supervised reference. HMT-DP, a self-supervised Hierarchical Motion Tokenizer for Digital Preservation, combines a part-aware graph transformer with masked motion reconstruction, kinematic consistency, view-invariant contrastive learning, and balanced prototype assignment. Frame-, motif-, and phrase-level tokens represent distinct semantic scales within fixed-duration input windows, and every token is linked to its source dataset, sequence, view, and temporal interval. With 100% target labels, HMT-DP achieved 89.1 ± 0.5% macro-F1 and 70.4 ± 0.6% cross-view mean average precision. In the low-label regime, macro-F1 was 42.6 ± 2.0%, 57.9 ± 1.3%, 66.8 ± 0.9%, and 78.4 ± 0.7% at 1%, 5%, 10%, and 25% labels, respectively. Under 20% random joint masking, HMT-DP achieved 84.6 ± 0.6%. Motif tokens provided 16-fold motion stream compression with a mean per joint position error of 45.8 ± 0.8 mm; complete repository accounting yielded an effective 3.1-fold reduction after codebooks, descriptors, provenance records, and the retrieval index were included. A manually segmented boundary subset yielded a tolerance-based boundary F1 of 79.8 ± 1.1%. Within the YNU-Dance Mongolian case study protocol, HMT-DP therefore provides a compact, searchable representation whose retrieval results remain linked to the recorded source intervals.

More from our Archive