Mitigating Data Imbalance in Head Pose Estimation Through Diffusion Model Generated Synthetic Data with Pose Reference

Many head pose estimation approaches are limited by the scarcity of diverse, accurately annotated datasets, especially for skewed facial orientations. Such extreme angles occur far less frequently in real-world data collection, leaving them severely underrepresented. This weakens trained models exactly at the extreme angles they had few chances to learn. Rather than perturbing existing samples, we generate genuinely new ones where data is sparse: a synthetic data framework using the SDXL diffusion model conditioned on OpenPose-derived keypoint maps, with InstantID for identity preservation. Because the produced pose can differ from the request, every image is reannotated by an independent 3D estimator (DAD-3DNet) rather than labeled by its target angle, yielding a more balanced pose distribution. We evaluate cross-dataset generalization using the Geodesic Error across two backbones, 6DRepNet and Hopenet. Adding the synthetic data to real training lowers cross-dataset error for both backbones, most clearly on unseen subjects, showing it is an effective complement to real data at rare orientations.