Development of two‐phase framework for driver behavior classification and representative motion prototyping for a reliable simulation‐based driver monitoring system

Abstract

Driver monitoring systems (DMS) are critical for safe autonomous driving, which requires accurate behavior recognition and reliable simulation environments for validation. This study proposes a robust two‐phase framework that combines a classification model and a prototype selection procedure to automatically generate realistic motion baselines from the acquired 3D skeleton data. Phase 1 uses a spatial temporal graph convolutional network to classify driver behaviors, and Phase 2 leverages class‐wise mean trajectories to interpret motion prototypes and select the closest real sequence, creating a verifiable motion trajectory library for simulation‐based safety testing. Our approach addresses DMS challenges through prototype visualization, robustness via discriminative embeddings, and standardized simulation scenarios, making it suitable for human–AI interactions in automotive safety systems. This framework produces a motion library of a real recorded sequence that can be directly used for simulation scenario generation and demonstrated through Metahuman retargeting and visualization in Unreal Engine.