BioEgo-HSG: A Memory-Augmented Hierarchical Skill-Graph Framework for Long-Horizon Biomedical Laboratory Automation from Egocentric Videos

Learning procedural knowledge from human egocentric videos offers a source of demonstrations for long-horizon biomedical laboratory automation. Such workflows span functional zones, depend on persistent object and workspace states, and include recovery from procedural deviations. This paper formulates BioEgo-HSG as a memory-augmented hierarchical skill-graph framework for this setting. Laboratory Interaction-Centric Tokens combine hand–object relations, object-state cues, zone information, task progress, and confidence. A multi-scale temporal parser organizes demonstrations into procedure phases, functional subtasks, and embodiment-agnostic atomic skills. Episodic procedural memory carries object, zone, task-progress, and failure-recovery information across changes in visibility. A hierarchical skill graph represents temporal, prerequisite, dependency, transition, alternative, and recovery relations. A graph-constrained planner uses this representation and the memory state to select semantic skills and handle postcondition mismatches through a local execution interface. The contribution is the framework formulation and its procedural representation, memory mechanism, and skill-graph planning interface. The reported offline summaries and physical demonstrations provide preliminary feasibility observations within the studied laboratory setting.