Cross-Reality Interaction Techniques: Interacting with Physical Reality while Staying in Virtual Reality

Head-mounted displays (HMDs) allow players to immerse themselves in virtual reality (VR). However, players often encounter interruptions that require them to temporarily leave the virtual environment (VE), such as responding to a message or a person entering the room. Interruptions can cause issues both upon leaving and re-entering VR, such as disorientation, frustration, and breaks in presence. To address these challenges, we designed and implemented five cross-reality interaction techniques that allow users to remain in VR while viewing the real world in alternative ways. We evaluated them in a mixed-methods user study (n = 21), where participants completed real-world interruption tasks during VR gameplay, and compared them to physically removing the HMD and using a controller-enabled, embedded full passthrough mode. All techniques were strongly preferred over removing the HMD and helped participants maintain awareness of the VE during interruptions. Participants particularly preferred embedded full passthrough because it enabled rapid transitions between realities. More broadly, instead of focusing on one reality at a time, participants frequently alternated attention between virtual and physical realities to maintain awareness of both. Our findings highlight the importance of lightweight cross-reality interactions for supporting interruptions, gameplay continuity, and mixed-attention play in future VR experiences.