Beyond Novice and Expert: Domain Proficiency in Simulation-Based Evaluation of Geo-Located Augmented Reality Interfaces

This paper investigates how Quality of Experience (QoE) and User Experience (UX) evaluations of a geo-located augmented reality (AR) interface designed to support navigation and situational awareness in a security context differ across participant groups with distinct domain-related backgrounds. To address the limitations of current wearable AR hardware, the study used a first-person video game simulation of a person using AR glasses in an urban security scenario. The proposed interface consisted of three core elements: Map in the Sky, navigational arrows, and tags for objects or persons of interest. Data from two user studies conducted under the same experimental protocol were combined, yielding three participant groups with different domain-related backgrounds: Civilians, Veterans, and Cadets. The interface was evaluated using the Acceptance Scale (AS), the System Usability Scale (SUS), NASA-RTLX, and qualitative interviews. The results showed generally positive evaluations of the interface and all individual elements, with high SUS-rated usability, relatively low-to-moderate reported workload, and predominantly favorable interview responses. However, the pattern of results differed across participant groups. Civilians provided the most positive overall usability ratings; Veterans also evaluated the interface positively but reported the highest workload, and Cadets produced the least favorable usability and acceptance scores despite greater self-reported familiarity with AR and video games. Across groups, Map in the Sky received the lowest ratings, whereas arrows and tags were evaluated more positively. These findings indicate that AR QoE and UX outcomes can differ across participant groups with distinct domain-related backgrounds in ways that do not follow a simple novice-to-expert ordering. The results also demonstrate the practical feasibility of using PC-based simulation to obtain early-stage subjective QoE/UX feedback, while highlighting the importance of participant selection when evaluating domain-specific AR systems. Because the study did not include objective performance measures or a comparison condition, the findings reflect perceived usability, acceptance, and workload rather than operational effectiveness or workload reduction.