Visual search in 3D-modelled rooms: comparing results from the same protocol run in VR and on 2D screens

Erwan David & Melissa L.-H. Võ
Scene Grammar Lab, Department of Psychology, Goethe University Frankfurt, Germany

A virtual reality headset is an incredibly useful experimental tool because it allows for unrestricted movements compared to on-screen protocols, giving experimenters full control over stimuli and measurements/tracking. Together with an eye tracking device, virtual reality opens the door to protocols that would have been impossible on-screen and very limited or laborious in field studies. However, eye tracking in virtual reality requires access to such equipment, particular programming knowledge and appropriate analysis skills. In a two-part study, we asked if gaze dynamics of visual search in VR could be replicated in more accessible online studies. We wanted to measure the suitability of online testing (without eye tracking) as a proxy for measurements of visual behaviors in 3D-environments. The protocol implemented had participants search for targets in fully-furnished 3D-modelled indoor rooms. Search targets were located in plain view or stored inside containers (e.g., closet). Participants could interact with containers to open and look inside them. In the online version of the protocol participants used their keyboard and mouse to control camera movement. Results from the VR and online study show comparable search performances. Interestingly, we show that camera movements in the online variant (keyboard/mouse) mimicked gaze properties. For example, camera velocity signals exhibited rests and peaks reminiscent of fixations and saccades. We processed camera motion signals as we would do for gaze signals to further compare virtual reality and online measurements and discuss the implications of this method to make it easier to study gaze behaviors in increasingly realistic, dynamic tasks.

Keywords: visual search, virtual reality, online study, gaze

Acknowledgements: This work was supported by SFB/TRR 26 135 project C7 to Melissa L.-H. Võ and the Hessisches Ministerium für Wissenschaft und Kunst (HMWK; project ‘The Adaptive Mind’).