Ken Pfeuffer
Daniel Buschek
Florian Alt
ABSTRACT
Every person is unique, with individual behavioural characteristics: how one moves, coordinates, and uses their body. In this paper we investigate body motion as behavioural biometrics for virtual reality. In particular, we look into which behaviour is suitable to identify a user. This is valuable in situations where multiple people use a virtual reality environment in parallel, for example in the context of authentication or to adapt the VR environment to users' preferences. We present a user study (N=22) where people perform controlled VR tasks (pointing, grabbing, walking, typing), monitoring their head, hand, and eye motion data over two sessions. These body segments can be arbitrarily combined into body relations, and we found that these movements and their combination lead to characteristic behavioural patterns. We present an extensive analysis of which motion/relation is useful to identify users in which tasks using classification methods. Our findings are beneficial for researchers and practitioners alike who aim to build novel adaptive and secure user interfaces in virtual reality.
Supplemental Material
Available for Download
zip
pn7792.zip (69 KB)
The zip file contains one PDF file. It provides the user identification accuracy and F1 scores. The paper does not report F1 to save space, thus it is here a supplementary material.
- Virginia O. Andersson and Ricardo M. Araujo. 2015. Person Identification Using Anthropometric and Gait Data from Kinect Sensor. In Behavioural Biometrics in VR: Identifying People from Body Motion and Relations in Virtual Reality CHI 2019, May 4--9, 2019, Glasgow, Scotland Uk Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI'15). AAAI Press, 425--431. http://dl.acm.org/citation. cfm?id=2887007.2887067 Google Scholar
Digital Library
- Corey Ashby, Amit Bhatia, Francesco Tenore, and Jacob Vogelstein. 2011. Low-cost electroencephalogram (EEG) based authentication. In Neural Engineering (NER), 2011 5th International IEEE/EMBS Conference on. IEEE, 442--445.Google ScholarCross Ref
- Kyle O Bailey, James S Okolica, and Gilbert L Peterson. 2014. User identification and authentication using multi-modal behavioral biometrics. Computers & Security 43 (2014), 77--89.Google ScholarCross Ref
- VE Belenkii, VS Gurfinkel, and EI Paltsev. 1967. ON THE CONTROL ELEMENTS OF VOLUNTARY MOVEMENTS. Biofizika (1967).Google Scholar
- Chiraz BenAbdelkader, Ross Cutler, and Larry Davis. 2002. Viewinvariant estimation of height and stride for gait recognition. In International Workshop on Biometric Authentication. Springer, 155--167. Google ScholarDigital Library
- Mark Billinghurst, Jerry Bowskill, Nick Dyer, and Jason Morphett. 1998. Spatial information displays on a wearable computer. IEEE Computer Graphics and Applications 18, 6 (1998), 24--31. Google ScholarDigital Library
- Cheng Bo, Lan Zhang, Xiang-Yang Li, Qiuyuan Huang, and Yu Wang. 2013. Silentsense: silent user identification via touch and movement behavioral biometrics. In Proceedings of the 19th annual international conference on Mobile computing & networking. ACM, 187--190. Google ScholarDigital Library
- Edward G Britton, James S Lipscomb, and Michael E Pique. 1978. Making nested rotations convenient for the user. In ACM SIGGRAPH Computer Graphics, Vol. 12. ACM, 222--227. Google ScholarDigital Library
- Grigore C Burdea and Philippe Coiffet. 2003. Virtual reality technology. John Wiley & Sons. Google ScholarDigital Library
- Daniel Buschek, Alexander De Luca, and Florian Alt. 2015. Improving Accuracy, Applicability and Usability of Keystroke Biometrics on Mobile Touchscreen Devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1393--1402. Google ScholarDigital Library
- Antitza Dantcheva, Carmelo Velardo, Angela D'angelo, and JeanLuc Dugelay. 2011. Bag of soft biometrics for person identification. Multimedia Tools and Applications 51, 2 (2011), 739--777. Google ScholarDigital Library
- Alexander De Luca, Alina Hang, Frederik Brudy, Christian Lindner, and Heinrich Hussmann. 2012. Touch Me Once and I Know It's You!: Implicit Authentication Based on Touch Screen Patterns. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 987--996. Google ScholarDigital Library
- Simon Eberz, Kasper B Rasmussen, Vincent Lenders, and Ivan Martinovic. 2017. Evaluating behavioral biometrics for continuous authentication: Challenges and metrics. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security. ACM, 386--399. Google ScholarDigital Library
- Shaun Gallagher and Daniel Schmicking. 2010. Handbook of phenomenology and cognitive science. Springer.Google Scholar
- Yves Guiard. 1987. Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. In Journal of Motor Behavior, Vol. 19. 486--517.Google ScholarCross Ref
- Christian Holz, Senaka Buthpitiya, and Marius Knaust. 2015. Bodyprint: Biometric User Identification on Mobile Devices Using the Capacitive Touchscreen to Scan Body Parts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 3011--3014. Google ScholarDigital Library
- ISO ISO. {n. d.}. 9241--9 Ergonomic requirements for office work with visual display terminals (VDTs)-Part 9: Requirements for non-keyboard input devices, 2000. International Organization for Standardization ({n. d.}).Google Scholar
- Robert J.K. Jacob, Audrey Girouard, Leanne M. Hirshfield, Michael S. Horn, Orit Shaer, Erin Treacy Solovey, and Jamie Zigelbaum. 2008. Reality-based Interaction: A Framework for post-WIMP Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 201--210. Google ScholarDigital Library
- Anil K Jain, Sarat C Dass, and Karthik Nandakumar. 2004. Soft biometric traits for personal recognition systems. In Biometric authentication. Springer, 731--738.Google Scholar
- Erno Jeges, Istvan Kispal, and Zoltan Hornak. 2008. Measuring human height using calibrated cameras. In Human System Interactions, 2008 Conference on. IEEE, 755--760.Google ScholarCross Ref
- Roland S Johansson, Göran Westling, Anders Bäckström, and J Randall Flanagan. 2001. Eye--hand coordination in object manipulation. Journal of Neuroscience 21, 17 (2001), 6917--6932.Google ScholarCross Ref
- Georgios Kambourakis, Dimitrios Damopoulos, Dimitrios Papamartzivanos, and Emmanouil Pavlidakis. 2016. Introducing touchstroke: keystroke-based authentication system for smartphones. Security and Communication Networks 9, 6 (2016), 542--554. Google ScholarDigital Library
- George H. Kohavi, Ron and John. 1997. Wrappers for feature subset selection. Artificial Intelligence 97, 1 (1997), 273--324. Google ScholarDigital Library
- Alexander Kupin, Benjamin Moeller, Yijun Jiang, Natasha Kholgade Banerjee, and Sean Banerjee. 2019. Task-Driven Biometric Authentication of Users in Virtual Reality (VR) Environments. In MultiMedia Modeling - 25th International Conference, MMM 2019, Thessaloniki, Greece, January 8--11, 2019, Proceedings, Part I. 55--67.Google Scholar
- Nurakhmed Nurislamovich Latypov and Nurulla Nurislamovich Latypov. 1999. Method for tracking and displaying user's spatial position and orientation, a method for representing virtual reality for a user, and systems of embodiment of such methods. US Patent 6,005,548.Google Scholar
- Sugang Li, Ashwin Ashok, Yanyong Zhang, Chenren Xu, Janne Lindqvist, and Macro Gruteser. 2016. Whose move is it anyway? Authenticating smart wearable devices using unique head movement patterns. In Pervasive Computing and Communications (PerCom), 2016 IEEE International Conference on. IEEE, 1--9.Google ScholarCross Ref
- Hanneke Liesker, Eli Brenner, and Jeroen BJ Smeets. 2009. Combining eye and hand in search is suboptimal. Experimental brain research 197, 4 (2009), 395--401.Google Scholar
- Feng Lin, Kun Woo Cho, Chen Song, Wenyao Xu, and Zhanpeng Jin. 2018. Brain Password: A Secure and Truly Cancelable Brain Biometrics for Smart Headwear. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 296-- 309. Google ScholarDigital Library
- Pedro Lopes, Alexandra Ion, Willi Müller, Daniel Hoffmann, Patrik Jonell, and Patrick Baudisch. 2015. Proprioceptive Interaction. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, New York, NY, USA, 175--175. Google ScholarDigital Library
- I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). ACM, New York, NY, USA, 754--755. Google ScholarDigital Library
- Jean Massion. 1992. Movement, posture and equilibrium: interaction and coordination. Progress in neurobiology 38, 1 (1992), 35--56.Google Scholar
- B Miller. 1988. Everything you need to know about biometric identification. Personal Identification News 1988 Biometric Industry Directory (1988).Google Scholar
- Mark R Mine, Frederick P Brooks Jr, and Carlo H Sequin. 1997. Moving objects in space: exploiting proprioception in virtual-environment interaction. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-Wesley Publishing Co., 19--26. CHI 2019, May 4--9, 2019, Glasgow, Scotland Uk Pfeuffer et al. Google ScholarDigital Library
- Brent C. Munsell, Andrew Temlyakov, Chengzheng Qu, and Song Wang. 2012. Person Identification Using Full-body Motion and Anthropometric Biometrics from Kinect Videos. In Proceedings of the 12th International Conference on Computer Vision - Volume Part III (ECCV'12). Springer-Verlag, Berlin, Heidelberg, 91--100. Google ScholarDigital Library
- Tahrima Mustafa, Richard Matovu, Abdul Serwadda, and Nicholas Muirhead. 2018. Unsure How to Authenticate on Your VR Headset?: Come on, Use Your Head!. In Proceedings of the Fourth ACM International Workshop on Security and Privacy Analytics (IWSPA '18). ACM, New York, NY, USA, 23--30. Google ScholarDigital Library
- Vitruvius Pollio and M. H. Morgan. 1960. De architectura. New York: Dover Publications.Google Scholar
- Daniel A Reid and Mark S Nixon. 2011. Using comparative human descriptions for soft biometrics. In Biometrics (IJCB), 2011 International Joint Conference on. IEEE, 1--6. Google ScholarDigital Library
- Ioannis Rigas, Oleg Komogortsev, and Reza Shadmehr. 2016. Biometric recognition via eye movements: Saccadic vigor and acceleration cues. ACM Transactions on Applied Perception (TAP) 13, 2 (2016), 6. Google ScholarDigital Library
- Yiran Shen, Hongkai Wen, Chengwen Luo, Weitao Xu, Tao Zhang, Wen Hu, and Daniela Rus. 2018. GaitLock: Protect virtual and augmented reality headsets using gait. IEEE Transactions on Dependable and Secure Computing (2018).Google Scholar
- Marco Speicher, Anna Maria Feit, Pascal Ziegler, and Antonio Krüger. 2018. Selection-based Text Entry in Virtual Reality. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 647, 13 pages. Google ScholarDigital Library
- Robert John Teather. 2013. Evaluating 3D pointing techniques. (2013).Google Scholar
- Robert J Teather and Wolfgang Stuerzlinger. 2011. Pointing at 3D targets in a stereo head-tracked virtual environment. In 3D User Interfaces (3DUI), 2011 IEEE Symposium on. IEEE, 87--94. Google ScholarDigital Library
- Colin Ware and Harutune H. Mikaelian. 1987. An Evaluation of an Eye Tracker As a Device for Computer Input. In Proceedings of the SIGCHI/GI Conference on Human Factors in Computing Systems and Graphics Interface (CHI '87). ACM, New York, NY, USA, 183--188. Google ScholarDigital Library
- J Wayman. 2000. A definition of biometrics. National Biometric Test Center Collected Works 1, 2 (2000), 21--23.Google Scholar
- Shanhe Yi, Zhengrui Qin, Ed Novak, Yafeng Yin, and Qun Li. 2016. Glassgesture: Exploring head gesture interface of smart glasses. In Computer Communications, IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on. IEEE, 1--9.Google ScholarDigital Library
- Yongtuo Zhang, Wen Hu, Weitao Xu, Chun Tung Chou, and Jiankun Hu. 2018. Continuous Authentication Using Eye Movement Response of Implicit Visual Stimuli. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 4 (2018), 177. Google ScholarDigital Library
- Zhaoxiang Zhang, Maodi Hu, and Yunhong Wang. 2011. A survey of advances in biometric gait recognition. In Chinese Conference on Biometric Recognition. Springer, 150--158. Google ScholarDigital Library
- Zhaoxiang Zhang, Kaiyue Wang, and Yunhong Wang. 2011. A survey of on-line signature verification. In Chinese Conference on Biometric Recognition. Springer, 141--149. Google ScholarDigital Library
Index Terms
- Behavioural Biometrics in VR: Identifying People from Body Motion and Relations in Virtual Reality
Recommendations
Touch and Explore: A VR Game Exploration, Based on Haptic Driven Game-play
ISS '21: Companion Proceedings of the 2021 Conference on Interactive Surfaces and SpacesIn digital games, visuals and audios are widely used to provide immersion; with an introduction to VR (Virtual Reality) in games, novel interaction opportunities are possible. Interactions in VR are similar to real-life. Hence haptics plays a crucial ...
VR Grabbers: Ungrounded Haptic Retargeting for Precision Grabbing Tools
UIST '18: Proceedings of the 31st Annual ACM Symposium on User Interface Software and TechnologyHaptic feedback in VR is important for realistic simulation in virtual reality. However, recreating the haptic experience for hand tools in VR traditionally requires hardware with precise actuators, adding complexity to the system. We propose Ungrounded ...
Virtually-Extended Proprioception: Providing Spatial Reference in VR through an Appended Virtual Limb
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsSelecting targets directly in the virtual world is difficult due to the lack of haptic feedback and inaccurate estimation of egocentric distances. Proprioception, the sense of self-movement and body position, can be utilized to improve virtual target ...
Comments