A realistic game system using multi-modal user interfaces

Hwan Heo, Eui Chul Lee, Kang Ryoung Park, Chi Jung Kim, Mincheol Whang

Research output: Contribution to journalArticlepeer-review

39 Scopus citations

Abstract

This study is to propose a realistic game system using a multi-modal interface, including gaze tracking, hand gesture recognition and bio-signal analysis. Our research is novel in the following four ways, compared to previous game systems. First, a highly immersive and realistic game is implemented on a head mounted display (HMD), with a gaze tracker, a gesture recognizer and a bio-signal analyzer. Second, since the camera module for eye tracking is attached below the HMD, a user's gaze position onto the HMD display can be calculated without wearing any additional eye tracking devices. Third, an aiming cursor in the game system is controlled by the gaze tracking. The grabbing and throwing behaviors toward a target are performed by the user's hand gestures using a data glove. Finally, the level of difficulty in the game system is adaptively controlled according to the measurement and analysis of a user's bio-signals. Experimental results show that the proposed method provides more effect on experience of immersion and interest than conventional device such as a keyboard and or a mouse.

Original languageEnglish
Article number5606271
Pages (from-to)1364-1372
Number of pages9
JournalIEEE Transactions on Consumer Electronics
Volume56
Issue number3
DOIs
StatePublished - Aug 2010

Keywords

  • Bio-signal Analysis
  • Gaze Tracking
  • Hand Gesture Recognition
  • Multi-modal Interface

Fingerprint

Dive into the research topics of 'A realistic game system using multi-modal user interfaces'. Together they form a unique fingerprint.

Cite this