TY - JOUR
T1 - A realistic game system using multi-modal user interfaces
AU - Heo, Hwan
AU - Lee, Eui Chul
AU - Park, Kang Ryoung
AU - Kim, Chi Jung
AU - Whang, Mincheol
PY - 2010/8
Y1 - 2010/8
N2 - This study is to propose a realistic game system using a multi-modal interface, including gaze tracking, hand gesture recognition and bio-signal analysis. Our research is novel in the following four ways, compared to previous game systems. First, a highly immersive and realistic game is implemented on a head mounted display (HMD), with a gaze tracker, a gesture recognizer and a bio-signal analyzer. Second, since the camera module for eye tracking is attached below the HMD, a user's gaze position onto the HMD display can be calculated without wearing any additional eye tracking devices. Third, an aiming cursor in the game system is controlled by the gaze tracking. The grabbing and throwing behaviors toward a target are performed by the user's hand gestures using a data glove. Finally, the level of difficulty in the game system is adaptively controlled according to the measurement and analysis of a user's bio-signals. Experimental results show that the proposed method provides more effect on experience of immersion and interest than conventional device such as a keyboard and or a mouse.
AB - This study is to propose a realistic game system using a multi-modal interface, including gaze tracking, hand gesture recognition and bio-signal analysis. Our research is novel in the following four ways, compared to previous game systems. First, a highly immersive and realistic game is implemented on a head mounted display (HMD), with a gaze tracker, a gesture recognizer and a bio-signal analyzer. Second, since the camera module for eye tracking is attached below the HMD, a user's gaze position onto the HMD display can be calculated without wearing any additional eye tracking devices. Third, an aiming cursor in the game system is controlled by the gaze tracking. The grabbing and throwing behaviors toward a target are performed by the user's hand gestures using a data glove. Finally, the level of difficulty in the game system is adaptively controlled according to the measurement and analysis of a user's bio-signals. Experimental results show that the proposed method provides more effect on experience of immersion and interest than conventional device such as a keyboard and or a mouse.
KW - Bio-signal Analysis
KW - Gaze Tracking
KW - Hand Gesture Recognition
KW - Multi-modal Interface
UR - http://www.scopus.com/inward/record.url?scp=78149276343&partnerID=8YFLogxK
U2 - 10.1109/TCE.2010.5606271
DO - 10.1109/TCE.2010.5606271
M3 - Article
AN - SCOPUS:78149276343
SN - 0098-3063
VL - 56
SP - 1364
EP - 1372
JO - IEEE Transactions on Consumer Electronics
JF - IEEE Transactions on Consumer Electronics
IS - 3
M1 - 5606271
ER -