Real-time single camera natural user interface engine development

Wei Song, Xingquan Cai, Yulong Xi, Seoungjae Cho, Kyungeun Cho

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

Natural user interfaces (NUIs) provide human computer interaction (HCI) with natural and intuitive operation interfaces, such as using human gestures and voice. We have developed a real-time NUI engine architecture using a web camera as a means of implementing NUI applications. The system captures video via the web camera, implements real-time image processing using graphic processing unit (GPU) programming. This paper describes the architecture of the engine and the real-virtual environment interaction methods, such as foreground segmentation and hand gesture recognition. These methods are implemented using GPU programming in order to realize real-time image processing for HCI. To verify the efficacy of our proposed NUI engine, we utilized it in the development and implementation of several mixed reality games and touch-less operation applications, using the developed NUI engine and the DirectX SDK. Our results confirm that the methods implemented by the engine operate in real time and the interactive operations are intuitive.

Original languageEnglish
Pages (from-to)11159-11175
Number of pages17
JournalMultimedia Tools and Applications
Volume76
Issue number9
DOIs
StatePublished - 1 May 2017

Keywords

  • Foreground segmentation
  • GPU programming
  • Mixed reality
  • Motion detection
  • NUI

Fingerprint

Dive into the research topics of 'Real-time single camera natural user interface engine development'. Together they form a unique fingerprint.

Cite this