Multimedia system for real-time photorealistic nonground modeling of 3D dynamic environment for remote control system

Phuong Minh Chu, Seoungjae Cho, Sungdae Sim, Kiho Kwak, Kyungeun Cho

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Nowadays, unmanned ground vehicles (UGVs) are widely used for many applications. UGVs have sensors including multi-channel laser sensors, two-dimensional (2D) cameras, Global Positioning System receivers, and inertial measurement units (GPS-IMU). Multi-channel laser sensors and 2D cameras are installed to collect information regarding the environment surrounding the vehicle. Moreover, the GPS-IMU system is used to determine the position, acceleration, and velocity of the vehicle. This paper proposes a fast and effectivemethod formodeling nonground scenes usingmultiple types of sensor data captured through a remote-controlled robot. Themulti-channel laser sensor returns a point cloud in each frame. We separated the point clouds into ground and nonground areas before modeling the three-dimensional (3D) scenes. The ground part was used to create a dynamic triangular mesh based on the height map and vehicle position. The modeling of nonground parts in dynamic environments including moving objects is more challenging than modeling of ground parts. In the first step, we applied our object segmentation algorithm to divide nonground points into separate objects. Next, an object tracking algorithm was implemented to detect dynamic objects. Subsequently, nonground objects other than large dynamic ones, such as cars, were separated into two groups: surface objects and non-surface objects. We employed colored particles to model the non-surface objects. To model the surface and large dynamic objects, we used two dynamic projection panels to generate 3D meshes. In addition, we applied two processes to optimize the modeling result. First, we removed any trace of the moving objects, and collected the points on the dynamic objects in previous frames. Next, these points were merged with the nonground points in the current frame. We also applied slide window and near point projection techniques to fill the holes in the meshes. Finally, we applied texture mapping using 2D images captured using three cameras installed in the front of the robot. The results of the experiments prove that our nonground modeling method can be used to model photorealistic and real-time 3D scenes around a remote-controlled robot.

Original languageEnglish
Article number83
JournalSymmetry
Volume10
Issue number4
DOIs
StatePublished - 1 Apr 2018

Keywords

  • Dynamic environment
  • Modeling
  • Multimedia
  • Remote-controlled system

Fingerprint

Dive into the research topics of 'Multimedia system for real-time photorealistic nonground modeling of 3D dynamic environment for remote control system'. Together they form a unique fingerprint.

Cite this