Continuous Image Generation from Low-Update-Rate Images and Physical Sensors through a Conditional GAN for Robot Teleoperation

Dae Kwan Ko, Dong Han Lee, Soo Chul Lim

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

When a robot is teleoperated, its operator control is based on transmitted images. Network limitations and/or a remote distance usually cause delays or interruptions of the image transmission, which is one of the reasons for the instability of teleoperation systems. In this article, we propose a high-update-rate image generation method using past low update image and current grip position and electrical motor current of gripper received by sensors during teleoperation via a conditional generative adversarial network. The main challenge is that such a network can generate current high-update-rate images from past low-update-rate one, the current high-update-rate grip force, and the grip angle. We equipped a robot gripper with a camera and a grip force sensor and collected a large data set of robot vision, grip force, and grip angle sequences; objects with deformation, including irregular deformation, and rigid objects were tested in the experiment to verify the possibility of high-update-rate image generation under various grip conditions. We found that the proposed network allows the generation of current images with high update rate.

Original languageEnglish
Article number9084292
Pages (from-to)1978-1986
Number of pages9
JournalIEEE Transactions on Industrial Informatics
Volume17
Issue number3
DOIs
StatePublished - Mar 2021

Keywords

  • Image generation
  • machine learning
  • neural networks
  • robot grasping
  • telerobotics

Fingerprint

Dive into the research topics of 'Continuous Image Generation from Low-Update-Rate Images and Physical Sensors through a Conditional GAN for Robot Teleoperation'. Together they form a unique fingerprint.

Cite this