Robust, Multimodal Capacitive Sensors Using Carbon Nanotube Paper Composites for Human and Robot Finger Tracking in Human–Robot Collaboration

Robust multimodal capacitive sensors can significantly enhance human–robot collaboration (HRC) across sectors such as healthcare, daily life support, and manufacturing. Despite this potential, achieving high-precision 3D recognition of fingertip movements using capacitive sensors remains challenging. This study demonstrates multimodal capacitive sensors for proximity, pressure, and permittivity sensing in HRC applications. The goal is to accurately detect subtle fingertip movements, enabling precise control of a robotic arm in 3D space. The circular capacitive sensors, made from a carbon nanotube-paper composite (CPC), generate a high electric field and proximity. These sensors are integrated into two types of controllers: a desktop controller for intuitive and robust control, and a handheld controller for enhanced portability. An algorithm is presented for accurate 3D fingertip tracking under scenarios involving wet- and gloved hands, to ensure a reliable interface in harsh environments. Additionally, the use of multimodal sensors integrated into robot fingers is demonstrated to detect pressure and permittivity changes, allowing the operator to identify objects. Two tasks of moving objects and distinguishing alcohol from water demonstrate the system's effectiveness for industrial applications, such as efficient HRC, hazardous material inspection, and remote work.