In a few words
Integration of under-actuated wearable haptic devices in a VR environment.
Context
From 2015 to 2016 I was working as a Post-Doc in IIT’s ADVR department, within the scope of the WearHap european project. In particular, I was part of the line of research dealing with the development of applications for novel wearable, under-actuated haptic devices. At the core of the WearHap project was the idea to create a new set of tools for the development of the next generation of haptic devices. This involved fields as neuroscience, physics, engineering, design. The IIT team I was working with was developing a wearable hand exoskeleton. The exoskeleton, named HEXOTRAC, was designed to be lightweight compared to other hand exoskeletons, but still precise in its tracking features. HEXOTRAC can track the position and orientation of the index, middle and thumb fingertips of its user, with respect to the exoskeleton frame reference. Three brushless motors are mounted on the back of the device, which can apply a pulling/pushing force at the fingertips, which can be used as haptic kinaesthetic feedback. Being a wearable device, HEXOTRAC is not grounded, so an external tool is needed in order to track the precise movements of the whole device (e.g. optical motion capture). Possible applications for such a device are VR interaction, Teleoperation and rehabilitation.
A VR framework for an under-actuated hand exoskeleton
In order to evaluate the HEXOTRAC device, I created a VR environment which could exploit all its functionalities, and test the limitations given by under-actuation. A strong feature of HEXOTRAC is the possibility to precisely detect fine movements performed with the fingertips. The VR environment was then designed to allow fingertip-based grasping of virtual objects, which takes into account the position and orientation of the three tracked fingertips, and of course of the hand/device. Touching, grasping or pushing a virtual objects resulted in kinaesthetic haptic feedback, applied by the device motors. Due to under-actuation, haptic feedback can only be provided along one direction per finger. This direction changes based on the current configuration of each finger, but the force can be generally thought as a push or pull at the fingertips. The VR environment isolates only the displayable part of the forces, and then feeds it to the exoskeleton.
How does this work
The VR enironment was created with the Unity game engine. Stereoscopic content is displayed on a 3D monitor. A set of 4 optitrack cameras is used to track the position of HEXOTRAC and for headtracking. A DLL is imported by Unity in order to establish a connection with the exoskeleton, receive its tracking data and control its force feedback. The exoskeleton is connected through an ethernet cable, and a trackable rigid body is mounted on its back, so that optitrack can track the device position and orientation. The grasping technique used to interact with the virtual objects is an adaptation of the solution proposed in Multi-Contact Grasp Interaction for Virtual Environments, by Holz et al. The setup has been presented at the IROS 2015 conference in Hamburg, Germany, at the COMPAMED Fair in Dusseldorf, Germany, and at the 2016 WearHap review meeting held at IIT, Genova, Italy.
Videos and pictures
Publication
HEXOTRAC: A Highly Under-Actuated Hand Exoskeleton for Finger Tracking and Force Feedback
Ioannis Sarakoglou, Anais Brygo, Dario Mazzanti, N.V. Garcia Hernandez, Darwin Caldwell, Nikos Tsagarakis
Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2016), 2016