In a few words
VR interface for Telepresence and remote Teleoperation.
Starting from February 2018 I have been working for the Robot Teleoperativo project, a collaboration between IIT and INAIL (“Istituto Nazionale per l’Assicurazione contro gli Infortuni sul Lavoro” – National Institute for Insurance against Workplace Injuries, Italy).
Aim of the project is “[…] the development of a novel, collaborative teleoperation hardware and software system dedicated to operating in certain hazard-prone industries, reducing the risks to the workers’ safety and well-being. It will utilize, develop, and integrate advanced technologies in tele-locomotion, tele-manipulation, and human-machine interaction.“
My team’s contribution is in the area of VR, Telepresence and Teleoperation. Our platform of choice for VR is Unreal. In particular, we are working on the tools needed to visualize and update the pose of 3D robot models, to playback real time video and audio, to show real time point clouds and to provide tools for the design of a 2D UI which can be used side by side with a VR HMD.
Most of these tasks required the development of C++ plugins for Unreal, which can deal with custom data and networking requirements. We are using ffmpeg, PCL, Boost and other libraries to help retrieving live streams of data from a NVIDIA Jetson, and to communicate with ROS-based systems.
A teaser video is available with footage taken from testing sessions which took place in 2019 and 2020:
The Vicarios Virtual Reality Interface for Remote Robotic Teleoperation
Abdeldjallil Naceri, Dario Mazzanti, Joao Bimbo, Yonas T Tefera, Domenico Prattichizzo, Darwin G Caldwell, Leonardo S Mattos, Nikhil Deshpande.
J. Intell. Robotic Syst. (2021)
Towards a Virtual Reality Interface for Remote Robotic Teleoperation
Abdeldjallil Naceri, Dario Mazzanti, Joao Bimbo, Domenico Prattichizzo, Darwin G Caldwell, Leonardo S Mattos, Nikhil Deshpande.
2019 19th International Conference on Advanced Robotics (ICAR)