In a few words
A HCI interface using a Kinect and a smartphone.
Context
The Distractive User Interface was born from a collaboration with Giacinto Barresi, who was researching the topic of assistance for patients in training and rehabilitation. His intention was to design an interface which could exploit dual task paradigms, in order to distract a patient from a fatiguing task through a secondary one. So, we designed and developed an interface which integrates Kinect tracking with the touch screen input and vibration feedback of a smartphone.
The Study
The Distractive User Interface was tested with an experimental study in which subjects performed a main control task with their right hand movements, and an accessory task with their right thumb sliding over the smartphone screen. The experiment was designed to prove whether the thumb task was capable of distracting the attention of the subject from the upper arm, and to support user performance during each trial. Participants had to drive a circle through a set of corridors, within a 2D Interactive Environment. Using their right hand movements, tracked with the Kinect, users could control the circle position on the screen. The circle diameter could be shrank by performing specific gestures with the thumb, over the smartphone screen. This allowed the circle to pass through the corridors. Two thumb interaction methods have been compared. Data collected during the experiments, including a questionnaire on User Experience, showed that the distraction given by the thumb task actually helped the participants in perceiving their primary task as less fatiguing. More information on the study itself can be found in the related publication (see page bottom).
How does this work
Both the 2D Interactive Environment and the application running on the smartphone have been developed using Processing. The Kinect sensor is connected to a Linux machine running the 2D environment Processing sketch, and its streamed data is directly accessed by Processing. The OSC protocol is used for wireless communication between the smartphone and the 2D Interactive Environment.
Publication
Distractive User Interface for Repetitive Motor Tasks: a Pilot Study, Barresi, G., Mazzanti, D., Caldwell, D. and Brogni, A.
Proceedings of 2013 IEEE International Conference on Complex, Intelligent and Software Intensive Systems – International Workshop on Intelligent Interfaces for Human-Computer Interaction, 3 – 5 July 2013, Taichung, Taiwan.