In a few words
An Immersive Virtual Musical Instrument installation.
Context
The use of VR in artistic expression have always intrigued new media artists, pushing them to explore the infinite possibilities of human-coded immersive digital worlds: unconventional rules, perceptions and unexplored interaction paradigms, far from our physical world, are possible in VR.
vrGrains is an Immersive Virtual Musical Instrument, designed as an installation in which the participant is surrounded by a dynamically generated cloud of sounds. Through natural interaction paradigms and gestures, vrGrains allows to explore and control these sound units in different ways.
These sound units are generated by means of corpus based concatenative synthesis (CBCS), which has been approached from different perspectives by many.
The Experience
The participant is free to explore a sound corpus represented by 3D units which physically surrounds her/him. By touching and pushing the audio units with hands and fingers, the user triggers the sound associated to them. When pushed, the sound units move, and may collide with other units, thus triggering their sound as well. Other than simply pushing and touching the units, the installation provides a number of interaction modalities which allow controllable and chaotic audio unit triggering and motion.
How does this work
The Virtual Environment in vrGrains has been developed using the Unity game engine. In order to give Unity access to stereoscopic rendering, the MiddleVR middleware was used.
Audio content was driven by a Max/MSP patch, taking care of Corpus Based Concatenative Synthesis. The communication between Unity and this patch was implemented using the OSC protocol.
Publication
Concatenative Synthesis Unit Navigation and Dynamic Rearrangement in vrGrains, Zappi, V., Mazzanti, D., Brogni, A. and Caldwell, D.
Proceedings of the 9th Sound and Music Computing Conference, 11 – 14 July 2012, Copenhagen, Denmark.
Also related to:
Point Clouds Indexing in Real Time Motion Capture, Mazzanti, D., Zappi, V., Brogni, A. and Caldwell, D.
Proceedings of the 18th International Conference on Virtual Systems and Multimedia, 2 – 5 September 2012, Milan, Italy.