Immersion (virtual reality)
Immersion into virtual reality is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system in images, sound or other stimuli that provide an engrossing total environment.
The name is a metaphoric use of the experience of submersion applied to representation, fiction or simulation. Immersion can also be defined as the state of consciousness where a "visitor" (Maurice Benayoun) or "immersant" (Char Davies)'s awareness of physical self is transformed by being surrounded in an artificial environment; used for describing partial or complete suspension of disbelief, enabling action or reaction to stimulations encountered in a virtual or artistic environment. The degree to which the virtual or artistic environment faithfully reproduces reality determines the degree of suspension of disbelief. The greater the suspension of disbelief, the greater the degree of presence achieved.
- Tactical immersion
- Tactical immersion is experienced when performing tactile operations that involve skill. Players feel "in the zone" while perfecting actions that result in success.
- Strategic immersion
- Strategic immersion is more cerebral, and is associated with mental challenge. Chess players experience strategic immersion when choosing a correct solution among a broad array of possibilities.
- Narrative immersion
- Narrative immersion occurs when players become invested in a story, and is similar to what is experienced while reading a book or watching a movie.
Staffan Björk and Jussi Holopainen, in Patterns In Game Design, divide immersion into similar categories, but call them sensory-motoric immersion, cognitive immersion and emotional immersion, respectively. In addition to these, they add a new category:
- Spatial immersion
- Spatial immersion occurs when a player feels the simulated world is perceptually convincing. The player feels that he or she is really "there" and that a simulated world looks and feels "real".
Presence, a term derived from the shortening of the original "telepresence", is a phenomenon enabling people to interact with and feel connected to the world outside their physical bodies via technology. It is defined as a person's subjective sensation of being there in a scene depicted by a medium, usually virtual in nature. Most designers focus on the technology used to create a high-fidelity virtual environment; however, the human factors involved in achieving a state of presence must be taken into account as well. It is the subjective perception, although generated by and/or filtered through human-made technology, that ultimately determines the successful attainment of presence.
Virtual reality glasses can produce a visceral feeling of being in a simulated world, a form of spatial immersion called Presence. According to Oculus VR, the technology requirements to achieve this visceral reaction are low-latency and precise tracking of movements.
- A wide field of view (80 degrees or better)
- Adequate resolution (1080p or better)
- Low pixel persistence (3 ms or less)
- A high enough refresh rate (>60 Hz, 95 Hz is enough but less may be adequate)
- Global display where all pixels are illuminated simultaneously (rolling display may work with eye tracking.)
- Optics (at most two lenses per eye with trade-offs, ideal optics not practical using current technology)
- Optical calibration
- Rock-solid tracking – translation with millimeter accuracy or better, orientation with quarter degree accuracy or better, and volume of 1.5 meter or more on a side
- Low latency (20 ms motion to last photon, 25 ms may be good enough)
Immersive virtual reality
Immersive virtual reality is a hypothetical future technology that exists today as virtual reality art projects, for the most part. It consists of immersion in an artificial environment where the user feels just as immersed as they usually feel in consensus reality.
Direct interaction of the nervous system
The most considered method would be to induce the sensations that made up the virtual reality in the nervous system directly. In functionalism/conventional biology we interact with consensus reality through the nervous system. Thus we receive all input from all the senses as nerve impulses. It gives your neurons a feeling of heightened sensation. It would involve the user receiving inputs as artificially stimulated nerve impulses, the system would receive the CNS outputs (natural nerve impulses) and process them allowing the user to interact with the virtual reality. Natural impulses between the body and central nervous system would need to be prevented. This could be done by blocking out natural impulses using nanorobots which attach themselves to the brain wiring, whilst receiving the digital impulses of which describe the virtual world, which could then be sent into the wiring of the brain. A feedback system between the user and the computer which stores the information would also be needed. Considering how much information would be required for such a system, it is likely that it would be based on hypothetical forms of computer technology.
- Understanding of the nervous system
A comprehensive understanding of which nerve impulses correspond to which sensations, and which motor impulses correspond to which muscle contractions will be required. This will allow the correct sensations in the user, and actions in the virtual reality to occur. The Blue Brain Project is the current, most promising research with the idea of understanding how the brain works by building very large scale computer models.
- Ability to manipulate CNS
The nervous system would obviously need to be manipulated. Whilst non-invasive devices using radiation have been postulated, invasive cybernetic implants are likely to become available sooner and be more accurate. Manipulation could occur at any stage of the nervous system – the spinal cord is likely to be simplest; as all nerves pass through here, this could be the only site of manipulation. Molecular nanotechnology is likely to provide the degree of precision required and could allow the implant to be built inside the body rather than be inserted by an operation.
- Computer hardware/software to process inputs/outputs
A very powerful computer would be necessary for processing virtual reality complex enough to be nearly indistinguishable from consensus reality and interacting with central nervous system fast enough.
Immersive digital environments
Immersive digital environments could be thought of as synonymous with virtual reality, but without the implication that actual "reality" is being simulated. An immersive digital environment could be a model of reality, but it could also be a complete fantasy user interface or abstraction, as long as the user of the environment is immersed within it. The definition of immersion is wide and variable, but here it is assumed to mean simply that the user feels like they are part of the simulated "universe". The success with which an immersive digital environment can actually immerse the user is dependent on many factors such as believable 3D computer graphics, surround sound, interactive user-input and other factors such as simplicity, functionality and potential for enjoyment. New technologies are currently under development which claim to bring realistic environmental effects to the players' environment – effects like wind, seat vibration and ambient lighting.
To create a sense of full immersion, the 5 senses (sight, sound, touch, smell, taste) must perceive the digital environment to be physically real. Immersive technology can perceptually fool the senses through:
- Panoramic 3D displays (visual)
- Surround sound acoustics (auditory)
- Haptics and force feedback (tactile)
- Smell replication (olfactory)
- Taste replication (gustation)
Once the senses reach a sufficient belief that the digital environment is real (it is interaction and involvement which can never be real), the user must then be able to interact with the environment in a natural, intuitive manner. Various immersive technologies such as gestural controls, motion tracking, and computer vision respond to the user's actions and movements. Brain control interfaces (BCI) respond to the user's brainwave activity.
Examples and applications
Training and rehearsal simulations run the gamut from part task procedural training (often buttonology, for example: which button do you push to deploy a refueling boom) through situational simulation (such as crisis response or convoy driver training) to full motion simulations which train pilots or soldiers and law enforcement in scenarios that are too dangerous to train in actual equipment using live ordinance.
Computer games from simple arcade to massively multiplayer online game and training programs such as flight and driving simulators. Entertainment environments such as motion simulators that immerse the riders/players in a virtual digital environment enhanced by motion, visual and aural cues. Reality simulators, such as one of the Virunga Mountains in Rwanda that takes you on a trip through the jungle to meet a tribe of mountain gorillas. Or training versions such as one which simulates taking a ride through human arteries and the heart to witness the buildup of plaque and thus learn about cholesterol and health.
In parallel with scientist, artists like Knowbotic Research, Donna Cox, Rebecca Allen, Robbie Cooper, Maurice Benayoun, Char Davies, and Jeffrey Shaw use the potential of immersive virtual reality to create physiologic or symbolic experiences and situations.
Other examples of immersion technology include physical environment / immersive space with surrounding digital projections and sound such as the CAVE, and the use of virtual reality headsets for viewing movies, with head-tracking and computer control of the image presented, so that the viewer appears to be inside the scene. The next generation is VIRTSIM, which achieves total immersion through motion capture and wireless head mounted displays for teams of up to thirteen immersants enabling natural movement through space and interaction in both the virtual and physical space simultaneously.
Use in medical care
New fields of studies linked to the immersive virtual reality emerges every day. Researchers see a great potential in virtual reality tests serving as complementary interview methods in psychiatric care. Immersive virtual reality have in studies also been used as an educational tool in which the visualization of psychotic states have been used to get increased understanding of patients with similar symptoms. New treatment methods are available for schizophrenia and other newly developed research areas where immersive virtual reality is expected to achieve melioration is in education of surgical procedures, rehabilitation program from injuries and surgeries and reduction of phantom limb pain.
Applications in the built environment
In the domain of architectural design and building science, immersive virtual environments are adopted to facilitate architects and building engineers to enhance the design process through assimilating their sense of scale, depth, and spatial awareness. Such platforms integrate the use of virtual reality models and mixed reality technologies in various functions of building science research, construction operations, personnel training, end-user surveys, performance simulations and building information modeling visualization. Head-mounted displays (with both 3 degrees of freedom and 6 degrees of freedom systems) and CAVE platforms are used for spatial visualization and building information modeling (BIM) navigations for different design and evaluation purposes. Clients, architects and building owners use derived applications from game engines to navigate 1:1 scale BIM models, allowing a virtual walkthrough experience of future buildings. For such use cases, the performance improvement of space navigation between virtual reality headsets and 2D desktop screens has been investigated in various studies, with some suggesting significant improvement in virtual reality headsets while others indicate no significant difference. Architects and building engineers can also use immersive design tools to model various building elements in virtual reality CAD interfaces, and apply property modifications to building information modeling (BIM) files through such environments.
In the building construction phase, immersive environments are used to improve site preparations, on site communication and collaboration of team members, safety and logistics. For training of construction workers, virtual environments have shown to be highly effective in skill transfer with studies showing similar performance results to training in real environments. Moreover, virtual platforms are also used in the operation phase of buildings to interact and visualize data with Internet of Things (IoT) devices available in buildings, process improvement and also resource management.
Occupant and end-user studies are performed through immersive environments. Virtual immersive platforms engage future occupants in the building design process by providing a sense of presence to users with integrating pre-construction mock-ups and BIM models for the evaluation of alternative design options in the building model in a timely and cost efficient manner. Studies conducting human experiments have shown users perform similarly in daily office activities (object identification, reading speed and comprehension) within immersive virtual environments and benchmarked physical environments. In the field of lighting, virtual reality headsets have been used investigate the influence of façade patterns on the perceptual impressions and satisfaction of a simulated daylit space. Moreover, artificial lighting studies have implemented immersive virtual environments to evaluate end-users lighting preferences of simulated virtual scenes with the controlling of the blinds and artificial lights in the virtual environment.
For structural engineering and analysis, immersive environments enable the user to focus on structural investigations without getting too distracted to operate and navigate the simulation tool. Virtual and augmented reality applications have been designed for finite element analysis of shell structures. Using stylus and data gloves as input devices, the user can create, modify mesh, and specify boundary conditions. For a simple geometry, real-time color-coded results are obtained by changing loads on the model. Studies have used artificial neural networks (ANN) or approximation methods to achieve real-time interaction for the complex geometry, and to simulate its impact via haptic gloves. Large scale structures and bridge simulation have also been achieved in immersive virtual environments. The user can move the loads acting on the bridge, and finite element analysis results are updated immediately using an approximate module.
Simulation sickness, or simulator sickness, is a condition where a person exhibits symptoms similar to motion sickness caused by playing computer/simulation/video games (Oculus Rift is working to solve simulator sickness).
Motion sickness due to virtual reality is very similar to simulation sickness and motion sickness due to films. In virtual reality, however, the effect is made more acute as all external reference points are blocked from vision, the simulated images are three-dimensional and in some cases stereo sound that may also give a sense of motion. Studies have shown that exposure to rotational motions in a virtual environment can cause significant increases in nausea and other symptoms of motion sickness.
Other behavioural changes such as stress, addiction, isolation and mood changes are also discussed to be side-effects caused by immersive virtual reality.
- Alternate reality game
- Cave automatic virtual environment
- Environmental sculpture
- Immersive design
- Immersive technology
- Interactive art
- Motion sickness
- Narrative transportation
- Neo-conceptual art
- Simulated reality
- Simulation sickness
- Sound art
- Sound installation
- Video installation
- Virtual art
- Adams, Ernest (July 9, 2004). "Postmodernism and the Three Types of Immersion". Gamasutra. Archived from the original on October 24, 2007. Retrieved 2007-12-26.
- Björk, Staffan; Jussi Holopainen (2004). Patterns In Game Design. Charles River Media. p. 206. ISBN 1-58450-354-8.
- "10.000 moving cities - same but different, interactive net-and-telepresence-based installation 2015". Marc Lee. Archived from the original on 2017-02-12. Retrieved 2017-03-12.
- Barfield, Woodrow; Zeltzer, David; Sheridan, Thomas; Slater, Mel (1995). "Presence and Performance Within Virtual Environments". In Barfield, Woodrow; Furness, III, Thomas A. Virtual Environments and Advanced Interface Design. Oxford University Press. p. 473. ISBN 0195075552.
- Thornson, Carol; Goldiez, Brian (January 2009). "Predicting presence: Constructing the Tendency toward Presence Inventory". International Journal of Human Computer Studies. 67 (1): 62–78. doi:10.1016/j.ijhcs.2008.08.006. Retrieved 19 June 2017.
- Seth Rosenblatt (19 March 2014). "Oculus Rift Dev Kit 2 now on sale for $350". CNET. CBS Interactive. Archived from the original on 28 March 2014.
- "Oculus Rift DK2 hands-on and first-impressions". SlashGear.
- "Announcing the Oculus Rift Development Kit 2 (DK2)". oculusvr.com. Archived from the original on 13 September 2014. Retrieved 3 May 2018.
- Abrash M. (2014). What VR could, should, and almost certainly will be within two years Archived 2014-03-20 at the Wayback Machine.
- Joseph Nechvatal, Immersive Ideals / Critical Distances. LAP Lambert Academic Publishing. 2009, pp. 367-368
- Joseph Nechvatal, Immersive Ideals / Critical Distances. LAP Lambert Academic Publishing. 2009, pp. 48-60
- pulseworks.com Archived 2009-05-05 at the Wayback Machine.
- "Thank You".
- Freeman, D.; Antley, A.; Ehlers, A.; Dunn, G.; Thompson, C.; Vorontsova, N.; Garety, P.; Kuipers, E.; Glucksman, E.; Slater, M. (2014). "The use of immersive virtual reality (VR) to predict the occurrence 6 months later of paranoid thinking and posttraumatic stress symptoms assessed by self-report and interviewer methods: A study of individuals who have been physically assaulted". Psychological Assessment. 26 (3): 841. doi:10.1037/a0036240.
- Freeman, D. (2007). "Studying and Treating Schizophrenia Using Virtual Reality: A New Paradigm". Schizophrenia Bulletin. 34 (4): 605. doi:10.1093/schbul/sbn020. PMC 2486455
- Virtual Reality in Neuro-Psycho-Physiology, p. 36, at Google Books
- De Los Reyes-Guzman, A.; Dimbwadyo-Terrer, I.; Trincado-Alonso, F.; Aznar, M. A.; Alcubilla, C.; Pérez-Nombela, S.; Del Ama-Espinosa, A.; Polonio-López, B. A.; Gil-Agudo, Á. (2014). "A Data-Globe and Immersive Virtual Reality Environment for Upper Limb Rehabilitation after Spinal Cord Injury". XIII Mediterranean Conference on Medical and Biological Engineering and Computing 2013. IFMBE Proceedings. 41. p. 1759. doi:10.1007/978-3-319-00846-2_434. ISBN 978-3-319-00845-5.
- Llobera, J.; González-Franco, M.; Perez-Marcos, D.; Valls-Solé, J.; Slater, M.; Sanchez-Vives, M. V. (2012). "Virtual reality for assessment of patients suffering chronic pain: A case study". Experimental Brain Research. 225: 105. doi:10.1007/s00221-012-3352-9.
- Kuliga, S.F.; Thrash, T.; Dalton, R.C.; Hölscher, C. "Virtual reality as an empirical research tool — Exploring user experience in a real building and a corresponding virtual model". Computers, Environment and Urban Systems. 54: 363–375. doi:10.1016/j.compenvurbsys.2015.09.006. Archived from the original on 2018-05-03.
- Kamat Vineet R.; Martinez Julio C. (2001-10-01). "Visualizing Simulated Construction Operations in 3D". Journal of Computing in Civil Engineering. 15 (4): 329–337. doi:10.1061/(asce)0887-3801(2001)15:4(329).
- Malkawi, Ali M.; Srinivasan, Ravi S. "A new paradigm for Human-Building Interaction: the use of CFD and Augmented Reality". Automation in Construction. 14 (1): 71–84. doi:10.1016/j.autcon.2004.08.001.
- "Revit Live | Immersive Architectural Visualization | Autodesk". Archived from the original on 2017-11-09. Retrieved 2017-11-09.
- "IrisVR - Virtual Reality for Architecture, Engineering, and Construction". irisvr.com. Retrieved 2017-11-09.
- Frost, P.; Warren, P. (2000). "Virtual reality used in a collaborative architectural design process". 2000 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics: 568–573. doi:10.1109/iv.2000.859814. Archived from the original on 2017-11-09.
- Santos, Beatriz Sousa; Dias, Paulo; Pimentel, Angela; Baggerman, Jan-Willem; Ferreira, Carlos; Silva, Samuel; Madeira, Joaquim (2009-01-01). "Head-mounted display versus desktop for 3D navigation in virtual reality: a user study". Multimedia Tools and Applications. 41 (1): 161. doi:10.1007/s11042-008-0223-2. ISSN 1380-7501.
- Ruddle, Roy A.; Payne, Stephen J.; Jones, Dylan M. (1999-04-01). "Navigating Large-Scale Virtual Environments: What Differences Occur Between Helmet-Mounted and Desk-Top Displays?". Presence: Teleoperators and Virtual Environments. 8 (2): 157–168. doi:10.1162/105474699566143. ISSN 1054-7460.
- Robertson, George; Czerwinski, Mary; van Dantzich, Maarten (1997). "Immersion in Desktop Virtual Reality". Proceedings of the 10th Annual ACM Symposium on User Interface Software and Technology. UIST '97. New York, NY, USA: ACM: 11–19. doi:10.1145/263407.263409. ISBN 0897918819.
- Ruddle, Roy A; Péruch, Patrick (2004-03-01). "Effects of proprioceptive feedback and environmental characteristics on spatial learning in virtual environments". International Journal of Human-Computer Studies. 60 (3): 299–326. doi:10.1016/j.ijhcs.2003.10.001.
- "vSpline". www.vspline.com. Archived from the original on 2017-09-19. Retrieved 2017-11-09.
- "VR - Gravity Sketch". Gravity Sketch. Archived from the original on 2017-01-15. Retrieved 2017-11-09.
- "VR Productivity for AEC". www.kalloctech.com. Archived from the original on 2017-11-09. Retrieved 2017-11-09.
- Colombo, Simone; Manca, Davide; Brambilla, Sara; Totaro, Roberto; Galvagni, Remo (2011-01-01). "Towards the Automatic Measurement of Human Performance in Virtual Environments for Industrial Safety": 67–76. doi:10.1115/winvr2011-5564.
- "DAQRI - Smart Helmet®". daqri.com. Archived from the original on 2017-11-09. Retrieved 2017-11-09.
- Messner, John I. (2006). Intelligent Computing in Engineering and Architecture. Lecture Notes in Computer Science. Springer, Berlin, Heidelberg. pp. 484–491. doi:10.1007/11888598_43. ISBN 9783540462460.
- Waller, David; Hunt, Earl; Knapp, David (1998-04-01). "The Transfer of Spatial Knowledge in Virtual Environment Training". Presence: Teleoperators and Virtual Environments. 7 (2): 129–143. doi:10.1162/105474698565631. ISSN 1054-7460.
- V.Whisker, A. Baratta, S. Yerrapathruni, J.Messner, T. Shaw,M.Warren, E. Rotthoff, J. Winters, J. Clelland, F. Johnson, (2003). "Using immersive virtual environments to develop and visualize construction schedules for advanced nuclear power plants". Proceedings of ICAPP. 3: 4–7. Archived from the original on 2017-11-09.
- Colombo, Simone; Nazir, Salman; Manca, Davide (2014-10-01). "Immersive Virtual Reality for Training and Decision Making: Preliminary Results of Experiments Performed With a Plant Simulator". SPE Economics & Management. 6 (04): 165–172. doi:10.2118/164993-pa. ISSN 2150-1173.
- Heydarian, Arsalan; Carneiro, Joao P.; Gerber, David; Becerik-Gerber, Burcin; Hayes, Timothy; Wood, Wendy. "Immersive virtual environments versus physical built environments: A benchmarking study for building design and user-built environment explorations". Automation in Construction. 54: 116–126. doi:10.1016/j.autcon.2015.03.020.
- Heydarian, Arsalan; Carneiro, Joao P.; Gerber, David; Becerik-Gerber, Burcin. "Immersive virtual environments, understanding the impact of design features and occupant choice upon lighting for building performance". Building and Environment. 89: 217–228. doi:10.1016/j.buildenv.2015.02.038.
- Mahdjoub, Morad; Monticolo, Davy; Gomes, Samuel; Sagot, Jean-Claude. "A collaborative Design for Usability approach supported by Virtual Reality and a Multi-Agent System embedded in a PLM environment". Computer-Aided Design. 42 (5): 402–413. doi:10.1016/j.cad.2009.02.009.
- Chamilothori, Kynthia; Wienold, Jan; Andersen, Marilyne. "Daylight patterns as a means to influence the spatial ambiance: a preliminary study". Proceedings of the 3rd International Congress on Ambiances.
- Huang, J.M.; Ong, S.K.; Nee, A.Y.C. "Visualization and interaction of finite element analysis in augmented reality". Computer-Aided Design. 84: 1–14. doi:10.1016/j.cad.2016.10.004.
- Liverani, A.; Kuester, F.; Hamann, B. (1999). "Towards interactive finite element analysis of shell structures in virtual reality". 1999 IEEE International Conference on Information Visualization (Cat. No. PR00210): 340–346. doi:10.1109/iv.1999.781580.
- Hambli, Ridha; Chamekh, Abdessalam; Salah, Hédi Bel Hadj. "Real-time deformation of structure using finite element and neural networks in virtual reality applications". Finite Elements in Analysis and Design. 42 (11): 985–991. doi:10.1016/j.finel.2006.03.008.
- Connell, Mike; Tullberg, Odd. "A framework for immersive FEM visualisation using transparent object communication in a distributed network environment". Advances in Engineering Software. 33 (7-10): 453–459. doi:10.1016/s0965-9978(02)00063-7.
- "Oculus Rift is working to solve simulator sickness". Polygon. Archived from the original on 2015-09-24. Retrieved 2015-05-05.
- So, R.H.Y. and Lo, W.T. (1999) "Cybersickness: An Experimental Study to Isolate the Effects of Rotational Scene Oscillations." Proceedings of IEEE Virtual Reality '99 Conference, March 13–17, 1999, Houston, Texas. Published by IEEE Computer Society, pp. 237–241
- "Archived copy" (PDF). Archived (PDF) from the original on 2014-12-18. Retrieved 2014-11-25.
- Reyes, Stephanie. "Ford brings virtual reality presentation to UCF.", September 2012.
- Christiane Paul, Digital Art, Thames & Hudson Ltd.
- Oliver Grau, "Virtual Art: From Illusion to Immersion" MIT-Press, Cambridge 2003
- Timothy Murray, Derrick de Kerckhove, Oliver Grau, Kristine Stiles, Jean-Baptiste Barrière, Dominique Moulon, Maurice Benayoun Open Art, Nouvelles éditions Scala, 2011, French version, ISBN 978-2-35988-046-5
- Allen Varney, (August 8, 2006). "Immersion Unexplained" in "The Escapist"
- Frank Popper, "From Technological to Virtual Art", MIT Press. ISBN 0-262-16230-X.
- Oliver Grau (Ed.), Media Art Histories, MIT-Press, Cambridge 2007
- Joseph Nechvatal, "Immersive Excess in the Apse of Lascaux", Technonoetic Arts 3, no3. 2005
- Adams, Ernest (July 9, 2004). "Postmodernism and the Three Types of Immersion". Gamasutra. Retrieved 2007-12-26.
- Björk, Staffan; Jussi Holopainen (2004). Patterns In Game Design. Charles River Media. p. 423. ISBN 1-58450-354-8.
- Edward A. Shanken, Art and Electronic Media. London: Phaidon, 2009. ISBN 978-0-7148-4782-5
- Joseph Nechvatal Towards an Immersive Intelligence: Essays on the Work of Art in the Age of Computer Technology and Virtual Reality (1993–2006). Edgewise Press. New York, N.Y. 2009
- Joseph Nechvatal, Immersive Ideals / Critical Distances. LAP Lambert Academic Publishing. 2009
|Wikimedia Commons has media related to Immersion (virtual reality).|
- Annual Summit on Immersive Technology
- pdf download of Joseph Nechvatal's text book: Immersive Ideals / Critical Distances. LAP Lambert Academic Publishing. 2009
- Audio and Game Immersion PhD thesis about game audio (the IEZA Framework) and immersion.
- "Improvising Synesthesia: Comprovisation of Generative Graphics and Music" by Joshua B. Mailman, in Leonardo Electronic Almanac v.19 no.3, Live Visuals, 2013, pp. 352–84, about two immersive systems for improvising music and graphics through dance-like motion detected by an infrared video camera and other sensors.