From Virtual Reality to Tangible Virtuality: An Inventory of the Technological Challenges

The paradigm of tangible virtuality (TV) has grown out from the paradigm of virtual reality (VR), leaving some key concepts of projective and immersive VR behind, but introducing many new ones. The ultimate objective of TV is to intuitively generate air-borne synthetic objects (SOs), share them between remote work environments in real time, simulate their physical behavior, and enable a physics-obeying interaction of humans with SOs. The major challenges for the implementation of TV originate in the need for multi-sensory integral rendering of SOs and un-instrumented generation of high fidelity sensations. These scientific and technological challenges have a strong influence on the current advancement of TV. Actually, implementation of fully featured TV environments is still in its infancy. The goal of this paper is to analyze the main implementation concepts and to make an inventory of the technological challenges and opportunities. Six major issues have been identified: (i) use of natural interaction modalities to express SOs, (ii) multi-aspect (e.g., tactile, haptic and auditory) volumetric rendering and sensation of air-borne SOs, (iii) simulation of physical behavior of interacting SOs, and (iv) manipulative human interaction with SOs. The paper investigates the opportunities of a short-term implementation based on the current technologies and of a near-future implementation based on emerging and anticipated technologies. The conclusion is that current technologies allow only a limited implementation of SOs and a low fidelity multi-sensorial experience. For a full scale implementation, radically new technologies and implementation concepts are needed, but it also requires a goal-driven extension of the existing scientific knowledge.