Tele-Immersive Applications On The Next Generation Internet. AUTHORS Joeseph Bannister USC/Information Sciences Institute joeseph@isi.edu (310) 822-1511 Tom DeFanti Electronic Visualization Laboratory University of Illinois at Chicago tom@eecs.uic.edu (312) 996-3002 Ian Foster Mathematics and Computer Science Division Argonne National Laboratory itf@mcs.anl.gov (630) 252-4619 Carl Kesselman USC/Information Sciences Institute carl@isi.edu (310) 822-1511 Rick Stevens Mathematics and Computer Science Division Argonne National Laboratory stevens@mcs.anl.gov (630) 252-3378 ABSTRACT Advances in computer visualization, virtual reality, high-performance computing and networking technology have opened up the possibility for entirely new modes of interaction and collaboration. The basic hardware technology exists for constructing shared interactive virtual reality simulations in which physical entities such as people, scientific instruments and machines can interact with synthetic entities, such as scientific simulations and agents acting as virtual people or teachers. Tele-immersive applications are those virtual reality simulations that couple many sites and users located at geographically distributed locations. A prototypical tele-immersive environment consists of a collection of CAVE-like virtual reality displays in which user position in a shared virtual space is determined via one or more trackers attached to the user. Multichannel audio or haptic devices provide additional sensory feedback. The images in the CAVE are produced from output of various simulations, such as one supporting molecular modeling. For example, a group of scientists could collaboratively perform rational drug design using the networked tele-immersive environment from different sites. Although the required computing and visualization technology are available on a stand-alone basis, fundamentally interactive modes of collaboration call for the interconnection of appropriate computers, visualization devices, information sources and people by high-speed networks. Consequently, a NGI is a prerequisite for wide-scale use of tele-immersion as a vehicle for collaboration, or education. NETWORKING REQUIREMENTS FOR TELE-IMMERSION Tele-immersive applications will stress the network like never before. Central to achieving usable multi-site tele-immersion is the synchronizing of flows of information from tracking, audio, video, data, and computing. These flows have widely differing bandwidth requirements and latency tolerance. For example, tracker data, critical to human-to-human interaction over distance, is low bandwidth but latency-intolerant, while video is high-bandwidth and somewhat latency tolerant, and both are tolerant of loss, needing only the latest information rather than the entire transmission history. Audio, critical to collaboration, is low-bandwidth but rather latency and jitter intolerant. Data and simulation can be extremely high bandwidth, somewhat latency tolerant, but must be guaranteed lossless. Control information for all these streams is low bandwidth, low latency and lossless as well. Supporting tele-immersive applications on the NGI will require that a number of fundamental issues be addressed. In particular, managing "bundled" flows composed from individual flows, each with diverse QoS requirements has not been explored, yet is critical for tele-immersive applications. In addition, security structures that support large scale, fluid collaborations and network management-like tools to establish and control collaboration sessions need to be developed and deployed. Additionally, we argue that there should be a significant effort in developing networking middleware to enable the rapid development and experimentation of tele-immersive applications. We believe that two distinct classes of middleware need to be developed. At the lower level, we need software that will enable the construction of flexible applications that can adapt their behavior to the characteristics of the underlying networking infrastructure. This adaption must be dynamic, allowing for robust application behavior in the face of dynamics in the underlying network. This middleware should serve as a mediator, translating application requirements into specific networking mechanisms and notifying the application of changes in the underlying characteristics of the network. While the lower level middleware mediates interactions between applications and the network, we need to have domain specific upper layer, capable of manipulating the abstractions of tele-immersive applications, such as virtual spaces, video conferencing sessions and simulations. This layer exploits the information provided by the adaption layer, conveying resource requirements to the adaption layer and making decisions about how to adjust application behavior to respond to information provided by the lower software layers. To promote the development of tele-immersive applications, we need to ensure that the high-end NGI testbeds focus not only on the communication infrastructure but the use of that infrastructure in a computational framework. This means that the network must be designed to include both computational and data resources. Specifically, we need to ensure that these testbeds include highend supercomputers, databases, high-performance network attached I/O devices and virtual reality devices such as CAVES, Immersi-desks and Reactive Workbenches. >From the perspective of the networking infrastructure, we need to ensure that we have the ability to perform experimentation with quality of service mechanisms. The need for experimentation will also impact the management policy of the NGI. For example, specific experiments may require the ability to carve out temporary private networks, or to have programmable routers designed for QoS experimentations. In summary, the NGI can enable tele-immersive application which offer radically new modes of interaction and collaboration. In order to make this a reality, we must 1) design a NGI that interconnects appropriate components, 2) develop techniques for supported advanced quality of service management, security and session management 3) explore new software design techniques and infrastructure that supports adaption, and 4) enable experimentation on the network.