During the last 10 years, several disparate technologies and science, notably in computer vision, computer graphics, distributed computing, and broadband networks have come together and facilitated geographically distributed tele-immersive environments in which people can interact and communicate.
We have built such 3D tele-immersive (TI) environments at the University of Illinois, Urbana-Champaign (UIUC) and University of California, Berkeley (UCB).1 The TI environments deploy large-scale 3D camera networks that capture, digitize, and reconstruct three dimensional images and sounds of moving people, as well as integrate and render the multimedia data from geographically distributed sites into a joint virtual space at each TI site. The reasons for deploying 3D TI environments, instead of just as an available video conferencing technology, are as follows: (1) the video conferencing displays only individual 2D video streams/pictures (though next to each other) on the screen, hence no joint virtual space can be created and no real interaction is feasible; (2) the 3D environment facilitates different views of interaction from any viewpoint, as the viewer desires, and with different digital options, such as different scales of participants and different people/scene orientations that create physically impossible views; and (3) with the 3D TI environment, one can easily integrate synthetic objects and other environments into the current immersive environment.
While there have been other demonstration projects of this nature (e.g., 2, 3, 4, 5, 6) they were restricted to the video sequences and special dedicated network connectivity. We are aiming to create 3D TI environments from COTS components for a common user who does not have the luxury of expensive supercomputing facilities, special purpose camera hardware and dedicated networks.
Yet we see the opportunity to use this kind of technology for exploring geographically distributed interaction and communication of people in which the physical/body interaction is important. We have explored this interaction in the domain of dance, since dance is a way of physical communication. However, we also see many other applications such as remote physiotherapy, collaboration between distributed scientists working on common multidimensional data sets, design of artifacts (architecture, mechanical, chemical and electrical designs), planning for coordinated activities and their like.
In this article, we will describe the individual components comprising the UIUC and UCB TI environments, as well as the challenges and few solutions that allowed us to execute one of the very first public performances in geographically distributed collaborative dancing that took place in December 2006. We will briefly describe the experiment as well as lessons learned from this exciting and very successful performance.
