A Cartoon That You Can Get Into
Content
A CARTOON THAT YOU CAN GET INTO.
An interactive computer system so fast and intuitive that the computer disappears from the mind of the user, leaving the computer-generated environment as reality. Virtual reality refers to computer-generated, interactive, three-dimensional environments into which people are immersed. It provides a way for people to visualize, manipulate, and interact with simulated environments through the use of computers and extremely complex data. In virtual reality, a "world" is created that exists entirely in the memory of a computer and in the equipment you are wearing. The computers control what you sense through simulations such as three-dimensional images, sound, artificial smell, and force feedback. VR is simple to understand. The idea is to show you, the user or viewer, a computer generated environment that adjusts to you, to your movements. To be specific the idea is that when you turn your head and for example, look up to see a computer generated sky, the sky is there. When you look down the ground is there. Now we got a problem though, a computer generated scene is going to be displayed on a monitor. If the monitor is on your desk and you turn your head up, it's kind of hard to see the scene. So now we have the HMD (Head Mounted Display) which basically are those cool looking helmets people wear. Inside the helmet is a display to that as you turn your head, the display is still in front of your eyeballs. Simple idea but not so simple to execute well. One of the big problems is that when you strap a monitor to your face, preferably one in front of each eyeball, the display's are small and usually of low resolution. High resolution displays are now available but expensive. They are getting better and cheaper all the time. You can also use the helmets to just sit and watch videos! I've even seen some airports rent portable DVD players with these displays that you can take with you on airplanes. You, in turn, are able to enter and interact with the virtual realities by controlling the computers through equipment such as head-mounted displays (which track your eye and head movements in relation to the simulations) and data gloves (which track your hand movements in relation to the simulations). Virtual reality can be delivered using a variety of systems. The "world" may be projected inside a 'cave' which users can move around. Or headsets and gloves may be worn so that users are immersed in a virtual world which they can move around and touch. But the most widely used form of virtual reality in use today is desk-top virtual reality. In these systems virtual reality worlds run from users' desk-top computers are displayed on a standard monitor and navigated using a mouse, or 3d space ball and keyboard. Desk top virtual reality systems can be distributed easily via the World Wide Web or on CD and users need little skill to install or use them. Generally all that is needed to allow
this type of virtual reality to run on a standard computer is a single piece of software in the form of a viewer. Desktop virtual reality is very accessible and is widely used. Virtual reality experiences can be described as passive, exploratory, or interactive. In passive virtual reality, you watch, hear, and possibly feel the environment move around you, which makes it appear as if you are moving through the environment. Nevertheless, you cannot control the environment; you are just a spectator. Exploratory virtual reality allows you to explore and move through space. For example, a chemical plant tour has been developed by John Bell, a lecturer in the Chemical Engineering Department, in the U-M College of Engineering's Virtual Reality in Chemical Engineering Lab. Instead of just seeing a three-dimensional room with a reactor in it, you can step inside the reactor and observe the chemical processes. Most architectural virtual walk- throughs and virtual art shows are exploratory virtual reality. The most powerful and complex type of virtual reality is interactive. Here, you can explore the environment and, most importantly, interact with and change it. For instance, in the virtual car interior, if you "touch" a radio button on the car control panel while wearing a data glove, the computer will generate a sound like a radio station. In the virtual chemical plant, students can operate controls to change reactor conditions.
Uses of virtual reality There are many common applications for virtual reality. They fall into the main categories of training, education, simulation, visualization, conceptual navigation, design and entertainment but there is much overlap between these categories:
•
•
•
•
•
Training applications include allowing users to practice a process repeatedly in a no-risk environment. For example, users might dig an archaeological site trying out different strategies without the risk of destroying important evidence. Educational applications include virtual visits and simulations. For example, a virtual visit to a museum that is too far away to visit or does not exist in the real world. Or historic battles may be simulated allowing users to see "what would have happened if?" Visualisation examples include an architect's design for a building or the reconstruction of ancient buildings from archaeological evidence. Such models also allow users to explore something too large or too small to explore in reality and can bring historical time-lines to life. Applications of virtual reality for conceptual navigation enable, for example, users of a library or archive to find the information they need at a logical or physical level. Virtual reality allows designs to be visualized and tested. For example, a design application might allow a choreographer to see a dance in action.
• •
Entertainment applications include virtual art galleries and games. Virtual reality may also be considered as an art form in its own right. Collaborative Virtual Environments (CVEs) allow users to interact with each other in a Virtual world allowing the development of virtual communities adding a new dimension to virtual reality.
Today, 'Virtual Reality' is used in a variety of ways and often in a confusing and misleading manner. Originally, the term referred to 'Immersive Virtual Reality.' In immersive VR, the user becomes fully immersed in an artificial, three-dimensional world that is completely generated by a computer. Head-Mounted Display (HMD) The head-mounted display (HMD) was the first device providing its wearer with an immersive experience. Evans and Sutherland demonstrated a head-mounted stereo display already in 1965. It took more then 20 years before VPL Research introduced a commercially available HMD, the famous "Eye Phone" system (1989). A head-mounted display (HMD):
A typical HMD houses two miniature display screens and an optical system that channels the images from the screens to the eyes, thereby, presenting a stereo view of a virtual world. A motion tracker continuously measures the position and orientation of the user's head and allows the image generating computer to adjust the scene representation to the current view. As a result, the viewer can look around and walk through the surrounding virtual environment.
Various Displays available are:
The helmet is only part of the VR equation. In order for the computer to adjust the display it had to know your head's position. Specifically it needs to know the position of your head in space and the orientation of your head.
Ascention Flock of Birds Motion Tracker Actually if you are sitting still in a chair and you turn your head around without getting up the position information isn't as important as the orientation information. So this magic is accomplished by devices called position trackers. Position trackers are not cheap. They
typically cost several thousand dollars. They work using a variety of technologies each with advantages and disadvantages. There are links to lots of these devices on the VR Hardware page. Getting back to basics though, if you're sitting in front of a computer with a helmet on and the helmet has a position tracker attached (a typical setup) you now have an immersive VR system. The computer graphics scene generated from the computer will be routed to the helmet and the position information of your head, will be routed via the position tracker to the computer. Often the position information can be communicated via a simple serial port, just like your modem. Although that is not the fastest way, which would be via a dedicated interface card in the computer. To overcome the often uncomfortable intrusiveness of a head-mounted display, alternative concepts (e.g., BOOM and CAVE) for immersive viewing of virtual environments were developed. BOOM The BOOM (Binocular Omni-Orientation Monitor) from Fakespace is a head-coupled stereoscopic display device. Screens and optical system are housed in a box that is attached to a multi-link arm. The user looks into the box through two holes, sees the virtual world, and can guide the box to any position within the operational volume of the device. Head tracking is accomplished via sensors in the links of the arm that holds the box. The BOOM, a head-coupled display device:
CAVE The CAVE (Cave Automatic Virtual Environment provides the illusion of immersion by projecting stereo images on the walls and floor of a room-sized cube. Several persons wearing lightweight stereo glasses can enter and walk freely inside the CAVE. A head tracking system continuously adjust the stereo projection to the current position of the leading viewer.
CAVE system (schematic principle):
Input Devices and other Sensual Technologies A variety of input devices like data gloves, joysticks, and hand-held wands allow the user to navigate through a virtual environment and to interact with virtual objects. Directional sound, tactile and force feedback devices, voice recognition and other technologies are being employed to enrich the immersive experience and to create more "sensualized" interfaces. A data glove allows for interactions with the virtual world:
The new 5DT Glove features advanced fiber-optic flex sensors to generate finger-bend data. Move easily through your virtual world by combining hand gestures with the pitch and roll of your hand. Breakthrough pricing, new features, open architecture and software support have made it the glove of choice.
5DT Glove
Cyber Glove
P5 Glove
Pinch Glove
CyberGlove is a low-profile, lightweight glove with flexible sensors which accurately and repeatably measure the position and movement of the fingers and wrist. CyberGlove's award-winning design incorporates the latest high-precision joint-sensing technology. CyberGlove is state-of-the-art in instrumented gloves. Pinch Glove is a remarkable new system for interacting with 3D simulation. This pair of stretch-fabric gloves contain sensors in each fingertip which detect contact between the digits of your hand. You can use these gestures for a wide range of control and interactive functions customized to your specifications. Any combination of single or multiple contacts between two or more digits can be programmed to have specific meanings, ranging from simple on/off to multi-part, multi-action commands. The gestures are not dependent on individual hand geometry - the Pinch never requires calibration. Ok so now we have a computer-generated scene which adjusts to the viewer’s head position. The next thing we need is the ability to interact with the scene. To touch stuff in the scene. This is where the infamous gloves come in. VR glove devices, also not cheap, are devices that are specialized position, orientation sensors. Specialized software reads the position of the hand along with the positions of the figures to let you make gestures of various sorts to let you fly around a scene, select objects, manipulate controls and so on. The final component for a good immersive VR system is spatialized sound. Often the helmets you get have earphones. The computing environment can have sounds associated directly with objects or people floating around in the virtual world. Attaching sounds and getting the sensation of object locations via sound cues is an effective technique to highten the immersive effect.
Characteristics of Immersive VR The unique characteristics of immersive virtual reality can be summarized as follows:
•
• • • • •
Head-referenced viewing provides a natural interface for the navigation in threedimensional space and allows for look-around, walk-around, and fly-through capabilities in virtual environments. Stereoscopic viewing enhances the perception of depth and the sense of space. The virtual world is presented in full scale and relates properly to the human size. Realistic interactions with virtual objects via data glove and similar devices allow for manipulation, operation, and control of virtual worlds. The convincing illusion of being fully immersed in an artificial world can be enhanced by auditory, haptic, and other non-visual technologies. Networked applications allow for shared virtual environments.
Shared Virtual Environments
In the example illustrated below, three networked users at different locations (anywhere in the world) meet in the same virtual world by using a BOOM device, a CAVE system, and a Head-Mounted Display, respectively. All users see the same virtual environment from their respective points of view. Each user is presented as a virtual human (avatar) to the other participants. The users can see each other, communicated with each other, and interact with the virtual world as a team.
Non-immersive VR Today, the term 'Virtual Reality' is also used for applications that are not fully immersive. The boundaries are becoming blurred, but all variations of VR will be important in the future. This includes mouse-controlled navigation through a three-dimensional environment on a graphics monitor, stereo viewing from the monitor via stereo glasses, stereo projection systems, and others. Apple's QuickTimeVR,for example, uses photographs for the modeling of three-dimensional worlds and provides pseudo lookaround and walk-trough capabilities on a graphics monitor. VRML Most exciting is the ongoing development of VRML (Virtual Reality Modeling Language) on the World Wide Web. In addition to HTML (HyperText Markup Language), that has become a standard authoring tool for the creation of home pages, VRML provides threedimensional worlds with integrated hyperlinks on the Web. Home pages become home spaces. The viewing of VRML models via a VRML plug-in for Web browsers is usually done on a graphics monitor under mouse-control and, therefore, not fully immersive. However, the syntax and data structure of VRML provide an excellent tool for the modeling of three-dimensional worlds that are functional and interactive and that can, ultimately, be transferred into fully immersive viewing systems
Rendering of Escher's Penrose Staircase
VR-related Technologies Other VR-related technologies combine virtual and real environments. Motion trackers are employed to monitor the movements of dancers or athletes for subsequent studies in immersive VR. The technologies of 'Augmented Reality' allow for the viewing of real environments with superimposed virtual objects. Telepresence systems (e.g., telemedicine, telerobotics) immerse a viewer in a real world that is captured by video cameras at a distant location and allow for the remote manipulation of real objects via robot arms and manipulators.
VIRTUAL REALITY: Applications for Grand Challenges
As the technologies of virtual reality evolve, the applications of VR become literally unlimited. It is assumed that VR will reshape the interface between people and information technology by offering new ways for the communication of information, the visualization of processes, and the creative expression of ideas. Note that a virtual environment can represent any three-dimensional world that is either real or abstract. This includes real systems like buildings, landscapes, underwater shipwrecks, spacecrafts, archaeological excavation sites, human anatomy, sculptures, crime scene reconstructions, solar systems, and so on. Of special interest is the visual and sensual representation of abstract systems like magnetic fields, turbulent flow structures, molecular models, mathematical systems, auditorium acoustics, stock market behavior, population densities, information flows, and any other conceivable system including artistic and creative work of abstract nature. These virtual worlds can be animated, interactive, shared, and can expose behavior and functionality. Real and abstract virtual worlds (Stadium, Flow Structure):
Useful applications of VR include training in a variety of areas (military, medical, equipment operation, etc.), education, design evaluation (virtual prototyping), architectural walk-through, human factors and ergonomic studies, simulation of assembly sequences and maintenance tasks, assistance for the handicapped, study and treatment of phobias (e.g., fear of height), entertainment, and much more. Grand Challenge research is employing high-performance computing and communications to build more energy-efficient cars and airplane, to design better drugs, to improve military surveillance systems and environmental monitoring, to create new materials. The 34 Grand Challenge projects now underway range from explorations of molecules to studies on the origins of galaxies. Cosmology One of the largest, 3-dimensional simulations of the universe is helping scientists refine theories about the origins of galaxies. By digitally altering the mix of stellar gas, ordinary matter, and dark matter created soon after the Big Bang, cosmologists are searching for the correct formula for replicating the universe as it exists. Knowing how the structures we see today emerged from the fireball of creation will reveal much about the future of the cosmos. Material sciences Researchers are modeling the more than 400 different hydrogen-nitrogen chemical reactions in an internal combustion engine to design cooler, more efficient car engines. Other researchers are analyzing the properties of compounds in a race to discover the next generation of superconducting material. The winner will revolutionize power transmission and transportation.
Molecular biosciences VR models of some of life's smallest structures are helping scientists decipher the precise mechanisms through which proteins communicate with each other, for instance, to bind antibody to antigen or signal a cell membrane to dilate. Knowledge of this kind is speeding the development of biological and industrial catalysts as well as therapeutic drugs. Relativity By simulating the gravitational ripples that would be generated if two black holes collided, researchers hope to confirm the existence of these elusive objects predicted as a consequence of Einstein's famous General Theory of Relativity. Should the simulated ripples precisely match gravitational waves detected by LIGO ,an array of sensing devices that will become operational in 2000, not only could the existence of black holes be confirmed but also Einstein's 80-year-old theory finally will be vindicated. Weather forecasting When Hurricane Emily approached the Atlantic coast in 1993, a new hurricane model accurately predicted 48 hours in advance that the hurricane would turn sharply back out to sea off Cape Hatteras without making landfall. Predicting longterm weather patterns is one of the outcomes of the new monitoring and instrumentation tools being developed at part of the HPCC. Atmospheric scientists are also turning to advanced computing tools and virtual environments like the CAVE to calculate the behavior of more local disturbances, particularly thunderstorms that spawn tornados
Modeling. We create mental and physical models to better understand our world. Virtual reality allows you to experience and manipulate more complex and sophisticated models than you might otherwise be able to create. Here are just a few examples:
•
•
•
The chemical plant tour and other virtual reality programs developed at the U-M Virtual Reality in Chemical Engineering Lab allow chemical engineering students to study chemical reactions, chemical plant safety, crystal structures, and fluid flow velocity profiles. Beier's virtual car interior will eventually replace a physical mockup for the analysis of design aspects such as layout; visibility of instruments, controls, and mirrors; reachability and accessibility; and human performance. Researchers in the U-M College of Engineering's Virtual Reality Laboratory created a virtual prototype of a cargo ship for a manufacturing company with such detailed and realistic three-dimensional interior and exterior representations that if
•
the design had been put on the Virtual Reality Laboratory Web page, competitors could have copied the hull design. Some medical schools use immersive, interactive visualizations of body parts and systems that students can see in a 360-degree fly-through to help them better understand concepts.
Communication. Under the best of circumstances, effective communication is difficult to achieve. The models created using virtual reality can improve communication. Virtual reality can also be used to create virtual meetings that participants attend via the Internet. Meeting attendees can access virtual meeting rooms, see demonstrations in virtual laboratories, and converse in real time with other attendees. Control. Virtual reality can help you organize, manage, and control large, complex information systems. For example, in the past, it was impossible to model a complex cargo ship in sufficient detail to reveal all potential design flaws. However, when Virtual Reality Laboratory researchers created a virtual prototype of a cargo ship at U-M and conducted a virtual walk-through, severe design flaws were immediately revealed. These findings avoided significant extra manufacturing costs. Virtual reality technology can also be used by people with severe physical handicaps to gain more control and independence. Using only their eyes, people wearing head-mounted displays can manipulate computers that direct robots to perform tasks. Arts, leisure, and entertainment. Virtual reality applications are most numerous and growing most rapidly in the arts, leisure, and entertainment. Video games are one of the largest markets in computer technology. In virtual museums, you can view art by "walking" through virtual galleries; in a virtual design studio, you can interact with remote production teams and examine the design object; in a virtual music room, you can see how to play instruments. Children can build structures using virtual blocks and then enter the structures created. CAVE The CAVE (CAVE Automatic Virtual Environment) is the cadillac of virtual reality systems. The CAVE is a 10 x 10 x 10 foot wholly immersive environment which allows for peripheral vision, multi-person use, and has full sound and visualization capabilities. A CAVE environment consists of a projectable floor and three rear-projection screen walls. Graphics in the CAVE are produced by a Silicon Graphics Onyx supercomputer that houses multiple Reality Engine graphics CPUs. Each Reality Engine is responsible for rendering a single wall. Two views are produced for each wall, one for the left eye and one for the right eye. Stereo glasses are used while in the CAVE in order to view the virtual environment in 3D. These shuttered LCD glasses are synchronized to the screen at an update rate of 96 Hz. The two halves of the stereo image are seen 48 times per second by each eye separately. The brain then combines these two views into one 3-dimensional image. To exploit the hardware capabilities of the CAVE, visualization software must be
developed with calls to the functions of the CAVE software library. This software may be developed and tested on any SGI computer using CAVE simulator mode. The CAVE simulator provides two-fold benefits to the CAVE software development process. First, it frees up the CAVE by allowing multiple developers to simultaneously develop and debug CAVE software on relatively inexpensive workstations. Secondly, it provides less expensive alternatives for organizations that may find the cost of the CAVE prohibitive. The Reactor Engineering Division has explored two such options. A personal desktop VR system has been developed by adding an emitter to an SGI Octane workstation. For group use, a single wall CAVE capability has been developed in the Advanced Simulations and Control Laboratory. Four examples that we have studied so far are: visualization of CAD data, computational crashworthiness and computational combustion.
Virtual Reality Visualization of CAD Data CAD/CAM software plays an integral role in nuclear power plant design. However, CAD/CAM software leaves some major issues in the design process unaddressed. We seek to fill the gap through the use of a virtual reality representation of the plant. Through the use of virtual reality, we plan to investigate the integration of components during construction. Unforeseen problems during construction often require field changes that introduce delays. In addition, virtual reality can be used for the training of plant maintenance personnel. The model we are using for proof-of-concept is a CAD generated model of the AP600 plant. The original AUTOCAD dxf file was passed through a filter to produce a data file compatible with our VR software. We have also developed a methodology for producing virtual models from PRO/E.
BFHS
Pro/Engineer CAD Model Mechanical components placed inside a hot cell must be carefully designed for
maintenance. Once placed inside the hot cell, they are only accessible indirectly through devices such as glove boxes, master slave manipulators and robots. The design of these components must take these restrictions into account. Through a virtual prototype, designers and plant operators can rehearse maintenance procedures and identify potential problems before equipment is built and placed into a hot cell.
Computational Crashworthiness In early 1999, the Transportation Research Center at Argonne hosted research from the
National Crash Analysis Center (NCAC) at George Washington University (GWU). The NCAC is interested in collaborating with Argonne. The specific research issues that we would explore in the course of our collaboration are studying deformations in regions of the car that are difficult to instrument and observe in physical experiments. A pilot project was subsequently launched to provide an initial proof-of-concept demonstration. Crashworthiness simulation results were received from the NCAC. The corresponding finite element model consisted of 81 frames each with 47856 elements and 49574 nodes. The model included an airbag and driver.
Underhood Thermal Management Next - generation automobiles need to meet stringent government standards for emissions and fuel economy. Increased attention has been turned towards high performance computing and simulation as tools to enable engine designers to reach these goals. In addition, these future vehicles will have expensive electronics modules for controlling vehicle behavior in response to road conditions and driving style. A novel issue in the design of these vehicles will be locating these modules away from extreme heat. The model we are using is from a 3 million-cell computation using STAR-CD on the IBM SP computer system. The capabilities that will be demonstrated are the visualization of
temperature fields as well as particle tracking in three dimensions.
An interactive computer system so fast and intuitive that the computer disappears from the mind of the user, leaving the computer-generated environment as reality. Virtual reality refers to computer-generated, interactive, three-dimensional environments into which people are immersed. It provides a way for people to visualize, manipulate, and interact with simulated environments through the use of computers and extremely complex data. In virtual reality, a "world" is created that exists entirely in the memory of a computer and in the equipment you are wearing. The computers control what you sense through simulations such as three-dimensional images, sound, artificial smell, and force feedback. VR is simple to understand. The idea is to show you, the user or viewer, a computer generated environment that adjusts to you, to your movements. To be specific the idea is that when you turn your head and for example, look up to see a computer generated sky, the sky is there. When you look down the ground is there. Now we got a problem though, a computer generated scene is going to be displayed on a monitor. If the monitor is on your desk and you turn your head up, it's kind of hard to see the scene. So now we have the HMD (Head Mounted Display) which basically are those cool looking helmets people wear. Inside the helmet is a display to that as you turn your head, the display is still in front of your eyeballs. Simple idea but not so simple to execute well. One of the big problems is that when you strap a monitor to your face, preferably one in front of each eyeball, the display's are small and usually of low resolution. High resolution displays are now available but expensive. They are getting better and cheaper all the time. You can also use the helmets to just sit and watch videos! I've even seen some airports rent portable DVD players with these displays that you can take with you on airplanes. You, in turn, are able to enter and interact with the virtual realities by controlling the computers through equipment such as head-mounted displays (which track your eye and head movements in relation to the simulations) and data gloves (which track your hand movements in relation to the simulations). Virtual reality can be delivered using a variety of systems. The "world" may be projected inside a 'cave' which users can move around. Or headsets and gloves may be worn so that users are immersed in a virtual world which they can move around and touch. But the most widely used form of virtual reality in use today is desk-top virtual reality. In these systems virtual reality worlds run from users' desk-top computers are displayed on a standard monitor and navigated using a mouse, or 3d space ball and keyboard. Desk top virtual reality systems can be distributed easily via the World Wide Web or on CD and users need little skill to install or use them. Generally all that is needed to allow
this type of virtual reality to run on a standard computer is a single piece of software in the form of a viewer. Desktop virtual reality is very accessible and is widely used. Virtual reality experiences can be described as passive, exploratory, or interactive. In passive virtual reality, you watch, hear, and possibly feel the environment move around you, which makes it appear as if you are moving through the environment. Nevertheless, you cannot control the environment; you are just a spectator. Exploratory virtual reality allows you to explore and move through space. For example, a chemical plant tour has been developed by John Bell, a lecturer in the Chemical Engineering Department, in the U-M College of Engineering's Virtual Reality in Chemical Engineering Lab. Instead of just seeing a three-dimensional room with a reactor in it, you can step inside the reactor and observe the chemical processes. Most architectural virtual walk- throughs and virtual art shows are exploratory virtual reality. The most powerful and complex type of virtual reality is interactive. Here, you can explore the environment and, most importantly, interact with and change it. For instance, in the virtual car interior, if you "touch" a radio button on the car control panel while wearing a data glove, the computer will generate a sound like a radio station. In the virtual chemical plant, students can operate controls to change reactor conditions.
Uses of virtual reality There are many common applications for virtual reality. They fall into the main categories of training, education, simulation, visualization, conceptual navigation, design and entertainment but there is much overlap between these categories:
•
•
•
•
•
Training applications include allowing users to practice a process repeatedly in a no-risk environment. For example, users might dig an archaeological site trying out different strategies without the risk of destroying important evidence. Educational applications include virtual visits and simulations. For example, a virtual visit to a museum that is too far away to visit or does not exist in the real world. Or historic battles may be simulated allowing users to see "what would have happened if?" Visualisation examples include an architect's design for a building or the reconstruction of ancient buildings from archaeological evidence. Such models also allow users to explore something too large or too small to explore in reality and can bring historical time-lines to life. Applications of virtual reality for conceptual navigation enable, for example, users of a library or archive to find the information they need at a logical or physical level. Virtual reality allows designs to be visualized and tested. For example, a design application might allow a choreographer to see a dance in action.
• •
Entertainment applications include virtual art galleries and games. Virtual reality may also be considered as an art form in its own right. Collaborative Virtual Environments (CVEs) allow users to interact with each other in a Virtual world allowing the development of virtual communities adding a new dimension to virtual reality.
Today, 'Virtual Reality' is used in a variety of ways and often in a confusing and misleading manner. Originally, the term referred to 'Immersive Virtual Reality.' In immersive VR, the user becomes fully immersed in an artificial, three-dimensional world that is completely generated by a computer. Head-Mounted Display (HMD) The head-mounted display (HMD) was the first device providing its wearer with an immersive experience. Evans and Sutherland demonstrated a head-mounted stereo display already in 1965. It took more then 20 years before VPL Research introduced a commercially available HMD, the famous "Eye Phone" system (1989). A head-mounted display (HMD):
A typical HMD houses two miniature display screens and an optical system that channels the images from the screens to the eyes, thereby, presenting a stereo view of a virtual world. A motion tracker continuously measures the position and orientation of the user's head and allows the image generating computer to adjust the scene representation to the current view. As a result, the viewer can look around and walk through the surrounding virtual environment.
Various Displays available are:
The helmet is only part of the VR equation. In order for the computer to adjust the display it had to know your head's position. Specifically it needs to know the position of your head in space and the orientation of your head.
Ascention Flock of Birds Motion Tracker Actually if you are sitting still in a chair and you turn your head around without getting up the position information isn't as important as the orientation information. So this magic is accomplished by devices called position trackers. Position trackers are not cheap. They
typically cost several thousand dollars. They work using a variety of technologies each with advantages and disadvantages. There are links to lots of these devices on the VR Hardware page. Getting back to basics though, if you're sitting in front of a computer with a helmet on and the helmet has a position tracker attached (a typical setup) you now have an immersive VR system. The computer graphics scene generated from the computer will be routed to the helmet and the position information of your head, will be routed via the position tracker to the computer. Often the position information can be communicated via a simple serial port, just like your modem. Although that is not the fastest way, which would be via a dedicated interface card in the computer. To overcome the often uncomfortable intrusiveness of a head-mounted display, alternative concepts (e.g., BOOM and CAVE) for immersive viewing of virtual environments were developed. BOOM The BOOM (Binocular Omni-Orientation Monitor) from Fakespace is a head-coupled stereoscopic display device. Screens and optical system are housed in a box that is attached to a multi-link arm. The user looks into the box through two holes, sees the virtual world, and can guide the box to any position within the operational volume of the device. Head tracking is accomplished via sensors in the links of the arm that holds the box. The BOOM, a head-coupled display device:
CAVE The CAVE (Cave Automatic Virtual Environment provides the illusion of immersion by projecting stereo images on the walls and floor of a room-sized cube. Several persons wearing lightweight stereo glasses can enter and walk freely inside the CAVE. A head tracking system continuously adjust the stereo projection to the current position of the leading viewer.
CAVE system (schematic principle):
Input Devices and other Sensual Technologies A variety of input devices like data gloves, joysticks, and hand-held wands allow the user to navigate through a virtual environment and to interact with virtual objects. Directional sound, tactile and force feedback devices, voice recognition and other technologies are being employed to enrich the immersive experience and to create more "sensualized" interfaces. A data glove allows for interactions with the virtual world:
The new 5DT Glove features advanced fiber-optic flex sensors to generate finger-bend data. Move easily through your virtual world by combining hand gestures with the pitch and roll of your hand. Breakthrough pricing, new features, open architecture and software support have made it the glove of choice.
5DT Glove
Cyber Glove
P5 Glove
Pinch Glove
CyberGlove is a low-profile, lightweight glove with flexible sensors which accurately and repeatably measure the position and movement of the fingers and wrist. CyberGlove's award-winning design incorporates the latest high-precision joint-sensing technology. CyberGlove is state-of-the-art in instrumented gloves. Pinch Glove is a remarkable new system for interacting with 3D simulation. This pair of stretch-fabric gloves contain sensors in each fingertip which detect contact between the digits of your hand. You can use these gestures for a wide range of control and interactive functions customized to your specifications. Any combination of single or multiple contacts between two or more digits can be programmed to have specific meanings, ranging from simple on/off to multi-part, multi-action commands. The gestures are not dependent on individual hand geometry - the Pinch never requires calibration. Ok so now we have a computer-generated scene which adjusts to the viewer’s head position. The next thing we need is the ability to interact with the scene. To touch stuff in the scene. This is where the infamous gloves come in. VR glove devices, also not cheap, are devices that are specialized position, orientation sensors. Specialized software reads the position of the hand along with the positions of the figures to let you make gestures of various sorts to let you fly around a scene, select objects, manipulate controls and so on. The final component for a good immersive VR system is spatialized sound. Often the helmets you get have earphones. The computing environment can have sounds associated directly with objects or people floating around in the virtual world. Attaching sounds and getting the sensation of object locations via sound cues is an effective technique to highten the immersive effect.
Characteristics of Immersive VR The unique characteristics of immersive virtual reality can be summarized as follows:
•
• • • • •
Head-referenced viewing provides a natural interface for the navigation in threedimensional space and allows for look-around, walk-around, and fly-through capabilities in virtual environments. Stereoscopic viewing enhances the perception of depth and the sense of space. The virtual world is presented in full scale and relates properly to the human size. Realistic interactions with virtual objects via data glove and similar devices allow for manipulation, operation, and control of virtual worlds. The convincing illusion of being fully immersed in an artificial world can be enhanced by auditory, haptic, and other non-visual technologies. Networked applications allow for shared virtual environments.
Shared Virtual Environments
In the example illustrated below, three networked users at different locations (anywhere in the world) meet in the same virtual world by using a BOOM device, a CAVE system, and a Head-Mounted Display, respectively. All users see the same virtual environment from their respective points of view. Each user is presented as a virtual human (avatar) to the other participants. The users can see each other, communicated with each other, and interact with the virtual world as a team.
Non-immersive VR Today, the term 'Virtual Reality' is also used for applications that are not fully immersive. The boundaries are becoming blurred, but all variations of VR will be important in the future. This includes mouse-controlled navigation through a three-dimensional environment on a graphics monitor, stereo viewing from the monitor via stereo glasses, stereo projection systems, and others. Apple's QuickTimeVR,for example, uses photographs for the modeling of three-dimensional worlds and provides pseudo lookaround and walk-trough capabilities on a graphics monitor. VRML Most exciting is the ongoing development of VRML (Virtual Reality Modeling Language) on the World Wide Web. In addition to HTML (HyperText Markup Language), that has become a standard authoring tool for the creation of home pages, VRML provides threedimensional worlds with integrated hyperlinks on the Web. Home pages become home spaces. The viewing of VRML models via a VRML plug-in for Web browsers is usually done on a graphics monitor under mouse-control and, therefore, not fully immersive. However, the syntax and data structure of VRML provide an excellent tool for the modeling of three-dimensional worlds that are functional and interactive and that can, ultimately, be transferred into fully immersive viewing systems
Rendering of Escher's Penrose Staircase
VR-related Technologies Other VR-related technologies combine virtual and real environments. Motion trackers are employed to monitor the movements of dancers or athletes for subsequent studies in immersive VR. The technologies of 'Augmented Reality' allow for the viewing of real environments with superimposed virtual objects. Telepresence systems (e.g., telemedicine, telerobotics) immerse a viewer in a real world that is captured by video cameras at a distant location and allow for the remote manipulation of real objects via robot arms and manipulators.
VIRTUAL REALITY: Applications for Grand Challenges
As the technologies of virtual reality evolve, the applications of VR become literally unlimited. It is assumed that VR will reshape the interface between people and information technology by offering new ways for the communication of information, the visualization of processes, and the creative expression of ideas. Note that a virtual environment can represent any three-dimensional world that is either real or abstract. This includes real systems like buildings, landscapes, underwater shipwrecks, spacecrafts, archaeological excavation sites, human anatomy, sculptures, crime scene reconstructions, solar systems, and so on. Of special interest is the visual and sensual representation of abstract systems like magnetic fields, turbulent flow structures, molecular models, mathematical systems, auditorium acoustics, stock market behavior, population densities, information flows, and any other conceivable system including artistic and creative work of abstract nature. These virtual worlds can be animated, interactive, shared, and can expose behavior and functionality. Real and abstract virtual worlds (Stadium, Flow Structure):
Useful applications of VR include training in a variety of areas (military, medical, equipment operation, etc.), education, design evaluation (virtual prototyping), architectural walk-through, human factors and ergonomic studies, simulation of assembly sequences and maintenance tasks, assistance for the handicapped, study and treatment of phobias (e.g., fear of height), entertainment, and much more. Grand Challenge research is employing high-performance computing and communications to build more energy-efficient cars and airplane, to design better drugs, to improve military surveillance systems and environmental monitoring, to create new materials. The 34 Grand Challenge projects now underway range from explorations of molecules to studies on the origins of galaxies. Cosmology One of the largest, 3-dimensional simulations of the universe is helping scientists refine theories about the origins of galaxies. By digitally altering the mix of stellar gas, ordinary matter, and dark matter created soon after the Big Bang, cosmologists are searching for the correct formula for replicating the universe as it exists. Knowing how the structures we see today emerged from the fireball of creation will reveal much about the future of the cosmos. Material sciences Researchers are modeling the more than 400 different hydrogen-nitrogen chemical reactions in an internal combustion engine to design cooler, more efficient car engines. Other researchers are analyzing the properties of compounds in a race to discover the next generation of superconducting material. The winner will revolutionize power transmission and transportation.
Molecular biosciences VR models of some of life's smallest structures are helping scientists decipher the precise mechanisms through which proteins communicate with each other, for instance, to bind antibody to antigen or signal a cell membrane to dilate. Knowledge of this kind is speeding the development of biological and industrial catalysts as well as therapeutic drugs. Relativity By simulating the gravitational ripples that would be generated if two black holes collided, researchers hope to confirm the existence of these elusive objects predicted as a consequence of Einstein's famous General Theory of Relativity. Should the simulated ripples precisely match gravitational waves detected by LIGO ,an array of sensing devices that will become operational in 2000, not only could the existence of black holes be confirmed but also Einstein's 80-year-old theory finally will be vindicated. Weather forecasting When Hurricane Emily approached the Atlantic coast in 1993, a new hurricane model accurately predicted 48 hours in advance that the hurricane would turn sharply back out to sea off Cape Hatteras without making landfall. Predicting longterm weather patterns is one of the outcomes of the new monitoring and instrumentation tools being developed at part of the HPCC. Atmospheric scientists are also turning to advanced computing tools and virtual environments like the CAVE to calculate the behavior of more local disturbances, particularly thunderstorms that spawn tornados
Modeling. We create mental and physical models to better understand our world. Virtual reality allows you to experience and manipulate more complex and sophisticated models than you might otherwise be able to create. Here are just a few examples:
•
•
•
The chemical plant tour and other virtual reality programs developed at the U-M Virtual Reality in Chemical Engineering Lab allow chemical engineering students to study chemical reactions, chemical plant safety, crystal structures, and fluid flow velocity profiles. Beier's virtual car interior will eventually replace a physical mockup for the analysis of design aspects such as layout; visibility of instruments, controls, and mirrors; reachability and accessibility; and human performance. Researchers in the U-M College of Engineering's Virtual Reality Laboratory created a virtual prototype of a cargo ship for a manufacturing company with such detailed and realistic three-dimensional interior and exterior representations that if
•
the design had been put on the Virtual Reality Laboratory Web page, competitors could have copied the hull design. Some medical schools use immersive, interactive visualizations of body parts and systems that students can see in a 360-degree fly-through to help them better understand concepts.
Communication. Under the best of circumstances, effective communication is difficult to achieve. The models created using virtual reality can improve communication. Virtual reality can also be used to create virtual meetings that participants attend via the Internet. Meeting attendees can access virtual meeting rooms, see demonstrations in virtual laboratories, and converse in real time with other attendees. Control. Virtual reality can help you organize, manage, and control large, complex information systems. For example, in the past, it was impossible to model a complex cargo ship in sufficient detail to reveal all potential design flaws. However, when Virtual Reality Laboratory researchers created a virtual prototype of a cargo ship at U-M and conducted a virtual walk-through, severe design flaws were immediately revealed. These findings avoided significant extra manufacturing costs. Virtual reality technology can also be used by people with severe physical handicaps to gain more control and independence. Using only their eyes, people wearing head-mounted displays can manipulate computers that direct robots to perform tasks. Arts, leisure, and entertainment. Virtual reality applications are most numerous and growing most rapidly in the arts, leisure, and entertainment. Video games are one of the largest markets in computer technology. In virtual museums, you can view art by "walking" through virtual galleries; in a virtual design studio, you can interact with remote production teams and examine the design object; in a virtual music room, you can see how to play instruments. Children can build structures using virtual blocks and then enter the structures created. CAVE The CAVE (CAVE Automatic Virtual Environment) is the cadillac of virtual reality systems. The CAVE is a 10 x 10 x 10 foot wholly immersive environment which allows for peripheral vision, multi-person use, and has full sound and visualization capabilities. A CAVE environment consists of a projectable floor and three rear-projection screen walls. Graphics in the CAVE are produced by a Silicon Graphics Onyx supercomputer that houses multiple Reality Engine graphics CPUs. Each Reality Engine is responsible for rendering a single wall. Two views are produced for each wall, one for the left eye and one for the right eye. Stereo glasses are used while in the CAVE in order to view the virtual environment in 3D. These shuttered LCD glasses are synchronized to the screen at an update rate of 96 Hz. The two halves of the stereo image are seen 48 times per second by each eye separately. The brain then combines these two views into one 3-dimensional image. To exploit the hardware capabilities of the CAVE, visualization software must be
developed with calls to the functions of the CAVE software library. This software may be developed and tested on any SGI computer using CAVE simulator mode. The CAVE simulator provides two-fold benefits to the CAVE software development process. First, it frees up the CAVE by allowing multiple developers to simultaneously develop and debug CAVE software on relatively inexpensive workstations. Secondly, it provides less expensive alternatives for organizations that may find the cost of the CAVE prohibitive. The Reactor Engineering Division has explored two such options. A personal desktop VR system has been developed by adding an emitter to an SGI Octane workstation. For group use, a single wall CAVE capability has been developed in the Advanced Simulations and Control Laboratory. Four examples that we have studied so far are: visualization of CAD data, computational crashworthiness and computational combustion.
Virtual Reality Visualization of CAD Data CAD/CAM software plays an integral role in nuclear power plant design. However, CAD/CAM software leaves some major issues in the design process unaddressed. We seek to fill the gap through the use of a virtual reality representation of the plant. Through the use of virtual reality, we plan to investigate the integration of components during construction. Unforeseen problems during construction often require field changes that introduce delays. In addition, virtual reality can be used for the training of plant maintenance personnel. The model we are using for proof-of-concept is a CAD generated model of the AP600 plant. The original AUTOCAD dxf file was passed through a filter to produce a data file compatible with our VR software. We have also developed a methodology for producing virtual models from PRO/E.
BFHS
Pro/Engineer CAD Model Mechanical components placed inside a hot cell must be carefully designed for
maintenance. Once placed inside the hot cell, they are only accessible indirectly through devices such as glove boxes, master slave manipulators and robots. The design of these components must take these restrictions into account. Through a virtual prototype, designers and plant operators can rehearse maintenance procedures and identify potential problems before equipment is built and placed into a hot cell.
Computational Crashworthiness In early 1999, the Transportation Research Center at Argonne hosted research from the
National Crash Analysis Center (NCAC) at George Washington University (GWU). The NCAC is interested in collaborating with Argonne. The specific research issues that we would explore in the course of our collaboration are studying deformations in regions of the car that are difficult to instrument and observe in physical experiments. A pilot project was subsequently launched to provide an initial proof-of-concept demonstration. Crashworthiness simulation results were received from the NCAC. The corresponding finite element model consisted of 81 frames each with 47856 elements and 49574 nodes. The model included an airbag and driver.
Underhood Thermal Management Next - generation automobiles need to meet stringent government standards for emissions and fuel economy. Increased attention has been turned towards high performance computing and simulation as tools to enable engine designers to reach these goals. In addition, these future vehicles will have expensive electronics modules for controlling vehicle behavior in response to road conditions and driving style. A novel issue in the design of these vehicles will be locating these modules away from extreme heat. The model we are using is from a 3 million-cell computation using STAR-CD on the IBM SP computer system. The capabilities that will be demonstrated are the visualization of
temperature fields as well as particle tracking in three dimensions.
