Fifth Generation Systems II Virtual Reality

Virtual reality visualization represents the next milestone in biomedical visualization. While stereoscopic visualization provided the 3D perception, such a presentation also required intrinsically intuitive 3D interaction with the display. The traditional interaction paradigm using a tool or stylus such as a mouse or pointer that is away from the display lacks the intuitive control of hand-eye coordination. A more natural and intuitive interaction can be obtained by manipulating the image where it is perceived in 3D space. This can be achieved with a stylus using 3D positional sensors. However, placing such a stylus that the hand can manipulate directly on the image would obstruct the view of the image.

Some virtual systems overcome this problem by using helmet-mounted displays that present a virtual world through stereo goggles mounted to the display. This is generally called an ''immersive'' virtual reality paradigm. It may be suitable for representing a virtual world where the physical scale of the objects is larger than the viewers, as in architectural visualization, where they can imagine immersing themselves into such a virtual world. However, such a paradigm may not be suitable for visualizing objects of similar or smaller physical scale, as is often the case in medicine. A more suitable virtual reality paradigm proposed for such interaction is the virtual workbench or virtual work volume concept (Figs 15 and 16). A mirror arrangement is placed at a suitable angle to the stereo display, reflecting the 3D image to the viewer, so that the viewer, looking at the mirror through stereo goggles, perceives 3D objects behind the mirror. The virtual object that appears within the ''reach-in'' distance of the hand can be manipulated exactly where it appears in the virtual work volume. The 3D stylus or mouse can be replaced by a suitable visual motif functionally representing the tool in the image. Augmented reality visualization encapsulates same principles, but instead of presenting a fully synthetic virtual world, it is optically superimposed on the real world, so that the viewer perceives the virtual objects placed in a real working space. This is useful during an intraoperative procedure, combining, for instance, a 3D MRI image that can show subtle differences in the brain tissues with a real image of the brain during the surgery where the naked eye would not be able to see the differences very well. Real-time registration of the actual anatomy with the virtual image is crucial for such integration [56-58]. For such purposes, a stereotaxic reference frame that is stationary with respect to the subject and remains physically attached to the subject during imaging and surgery is used. Another issue that needs to be addressed is the change in the shape of the brain during the craniotomy as the skull is opened for surgery.

FIGURE 12 Finite-element modeling of tumor growth and its visualization using 3D morphing to visualize the morphological changes in the surrounding anatomy. (Top) FEM mesh (white: white matter, gray: gray matter, red: boundary nodes). (Bottom) FEM embedded 3D visualization. (Left) Undeformed brain. (Right) Deformed brain. See also Plate 116. (Images courtesy of Stelios Kyriacou, M. Solaiyappan, Christos Davatzikos.)

FIGURE 12 Finite-element modeling of tumor growth and its visualization using 3D morphing to visualize the morphological changes in the surrounding anatomy. (Top) FEM mesh (white: white matter, gray: gray matter, red: boundary nodes). (Bottom) FEM embedded 3D visualization. (Left) Undeformed brain. (Right) Deformed brain. See also Plate 116. (Images courtesy of Stelios Kyriacou, M. Solaiyappan, Christos Davatzikos.)

FIGURE 13 Finite element modeling of the electrical activity of the heart and its 3D visualization. Depolarization of the heart with both normal and abnormal cells, with the abnormal cells having ionic properties similar to those seen in patients with congestive heart failure. This simulation leads to a sustained reentrant wave of activation. The ECG of this arrhythmia is similar to those seen in CHF patients. (Images courtesy of Raimond Winslow, Dave Scollan, Prasad Gharnpure, M. Solaiyappan.)

FIGURE 14 Real-time interactive surgical simulator. Finite element based modeling of the interaction of catheter and guide-wire devices with the vascular structures of the human body. Visualization provides training and planning information through the simulated fluoroscopic display. (Image courtesy of Yao Ping Wang, Chee Kong Chui, H. L. Lim, Y. Y. Cai, K. H. Mak, R. Mullick, R. Raghavan, James Anderson.)

FIGURE 14 Real-time interactive surgical simulator. Finite element based modeling of the interaction of catheter and guide-wire devices with the vascular structures of the human body. Visualization provides training and planning information through the simulated fluoroscopic display. (Image courtesy of Yao Ping Wang, Chee Kong Chui, H. L. Lim, Y. Y. Cai, K. H. Mak, R. Mullick, R. Raghavan, James Anderson.)

FIGURE 15 Virtual workbench. (Left) Schematic diagram illustrating the concept of reach-in work space. (Right) Prototype model of the workbench. (Photos courtesy of Timothy Poston, Luis Serra.)

FIGURE 16 Presurgical planning: separation of siamese twins. 3D visualization and 3D interactions in the virtual workspace environment using MRA and MR volumes to visualize vascular and brain morphology assisted surgeons in devising strategies in planning the extremely complex and most delicate neurosurgical procedure of separating craniopagus Siamese twins (under the notably challenging circumstance that the patient was in a distant location and could not be reached prior to surgery). Dec. 1997, Zambia. Pediatric neurosurgeon: Benjamin Carson (JHU). Software (VIVIAN): Luis Serra, Ng Hern, (KRDL, Singapore). Tushar Goradia, James Anderson (JHU).

FIGURE 16 Presurgical planning: separation of siamese twins. 3D visualization and 3D interactions in the virtual workspace environment using MRA and MR volumes to visualize vascular and brain morphology assisted surgeons in devising strategies in planning the extremely complex and most delicate neurosurgical procedure of separating craniopagus Siamese twins (under the notably challenging circumstance that the patient was in a distant location and could not be reached prior to surgery). Dec. 1997, Zambia. Pediatric neurosurgeon: Benjamin Carson (JHU). Software (VIVIAN): Luis Serra, Ng Hern, (KRDL, Singapore). Tushar Goradia, James Anderson (JHU).

0 0

Post a comment