FIGURE 2 Physical setup fi highlighted in the scans (see also the chapter "Across-Modality Registration using Intensity-Based Cost Functions").
The result is an augmented, patient-specific, geometric model of relevant structural and functional information. Examples are shown in Fig. 3.
Registration is the process by which the MRI or CT data is transformed to the coordinate frame of the patient. Excellent reviews of registration methods include [22,24,32].
Extrinsic forms of registration use fiducials (e.g., [1,21,31,35]): either markers attached to the skin or bone prior to imaging or anatomically salient features on the head. The fiducials are manually localized in both the MR or CT imagery and on the patient, and the resulting correspondences are used to solve for the registration. Fiducial systems may not be as accurate as frame-based methods — Peters  reports fiducial accuracy about an order of magnitude worse than frame-based methods, but Maciunas  reports high accuracy achieved with novel implantable fiducials. The chapter "Validation of Registration Accuracy" addresses the issues of fiducial-based accuracy in much more detail.
Intrinsic registration is often based on surface alignment, in which the skin surface extracted from the MRI data is aligned with the patient's scalp surface in the operating room. Ryan  generates the patient's scalp surface by probing about 150
image guided surgery system.
points with a trackable medical instrument. Colchester  uses an active stereo system to construct the scalp surface. We also perform the registration using surface alignment , benefiting from its dense data representation, but use either a laser scanner to construct the patient's scalp surface or a trackable probe to obtain data points from the patient's skin surface for registration.
We have used two related methods to register the reconstructed model to the actual patient position. In the first method, we use a laser scanner to collect 3D data of the patient's scalp surface as positioned on the operating table. The scanner is a laser striping triangulation system consisting of a laser unit (low-power laser source and cylindrical lens mounted on a stepper motor) and a video camera. The laser is calibrated a priori by using a calibration gauge of known dimensions to calculate the camera parameters and the sweeping angles of the laser. In the operating room the laser scanner is placed to maximize coverage of the salient bony features of the head, such as nose and eye orbits. To ensure accurate registration we can supplement the laser data with points probed with a Flashpoint pointer, similar to Ryan , to include skin points that are not visible to the laser in the registration. The acquired laser data is overlaid on the laser scanner's video image of the patient for specification ofthe region ofinterest. This process uses a simple mouse interface to outline the region of the head on which we want to base the registration. This process need not be perfect — the registration is designed to deal robustly with outliers. The laser scan takes about 30 seconds once the sensor is appropriately placed above the patient.
An alternative method is to simply use a trackable probe to acquire data. In this case, we trace paths on the skin of the patient with the trackable probe, recording positional information at points along each path. These points are not landmarks, but simply replace the lines of laser data. The registration process is the same, whether matching laser data or trackable probe data to the skin surface of the MRI model.
The key to our system is the integration of a reliable and accurate data-to-MRI registration algorithm. Our registration process is described in detail in Crimson . It is a three-step process performing an optimization on a six-parameter rigid transformation, which aligns the data surface points with the MRI skin surface.
A manual initial alignment can be used to roughly align the two surfaces. Accurate manual alignment can be very difficult, but we aim only to be within 20° of the correct transformation, for which subsequent steps will solve. One method for achieving this uses a pair of displays and takes about 60 seconds. In one display, the rendered MRI skin is overlaid on the laser scanner's video view of the patient, and the MRI data is rotated and translated in three dimensions to achieve a qualitatively close alignment. In the second display, the laser data is projected onto three orthogonal projections of the MRI data. The projected MRI data is colored such that intensity is inversely proportional to distance from the viewer. In each overlay view, the laser data may be rotated and translated in two dimensions to align the projections. An alternative to manual initial alignment is to record three known points using the trackable probe (e.g., tip of the nose, tip of the ear), then identify roughly the same point in the MRI model, using a mouse-driven graphical interface. This process determines a rough initial alignment of the data to the MR reconstruction and typically takes less then 5 seconds.
It is also possible to automate this process, by using search methods from the computer vision literature. In , we describe an efficient search algorithm that matches selected points from the patient's skin to candidate matches from the skin surface of the MRI model. By using constraints on the distance and orientation between the sets of points, these algorithms can quickly identify possible registrations of the two data sets. Applying the coordinate frame transformation defined by each match, the full set of data points from the patient's skin surface can then be transformed to the MRI frame of reference. Residual distances between the transformed data points and the MRI skin surface serve as a measure of fit and can be used to determine good candidate initial alignments.
Civen the initial alignment of the two data sets, we typically have registrations on the order of a few centimeters and a few tens of degrees. We need to automatically refine this alignment to a more accurate one. Ideally, we need algorithms that can converge to an optimal alignment from a large range of initial positions [12-14].
Our method iteratively refines its estimate of the transformation that aligns patient data and MRI data. Civen a current estimate of the transformation, it applies that estimate to the patient data to bring it into the MRI coordinate frame. For each transformed data point, it then measures a Caussian weighted distance between the data point and the nearest surface point in the MRI model. These Caussian weighted distances are summed for all data points, which defines a measure of the goodness of fit ofthe current estimated transformation. This objective function is then optimized using a gradient descent algorithm. The role of the Caussian weighting is to facilitate "pulling in'' of one data set to the other, without needing to know the exact correspondence between data points. The process can be executed in a multiresolution manner, by first using Caussian distributions with large spreads (to get the registration close), then reducing the spread of the distribution, and resolving in a sequence of steps.
This process runs in about 10 seconds on a Sun UltraSPARC workstation. The method basically solves for the transform that optimizes a Caussian weighted least-squares fit of the two data sets.
Automated detailed alignment then seeks to accurately localize the best surface data to MRI transformation [12-14]. Starting from the best transformation of the previous step, the method then solves a second minimization problem. In this case it measures the least-squares fit of the two data sets under the current estimated transformation (subject to a maximum distance allowed between a transformed data point and the nearest point on the skin surface, to discount the effects of outliers in the data). This minimization can again be solved using a gradient descent algorithm.
This process runs in about 10 seconds on a Sun UltraSPARC workstation. The method basically solves a truncated least-squares fit of the two data sets, refining the transformation obtained in the previous step.
To ensure that the solution found using this process is not a local minimum, the method arbitrarily perturbs the transformation and reruns the process. If the system converges to the same solution after several trials, the system terminates with this registration.
The final stage of the process is to determine the relationship between a video camera viewing the patient, and the patient position. This can be accomplished by using a trackable probe to identify the positions of points on a calibration object in patient coordinates. By relating those coordinates to the observed positions in the video image, one can solve for the transformation relating the camera to the patient [12-14].
Augmented Reality Visualization
By coupling all of these transformations together, we can provide visualizations of internal structures to the surgeon. In particular, we can transform the segmented MRI model (or any portions thereof) into the coordinate frame ofthe patient, then render those structures through the camera transformation, to create a synthetic image of how those structures should appear in the camera. This can then be mixed with a live video view to overlay the structures onto the actual image (Fig. 4).
Three verification tools are used to inspect the registration results, as the objective functions optimized by the registration algorithm may not be sufficient to guarantee the correct solution. One verification tool overlays the MRI skin on the video image of the patient (Fig. 5), except that we animate the visualization by varying the blending of the MRI skin and video image. A second verification tool overlays the sensed data on the MRI skin by color-coding the sensed data by distance between the data points and the nearest MRI skin points. Such a residual error display identifies possible biases remaining in the registration solution. A third verification tool compares locations of landmarks. Throughout the surgery, the surgeon uses the optically tracked probe to point to distinctive anatomical structures. The offset of the probe position from the actual point in the MR volume is then observed in the display. This serves to measure residual registration errors within the surgical cavity.
Tracking is the process by which objects are dynamically localized in the patient's coordinate system. Of particular interest to us is the tracking of medical instruments and the patient's head. The two most common methods of tracking are articulated arms and optical tracking. Articulated arms are attached to the head clamp or operating table and use encoders to accurately compute the angles of its joints and the resulting 3D position of its end point. Such devices, though, maybe bulky in the operating room and, because of their mechanical nature, are not as fault tolerant as other methods. Optical trackers use multiple cameras to triangulate the 3D location of flashing LEDs that may be mounted on any object to be tracked. Such devices are generally perceived as the most accurate, efficient, and reliable localization system [2, 5]. Other methods such as acoustic or magnetic field sensing are being explored as well, but can be more sensitive to environmental effects. We use optical tracking (the Flashpoint system by IGT Inc., Boulder, CO, USA) because of its accuracy and ease-of-use benefits, though magnetic tracking systems are of similar capability.
Tracking patient head motion is often necessary for a variety of reasons. The head is not always clamped to the operating table, the head may move relative to the clamp, the
operating table may be moved, or the hardware performing the tracking may be moved to rearrange lights or other equipment in the operating room. Although not all image-guided surgery systems account for patient motion, [1,2,6,21,29] solve this problem by attaching trackable markers to the head or clamp. We currently utilize an optically trackable configuration of markers attached to a Mayfield clamp (Fig. 6). We have also experimented with directly attaching trackable LEDs to the skin surface of the patient. Our experience is that while in most cases this worked well, it required that the surgeon carefully plan the
location of the LEDs to ensure that they did not move between initial placement and opening of the skin flap.
We require direct line-of-sight from the Flashpoint cameras to the LEDs at times when the surgeon requires image guidance. In order to maintain such line-of-sight, we can relocate the scanning bar such that it is out of the way of the surgeon but maintains visibility of the LEDs. Such dynamic reconfiguration of the scanning bar is a benefit of the head tracking process.
Instrument tracking is performed by attaching two LEDs to a sterile pointer. The two LEDs allow us to track the 3D position
FIGURE 6 Trackable configuration of LEDs attached to head clamp, or to the skin flap.
of the tip of the pointer as well as its orientation, up to the twist angle, which is not needed for this application. Figure 6 shows the surgeon using the trackable pointer in the opened craniotomy.
Two types of visualizations are provided to the surgeon on the workstation monitor. One is an enhanced reality visualization in which internal structures are overlaid on the video image of the patient. The video image is set up to duplicate the surgeon's view of the patient. Any segmented MR structures may be displayed at varying colors and opacities (see Fig. 5).
A second visualization shows the location of the pointer tip in a 3D rendering of selected MRI structures and in three orthogonal MRI slices (see Fig. 7). These visualizations are updated twice per second as the pointer is moved.
Was this article helpful?