## Segmentation Based on Statistical Models

Anatomical structures show a natural variation for different individuals (interindividual) and also for the same individual (intraindividual) over time. Obvious examples for intraindividual variation of organ shape are the lungs or the heart that both show cyclic variation of their shape. In contrast the bladder shows noncyclic shape variations that mainly depend on its filling. Several researchers have proposed to model the natural (large, but still strongly limited) variability of inter- as well as intraindividual organ shapes in a statistical framework and to utilize these statistics for model-based segmentation. The idea is to code the variations of the selected shape parameters in an observed population (the training set) and characterize this in a possibly compact way. This approach overcomes the limitations of the basically static view of the anatomy provided by the altases from the preceding section.

Such methods fundamentally depend on the availability of parametric models suitable to describe biological shapes. These frameworks always consider variation of shape, but may also include other characteristics, such as the variation of intensity values. Several methods have been proposed for such parametric shape descriptions, as deformable superquadrics augmented with local deformation modeling [21,22], series expansions [23,24], or simply using the coordinates of selected organ surface points as used by Cootes in [25] for his point distribution models.

To model these statistics the a priori probability p(p) of a parameter vector p and eventually the conditional probability p(p | D) under the condition of the sensed data D are estimated by a set of samples. Estimation of the probability p(p), however, requires that the entities of the sample set are described in the same way. If, for example, the parameter vector p simply consists of voxel coordinates then it is essential, that the elements of the parameter vectors of the different entities always describe the position of the same point on the different entities at the same position in the vector. To find these corresponding points on the different realizations is an important prerequisite for the generation of statistical shape models. However, it proves difficult, especially as there in no real agreement on what correspondence exactly is, and how it can be measured.

Correspondence can be established in two ways, either discrete or continuously. In the discrete case the surfaces are represented as point sets and the correspondence is defined by assigning points in different point sets to each other. In the continuous case, parameterizations of the surfaces are defined such that the same parameter values parameterize corresponding positions on the surface.

Looking at the discrete case the most obvious method is to define correspondences manually. To do so a number of landmarks need to be defined on each sample by the user. This method has been successfully used by [26]; however, this technique clearly requires extensive user input, making it only suitable when very limited number of points is regarded. Another possibility when dealing with discrete point sets offers the softassign Procrustes matching algorithm as described by [27]. The algorithm tackles the problem of finding correspondences in two point sets using the Procrustes distance shape similarity measure [18] that quantifies shape similarity.

The most common approach for the approximation of continuous correspondences in 2D is arc-length parameterization. Thus, points of the same parameter on different curves are then taken to be corresponding. This approach heavily depends on the availability of features and is thus bound to fail if the same features can not be located in both modalities. An other interesting view on continuous correspondences in 2D is given by [28], who defines correspondence between closed curves C1 and C2 as a subset of the product space C1 x C2. Kotcheff and Taylor presents in [29] an algorithm for automatic generation of correspondences based on eigenmodes. In [30] Kelemen et al. shows a straight forward expansion of arc-length parameterization based correspondence of curves to establish correspondences on surfaces. They establish correspondence by describing surfaces of a training set using a shape description invariant under rotation and translation presented in [24].

Once the parameterization is selected, the anatomical objects of interest are fully described (at least from the point of view of the envisioned segmentation procedure) by the resulting parameter vector p = {p1, p2,..., pn}, where n can of course be fairly large for complex shapes. Possible variations of the anatomy can be precisely characterized by the joint probability function of the shape parameters pi, information of which can be integrated into a stochastic Bayesian segmentation framework as a prior utilizing the knowledge gained from the training data for constraining the image analysis process [22,31]. It has to be, however, realized that the usually very limitednumber of examples in the training set forces us to very strongly limit the number of parameters involved in a fitting procedure. A very substantial reduction of the number of parameters can be achieved based on the fact that the single components of the vector p are usually highly correlated. A simplified characterization of the probability density is possible based on the first- and second-order moments of the distribution (for a multivariate Gaussian distribution this description is exact). The resulting descriptors are

where the training set consists of the N examples described by the parameter vectors p;;

• the covariance matrix of the components of the parameter vectors:

The existing correlations between the components of the vectors p can be removed by principal component analysis, providing the matrix Pp constructed from the eigenvectors of the covariance matrix £. Experience shows that even highly complex organs can well be characterized by the first few eigenvectors with the largest eigenvalues. This results in a description called active shape model [32], which allows to reasonably approximate the full variability of the anatomy by the deviation from the mean shape as a linear combination of a few eigenmodes of variation. The coefficients of this linear combination provide a very compact characterization of the possible organ shapes.

The automatic extraction of the outline of the corpus callosum on midsagit-tal MR images [33] nicely illustrates the basic ideas of using active shape models for segmentation. Figure 14.4 shows the region of interest covering the corpus callosum on a brain section (a) and on an MR image slice (b). Several examples have been hand-segmented, providing a training set of 71 outlines, which have been parameterized by Fourier coefficients up to degree 100. In order to incorporate not only shape-related but also positional variations into the statistical model, the contours have been normalized relative to a generally accepted neu-roanatomical coordinate system, defined by the anterior and posterior commissures (Fig. 14.4). The training data used and the shape model resulting from the principle component analysis is illustrated in Fig. 14.5. As image (b) illustrates,

Figure 14.4: (a) The corpus callosum from an anatomical atlas and (b) the corresponding region of interest in a midsagittal MR image. On the left image the connecting line between the anterior commissure (AC) and the posterior commissure (PC), which is used for normalization, is also shown.

Figure 14.4: (a) The corpus callosum from an anatomical atlas and (b) the corresponding region of interest in a midsagittal MR image. On the left image the connecting line between the anterior commissure (AC) and the posterior commissure (PC), which is used for normalization, is also shown.

the largest 12 eigenvalues (defined by the 400 original parameters) already reasonably represent the variability (covering about 95% of the full variance).

This statistical description can easily be used as a parametric deformable model allowing the fully automatic segmentation of previously unseen images (apart from the definition of the stereotactic reference system). Based on the

Figure 14.5: Building the active shape model for the corpus callosum. (a) The 71 outlines of the training set normalized in the anatomical coordinate system defined by the anterior and posterior commissures (AC/PC). The eigenvalues resulting from the principal component analysis are plotted in (b), while the eigenvectors corresponding to the three largest eigenvalues are illustrated in (c), (d), and (e). The deformations which correspond the eigenmodes cover the range - (light gray) to +V2Xk (dark gray).

Figure 14.5: Building the active shape model for the corpus callosum. (a) The 71 outlines of the training set normalized in the anatomical coordinate system defined by the anterior and posterior commissures (AC/PC). The eigenvalues resulting from the principal component analysis are plotted in (b), while the eigenvectors corresponding to the three largest eigenvalues are illustrated in (c), (d), and (e). The deformations which correspond the eigenmodes cover the range - (light gray) to +V2Xk (dark gray).

Figure 14.6: Segmentation of the corpus callosum. The top-left image shows the initialization, resulting from the average model and a subsequent match in the subspace of the largest four deformation modes. The other images (top right, lower left, and lower right) illustrate the deformation of this model during optimization using all selected deformation modes, allowing fine adjustments. The black contour is the result of a manual expert segmentation.

Figure 14.6: Segmentation of the corpus callosum. The top-left image shows the initialization, resulting from the average model and a subsequent match in the subspace of the largest four deformation modes. The other images (top right, lower left, and lower right) illustrate the deformation of this model during optimization using all selected deformation modes, allowing fine adjustments. The black contour is the result of a manual expert segmentation.

concept of deformable contour models or snakes [34] (see section 14.4.3), the corpus callosum outline can be searched in the subspace spanned by the selected number of largest eigenmodes using the usual energy minimization scheme as illustrated in Fig. 14.6. The efficiency of the fit can be vastly increased by incorporating information about the actual appearance of the organ on the radiological image, for example, in the form of intensity profiles along its boundary, as illustrated in Fig. 14.7(a), leading ultimately to the usage of integrated active appearance models [35] incorporating the shape and gray-level appearance of the anatomy in a coherent manner.

The illustrated ideas generalize conceptually very well to three dimensions, as illustrated on the anatomical model of the basal ganglia of the human brain shown in Fig. 14.8. The corresponding active shape model has been successfully applied for the segmentation of neuroradiological MR volumes [36]. Remaining interactions needed for the establishment of the anatomical coordinate system

Figure 14.7: Intensity profiles along the boundary of a (a) 2-D and a (b) 3-D object.

can be eliminated using automated adaptation of the stereotactical coordinate system [37].

The approach presented in this chapter can be extended to multiorgan matching, since they are spatially related to each other. The prostate, for example, is placed at an inferior dorsal position to the bladder. The bladder broadly changes its shape due to its filling and this deformation also influences the shape and position of the prostate. This correlation of the position and shape of the organs can be modeled by incorporating multiple organs in the shape statistics. To do so the coefficient vectors pj of the n incorporated organs can be gathered in one single coefficient vector pcomb = (p1, pn). As modeling the relative position of the organs is believed to be one of the major benefits of multiorgan modeling, the centers of gravity must be considered in the statistics. In

•à T |
Q * | |||

w m—'m m m m ■ ■ ■ |

Kl | |

m |

Mean model Correct eignmodes for bladder

Figure 14.9: Prediction of the position of the prostate for a known bladder shape.

particular the relative positions with respect to the origin of a common coordinate system of the combined organs must be modeled. There are different possibilities to choose this reference coordinate system. One possible and intuitive choice for a reference coordinate system would be the center of gravity of one of the organs.

Figure 14.9 shows that the position of the prostate depending on the shape of the bladder is modeled reliably. Here, the mean bladder-prostate model is shown on the left. In the right the first 10 eigenmodes were added, so that they best approximate the bladder. As can be seen in the figure, the position of the prostate is also approximately found, although no information on the prostate was included.

It should be noted that the establishment of correspondence is still a major matter of concern while the training set is created, which further complicates the generation of suitable data collections for training. The intensive manual work needed is, however, limited to the training phase, while the actual segmentation of the unseen data is fully automatic. The correspondences including the spatial variability and radiological appearance of the anatomical landmarks are integrated into the statistical model and will be transfered to the new images during the fitting process.

## Post a comment