Validation

Our pilot study on pancreatic cancer did not include a validation step due to the small number of the tested images to date. However, the evaluation of the clustering and segmentation outputs is expected to be a major part of this application. Hence, we will close our algorithm description with few remarks on segmentation validation issues and a summary of the measures proposed for this purpose.

Validation requires a gold standard segmentation that represents the "absolute truth" on the size and shape of the object of interest. The lack of a gold standard or absolute ground truth in most medical imaging applications does not allow an absolute quantitative evaluation of the segmentation output. The best and often only, option available is segmentations generated by expert observers that may be biased and also exhibit significant inter- and intraobserver variability. In some cases, an alternative approach to the direct evaluation of segmentation results is the use of simulation or phantom studies, [69] the use of relative performance measures, or the use of classification outcomes [70].

The goal of validation in our application is to demonstrate that the automatic methods proposed for the segmentation of pancreatic tumors will lead to standardized and more reproducible tumor measurements than the manual and visual estimates performed traditionally by experts. Tumor size, area, and volume are parameters currently used to determine tumor resectability, and response to treatment. Greater accuracy, less variability, and greater reproducibility in these measurements is expected to have a significant impact on the diagnosis and treatment of pancreatic cancer [71].

An indicated in Fig. 4.8, a postprocessing step is usually applied to the clustered data prior to validation in order to generate smooth contours of the organs and tumors that can then be compared to those in the truth files; see for example the truth files in Fig. 4.10 and the FCM segmentations (white outlines) of Figs. 4.15 and 4.16). From the measures available for segmentation validation, [72] we have selected and implemented those that are recommended for medical imaging applications and are particularly suited for the comparison of computer-generated to hand-drawn boundaries [73-75]. In addition, they are relatively computationally efficient and are not limited to specific shape patterns. These measures are as follows [74]:

1. The Hausdorff distance h(A, B) between two contours of the same object (tumor), one generated by an expert (A) and one generated by the computer (B).

Let A ={a1, a^,..., am} and B = {61, b2,..., bm} be the set of points on the two contours (each point representing a pair of x and y coordinates) then the distance of a point ai to the closest point on curve B is defined as d(ai, B) = min \\bj — ai|| j

Similarly the distance of a point bj to the closest point on curve A is defined as d(bj, A) = min \ai — bj\\

The Hausdorff distance h(A, B) is defined as the maximum of the above distances between the two contours, i.e.

2. The degree of overlap OL between the areas G and E encompassed by contours A and B. The overlap is defined as the ratio of the intersection and the union of the two areas, i.e, the ground truth area G and the experimental computer generated area E:

The ratio is 1 if there is perfect agreement and 0 if there is complete disagreement.

3. The mean absolute contour distance (MACD). MACD is a measure of the difference between the two contours. To estimate MACD, a one-to-one correspondence between the points of the two curves is required. Once this correspondence is established, the distances between corresponding points are estimated; their average corresponds to MACD. In addition to the absolute differences entering the MACD calculation, the signed distances between the curves may also be computed and used to determine the bias of an algorithm or any regional effects on the segmentation process, i.e., pancreatic areas closer to the liver may be less accurately segmented than areas away from large organs [74].

The first two measures above are sensitive to the size and shape of the segmented objects and also depend on the image spatial resolution. The third measure is independent of object size and image resolution and preferred if images from different sources are to be compared.

Alternatively to custom-made routines, the VALMET segmentation validation software tool that is publicly available could be used to generate these metrics in 2D and 3D [73]. Tools such as VALMET and ITK may offer the standardization missing from the validation of segmentation algorithms and reduce variability. Currently, there is no agreement on the "best method" or "best methods" for analyzing and validating segmentation results. The need for standardized measures that are widely acceptable is significant as is the need for establishing conventions on how to use expert-generated ground truth data in the evaluation process.

In a final note, the reader is reminded that a statistical analysis that measures the agreement between the measured parameters from different segmentation algorithms or the agreement between computer and observer performances should be part of the validation process. Computer and expert data are compared with a variety of statistical tools. The most frequently reported ones include (a) linear regression analysis to study the relationship of the means in the various segmentation sets [76, 77], (b) paired t test to determine agreement between the computer method(s) and the experts [76, 77], (c) Williams index to measure interobserver or interalgorithm variability in the generation of manual outlines [74], and (d) receiver operating characteristic analysis and related methods to obtain sensitivity and specificity indices by estimating the true positive and false positive fractions detected by the algorithm and/or the observer [78].

10 Ways To Fight Off Cancer

10 Ways To Fight Off Cancer

Learning About 10 Ways Fight Off Cancer Can Have Amazing Benefits For Your Life The Best Tips On How To Keep This Killer At Bay Discovering that you or a loved one has cancer can be utterly terrifying. All the same, once you comprehend the causes of cancer and learn how to reverse those causes, you or your loved one may have more than a fighting chance of beating out cancer.

Get My Free Ebook


Post a comment