Segmentation and Classification of Multiple Intensity Images

Introduction

In the previous case study we demonstrated the capability of segmentation to improve the classification of a simulated 3-look SAR image. In some applications a sequence of such images is available. We assume as prior knowledge that every image in the sequence has fundamentally the same regions and boundaries with only variation in the RCS within each region. Under these conditions we may perform joint segmentation over every image in the sequence. If we do this, the common boundaries reinforce one another and help to overcome speckle noise. One would therefore expect this to be reflected in improved segmentation and classification performance.

To demonstrate this we generate another 9 simulated images, with the same underlying segments as those introduced in the previous study. These then differ only in terms of speckle; they simulate a set of images obtained by repeated passes of a SAR over the same scene. A similar analysis to that performed previously can be carried out on this set of intensity images. The ground truth training and test masks are identical to the previous analysis.

Comparison of classification methods

We now compare classification of the test image using the same methods as before:

• Upper Limit - classifying the average intensity in each ground truth region
• Joint Segmentation - classifying the average intensity in each segment
• 3x3 window - classifying the average intensity over a 3x3 window
• Lower Limit - classifying the intensity in each pixel.

Note that the averages are now taken over each pixel within each segment and over all images in the sequence. However, there is another option available for this particular case where the boundaries and underlying cross-section is identical in each image. The Maximum Likelihood reconstruction is obtained by averaging all the intensities, i.e. by summing the images in the first place. Only a single image is then needed for segmentation, rather than joint segmentation over the complete set. We shall call this method the High Limit, since it would be expected to be lower than the Upper Limit but (since it is a maximum likelihood algorithm) higher than joint segmentation . In each case we calculate the likelihood that each region is a member of a gamma-distributed PDF of defined mean, depending on the class. The region is then assigned to that class which has the maximum likelihood. We assume for this comparison that the order parameter, determined by the speckle, is uniform across the image, as before.

The classification results are shown in Figure 2. Classification using a single pixel, shown in Figure 2(a) is still very poor. Indeed, the average probability of correctly classifying each category is only 0.673 and the associated confusion matrix, shown in Table 2(a) reveals major ambiguities in assigning the intermediate classes. Classification over a 3x3 window, shown in Figure 2(b) shows less improvement than might be expected, with the average fraction of correctly classified pixels in each class only 0.707. The associated confusion matrix in Table 2(b) again reveals considerable spread in the intermediate classes. Closer study of Figure 2(b) suggests that the poor performance is due to the averaging filter. This means that each change in cross-section is surrounded by a band of intermediate value which are classified into other categories. The result of classification following segmentation, shown in Figure 1(c) is very close to the ground truth, with the average fraction of correctly classified categories rising to 0.984 and the confusion matrix showing much less ambiguity between classes. The Higher Limit result, shown in Figure 1(d), yields an identical value for the average probability of correct classification. The fact that there is no advantage over joint segmentation reveals that the latter is capable of extracting all the information within the set of identical images. Finally, the Upper Limit, provided by the ground truth itself, leads to the results shown in Figure 1(e). The average probability of correct classification is now 0.992 with the confusion matrix showing very little ambiguity.

Figure 2(a): Classification using single pixels

Figure 2(b): Classification using 3 by 3 filter

Figure 2(c): Classification using joint segmentation

Figure 2(d): Classification using segmentation of the averaged images

Figure 2(d): Classification using Ground Truth segments

Table 2(a) Confusion Matrix for Classification using Single Pixels

Assigned class:	0	1	2	3	4	5
True class/Prob
0	0.9696	0.0304	0.0	0.0	0.0	0.0
1	0.0346	0.8409	0.1236	0.0	0.0	0.0
2	0.0073	0.1455	0.6641	0.1821	0.0063	0.0
3	0.0	0.0085	0.2299	0.5036	0.2260	0.0320
4	0.0	0.0	0.0325	0.2767	0.4069	0.2839
5	0.0	0.0	0.0037	0.0695	0.2726	0.6543

Table 2(b) Confusion Matrix for Classification using 3 by 3 filter

Assigned class:	0	1	2	3	4	5
True class/Prob
0	0.8291	0.1020	0.0628	0.0070	0.0	0.0
1	0.0059	0.9769	0.0171	0.0	0.0	0.0
2	0.0	0.1004	0.7574	0.1394	0.0028	0.0
3	0.0	0.0060	0.1883	0.5374	0.2670	0.0013
4	0.0	0.0037	0.0640	0.2505	0.3766	0.3053
5	0.0	0.0	0.0047	0.0421	0.1864	0.7667

Table 2(c) Confusion Matrix for Classification using Joint Segmentation

Assigned class:	0	1	2	3	4	5
True class/Prob
0	0.9777	0.0176	0.0047	0.0	0.0	0.0
1	0.0064	0.9873	0.0062	0.0002	0.0	0.0
2	0.0006	0.0102	0.9823	0.0053	0.0014	0.0003
3	0.0	0.0020	0.0078	0.9830	0.0059	0.0013
4	0.0	0.0	0.0036	0.0054	0.9837	0.0072
5	0.0	0.0	0.0	0.0031	0.0084	0.9885

Table 2(d) Confusion Matrix for Classification using Segmentation of the averaged images

Assigned class:	0	1	2	3	4	5
True class/Prob
0	0.9574	0.0257	0.0108	0.0061	0.0	0.0
1	0.0041	0.9948	0.0010	0.0	0.0	0.0
2	0.0003	0.0061	0.9934	0.0	0.0	0.0003
3	0.0	0.0013	0.0372	0.9615	0.0	0.0013
4	0.0	0.0018	0.0	0.0	0.9982	0.0
5	0.0	0.0	0.0	0.0	0.0	1.0

Table 2(e) Confusion Matrix for Classification using the Ground Truth segments

Assigned class:	0	1	2	3	4	5
True class/Prob
0	1.0	0.0	0.0	0.0	0.0	0.0
1	0.0	1.0	0.0	0.0	0.0	0.0
2	0.0	0.0003	0.9997	0.0	0.0	0.0
3	0.0	0.0	0.0	1.0	0.0	0.0
4	0.0	0.0	0.0	0.0	0.9992	0.0008
5	0.0	0.0	0.0	0.0	0.0	1.0

Conclusions

On comparing the results for classification over 10 images with the equivalent results for a single image we note that the classification performance is improved considerably for all except the 3x3 window result.

The 3x3 window method is shown to be a poor approach because the window introduces correlations at the boundaries of regions leading to misclassification. Joint segmentation followed by classification yields a considerable improvement, since it provides a filter whose window size and shape is adapted to the data. The fact that the segments are not matched exactly to the ground truth regions is an inevitable consequence of speckle noise. In this instance we know that the simulated image had a uniform RCS within each region. The speckle statistics have of course spoilt this.

The High Limit uses the fact that the average intensity in every region is the same. This allows the average intensity to be segmented and classified. However, the performance is essentially the same as that achieved with joint segmentation, revealing that the latter is able to use all the information in the sequence of images. The Upper Limit, as one would expect, provides the highest statistical accuracy.

There are two important caveats to note, however.

1. This result is based on the assumption that the RCS within the region is indeed homogeneous. In real applications this will not generally be the case.
2. The ``Upper Limit'' classification can only be applied where region boundaries are known. It cannot extrapolate beyond the ground truth. Segmentation, on the other hand, can use the training set to enable it to classify over as large an image as is required.

Thus, the overall conclusion is as in Study 1: segmentation provides an essential precursor to intensity classification.