Segmentation and Classification of Intensity Images

In many remote sensing applications it is important to be able to classify SAR images in terms of their intensity. The difficulty is the effect of speckle noise which, as we shall see, makes assigning the intensity within a single pixel to a given class very inaccurate. The situation can be somewhat improved, in general, by averaging over a window of (say) 3x3 pixels. This averages out the edges between regions, which would be undesirable in a complicated scene or if small objects are of interest.

Provided that we can be certain that we have known (ground truth) boundaries that actually apply to the data, with complete homogeneity within each region, classification can be carried out by averaging the intensity within each ground truth region. This represents what we shall term the Upper Limit of classification capability. However, this method can only classify images with known regions; it has no ability to deal with raw imagery. Furthermore, in many situations the contents of the regions in an image,whether it's region boundaries are known or not, are not uniform. Crops, for example, grow at different rates in different parts of a field, and/or may not actually fill the specified region.

Segmentation provides a tool for deriving homogeneous segments within the data, based on the information within the data. The intensity within each segment then represents the most reliable information about the cross-section within that region. Inhomogeneities arising from any cause will be detected. In this study we compare the results of several possible approaches to the classification of regions, including segmentation. To ensure that we have ground truth to compare the results obtained, we use a simulated three-look SAR image containing a known pattern of fields. This is representative of the SAR imagery obtained with the ERS1 and ERS2 satellites.

**Figure 6.1:** Simulated 3-look SAR data.
$\includegraphics[width=0.6\textwidth]{figures/study1/Image1.eps}$

Ground truth is available for this simulated image and is divided into a training and test set, illustrated in Figures 6.1.1(a) and 6.1.1(b).

$\includegraphics[width=0.6\textwidth]{figures/study1/Image2.eps}$

Figure 6.1.1(a): Training Ground Truth

$\includegraphics[width=0.6\textwidth]{figures/study1/Image3.eps}$

Figure 6.1.1(b): Test data Ground Truth. The classes are represented by: 0 - magenta; 1- blue; 2 - cyan; 3 - green; 4 - yellow; 5 - red.

Comparison of classification methods

We can now compare the results obtained classifying the data in the test image using the following methods:

We expect the Upper Limit results (unachievable on ``real'' data) to be best; the Lower Limit results to be the worst. The major interest is in seeing how much better are the segmentation-based results than those achieved with a traditional local (3 by 3) filter.

In each case we calculate the likelihood that each region is a member of a gamma-distributed PDF of defined mean, depending on the class. The region is then assigned to that class which has the maximum likelihood. We assume for this comparison that the order parameter, determined by the speckle, is uniform across the image, as would be expected in practice.

The classification results are shown in Figure 6.1.2. Classification of a single pixel, shown in Figure 6.1.2(a) is clearly very poor, as expected. Indeed, the average probability of correctly classifying each category is only 0.364 and the associated confusion matrix, shown in Table 6.1.2(a) reveals how poorly the intermediate classes are assigned. Classification over a 3x3 window, shown in Figure 6.1.2(b) shows considerable improvement. However, the average fraction of correctly classified pixels in each class is still only 0.571 and the associated confusion matrix in Table 6.1.2(b) reveals considerable spread in the intermediate classes.

The result of classification following segmentation, shown in Figure 6.1.2(c) is much closer to the ground truth, with the average fraction of correctly classified categories rising to 0.891 and the confusion matrix showing much less ambiguity between classes.

Finally, the Upper Limit, provided by the ground truth itself, leads to the results shown in Figure 6.1.2(d). The average probability of correct classification is now 0.993, and the confusion matrix shows very little ambiguity.

The conclusions are clear: if you have an accurate region map (ground truth) use it. If not: segment the image and classify the computed segmentation.

$\includegraphics[width=0.6\textwidth]{figures/study1/Image4.eps}$

Figure 6.1.2(a): Classification using single pixels

$\includegraphics[width=0.6\textwidth]{figures/study1/Image5.eps}$

Figure 6.1.2(b): Classification using a 3 by 3 filter

$\includegraphics[width=0.6\textwidth]{figures/study1/Image6.eps}$

Figure 6.1.2(c): Classification using segmentation

$\includegraphics[width=0.6\textwidth]{figures/study1/Image7.eps}$

Figure 6.1.2(d): Classification from the Ground Truth

Assigned class:	0	1	2	3	4	5
True class/Prob
0	0.8000	0.3445	0.1699	0.0732	0.0362	0.0324
1	0.1777	0.3522	0.2712	0.1809	0.1447	0.0956
2	0.0203	0.1898	0.2280	0.2234	0.1338	0.1211
3	0.0020	0.0782	0.1525	0.1659	0.1646	0.1436
4	0.0	0.0248	0.0866	0.1300	0.1555	0.1258
5	0.0	0.0105	0.0919	0.2266	0.3653	0.4815

Table 6.1.2(b) Confusion Matrix for Classification using a 3 by 3 Filter

Assigned class:	0	1	2	3	4	5
True class/Prob
0	0.8196	0.0870	0.0059	0.0	0.0	0.0
1	0.1236	0.7941	0.2886	0.0614	0.0457	0.0032
2	0.0466	0.1177	0.5413	0.32440	0.2057	0.0442
3	0.0095	0.0012	0.1514	0.4286	0.2815	0.1627
4	0.0007	0.0	0.0128	0.1629	0.2998	0.2491
5	0.0	0.0	0.0	0.0227	0.2011	0.5408

Table 6.1.2(c) Confusion Matrix for Classification using Segmentation

Assigned class:	0	1	2	3	4	5
True class/Prob
0	0.9243	0.0472	0.0329	0.0222	0.0307	0.0099
1	0.0277	0.8908	0.1205	0.0258	0.0579	0.0099
2	0.0345	0.0501	0.7894	0.0490	0.0	0.0385
3	0.0122	0.0010	0.0495	0.8928	0.0	0.0005
4	0.0014	0.0003	0.0033	0.0039	0.9096	0.0
5	0.0	0.0005	0.0042	0.0052	0.0018	0.9405

Table 6.1.2(d) Confusion Matrix for Classification using Ground Truth

Assigned class:	0	1	2	3	4	5
True class/Prob
0	0.9899	0.0002	0.0	0.0	0.0	0.0
1	0.0095	0.9998	0.0	0.0	0.0	0.0
2	0.0007	0.0	1.0000	0.0	0.0	0
3	0.0	0.0	0.0	1.0000	0.0	0.0
4	0.0	0.0	0.0	0.0	0.9982	0.0287
5	0.0	0.0	0.0	0.0	0.0018	0.9713

Conclusions

It is apparent from this comparison that it is pointless attempting classification on individual intensity pixels in SAR. This example was for a 3 look system; with single look data the result would be much worse.

Averaging over a 3x3 window improves the result. The accuracy obtained in this way could be further improved by increasing the size of the window. However, there is then a problem that small regions will get averaged out, and disappear. What is required is a method which adapts the region size to the data; this is segmentation.

Segmentation followed by classification yields a considerable improvement, since it provides a filter whose window size and shape is adapted to the data. The fact that the segments are not matched exactly to the ground truth regions is a consequence of speckle noise. In this instance we know that the simulation was of a scene with uniform RCS within each region. The characteristic SAR speckle spoils this, and is the major hurdle to be overcome in handling SAR images.

The Upper Limit, as one would expect, provides the highest statistical accuracy. There are two important caveats to note, however:

Thus, the overall conclusion is that segmentation provides an essential precursor to intensity classification. This is the approach taken by the classification routines in InfoPACK.

Segmentation and Classification of Intensity Images

Introduction

Comparison of classification methods

Conclusions