Segmentation and Classification of Polarimetric Data

Introduction

In many remote sensing applications important additional information can be derived from multipolarised imagery. The different polarisations respond in different ways to the orientation and shape of the objects from which scattering takes place. It is obviously important to have tools that make full use of this information, and in this Case Study we look at the steps needed. As previously, we simulate data of the required type, in this case HH, HV and VV complex imagery. The underlying class structure in the simulated image is shown in Figure 6.19(a). The specified ground truth is illustrated in Figure 6.19(b) while the training and test masks are shown in Figures 6.19(c) and (d) respectively.

Figure 6.19: (a) Simulation classes, (b) ground truth, (c) training mask, (d) test mask. The classes are denoted by 0 (magenta), 1 (blue), 2 (cyan), 3 (green), 4 (yellow) and 5 (red).

Figure 6.19(a) Figure 6.19(b)

Figure 6.19(c) Figure 6.19(d)

In order to simulate polarised imagery we specify the class covariance matrix for each of the six classes shown in Figure 6.19. The covariance matrix has the form:

$$\left\vert \begin{array}{ccc} E_{HH}E_{HH}^{*}&E_{HH}E_{HV}^{... ...*}&E_{VV}E_{HV}^{*}&E_{VV}E_{VV}^{*}\\ \end{array}\right\vert$$

The values of the elements for each class are listed in Table 6.8.1. Note that the element (0,0) is identical for each class. This means that the HH image will show no structure.

1:	1+0j	0j	0j
	0j	1+0j	0j
	0j	0j	0.1+0j
2:	1+0j	0.68+0j	0.21+0j
	0.68+0j	1.8+0j	0.12+0j
	0.21+0j	0.12+0.01j	0.15+0j
3:	1+0j	0.43-0.06j	0.1-0.2j
	0.43+0.06j	1.2+0j	0.4-0.01j
	0.1+0.2j	0.4+0.01j	0.18+0j
4:	1+0j	0.73+0.73j	0.092+0.046j
	0.73-0.73j	2.1+0j	0.23+0.15j
	0.092-0.046j	0.23-0.15j	0.08+0j
5:	1+0j	1.2-0.2j	0.05-0.05j
	1.2+0.2j	2.6+0j	0.05+0.1j
	0.05+0.05j	0.05-0.1j	0.11+0j
6:	1+0j	0.3-0.6j	0.1+0.2j
	0.3+0.6j	1.5+0j	0.09+0.02j
	0.1-0.2j	0.09-0.02j	0.14+0j

Table 6.8.1: Values of the covariance matrix elements for the 6 classes.

An example of simulated imagery for the HH, HV and VV intensities is shown in Figure 6.20. The HH image in Figure 6.20(a) shows no structure, as expected. There is some evidence of structure, however, in the HV and VV images, in Figures 6.20(b) and (c) respectively. Since there is visible structure, we might expect that these images could be segmented jointly in order to determine the structure within the scene. We shall show later that the sensitivity of this measure is poor when compared with full polarised segmentation. The sum of these three images represent the span measure, shown in Figure 6.20(d), which has often been adopted in the analysis of polarised images. It too will be shown to result in poor classification performance.

Figure 6.20: Amplitude of simulated HH (a), HV (b) VV (c) and span (d) images

Figure 6.20(a) Figure 6.20(b)

Figure 6.20(c) Figure 6.20(d)

With the exception of the three individual images or the span measure, shown above, other measures for polarised segmentation are all based on the covariance matrix of the image. This is constructed, as shown above, using data from the three complex simulated images, $E_{HH}$, $E_{HV}$, $E_{VV}$. The covariance matrix carries all the information about the polarised image. Therefore it is important to process the full complex data for each element. This is implemented in the Maximum Likelihood classifier and the polarised segmentation algorithm, which will be described later. This ML polarimetric classifier takes averages of the polarisation covariance matrix over regions, which ideally correspond to areas of uniform polarisation properties. Various algorithms are used to derive this region map. We either:

• treat the single pixel covariance matrix directly
• apply a 3x3 window to the covariance matrix.
• perform polarised segmentation on the data
• calculate the Upper Limit by averaging the covariance matrix over the original ground truth mask.

Two different forms of polarised segmentation will be compared:

• ML polarimetric segmentation
• segmentation of the span measure.

The first method provides the Maximum Likelihood segmentation solution. The latter is included to demonstrate how much poorer is segmentation which only uses incoherent information, namely the intensity in each polarisation.

The ML polarimetric segmentation algorithm requires the operating parameters to be tuned for ``best'' performance. As we found previously, segmentation is controlled by three basic parameters:

1. The shape parameter attaches a penalty to excessive curvature in the reconstruction. Large values of the parameter allow many changes of direction of the edge. Small values bias the reconstruction towards smoother edges. Since speckle yields very crinkled edges to regions of similar intensity it is important to adopt of large curvature penalty, i.e. a smaller shape parameter.
2. The annealing method has two further parameters attached. The first is the stopping parameter to assess when the reconstruction is complete. For noisier data one would expect this parameter to be increased. Smoother data, such as that encountered on averaging speckle, should allow a reduced value of the stopping parameter.
3. The second parameter in the annealing method is the cooling parameter which controls the rate at which the annealing converges on the final solution. If this is made too large the solution may freeze out away from the expected result.

It is important to derive values for these parameters that yield the``best'' segmentation. Since we have started with simulated test data we can tune the parameters in terms of the final classification performance. This avoids somewhat arbitrary decisions about what constitutes a``good'' segmentation.

Segmentation performance

A crucial issue with applying segmentation is the selection of operating parameters. In Figure 6.21 we show the result of applying ML polarimetric segmentation with different values of the shape parameter. In Figure 6.21(a) a shape parameter of 0.1 is adopted. It is apparent that this allows small regions with a sharply-varying boundary to be reconstructed However, there is some evidence for flecks of speckle-dominated segments. By increasing the penalty for non-smooth boundaries (reducing the shape parameter) this type of reconstruction can be suppressed. Figures 6.21(b) to (d) show the corresponding results for shape parameters of 0.01, 0.001 and 0.0001. If you compare these with the original test pattern in Figure 6.19(a) it is clear that reconstruction is generally very good. The shape of the fields around the water course are well represented. Figure 6.21(b), with a shape parameter of 0.01 appears to offer a good compromise between smooth edges to regions and reproducing small structures. We need to increase the shape parameter to follow weak small structure but, on the other hand, we need to reduce the shape parameter to eliminate speckle regions. As before, the only reasonable test is to compare the classification probability, as will be discussed below.

Figure 6.21: Segmentation of a single polarised image using different shape parameters. (a) 0.1, (b) 0.01, (c) 0.001, (d) 0.0001.

Figure 6.21(a) Figure 6.21(b)

Figure 6.21(c) Figure 6.21(d)

It is instructive to compare ML segmentation results with those derived using the span measure, shown in Figure 6.22. Figure 6.22(a) shows the ML segmentation using the ``normal'' parameters, discussed later. When the segment boundaries are overlaid onto the original classes, in Figure 6.22(b), two things are immediately obvious. Firstly, the segment boundaries are closely aligned with the region boundaries. This means that there is little blurring of the properties of the different classes. Secondly, quite large segments are reconstructed. Often a complete region of the original scene is represented by a single segment. Even where large homogeneous regions are broken up in segmentation, the individual segments are sufficiently large that the statistical properties will be well determined. On the other hand, there is no appreciable evidence of the field structure in the segmented span image, in Figure 6.22(c), revealing that there is very little information carried by the span measure. Indeed, the overlay of the segmentation boundaries onto the original classes, illustrated in Figure 6.22(d), confirm the inadequacy of the span measure in this context.

Figure 6.22: (a) ML segmentation (b) Overlay of the segmentation boundaries on the original classes (c) Segmentation of the span measure; (d) overlay of the span segment boundaries on the original classes.

Figure 6.22(a) Figure 6.22(b)

Figure 6.22(c) Figure 6.22(d)

The Lower Limit corresponds to attempting ML polarimetric classification on individual pixels. There is no reduction in the statistical uncertainty so that speckle noise would be expected to dominate. Because of this, much research has depended on taking 3x3 averages of the covariance matrix before classification. This reduces statistical uncertainty at the expense of blurring region boundaries. Thus the impact of such a process is scene dependent.

The purpose of the comparison is to reveal:

1. the importance of using the full complex polarimetric data.
2. the considerable advantage of segmentation methods over the 3x3 window and the Lower Limit.

Comparison of classification methods

Optimum classification of polarised images is based on the covariance matrix, which contains all the information in the scattering process. The ML classifier can be defined in terms of the likelihood

$$\lambda = -ln\vert{\bf R}_{c}\vert - Trace \lfloor{\bf R}_{c}^{-1} {\bf\hat{R}}\rfloor$$

where ${\bf R}_{c}$ is the reference covariance matrix for class C and $\vert{\bf R}{c}\vert$ denotes its determinant and $\hat{{\bf R}}_{c}$ denotes the estimated covariance matrix from the data. The maximum value of the likelihood, when all possible classes are compared, indicates which class the particular regions should be assigned to.

We now compare classification of the covariance matrix using the following methods:

1. Upper Limit - classifying the average in each ground truth region
2. Segmentation - classifying the average in each segment
3. 3x3 window - classifying the average over a 3x3 window
4. Lower Limit - classifying thqe covariance matrix in each pixel.

The classification results are shown in Figure 6.23. Classification of a single pixel, shown in Figure 6.23(a) is comparatively poor, with the average probability of correctly classifying each category 0.615. The associated confusion matrix, shown in Table 6.20(a) reveals how poorly the classes 0 (magenta), 1 (blue) and 4 (yellow) are assigned, as can be seen in Figure 6.23(a). Nevertheless, the underlying structure of the original classes in Figure 6.19(a) is still visible. Classification over a 3x3 window, shown in Figure 6.23(b) yields considerable improvement with the average fraction of correctly classified pixels in each class rising to 0.798. However, the confusion matrix shows that there is still ambiguity between class 2 (cyan) and 0 (magenta) and between class 1 (blue) and class 4 (yellow).

The results of classification following ML polarised segmentation depend on the selection of control parameters. Three basic default sets are defined, sensitive, normal and fast. The ``sensitive'' parameters should be employed where there is little scene structure, or when it is on a large scale so that large segments are to be preferred. The ``normal'' parameters are for general use, representing a compromise between sensitivity and speed. Finally, the ``fast'' parameters are intended for speedy, but not necessarily accurate determination of segments. Note that when we are concerned with classification after a segmentation stage it is not essential that the segmentation should completely fill the homogeneous regions. Provided that segments are large enough for the statistical uncertainty to be small and that the segments do not go over region boundaries the results will be indistinguishable. Indeed, there is no appreciable difference between the classification performance for the three modes, in this example. For the present comparison, we adopt the ``normal'' mode. The classified results, shown in Figure 6.23(c) is much closer to the ground truth, with the average fraction of correctly classified categories rising to 0.941 and the confusion matrix, in Table 6.20(c) showing very little ambiguity between classes. There is an obvious over-smoothing of some of the boundaries so that details of the structure have been lost, at the expense of rejecting unwanted flecks in the reconstruction. For comparison we show the corresponding result for ML classification following segmentation of the span measure, in Figure 6.23(d). As already noted, there is little correspondence between the segment boundaries and the original scene structure. This mixes the properties of the different classes and results in very poor classification. The average fraction of correctly classified categories is only 0.373, which is worse than all the others. Obviously the span measure is not worth pursuing.

Finally, the Upper Limit, provided by the ground truth itself, leads to the results shown in Figure 6.23(e). The average probability of correct classification is now 0.996, with only a few pixels in class 0 (magenta) incorrectly assigned. Indeed, the confusion matrix shown in Table 6.20(d) reveals very little ambiguity.

Figure 6.23: Classification based on (a) single pixel, (b) 3x3 window, (c) ML polarimetric segmentation, (d) span segmentation, (e) Upper Limit.

Figure 6.23(a) Figure 6.23(b)

Figure 6.23(c) Figure 6.23(d)

Figure 6.23(e)

Table 6.8.3: Confusion matrix for the different classification methods. The fraction of the true class assigned to each category is given

(a) Single Pixel

	Assigned class
True
Class	0	1	2	3	4	5
0	0.4087	0.0946	0.0662	0.1108	0.1345	0.1851
1	0.1180	0.3069	0.0517	0.1300	0.2070	0.1865
2	0.0119	0.0039	0.9106	0.0628	0.0053	0.0055
3	0.0274	0.0209	0.1234	0.7596	0.0431	0.0255
4	0.0705	0.0886	0.0579	0.1284	0.5533	0.1013
5	0.06663	0.0371	0.0397	0.0522	0.0527	0.7520

(b) 3x3 window

	Assigned class
True
Class	0	1	2	3	4	5
0	0.8858	0.0858	0.0034	0.0101	0.0149	0
1	0.0995	0.8687	0	0.0002	0.0283	0.0033
2	0.1902	0.0959	0.6801	0.0053	0.0284	0
3	0.1135	0.1095	0	0.7690	0.0080	0
4	0.0713	0.1846	0	0.0037	0.7367	0.0037
5	0.1169	0.0384	0	0	0	0.8447

(c) Segmentation

	Assigned class
True
Class	0	1	2	3	4	5
0	0.9108	0.0358	0.0149	0.0365	0.0020	0
1	0.0646	0.9263	0.0009	0.0015	0.0067	0.033
2	0.0445	0.0003	0.9552	0.0003	0	0
3	0.0105	0.0059	0.0026	0.9811	0	0
4	0.1049	0.0018	0	0.0054	0.8879	0
5	0.0094	0.0021	0.0052	0	0	0.9833

(d) Upper Limit

	Assigned class
True
Class	0	1	2	3	4	5
0	0.9757	0.0101	0.0014	0.0014	0.0055	0.0061
1	0	1.0000	0	0	0	0
2	0	0	1.0000	0	0	0
3	0	0	0	1.0000	0	0
4	0	0	0	0	1.0000	0
5	0	0	0	0	0	1.0008

Conclusion

• It is apparent from this comparison that it is pointless attempting classification on individual polarised pixels in SAR.
• Averaging over a 3x3 window improves the result. The accuracy could be further improved by increasing the size of the window. However, there is then a problem of averaging out small regions. What is required is a method that adapts the region size to the data; this is segmentation.
• Segmentation followed by classification yields a considerable improvement, since it provides a filter whose window size and shape is adapted to the data. The fact that the segments are not matched to the ground truth regions is a consequence of speckle noise. In this instance we know that the simulation was for uniform properties within each region. The speckle statistics have spoilt this. Larger regions could be grown, by changing the curvature parameter. However, this would tend to reject small regions.
• Only the full ML polarised segmentation method offers a significant improvement over the 3x3 window. It reduces the false classifications from about 20% to about 6%.
• The Upper Limit, as one would expect, provides the highest statistical accuracy. There are three important caveats to note, however.
  1. The result is based on the assumption that the properties within the region are indeed homogeneous. In real applications this will not generally be the case.
  2. The Upper Limit is incapable of showing if there are inhomogeneities within the defined reference region. The consequence of any such variation is a degradation in classification performance.
  3. The classification can only be applied where region boundaries are known. It cannot extrapolate beyond the ground truth. Segmentation, on the other hand, can use the training set to enable it to classify over as large an image as is required.

Thus, the overall conclusion is that segmentation provides an essential precursor to polarised classification.