Temporal Signatures of Intensity Images

Introduction

In the previous case study we demonstrated the capability of segmentation to improve the classification of multiple simulated 3-look SAR images. Here we extend this study to the matching of regions to a known temporal signature - a practical example would be the delineation of a set of different crops whose growth pattern is known. This growth pattern (signature) will often in practice be derived from a previous data or (as in this model study) from a defined training mask so that the process is one of supervised classification.

Joint segmentation is well matched to this process since it has constructs a set of segments that are common to each image. It is therefore possible to study the temporal development of identical regions of these images.

In this study we simulate a sequence of ten images with the same six classes as in the previous study, shown in Figure 6.5, but with cross-section variation from sample to sample as shown in Figure 6.6.

Figure 6.5: Test pattern of 6 defined classes used in this study for simulating image intensity.

Figure 6.6: Plot of the mean intensity in each of the classes from Figure 6.5 during the sequence of images

From Figure 6.6 we see that, with the exception of the maroon class, all the classes show a variation in RCS over the sequence. Typical variations include a gradual increase, analogous to crop growth, for example, and a sudden drop, analogous to the reaping of crops. The purpose of this study is to classify regions in the sequence of images in terms of this signature. In fact, we do not use this signature directly, but train the temporal signature of each class using the training masks shown in Figure 6.4.2(a). The validation is then performed by testing over the regions denoted by the test masks in Figure 6.4.2(b).

Classification methods

As before we simulate a series of images having random 3-look speckle with the underlying RCS shown in Figure 6.6. We then perform Maximum Likelihood temporal signature classification on these images, after different processing techniques have been applied, namely:

Upper Limit:

classification on the average intensity in each ground truth region over the sequence

Joint Segmentation:

classification on the average intensity in each segment over the sequence

3x3 window:

classification on the average intensity over a 3x3 window over the sequence

Lower Limit:

classification on the intensity in each pixel over the sequence.

Figure 6.4.2(a) Ground truth training masks for the simulated data. The classes are represented by: 0 - magenta; 1- blue; 2 - cyan; 3 - green; 4 - yellow; 5 - red.

Figure 6.4.2(b) Ground truth test masks for the simulated data.

The first stage is to perform the pre-processing stage described above. Next we train the temporal signature by estimating the properties of each class over the regions defined by the training mask in Figure 6.4.2. these properties are next used to provide Maximum Likelihood classification over the different temporal signatures. The underlying model for the speckled data is that it is gamma-distributed with order 3, corresponding to 3-look SAR, and varying mean from image to image. The likelihood we wish to calculate therefore corresponds to data having order 3 and pre-trained mean over the sequence. The likelihood of each segment belonging to each of the temporal signature categories can be calculated. The ML solution for each segment then denotes the assigned class for that segment.

When this analysis is applied to the sequence of images we obtain the results shown in Figure 6.4.2. Corresponding average probabilities of correct classification are summarised in Table 6.4.2. With single pixels (a) the classification is clearly similar to the expected result in Figure 6.5. However, there are spots of misclassified data visible, caused by speckle noise. The average probability of correct classification is 0.899. After averaging over a 3x3 window, the result in (b) shows that these spots have been averaged out. However, the correlations introduced by the filter leave a ring of misclassified regions around each large area. Thus, although the probability of correct classification is reasonable within large areas of homogeneous RCS, the average for this test scene, 0.954, is not as good as one might expect. Indeed, an image with more structure would yield an even worse result. The result after joint segmentation (c) is considerably better with all classes being separated, including most of the narrow sections between large areas. The average probability of correct classification has risen considerably, to 0.992. Finally, the Upper Limit yields essentially perfect classification.

Figure 6.4.2(a): Classification using single pixels

Figure 6.4.2(b): Classification using 3 by 3 filter

Figure 6.4.2(c): Classification using Joint Segmentation

Figure 6.4.2(d): Classification using Upper Limit

Figure 4: Classified results from (a) single pixels, (b) 3x3 window, (c) joint segmentation, (d) Upper Limit.

Table 6.4.2: Average probability of correct classification for the different processing methods

Method	Pcor
Single pixel	0.899
3 by 3 Window	0.954
Joint Segmentation	0.992
Upper Limit	1.000

If we compare the probability of false classification (1-Pcor), we see that this progresses from 10% for single pixels to 4.6% for 3x3 window, 0.8% for joint segmentation and 0% for the Upper Limit. The considerable advantage offered by joint segmentation before classification is obvious.

As a final comparison we show the confusion matrices for the different methods in Table 2. Generally speaking the classification results are better than those for the 10 scenes considered when we introduced joint segmentation and classification. This is because the average RCS in each class is higher than before.

These detailed results bear out the observations made from the classified images. In particular, the averaging window seems to have the most serious degrading effect on the classification of class 0. This is because all the other classes tend to be averaged down toward this class.

Table 6.4.2 Confusion matrices for temporal signature classification using the different processing methods. (a) single pixel, (b) 3 x3 window, (c) joint segmentation (d) Upper Limit

Table 6.4.2(a): Single Pixel

	Assigned class
True Class	0	1	2	3	4	5
0	0.9993	0	0.0007	0	0	0
1	0.0009	0.9404	0.0215	0.0245	0.0072	0.0055
2	0.0017	0.0210	0.8290	0.0706	0.0761	0.0017
3	0	0.0170	0.0660	0.9059	0.0105	0.0007
4	0	0.0072	0.1193	0.0199	0.7811	0.0723
5	0	0.0104	0.0042	0.0026	0.0439	0.9389

Table 6.4.2(b): 3 by 3 Window

	Assigned class
True Class	0	1	2	3	4	5
0	0.8581	0.0351	0.0689	0.0372	0.0007	0
1	0.0002	0.9984	0.0012	0	0.0002	0
2	0.0006	0.0014	0.9802	0	0.0178	0
3	0	0.0060	0.0060	0.9880	0	0
4	0	0	0.0823	0	0.9177	0
5	0	0	0.0005	0	0.0179	0.9816

Table 6.4.2(c): Joint Segmentation

	Assigned class
True Class	0	1	2	3	4	5
0	0.9723	0.0041	0.0122	0.0115	0	0
1	0.0012	0.9960	0.0015	0.0007	0.0003	0.0002
2	0.0003	0.0019	0.9972	0.0003	0	0.0003
3	0	0.0013	0.0026	0.9961	0	0
4	0	0.0018	0.0018	0	0.9964	0
5	0.0005	0.0010	0.0010	0	0.0010	0.9963

Table 6.4.2(d): Upper Limit

	Assigned class
True Class	0	1	2	3	4	5
0	1.0000	0	0	0	0	0
1	0	0.9998	0.0002	0	0	0
2	0	0.0003	0.9997	0	0	0
3	0	0	0	1.0000	0	0
4	0	0	0	0	1.0000	0
5	0	0	0	0	0	1.0000

Conclusions

• Joint segmentation offers a valuable precursor to temporal signature classification because it adapts the region size to the data and presents a common set of boundaries for the whole sequence of images.

• The Upper Limit provides the most sensitive method for classification. However, it suffers from three disadvantages:
  • It cannot detect inhomogeneities within the defined regions.
  • The sensitivity is degraded by any inhomogeneities.
  • It only process within defined regions, with no extrapolation capability such as that possessed by segmentation.

• Joint segmentation is somewhat less sensitive than the Upper Limit but is considerably better than using a 3x3 window, which in turn is better than single pixels.
• The 3x3 window method suffers from edge effects due to averaging over the boundaries.
• Therefore temporal signature classification should adopt joint segmentation as an initial preprocess.