Multisensor Classification of Wetland Environments Using
Airborne Multispectral and SAR Data

Michael R. Ricard, Amy L. Neuenschwander, Melba M. Crawford, and James C. Gibeaut
Center for Space Research, University of Texas at Austin, 3925 W. Braker Ln., Suite 200, Austin, TX 78759-5321
Bureau of Economic Geology, University of Texas at Austin, Campus Mail Code: E0630, Austin, TX 78712

This work was supported in part by the NASA Topography and Surface Change Program (Grant NAG5-2954) and by the NASA National Space Grant Consortium (Grant NGT40003).

Abstract - Near concurrent airborne data were acquired over the wetlands of the Bolivar Peninsula on the Texas coast by the NASA/JPL AIRSAR (June 28, 1996) and NASA/Stennis Space Center Calibrated Airborne Multispectral Scanner (CAMS) (July 3, 1996), both at 4m spatial resolution. Several approaches which utilize information from both sensors are investigated for classifying the landcover in these data sets. Differences in statistical characteristics of the data necessitate individual parametric models for observations from each sensor, so data are initially classified separately, then a final classification is obtained by combining results from the statistical models using different multisensor integration techniques. These integrated results are compared to single-sensor classification results, as well as to a multisensor classification based on artificial neural networks.

INTRODUCTION

The primary objective of classification of remotely sensed data is often to map landcover. Because different information is provided by various sensors, it can be advantageous to jointly utilize the information of the multisensor data in the classification process. In order to optimally exploit this potentially expanded information set in the classification framework, issues of sensor characteristics, differences in time of acquisition, and target/sensor dependent information content must be addressed.

Over the past several years, a significant amount of research has focused on multisource and/or multisensor classification for remote sensing applications. In [1] and [2], the authors classify multisource data consisting of digital imagery (Landsat MSS) and ancillary information (elevation, slope, and aspect data). Since these data cannot be represented by a single multivariate statistical model, the authors utilize consensus theoretic methods to combine the results of single-source statistical classifiers. In [3], the authors classify multisensor data (optical and SAR) using structured classifiers based on artificial neural networks, thus avoiding the need for modeling the statistical distribution of the data and treating each source or sensor separately.

Based on these issues, there were three objectives of this study. The first was to classify the landcover present in a wetland environment using remotely sensed data from several sensors. Part of this process involved assessing the accuracy of single-sensor classification, as well as determining the advantages and potential problems associated with the use of data from each sensor. By performing multisensor integration of single-sensor classifier outputs, it could be determined whether an improved classification was achieved, as well as whether sensor integration enabled the detection of "hard" classes, i.e. those classes which had lower probabilities of correct classification for a given sensor. The final objective was to determine, based on the data and single-sensor classifier architecture adopted, how to best integrate the multisensor data for classification of the study area.

The following sections contain descriptions of the test site, the multisensor data acquired for the project, and the methodology used to combine the information from these data sets for the purpose of multisensor classification, as well as preliminary results from analysis of the imagery.

STUDY SITE

Bolivar Peninsula is part of the low relief barrier islands of the Texas coast located at the mouth of GalvestonBay. The test site chosen for this study consists of a salt marsh located at a washover fan on southern Bolivar Peninsula.

For classification purposes, this salt marsh study area is characterized in terms of sub-environments defined by wetland maps [4]. The various landcover types present in these environments include low proximal marsh, high proximal marsh, high distal marsh, and spoil/barren flats, as well as areas consisting of water and trees. The low proximal marsh corresponds to tidal flats comprised of spartina alterniflora which experience frequent flooding. High proximal marsh is defined as more continuos areas of spartina alterniflora and salicornia virginica and are less frequently flooded. High distal marsh is comprised of spartina patens, salicornia virginica, juncus roemerianus and lies adjacent to barren sand flats. This area is flooded less frequently than proximal marshes.

MULTISENSOR DATA DESCRIPTION

Two near concurrent airborne data sets were acquired over the study site for the purpose of mapping wetland vegetation. Both 20 MHz and 40 MHz AIRSAR data were acquired by NASA/JPL on June 28, 1996 with a ground resolution of approximately 8m and 4m respectively. Additionally, Calibrated Airborne Multispectral Scanner (CAMS) was flown by NASA/Stennis Space Center on July 3, 1996 with approximately 4m spatial resolution. The CAMS data and the 40 MHz AIRSAR data were selected for multisensor classification due to their common coverage and comparable spatial resolution. The multisensor classification system analyzed data from three sensors: optical, thermal, and radar. The "optical sensor" consisted of the six Landsat bands of the CAMS instrument (Blue-NIR) plus a vegetation index, the thermal sensor recorded the ninth band of the CAMS data, and the NASA AIRSAR system acquired two frequency bands (C,L) of fully polarimetric radar data (six channels total).

CLASSIFICATION METHODOLOGY

An ensemble based approach was adopted for classification of the test site data. Data from each sensor were classified separately, then single-sensor classifier outputs were combined at the sensor integration stage.

Pre-Processing
During the pre-processing phase, radiometric and geometric corrections were applied to the data sets. The CAMS Optical data were corrected for bi-directional reflectance The CAMS Thermal data were empirically corrected for radiometric curvature present as a function of scan angle. The AIRSAR data was passed through a 5x5 enhanced Lee filter to reduce the effects of speckle in the imagery. Geometrically, the AIRSAR data was slant-to-ground range corrected. To enable multisensor classification, the three sensor data sets were co-registered. Finally, each sensor band was normalized to zero mean and standard deviation one for input to the classifiers.

Single-Sensor Classifier
For each sensor, the modular classifier architecture employs an expert classifier trained for each output class. The modularized class-specific expert classifiers are chosen to increase the rate of correct classification since the sensor classifier is not trained to solve the whole problem, just to identify a particular class from all the remainder [5].

A separate radial basis function (RBF) network, based on a mixture of Gaussians distribution for each sensor's class, is used to obtain an estimate of the posterior probability for each class
(1) where fi(x) are the local basis functions, wkj are the weights of the network, and M is the number of basis functions [6]. These class distributions are modeled as local kernel functions, in this case as mixtures of Gaussians. Based on this framework, each class-specific RBF network was trained to provide estimates of the posteriors using Moody-Darken three-phase learning.

Sensor Integration
Sensor integration techniques are investigated as ensemble approaches to combining classifiers with the goal to incorporate information from each sensor and thereby increase the performance over that achieved by single-sensor classifiers [5]. Since the classifiers utilized for this study provide estimates of the posterior probabilities for each class, information can be combined via either the sum rule or the product rule [1,7]. The sum rule, or weighted average, is based on a weighted sum of the posterior probabilities of a class for each sensor, whereas the product rule is based on a weighted product of the posterior probabilities of a class for each sensor. The weights can either be chosen to be equal for each sensor, in which case just a simple average of the posteriors is performed, or they can be chosen to represent, for instance, the reliability of a given sensor [1]. A further extension would be to weight the posteriors by the sensor's reliability for a given class, not just its overall reliability.

The final technique employed for sensor integration utilizes an artificial neural network, here an RBF network, trained on the outputs of the single-sensor classifiers.

Multisensor Classifier
For comparison to ensemble based sensor integration techniques, a multisensor classifier was tested to determine if information was lost through the single-sensor classification process. Since the data from each sensor were modeled using a mixture of Gaussians model, a classifier using an expert RBF network for each class was again used to classify the multisensor data jointly from a single input vector.

RESULTS

Single-sensor classifiers based on RBF networks and multisensor integrated classifiers based on ensemble approaches to combining classifiers were used for the classification of the CAMS Optical, CAMS Thermal, and AIRSAR data sets. These results are shown in Table 1.

Single-Sensor Classification
Each single-sensor classifier was trained, validated, and tested on separate data sets consisting of 267 ground truth points collected from each of the six classes: water (1), low proximal marsh (2), high proximal marsh (3), high distal marsh (4), spoil/barren flats (5), and trees (6).

The CAMS Optical data and AIRSAR data were both trained using expert RBFs with a total of 50 basis functions for each, while the CAMS Thermal data were trained using expert RBFs with a total of 40 basis functions for each.

CAMS Optical CAMS Thermal AIRSAR

Overall, CAMS Optical performed the best of three sensors, with the only difficulty coming in misclassifying 8% of the low proximal marsh as water. Given the amount of water in the low marshes, this is not surprising.

AIRSAR classified water and trees reasonably well, but had trouble separating both the high proximal marsh from the high distal marsh, as well as, separating the spoil/barren flats from the three marsh types. The similar moisture content and vegetation geometry in the high proximal marsh and high distal marshes are likely the cause of this result.

The CAMS Thermal sensor had trouble separating water and high proximal marsh, separating low proximal marsh and high distal marsh, and performed poorly for trees.

Multisensor Integration Results
A simple average and simple product of the single-sensor classifier results were computed with equal weights for each sensor, 95.2% and 94.9% overall classification rate respectively. Both performed better than the best single-sensor classifier, CAMS Optical, indicating the potential increase in performance through combining classifiers for different sensors, even with naive rules.

A weighted average and weighted product were then computed, with the weights for each sensor based on reliability factors obtained from the validation set's overall classification accuracy for each sensor. These sensor weighted results showed improvement over their equally weighted counterparts, thereby giving credence to influencing the sensor integration process based upon the reliability of a given sensor. Weights based on the reliability of a sensor for a given class were also chosen from the sensorv alidation set's probability of correct classification for that class. There was no significant improvement in results.

Simple Average Sensor Weighted Average

Another sensor integration technique involved using a single RBF network with 100 basis functions trained on the outputs of the single-sensor classifiers. These results were comparable overall to both sensor weighted results.

The final multisensor classification results were obtained from combining the single-sensor data prior to classification and then using them as inputs to the class-specific expert RBF classifiers. The results from this method were the best overall at 96.0%. This is because no information was lost from each of the sensors by classifying them separately.

Multisensor Joint Classifier

By utilizing the multisensor data, noticeable improvements were made in the classification accuracy for high proximal marsh, high distal marsh, and trees. This is due to the added information AIRSAR and CAMS Thermal data provide about these classes when used in conjunction with CAMS Optical.

CONCLUSIONS

Remotely sensed data from multiple sensors were classified both on a single-sensor and multisensor basis. Of the single-sensor classifiers, the CAMS Optical performed the best for each individual class and overall. When multisensor integration was performed on single-sensor classifiers, increases in classification rates were obtained for all techniques when compared to the best single-sensor classifier, CAMS Optical. This highlights the fact that additional information can be gained by combining the results from the classification of individual sensors.

Comparing the multisensor integration techniques, sensor weighted sum and product rules performed better than their equally weighted versions, demonstrating the need for utilizing sensor reliability measures into the classification scheme. Of these sensor integrated results, in addition to the RBF network integrator, all produced comparable results.

The best overall classification rate was obtained from the joint classification of the three sensors using an RBF network based on a mixture of Gaussians distribution for each class. While the percent increase was not sizable, it shows that some information was lost in classifying each sensor separately; that by combining the three sensors into a single classifier input vector, the CAMS Thermal and AIRSAR were able to provide useful information to the classification of the CAMS Optical data set. However, in general, flexibility is lost in classification of a combined data set in terms of the potential use of statistical classification techniques in conjunction with fusion of results via neural networks.

REFERENCES

[1] J.A. Benediktsson and P.H. Swain, "Consensus Theoretic Classification Methods", IEEE Transactions on Systems Man and Cybernetics, vol. 22, no. 4, pp. 688-704, July/August 1992.

[2] J.A. Benediktsson, J.R. Sveinsson, and P.H. Swain, "Hybrid Consensus Theoretic Classification", In Proceedings of International Geoscience and Remote Sensing Symposium, vol. III, pp. 1848-1850, 1996.

[3] S.B. Serpico and F. Roli, "Classification of Multisensor Remote-Sensing Images by Structured Neural Networks", IEEE Transactions on Geoscience and Remote Sensing, vol. 33, no. 3, pp. 562-578, May 1995.

[4] W.A. White, T.R. Calnan, R.A. Morton, R.S. Kimble, T.G. Littleton, J.H. McGowen, H.S. Nance, and K.E. Schmedes, Submerged Lands of Texas, Galveston-Houston Area, Bureau of Economic Geology, University of Texas at Austin, 1985.

[5] A.J.C. Sharkey, "On Combining Artificial Neural Nets", Connection Science, 8(3/4), 1996.

[6] C.M. Bishop, Neural Networks for Pattern Recognition, Oxford: Clarendon Press, 1995.

[7] J. Kittler, M. Hatef, and R.P.W. Duin, "Combining Classifiers", In Proceedings of International Conference on Pattern Recognition, vol. II, pp. 897-901, 1996.