Morphological feature extraction for automatic registration of multispectral images


The task of image registration can be divided into matched relative to the reference data. The general process of two major components, i.e., the extraction of control points or features from images, and the search among the extracted features for the matching pairs that represent the same feature in the images to be matched. Manual extraction of control features can be subjective and extremely time consuming, and often results in few usable points. On the other hand, automated feature extraction allows using invariant target features such as edges, corners, and line intersections as relevant landmarks for registration purposes. In this paper, we present an extension of a recently developed morphological approach for automatic extraction of landmark chips and corresponding windows in a fully unsupervised manner for the registration of multispectral images. Once a set of chip-window pairs is obtained, a (hierarchical) robust feature matching procedure, based on a multiresolution overcomplete wavelet decomposition scheme, is used for registration purposes. The proposed method is validated on a pair of remotely sensed scenes acquired by the Advanced Land Imager (ALI) multispectral instrument and the Hyperion hyperspectral instrument aboard NASA's Earth Observing-1 satellite. Many current and future applications in Earth Science, Space Science, and Exploration Science (will) require integration of data acquired from multiple sources. This type of integration provides necessary information for intelligent navigation and decision support systems in many real-time applications. Depending on the application at hand, the integration may be performed on-board or on-the-ground. In the case of Earth Science, for example, multiple data sources are due to satellite, aircraft, and ground measurement systems, where the data represent multiple temporal, spectral, and spatial resolutions. For the multisource data integration described above, a fast, automatic, reliable, and accurate image registration is essential. Assuming that image data have been systematically corrected through prior navigation, image registration (or "precision correction") corresponds to feature-based matching of the image data, which refines georegistration to subpixel accuracy. Image registration is thus defined as the process that determines the best match between two or more images acquired at the same time or later, and wigth the same sensor of a different one. One dataset is regarded as the reference data and all other data, called input data (or sensed data), are image registration includes three main steps: 1) Extraction of features to be used in the matching process; 2) application of a feature matching strategy subject to a specific metric; 3) resampling of the data based on the correspondence computed from matched features. A large number of automatic image registration methods have been proposed and surveys can be found in [I], [2]. In particular, previous work has focused on the comparison of different choices for Steps 1, 2 above, where either an entire scene was registered (against a reference scene), or where several (small) chips, extracted typically around characteristic image features or landmarks from the reference scene, were registered initially against corresponding windows from the input scene [3]. The working assumption was that, once a registration system became operational, a database of such landmark chips would be available for use. Developing the capability of extracting these chips automatically and independently of any database presents the following relevant advantages: Elimination of a chip database that needs to be maintained regularly and adapted to each sensor; extracting automatically chip/windows that contain a very small amount of clouds, provided that preprocessing (of reference and input data) includes cloud detection; . processing images of any size; reducing the initial registration error by extracting simultaneously chips (from the reference image) and corresponding windows (from the input image), thereby enhancing feature matching (or any optimization technique). as far as accuracy and speed are concerned; given that the initial conditions are closer to the parameters of the final transformation, these techniques serve as "refinement methods." In this paper, we expand a recently proposed morpho1ogical method for automatic extraction of reference chips and corresponding input windows from remotely sensed image data 141. Here, this method is adapted to the specific case of automatic registration of multispectral datasets. The paper is organized as follows. Section I1 outlines a previously developed morphological method which performs automated feature extraction for registration purposes. Section III addresses the modifications introduced in the morphological extraction algorithm in order to adapt it to processing of multispectral data sets. Section IV provides experimental results to validate the proposed approach. Finally, Section V concludes the paper with some remarks and hints at plausible future research. 11. MORPHOLOGICAL FEATURE XTRACTION Mathematical morphology [5] is a nonlinear image processing technique that was originally established by introducing fundamental operators applied to two sets. One set is processed by another set with certain spatial properties (i.e., shape and size), known as structuring element, which is translated over the spatial domain of the image. The structuring element acts as a filter for extracting or suppressing specific image structures. Following standard notation [5], let us consider a grayscale image f, defined on the discrete space Z2, and a structuring element designed by B E Z2. The latter is usually 'flat' in the sense that it is defined in the spatial domain of the image (the x-y plane). The morphological erosion of f by B is defined as where (x, y) E Z2 and Z 2 ( B ) denotes the set of discrete spatial coordinates associated with pixels Iying within the neighborhood defined by B. In contrast, the morphological dilation of f by B is defined by (f @ B)(x, Y) = Max( , , t )~zz (~ ) f (a:s, Y t). (2) Using the same notation above, morphological opening and closing filters [5] can be respectively defined by: The two operators above have been used successfully to build morphological profiles by using combinations of structuring elements of progressively increased size 161. However, these profiles cannot capture the directional information which is crucial to extract features such as edges, corners, and line intersections. To address this issue, we resort to so-called scale-orientation morphological projles (SOMPs). In order to mathematically define the concept of SOMPs, we first denote by Bp,(dx,dyj a line segment structuring element with p pixels along the line (in this work, we set p = 5) and slope dyldx. Using these notations, we can define the SOMPs by opening at a given pixel (x, y) of an image f as where p = {1,2, . . , k) and k is the maximum number of considered scales. In similar terms, we can define the SOMPs by closing at f (x, y) as follows: In both cases, a measure of line strength can be computed for each scale and orientation by calculating a point-wise distance dist between the pixel f (x, y) in the original imagc and the pixel at the same location in the image filtered by the considered line segment structuring element as follows: The resulting values from Eqs. (7) and (8) are combined into a feature vector D(x, y) = sO(x, y) U f ( x , y) U s" (x, y) with dimensionality L = 2k x m, where k is the number of scales and m is the number of orientations. In order to automatically extract significant points of interest for registration, we use the concept of self-infomation 171. (The more irregular the SOMP associated with a certain pixel, the higher the chance that it represents a corner pixel of an object.) Let the L components of D(x, y) be denoted by {dl(x, y))Ll; then the self-infomation provided by the L th component (1 < 1 5 L), can be defined by IZ (s, y) = -log PZ (2, Y), where Using the above definitions, the entropy of D(x , y) can be obtained from The SOMP operators described in Section I1 have been used in previous work to automatically extract a set of landmark chips from a reference scene, and to find corresponding windows in the input scene in a fully unsupervised manner [4]. A measure of spectral information divergence (SID) 171 was first used to establish a preliminary matching of SONIP-based profiles in the input scene with those in the reference scene, followed by a (hierarchical) robust feature matching (RFM) procedure that makes use of a multiresolution overcomplete wavelet decomposition scheme to extract chip and window features at the various levels of the decomposition 181. Although the proposed approach was successfu"aor the registration of single spectral bands extracted from Landsat data sets [4], the following important aspects, which are crucial for the registration of multi/hyperspectral image datasets, were not addresed: Our previous approach did not incorporate any cloud detection procedure and the registration process was strongly affected by atmospheric interferers such as clouds and their shadows. In order to address this issue, we have resorted to spectral unmixing techniques 171 to automatically detect clouds (and their shadows) from the input scenes. Specifically, we considered the full spectral information in the data and extracted the brightest pixel vector using the automatic target generation process (ATGP) described in [7], [9]. The second iteration of ATGP provides the pixel which is most different, in an orthogonal sense, from the brightest pixel (essentially, the darkest pixel in the scene). These pixels (often associated with clouds and (cloud) shadows) are used to mask out cloudy/shaded areas by unmixing the scenes using linear unmixing [9] and removing pixels with very high abundance (90% or more) of either cloud or shadow prior to registration. . If the pair of images to be registered are rotated against each other, as is indeed the case with ALI-Hyperion pairs, comparing the SOMPs directly (by using, e.g., a measure of spectral similarity) does not make sense, as opposed to the registration of Landsat data, for which a trivial rotation close to 0 degrees is expected. In fact, the SOMPs can be seen as histograms that indicate the degree of variation at different scales and orientations (see [6] for a more detailed explanation). Thus, if we calculate such a histogram for a certain pixel, then rotate the image, and recalculate the histogram, the order of the histogram bins (which show a measure of change of the pixel with regards to surrounding features in the scene at different scales and orientations) would be different, even though they refer to the same image pixel. To address this issue, we simply order the bins in the histogram in descending order (i.e., the lowest measures of change are placed first and the higher measures of change are placed last in the histogram) prior to spectral similarity matching using the SID measure. This issue is further discussed in [6]. Taking in mind the issues above, the sequence of steps implemented by our revised SOMP-based algorithm is: 1) Remove clouds and shadows in both the reference and input multispectral scenes using spectral unmixing. 2) Obtain the SOMPs for each pixel in the input scene, denoted by D,(x, y), and order the bins in descending order for each calculated SOMP. 3) For each pixel f ( x , y) in the reference scene, construct its SOMP denoted by D ( x , y) and calculate its entropy H ( D f ( x , Y ) ) . 4) Select the pixel f (x ' , y') with the highest H ( D ( x , y ) ) score (i.e., with the maximum entropy) in the reference scene and extract a reference chip centered on f ( X I , y'), where the chip size is defined by an input parameter. 5) Order the bins of the histogram given by D ( X I , y') in descending order and compute the SID between Df (x', y') and all the SOMPs in the input scene obtained in Step 2, retaining the pixel coordinates (x", y") of the pixel in the input scene for which the SID value is the smallest. 6) Extract a window centered on pixel (x" , y") in the input scene, where the window size is defined by an input parameter. 7) Return to Step 4 until a predefined number of different chip-window pairs is extracted. TABLE I ALI-HYPERION SPECTRAL BAND CORRESPONDENCES. TABLE I1 REGISTRATION RESULTS OBTAINED FOR FIVE CHIP-WINDOW PAIRS FROM A PAIR OF ALI-HYPERION MULTI/HYPERSPECTRAL IMAGES. ALI Chip-window Size Rotation Initial shift Initial shift pair (pixels) (degrees) t , t 7, 1 128 x 128 11.28 -1.442 0.153 Hyperion Band number IV. EXPERIMENTAL RESULTS Band number Spectral range (,urn) We have tested the fully automated system described in section 111 using a pair of remotely sensed scenes collected by the Advanced Land Imager (ALI) miultispectral instrument and the Hyperion hyperspectral instrument aboard NASA's Earth Observing-1 satellite. Although the instruments are mounted on the same platform, their spectral resolutions are different. For illustrative purposes, Table B shows the correspondences established between the 9 sgectral bands provided by ALI and the corresponding Hyperion bands. A detailed description of band coverage and correspondences for ALI-Hyperion is available online (from http://eo 1 -Coverage.htn?). In this study, we have selected a single pair of spectral bands from the considered ALI-Hyperion scenes for validation. The selected pair, displayed in Fig. 1, consists of ALI's band 7 and Hyperion's band 106. Both images are of size 256 x 3352 pixels. Strong atmospheric interferers (i.e., clouds and shadows) can be observed in these ALI and Hyperion images depicted in the figure. The automatic algorithm described in Section ID was applied to the above scenes (with k = 5 scales and m = 8 oricntations) to obtain a set of five chip-window pairs displayed in Fig. 2. Due to the narrow width of the ALI-Hyperion scenes (only 256 pixels), we had to carefully adjust the desired size of the extracted chip-window pairs. For all chip-window pairs, the size was set to 128 x 128 pixels, except for pair 2 (in Fig. 2), whose size was set to 96 x 96 pixels. In this particular case, the proximity to the image border prevented us from extracting a 128 x 128-pixel window around the image feature provided by the morphological operators. As can be observed from Fig. 2, all selected chip-window pairs are free from clouds/shadows. After applying the hierarchical version of the RFM techAverage wavelength (nm) 1 1 0.433 0.453 9 1 436.99

DOI: 10.1109/IGARSS.2007.4422820

Extracted Key Phrases

4 Figures and Tables

Cite this paper

@inproceedings{Plaza2007MorphologicalFE, title={Morphological feature extraction for automatic registration of multispectral images}, author={Antonio J. Plaza and Jacqueline Le Moigne and Nathan S. Netanyahu}, booktitle={IGARSS}, year={2007} }