Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network

  title={Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network},
  author={Frosti Palsson and Johannes R. Sveinsson and Magnus Orn Ulfarsson},
  journal={IEEE Geoscience and Remote Sensing Letters},
In this letter, we propose a method using a 3-D convolutional neural network to fuse together multispectral and hyperspectral (HS) images to obtain a high resolution HS image. Dimensionality reduction of the HS image is performed prior to fusion in order to significantly reduce the computational time and make the method more robust to noise. Experiments are performed on a data set simulated using a real HS image. The results obtained show that the proposed approach is very promising when… 

Figures and Tables from this paper

An Unsupervised Hyperspectral Image Fusion Method Based on Spectral Unmixing and Deep Learning

Due to the limitations of various hardware conditions, in practice, only high resolution multispectral and low resolution hyperspectral images are usually captured. In order to apply hyperspectral

Hyperspectral and Multispectral Image Fusion by Deep Neural Network in a Self-Supervised Manner

A novel fusion method, which operates in a self-supervised manner, to the task of hyperspectral and multispectral image fusion without training datasets, and results indicate that the proposed method outperforms those traditional methods by a large extent.

A model-guided deep convolutional sparse coding network for hyperspectral and multispectral image fusion

ABSTRACT Although high-resolution hyperspectral images are of great significance in various applications, the manner in which to acquire such images remains a fundamental problem due to sensors’

Coupled Convolutional Neural Network-Based Detail Injection Method for Hyperspectral and Multispectral Image Fusion

Compared with some state-of-the-art HS and MS image fusion methods, the proposed method achieves better fusion results, provides excellent spectrum preservation ability, and is easy to implement.

Variational Fusion of Hyperspectral Data by Non-Local Filtering

A variational fusion model with a nonlocal regularization term that encodes patch-based filtering conditioned to the geometry of the multispectral data and a radiometric constraint that injects the high frequencies of the scene into the fused product with a band per band modulation according to the energy levels of the multi-spectral and hyperspectral images is proposed.

Deep Hyperspectral and Multispectral Image Fusion with Inter-image Variability

An original image fusion method is introduced that solves the optimization problem accounting for inter-image variability with an iteratively reweighted scheme and that leverages light-weight CNN-based networks to learn realistic image priors from data.

A deep image prior-based interpretable network for hyperspectral image fusion

A new image fusion network which use deep prior information as spatial guide information as regularization term is proposed and an aggregation module is added at the end of the network.

Hyperspectral and Multispectral Data Fusion with 1D-Convolution on Spectrum

  • Jinchi XieYing WangJ. Li
  • Environmental Science, Computer Science
    IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium
  • 2022
A ID-convolutional neural network is proposed, it can extract hyperspectral and multispectral features, simultaneously, and also can capture the spectral correlation and cross-correlation between hyperspectrals and mult ispectral spectrum.

Hyperspectral and Multispectral Image Fusion Using a Multi-Level Propagation Learning Network

This work presents a Multi-Level Propagation Learning Network (MLPLN) based on a LSTM model but that can be trained with variable data sizes in order to achieve the fusion process and provides an intrinsic data augmentation feature that reduces the required number of training samples.

Hyperspectral and Multispectral Image Fusion Based on Band Simulation

A novel HSI fusion method based on band simulation is proposed that expands MSI using spectral unmixing and acquires high-resolution images based on linear least squares and outperforms the competitors.



Model-Based Fusion of Multi- and Hyperspectral Images Using PCA and Wavelets

This paper proposes a novel method for fusion of MS/HS and PAN images and of MS and HS images in the lower dimensional PC subspace, with substantially lower computational requirements and very high tolerance to noise in the observed data.

Noise-Resistant Wavelet-Based Bayesian Fusion of Multispectral and Hyperspectral Images

The technique is compared to several state-of-the-art pansharpening techniques, in the case where the MS image becomes a panchromatic image, and to MS and HS image fusion techniques from the literature.

Fusion of Hyperspectral and Multispectral Images Using Spectral Unmixing and Sparse Coding

Compared with other state-of-the-art algorithms based on pansharpening, spectral unmixing, and SC methods, the proposed method is shown to significantly increase the spatial resolution while perserving the spectral content of the HSI.

Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods

A novel framework that generalizes well-established pan-sharpening algorithms to HSI and MSI fusion of the same scene collected by the coupled sensors is presented and results show that the proposed methods excel the state-of-the-art methods in terms of simplicity, feasibility, efficiency, and effectiveness.

Pansharpening by Convolutional Neural Networks

A new pansharpening method is proposed, based on convolutional neural networks, which is largely competitive with the current state of the art in terms of both full-reference and no-reference metrics, and also at a visual inspection.

Multi-spectral and hyperspectral image fusion using 3-D wavelet transform

Image fusion is performed between one band of multi-spectral image and two bands of hyperspectral image to produce fused image with the same spatial resolution as source multi-spectral image and the

A New Pansharpening Method Based on Spatial and Spectral Sparsity Priors

A new variational model based on spatial and spectral sparsity priors for the fusion of panchromatic and fused multispectral images is introduced and results show the effectiveness of the proposed pansharpening method compared with the state-of-the-art.

A New Pan-Sharpening Method With Deep Neural Networks

Comparative experimental results with several quality assessment indexes show that the proposed method outperforms other pan-sharpening methods in terms of visual perception and numerical measures.

Fusion of Multispectral Images by Extension of the Pan-Sharpening ARSIS Method

The results showed that the ARSIS method extended to the fusion of two multispectral images provides better statistical and visual results than the two other methods, on both synthetic and real datasets, and it is better adapted to water than to land applications.

A New Pansharpening Algorithm Based on Total Variation

The method uses total variation to regularize an ill-posed problem dictated by a widely used explicit image formation model and produces images of excellent spatial and spectral quality.