Focal Attention Networks: Optimising Attention for Biomedical Image Segmentation

  title={Focal Attention Networks: Optimising Attention for Biomedical Image Segmentation},
  author={Michael Yeung and Leonardo Rundo and Evis Sala and Carola-Bibiane Sch{\"o}nlieb and Guang Yang},
  journal={2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI)},
In recent years, there has been a rising interest to incorporate attention into deep learning architectures for biomedical image segmentation. The modular design of attention mechanisms enable flexible integration into convolutional neural network architectures such as the U-Net. Whether attention is appropriate to use, what type of attention to use, and where in the network to incorporate attention modules, are all important considerations that are currently overlooked. In this paper, we… 

Figures and Tables from this paper


Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images
U-Net: Convolutional Networks for Biomedical Image Segmentation
It is shown that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.
Focus U-Net: A novel dual attention-gated CNN for polyp segmentation during colonoscopy
Loss Weightings for Improving Imbalanced Brain Structure Segmentation Using Fully Convolutional Networks
The results suggested that the distance map-based and the focal weightings could boost the performance of cross-entropy and Dice loss functions in class imbalanced segmentation tasks, respectively.
Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks
This paper introduces three variants of SE modules for image segmentation, and effectively incorporates these SE modules within three different state-of-the-art F-CNNs (DenseNet, SD-Net, U-Net) and observes consistent improvement of performance across all architectures, while minimally effecting model complexity.
MIScnn: A Framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning
The MIScnn framework enables researchers to rapidly set up a complete medical image segmentation pipeline by using just a few lines of code, and is a powerful predictor based on the standard 3D U-Net model.
Distance Map Loss Penalty Term for Semantic Segmentation
A novel distance map derived loss penalty term for semantic segmentation is proposed, using distance maps, derived from ground truth masks, to create a penalty term, guiding the network's focus towards hard-to-segment boundary regions.
CBAM: Convolutional Block Attention Module
The proposed Convolutional Block Attention Module (CBAM), a simple yet effective attention module for feed-forward convolutional neural networks, can be integrated into any CNN architectures seamlessly with negligible overheads and is end-to-end trainable along with base CNNs.