Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks

  title={Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networks},
  author={Richard J. Chen and Ming Y. Lu and Muhammad Shaban and Chengkuan Chen and Tiffany Y. Chen and Drew F. K. Williamson and Faisal Mahmood},
Cancer prognostication is a challenging task in computational pathology that requires context-aware representations of histology features to adequately infer patient survival. Despite the advancements made in weakly-supervised deep learning, many approaches are not context-aware and are unable to model important morphological feature interactions between cell identities and tissue types that are prognostic for patient survival. In this work, we present Patch-GCN, a contextaware, spatially… Expand
1 Citations

Figures and Tables from this paper

Pan-Cancer Integrative Histology-Genomic Analysis via Interpretable Multimodal Deep Learning
A pan-cancer integrative platform for biomarker discovery in both histology slides and molecular profile data is presented and performance increase in risk stratification on 9 out of 14 cancers is demonstrated, as well as shifts in feature importance when conditioned on more than one modality. Expand


Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks
This work proposes Deep Attention Multiple Instance Survival Learning (DeepAttnMISL) by introducing both siamese MI-FCN and attention-based MIL pooling to efficiently learn imaging features from the WSI and then aggregate WSI-level information to patient-level. Expand
WSISA: Making Survival Prediction from Whole Slide Histopathological Images
This paper proposes an effective Whole Slide Histopathological Images Survival Analysis framework (WSISA) that can efficiently exploit and utilize all discriminative patterns in WSIs to predict patients survival status and applies it to the survival predictions of glioma and non-small-cell lung cancer. Expand
Weakly Supervised Prostate Tma Classification Via Graph Convolutional Networks
This work proposes a weakly-supervised approach for grade classification in tissue micro-arrays (TMA) using graph convolutional networks (GCNs), in which the spatial organization of cells as a graph to better capture the proliferation and community structure of tumor cells. Expand
Human-interpretable image features derived from densely mapped cancer pathology slides predict diverse molecular phenotypes
This work presents an approach for predicting clinically-relevant molecular phenotypes from whole-slide histopathology images using human-interpretable image features (HIFs), and demonstrates that these HIFs correlate with well-known markers of the tumor microenvironment and can predict diverse molecular signatures with performance comparable to ‘black-box’ methods. Expand
Feature-Enhanced Graph Networks for Genetic Mutational Prediction Using Histopathological Images in Colon Cancer
This study proposes a feature-enhanced graph network (FENet) for genetic mutation prediction using histopathological images in colon cancer that incorporates feature enhancement in convolutional graph neural networks to aggregate discriminative features for capturing gene mutation status. Expand
Capturing Cellular Topology in Multi-Gigapixel Pathology Images
A novel graph convolutional neural network based model (called Slide Graph) is proposed which overcomes limitations by building a graph representation of the cellular architecture in an entire WSI in a bottom-up manner. Expand
Predicting Lymph Node Metastasis Using Histopathological Images Based on Multiple Instance Learning With Deep Graph Convolution
  • Yu Zhao, Fan Yang, +8 authors Jianhua Yao
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
This paper develops a self-supervised learning mechanism to train the feature extractor based on a combination model of variational autoencoder and generative adversarial network (VAE-GAN) and proposes a novel instance-level feature selection method to select the discriminative instance features. Expand
Graph CNN for Survival Analysis on Whole Slide Pathological Images
This work proposes to model WSI as graph and then develop a graph convolutional neural network (graph CNN) with attention learning that better serves the survival prediction by rendering the optimal graph representations of WSIs. Expand
Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images
This work uses a conditional generative adversarial network (cGAN) trained with synthetic and real data to train a conditional GAN with spectral normalization and gradient penalty for nuclei segmentation that outperforms conventional approaches, especially in isolating individual and overlapping nuclei. Expand
Graph Attention Multi-instance Learning for Accurate Colorectal Cancer Staging
A Graph Attention Multi-instance Learning (Graph Attention MIL) with texture features, which encodes a spatial structure between patches and jointly predicts the TNM staging, improves the performance over the existing state-of-the-art methods and indicates the future research towards graph based learning forTNM staging. Expand