Test Automation with Grad-CAM Heatmaps - A Future Pipe Segment in MLOps for Vision AI?

  title={Test Automation with Grad-CAM Heatmaps - A Future Pipe Segment in MLOps for Vision AI?},
  author={Markus Borg and Ronald Jabangwe and Simon {\AA}berg and Arvid Ekblom and Ludwig Hedlund and August Lidfeldt},
  journal={2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)},
Machine Learning (ML) is a fundamental part of modern perception systems. In the last decade, the performance of computer vision using trained deep neural networks has outperformed previous approaches based on careful feature engineering. However, the opaqueness of large ML models is a substantial impediment for critical applications such as in the automotive context. As a remedy, Gradient-weighted Class Activation Mapping (Grad-CAM) has been proposed to provide visual explanations of model… 

Figures from this paper

LAOps: Learning Analytics with Privacy-aware MLOps
: The intake of computer science faculty has rapidly increased with simultaneous reductions to course personnel. Presently, the economy is recovering slightly, and students are entering the working
Exploring ML testing in practice – Lessons learned from an interactive rapid review with Axis Communications
A taxonomy for the communication around ML testing challenges and results was developed and a list of 12 review questions relevant for Axis Communications was identified and extracted relevant approaches from the five studies on a conceptual level to support later context-specific improvements.
Agility in Software 2.0 - Notebook Interfaces and MLOps with Buttresses and Rebars
This keynote address presents a solution that can remedy some of the intrinsic weaknesses of working in notebooks by supporting easy transitions to integrated development environments and proposes reinforced engineering of AI systems by introducing metaphorical buttresses and rebars in the MLOps context.


Towards an Operational Design Domain That Supports the Safety Argumentation of an Automated Driving System
The operational design domain (ODD) of the automated driving system (ADS) can be used to restrict where the ADS is valid and thus confine the scope of the safety case as well as the verification.
Assessment List for Trustworthy Artificial Intelligence
  • European Commission, Brussels, Belgium, Tech. Rep., 2020.
  • 2020
Enabling Image Recognition on Constrained Devices Using Neural Network Pruning and a CycleGAN
Two approaches to reduce the need for compute in contemporary image recognition in an underpass with successful neural network pruning and how a CycleGAN can be used to transform out-of-distribution images to the operational design domain are explored.
  • 2017. [Online]. Available: https://github.com/fchollet/deep-learning-with-pythonnotebooks/blob/master/5.4-visualizing-what-convnets-learn.ipynb
  • 2017
Evaluating Deep Learning Classification Reliability in Android Malware Family Detection
A malicious family detector based on deep learning is proposed, providing a mechanism aimed to assess the prediction reliability, and it is shown how the proposed method can assist the security analyst to interpret the output classification and verify the predictions reliability by exploiting activation maps.
Towards Grad-CAM Based Explainability in a Legal Text Processing Pipeline
This paper provides the first approaches to using a popular image processing technique, Grad-CAM, to showcase the explainability concept for legal texts and shows the interplay between the choice of embeddings, its consideration of contextual information, and their effect on downstream processing.
Comparison of Faster-RCNN, YOLO, and SSD for Real-Time Vehicle Type Recognition
The Yolov4 model outperforms other methods, showing 93% accuracy in recognizing the vehicle model and the Faster-RCNN, YOLO, and SSD are presented.
The AIQ Meta-Testbed: Pragmatically Bridging Academic AI Testing and Industrial Q Needs
A working definition of artificial intelligence (AI) and a pragmatic approach to address the corresponding quality assurance with a focus on testing are shared.
Adaptive Adversarial Videos on Roadside Billboards: Dynamically Modifying Trajectories of Autonomous Vehicles
The effectiveness of an adversarial dynamic attack on an end-to-end trained DNN controlling an autonomous vehicle is shown and the approach enables dynamic adversarial perturbation that adapts to the relative pose of the vehicle and uses the dynamics of the Vehicle to steer it along adversary-chosen trajectories while being robust to variations in view, lighting, and weather.
The EU Approach to Ethics Guidelines for Trustworthy Artificial Intelligence
As part of its European strategy for Artificial Intelligence (AI), and as a response to the increasing ethical questions raised by this technology, the European Commission established an independent