Exploring Fault-Energy Trade-offs in Approximate DNN Hardware Accelerators

  title={Exploring Fault-Energy Trade-offs in Approximate DNN Hardware Accelerators},
  author={Ayesha Siddique and Kanad Basu and Khaza Anuarul Hoque},
  journal={2021 22nd International Symposium on Quality Electronic Design (ISQED)},
Systolic array-based deep neural network (DNN) accelerators have recently gained prominence for their low computational cost. However, their high energy consumption poses a bottleneck to their deployment in energy-constrained devices. To address this problem, approximate computing can be employed at the cost of some tolerable accuracy loss. However, such small accuracy variations may increase the sensitivity of DNNs towards undesired subtle disturbances, such as permanent faults. The impact of… 

Exposing Reliability Degradation and Mitigation in Approximate DNNs under Permanent Faults

An extensive fault resilience analysis of approximate multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs) using the state-of-the-art Evoapprox8b multipliers in GPU and TPU accelerators and a novel fault mitigation method, i.e., fault-aware retuning of weights (Fal-reTune), are presented.

Improving Reliability of Spiking Neural Networks through Fault Aware Threshold Voltage Optimization

The proposed FalVolt mitigation method improves the performance of systolicSNNs by enabling them to operate at fault rates of up to 60%, with a negligible drop in classi-cation accuracy, and is 2x faster compared to other state-of-the-art techniques common in artificial neural networks (ANNs), such as fault-aware pruning and retraining without threshold voltage optimization.

Is Approximation Universally Defensive Against Adversarial Attacks in Deep Neural Networks?

  • A. SiddiqueK. A. Hoque
  • Computer Science
    2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)
  • 2022
It is demonstrated that adversarial attacks on AxDNNs can cause 53% accuracy loss whereas the same attack may lead to almost no accuracy loss (as low as 0.06%) in the accurate DNN, thus, approximate computing cannot be referred to as a universal defense strategy against adversarial attack.

A design methodology for fault-tolerant computing using astrocyte neural networks

A many-core fault-tolerant neuromorphic hardware design, where neuron and synapse circuitries in each neuromorphic core are enclosed with astrocyte circuitries, the star-shaped glial cells of the brain that facilitate self-repair by restoring the spike firing frequency of a failed neuron using a closed-loop retrograde feedback signal is implemented.



Analyzing and mitigating the impact of permanent faults on a systolic array based neural network accelerator

This paper deals with the design of fault-tolerant, systolic array based DNN accelerators for high defect rate technologies and proposes two novel strategies, fault-aware pruning (FAP) and fault- aware pruning+retraining (F AP+T), that enable the TPU to operate at fault rates of up to 50%, with negligible drop in classification accuracy.

Understanding Error Propagation in Deep Learning Neural Network (DNN) Accelerators and Applications

  • Guanpeng LiS. Hari S. Keckler
  • Computer Science
    SC17: International Conference for High Performance Computing, Networking, Storage and Analysis
  • 2017
It is found that the error resilience of a DNN system depends on the data types, values, data reuses, and types of layers in the design, and two efficient protection techniques for DNN systems are proposed.

CANN: Curable Approximations for High-Performance Deep Neural Network Accelerators

This paper presents a novel method to design high-performance DNN accelerators where approximation error from one stage/ part of the design is "completely" compensated in the subsequent stage/part while offering significant efficiency gains.

Assessing the Reliability of Successive Approximate Computing Algorithms under Fault Injection

This work presents two fault injection and dependability test methodologies exploring the fault tolerance of successive approximation algorithms, showing that successive approximation is effective in protecting the output from faults injected at the data cache memory, but not from the ones injections at the register file.

RAPIDNN: In-Memory Deep Neural Network Acceleration Framework

A novel framework, called RAPIDNN, is proposed, which processes all DNN operations within the memory to minimize the cost of data movement and achieves 68.4x, 49.5x energy efficiency improvement and 48.9x speedup as compared to ISAAC and PipeLayer, the state-of-the-art DNN accelerators, while ensuring less than 0.3% of quality loss.

Toolflows for Mapping Convolutional Neural Networks on FPGAs

A survey of the existing CNN-to-FPGA toolflows is presented, comprising a comparative study of their key characteristics, which include the supported applications, architectural choices, design space exploration methods, and achieved performance.

Approximate Random Dropout for DNN training acceleration in GPGPU

This paper proposes the Approximate Random Dropout that replaces the conventional random dropout of neurons and synapses with a regular and online generated patterns to eliminate the unnecessary computation and data access and proves the approach is statistically equivalent to the previous dropout method.

Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks

The impact of an exemplary hardware fault attack, Rowhammer, is studied to show that a Rowhammer enabled attacker co-located in the same physical machine can inflict significant accuracy drops even with single bit-flip corruptions and no knowledge of the model.

EvoApproxSb: Library of approximate adders and multipliers for circuit design and benchmarking of approximation methods

The EvoApprox8b library provides Verilog, Matlab and C models of all approximate circuits and the error is given for seven different error metrics.

Embracing approximate computing for energy-efficient motion estimation in high efficiency video coding

A novel approximate architecture for energy-efficient motion estimation (ME) in high efficiency video coding (HEVC) is presented and designs for both ASIC and FPGA design flows are synthesized.