SATBench: Benchmarking the speed-accuracy tradeoff in object recognition by humans and dynamic neural networks

@article{Subramanian2022SATBenchBT,
  title={SATBench: Benchmarking the speed-accuracy tradeoff in object recognition by humans and dynamic neural networks},
  author={Ajay Subramanian and Sara A. Price and Omkar Kumbhar and Elena Sizikova and Najib J. Majaj and D. Pelli},
  journal={ArXiv},
  year={2022},
  volume={abs/2206.08427}
}
The core of everyday tasks like reading and driving is active object recognition. Attempts to model such tasks are currently stymied by the inability to incorporate time. People show a flexible tradeoff between speed and accuracy and this tradeoff is a crucial human skill. Deep neural networks have emerged as promising candidates for predicting peak human object recognition performance and neural activity. However, modeling the temporal dimension i.e., the speed-accuracy tradeoff (SAT), is… 

RTNet: A neural network that exhibits the signatures of human perceptual decision making

Overall, RTNet is the first neural network that exhibits all basic signatures of perceptual decision making, and therefore provides the most detailed model of all critical features of human behavior for novel images.

References

SHOWING 1-10 OF 55 REFERENCES

Anytime Prediction as a Model of Human Reaction Time

This work considers a classification network that uses early-exit classifiers to make anytime predictions and concludes that Anytime classification is a promising model for human reaction time in recognition tasks.

Generalisation in humans and deep neural networks

The robustness of humans and current convolutional deep neural networks on object recognition under twelve different types of image degradations is compared and it is shown that DNNs trained directly on distorted images consistently surpass human performance on the exact distortion types they were trained on.

Recurrent Convolutional Neural Networks: A Better Model of Biological Object Recognition

It is found that recurrent neural networks outperform feedforward control models at recognising objects, both in the absence of occlusion and in all occlusions conditions, and suggests that the ubiquitous recurrent connections in biological brains are essential for task performance.

Large-Scale, High-Resolution Comparison of the Core Visual Object Recognition Behavior of Humans, Monkeys, and State-of-the-Art Deep Artificial Neural Networks

The results show that current DCNNIC models cannot account for the image-level behavioral patterns of primates and that new ANN models are needed to more precisely capture the neural mechanisms underlying primate object vision.

Simple Learned Weighted Sums of Inferior Temporal Neuronal Firing Rates Accurately Predict Human Core Object Recognition Performance

A database of images for evaluating object recognition performance was designed and it was found that simple learned weighted sums of firing rates of neurons in monkey inferior temporal (IT) cortex accurately predicted human performance.

Partial success in closing the gap between human and machine vision

The longstanding distortion robustness gap between humans and CNNs is closing, with the best models now exceeding human feedforward performance on most of the investigated OOD datasets, and the behavioural difference between human and machine vision is narrowing.

Recurrent computations for visual pattern completion

The recurrent model was able to predict which images of heavily occluded objects were easier or harder for humans to recognize, could capture the effect of introducing a backward mask on recognition behavior, and was consistent with the physiological delays along the human ventral visual stream.

Understanding the computation of time using neural network models

It is found that neural networks perceive time through state evolution along stereotypical trajectories and produce time intervals by scaling evolution speed and four factors that facilitate strong temporal signals in nontiming tasks, including the anticipation of coming events are identified.

Performance-optimized hierarchical models predict neural responses in higher visual cortex

This work uses computational techniques to identify a high-performing neural network model that matches human performance on challenging object categorization tasks and shows that performance optimization—applied in a biologically appropriate model class—can be used to build quantitative predictive models of neural processing.

Multi-Scale Dense Networks for Resource Efficient Image Classification

Experiments demonstrate that the proposed framework substantially improves the existing state-of-the-art in both image classification with computational resource limits at test time and budgeted batch classification.
...