In Defence of Visual Analytics Systems: Replies to Critics

  title={In Defence of Visual Analytics Systems: Replies to Critics},
  author={Aoyu Wu and Dazhen Deng and Furui Cheng and Yingcai Wu and Shixia Liu and Huamin Qu},
  journal={IEEE Transactions on Visualization and Computer Graphics},
  • Aoyu WuDazhen Deng Huamin Qu
  • Published 24 January 2022
  • Computer Science
  • IEEE Transactions on Visualization and Computer Graphics
The last decade has witnessed many visual analytics (VA) systems that make successful applications to wide-ranging domains like urban analytics and explainable AI. However, their research rigor and contributions have been extensively challenged within the visualization community. We come in defence of VA systems by contributing two interview studies for gathering critics and responses to those criticisms. First, we interview 24 researchers to collect criticisms the review comments on their VA… 

Figures and Tables from this paper

MetaGlyph: Automatic Generation of Metaphoric Glyph-based Visualization

This paper proposes MetaGlyph, an automatic system for generating MGVs from a spreadsheet that automatically selects metaphors with corresponding images from online resources based on the input data semantics and integrates a Monte Carlo tree search algorithm that explores the design of an MGV.

KB4VA: A Knowledge Base of Visualization Designs for Visual Analytics

This work builds KB4VA, a knowledge base of visualization designs in VA systems with comprehensive labels about their analytical tasks and visual encodings, and extends Vega-Lite specifications for describing advanced and composited visualization Designs in a declarative manner, thus facilitating human understanding and automatic indexing.

HardVis: Visual Analytics to Handle Instance Hardness Using Undersampling and Oversampling Techniques

This paper introduces HardVis, a visual analytics system designed to handle instance hardness mainly in imbalanced classification scenarios, and assists users in visually comparing different distributions of data types, selecting types of instances based on local characteristics that will later be affected by the active sampling method, and validating which suggestions from undersampling or oversampling techniques are bene-cial for the ML model.

Revisiting the Design Patterns of Composite Visualizations

This paper constructed a corpus of composite visualizations from the publications and analyzed common practices, such as the pattern distributions and co-occurrence of visualization types, to obtain insights into different design patterns on the utilities and their potential pros and cons.



A Systematic Review on the Practice of Evaluating Visualization

An assessment of the state and historic development of evaluation practices as reported in papers published at the IEEE Visualization conference found that evaluations specific to assessing resulting images and algorithm performance are the most prevalent and generally the studies reporting requirements analyses and domain-specific work practices are too informally reported.

Design Study Methodology: Reflections from the Trenches and the Stacks

This paper reflects on the combined experience of conducting twenty-one design studies, conducts an extensive literature survey of related methodological approaches that involve a significant amount of qualitative field work, and compares design study methodology to that of ethnography, grounded theory, and action research.

MiningVis: Visual Analytics of the Bitcoin Mining Economy

A visual analytics tool, MiningVis, is presented to explore the long-term historical evolution and dynamics of the Bitcoin mining ecosystem, including mining market statistics, multi-measure mining pool rankings, and pool hopping behavior.

IRVINE: A Design Study on Analyzing Correlation Patterns of Electrical Engines

The results suggest a high usability and usefulness of IRVINE as part of the improvement of a real-world manufacturing process, and domain experts were able to label and annotate produced electrical engines more than 30% faster.

On Validity

Employing criterion validation procedures in cases where constructs are being assessed has resulted in contamination of many validation efforts by mono-operation bias. One result of this mistake is

An Ontological Framework for Supporting the Design and Evaluation of Visual Analytics Systems

This paper presents an ontological framework for recording and categorizing technical shortcomings to be addressed in a VA workflow, reasoning about the causes of such problems, identifying technical solutions, and anticipating secondary effects of the solutions.

The Validity, Generalizability and Feasibility of Summative Evaluation Methods in Visual Analytics

This paper provides a survey and taxonomy of the evaluation methods that have appeared in the VAST literature in the past two years, and proposes a new metric called summative quality to compare evaluation methods according to their ability to prove usefulness.

Criteria for Rigor in Visualization Design Study

We develop a new perspective on research conducted through visualization design study that emphasizes design as a method of inquiry and the broad range of knowledge-contributions achieved through it

Reflection on Reflection in Applied Visualization Research

Reflection is a core method used by visualization researchers to generate knowledge from design practice. There is, however, a lack of standards to inform reflective practice and through which we can

Visual Analytics: Definition, Process, and Challenges

The possibilities to collect and store data increase at a faster rate than the ability to use it for making decisions, and in most applications, raw data has no value in itself; instead the authors want to extract the information contained in it.