Touchstone2: An Interactive Environment for Exploring Trade-offs in HCI Experiment Design

@article{Eiselmayer2019Touchstone2AI,
  title={Touchstone2: An Interactive Environment for Exploring Trade-offs in HCI Experiment Design},
  author={Alexander Eiselmayer and Chat Wacharamanotham and Michel Beaudouin-Lafon and Wendy E. Mackay},
  journal={Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems},
  year={2019}
}
Touchstone2 offers a direct-manipulation interface for generating and examining trade-offs in experiment designs. Based on interviews with experienced researchers, we developed an interactive environment for manipulating experiment design parameters, revealing patterns in trial tables, and estimating and comparing statistical power. We also developed TSL, a declarative language that precisely represents experiment designs. In two studies, experienced HCI researchers successfully used… 
View on ACM
zora.uzh.ch
28 Citations
Background Citations
17
Methods Citations
5

Figures and Tables from this paper

28 Citations

Citation Type

Citation Type

Supporting the Design and Analysis of HCI Experiments
TLDR
This dissertation contributes an empirical understanding of how researchers design and analyze experiments, as well as a set of tools that support researchers during that process.
Flex-ER
TLDR
Flex-ER, a flexible web-based environment that enables users to prototype, debug and share experimental conditions and results, is proposed and it is suggested that Flex-ER can be helpful to prototype and explore different interaction techniques for immersive analytics.
A Meta-Analysis of Effect Sizes of CHI Typing Experiments
TLDR
This work presents the reference for small, medium, and large effect sizes for typing experiments based on a meta-analysis of well-cited papers from CHI conference, which can be used to conduct a priori power analysis or assess the magnitude of the found effect.
Data Comics for Reporting Controlled User Studies in Human-Computer Interaction
TLDR
This paper provides examples of data comics study reports alongside a graphical repertoire of examples, embedded in a framework of guidelines for creating comics reports which was iterated upon and evaluated through a series of collaborative design sessions.
Interactive Tools for Reproducible Science - Understanding, Supporting, and Motivating Reproducible Science Practices
TLDR
This thesis paves new ways for interaction with RDM tools that support and motivate reproducible science, and advocates the unique role of HCI in supporting, motivating, and transforming reproducible research practices through the design of tools that enable effective RDM.
Gesture Elicitation as a Computational Optimization Problem
TLDR
This paper treats the problem of deriving a gesture vocabulary from gesture elicitation data as a computational optimization problem and shows how to formalize it as an optimal assignment problem and discusses how to express objective functions and custom design constraints through integer programs.
How Subtle Can It Get?
TLDR
This paper investigates one-handed human-to-drone-interaction by leveraging three modalities: force, touch, and IMU, and implements a method that achieves faster task completion times than the two-handed commercial baseline with the merits of subtle user behaviours inside a small-size ring-form device.
Tea: A High-level Language and Runtime System for Automating Statistical Analysis
TLDR
This work introduces Tea, a high-level declarative language and runtime system that generally matches the choices of experts while automatically switching to non-parametric tests when parametric assumptions are not met, and demonstrates the effect of mistakes made by non-expert users.
Putting scientific results in perspective: Improving the communication of standardized effect sizes
How do people form impressions of effect size when reading scientific results? We present a series of studies on how people perceive treatment effectiveness when scientific results are summarized in…
Hypothesis Formalization: Empirical Findings, Software Limitations, and Design Implications
TLDR
A mixed-methods study aimed at identifying the steps, considerations, and challenges involved in operationalizing hypotheses into statistical models, a process referred to as hypothesis formalization is presented.
...
1
2
3
...

References

SHOWING 1-10 OF 41 REFERENCES
Touchstone: exploratory design of experiments
TLDR
An open-source experiment design platform designed to help establish a solid research foundation for HCI in the area of novel interaction techniques, Touchstone facilitates the process of creating new experiments, as well as replicating and extending experiments in the research literature.
Using video to support interaction design
TLDR
The purpose of these exercises is to create an innovative design for an electronic Post-it® note, in the context of how it might really be used, using scenarios, storyboards and video prototypes.
Fair Statistical Communication in HCI
TLDR
This chapter explains in a non-technical manner why it would be beneficial for HCI to switch to an estimation approach, i.e., reporting informative charts with effect sizes and interval estimates, and offering nuanced interpretations of the results.
NexP: A Beginner Friendly Toolkit for Designing and Conducting Controlled Experiments
TLDR
NexP (Next Experiment Toolkit), an open-source toolkit for designing and running controlled experiments, is introduced and compared against a state-of-the-art experimental design toolkit, showing that novices considered NexP more intuitive and easier to use.
Researcher-Centered Design of Statistics: Why Bayesian Statistics Better Fit the Culture and Incentives of HCI
TLDR
It is shown that Bayesian analysis yields more precise effects with each new study, facilitating knowledge accrual without traditional meta-analyses, and Bayesian methods can be adopted from the bottom up without the need for new incentives for replication or meta-analysis.
VEEVVIE: Visual Explorer for Empirical Visualization, VR and Interaction Experiments
TLDR
VEEVVIE, the Visual Explorer for Empirical Visualization, VR and Interaction Experiments, is introduced, comprised of a back-end ontology which can model several experimental designs encountered in these fields.
Effect Sizes and Power Analysis in HCI
TLDR
This chapter discusses common misunderstandings of NHST and p value, and how effect sizes can complement them, and presents methods for calculating effect sizes with examples and describes another closely related topic, power analysis.
HARK No More: On the Preregistration of CHI Experiments
TLDR
How experimental preregistration could be adapted to the distinctive characteristics of Human-Computer Interaction empirical research, to the betterment of the discipline is discussed.
The table lens: merging graphical and symbolic representations in an interactive focus + context visualization for tabular information
TLDR
The Table Lens fuses symbolic and graphical representations into a single coherent view that can be fluidly adjusted by the user and enables an extremely rich and natural style of direct manipulation exploratory data analysis.
The Design of Experiments
  • J. I
  • Economics
    Nature
  • 1936
AbstractREADERS of “Statistical Methods for Research Workers” will welcome Prof. Fisher's new book, which is partly devoted to a development of the logical ideas underlying the earlier volume and…
...
1
2
3
4
5
...