Graphical tests for power comparison of competing designs

Heike Hofmann, Lendie Follett, Mahbubul Majumder, Dianne Cook

Research output: Contribution to journalArticle

19 Scopus citations

Abstract

Lineups [4, 28] have been established as tools for visual testing similar to standard statistical inference tests, allowing us to evaluate the validity of graphical findings in an objective manner. In simulation studies [12] lineups have been shown as being efficient: the power of visual tests is comparable to classical tests while being much less stringent in terms of distributional assumptions made. This makes lineups versatile, yet powerful, tools in situations where conditions for regular statistical tests are not or cannot be met. In this paper we introduce lineups as a tool for evaluating the power of competing graphical designs. We highlight some of the theoretical properties and then show results from two studies evaluating competing designs: both studies are designed to go to the limits of our perceptual abilities to highlight differences between designs. We use both accuracy and speed of evaluation as measures of a successful design. The first study compares the choice of coordinate system: polar versus cartesian coordinates. The results show strong support in favor of cartesian coordinates in finding fast and accurate answers to spotting patterns. The second study is aimed at finding shift differences between distributions. Both studies are motivated by data problems that we have recently encountered, and explore using simulated data to evaluate the plot designs under controlled conditions. Amazon Mechanical Turk (MTurk) is used to conduct the studies. The lineups provide an effective mechanism for objectively evaluating plot designs.

Original languageEnglish (US)
Article number6327249
Pages (from-to)2441-2448
Number of pages8
JournalIEEE Transactions on Visualization and Computer Graphics
Volume18
Issue number12
DOIs
StatePublished - 2012

Keywords

  • Efficiency of displays
  • Lineups
  • Power comparison
  • Visual inference

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Fingerprint Dive into the research topics of 'Graphical tests for power comparison of competing designs'. Together they form a unique fingerprint.

  • Cite this