Assessing the type annotation burden

John Paul Ore, Carrick Detweiler, Sebastian Elbaum, Lambros Karkazis

Research output: Chapter in Book/Report/Conference proceedingConference contribution

8 Scopus citations

Abstract

Type annotations provide a link between program variables and domain-specific types. When combined with a type system, these annotations can enable early fault detection. For type annotations to be cost-effective in practice, they need to be both accurate and affordable for developers. We lack, however, an understanding of how burdensome type annotation is for developers. Hence, this work explores three fundamental questions: 1) how accurately do developers make type annotations; 2) how long does a single annotation take; and, 3) if a system could automatically suggest a type annotation, how beneficial to accuracy are correct suggestions and how detrimental are incorrect suggestions? We present results of a study of 71 programmers using 20 random code artifacts that contain variables with physical unit types that must be annotated. Subjects choose a correct type annotation only 51% of the time and take an average of 136 seconds to make a single correct annotation. Our qualitative analysis reveals that variable names and reasoning over mathematical operations are the leading clues for type selection. We find that suggesting the correct type boosts accuracy to 73%, while making a poor suggestion decreases accuracy to 28%. We also explore what state-of-the-art automated type annotation systems can and cannot do to help developers with type annotations, and identify implications for tool developers.

Original languageEnglish (US)
Title of host publicationASE 2018 - Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering
EditorsChristian Kastner, Marianne Huchard, Gordon Fraser
PublisherAssociation for Computing Machinery, Inc
Pages190-201
Number of pages12
ISBN (Electronic)9781450359375
DOIs
StatePublished - Sep 3 2018
Event33rd IEEE/ACM International Conference on Automated Software Engineering, ASE 2018 - Montpellier, France
Duration: Sep 3 2018Sep 7 2018

Publication series

NameASE 2018 - Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering

Conference

Conference33rd IEEE/ACM International Conference on Automated Software Engineering, ASE 2018
Country/TerritoryFrance
CityMontpellier
Period9/3/189/7/18

Keywords

  • Abstract type inference
  • Dimensional analysis
  • Physical units
  • Program analysis
  • Robotic systems
  • Static analysis
  • Type checking
  • Unit consistency

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Human-Computer Interaction
  • Software

Fingerprint

Dive into the research topics of 'Assessing the type annotation burden'. Together they form a unique fingerprint.

Cite this