Expanded understanding of student conceptions of engineers: Validation of the modified draw-an-engineer test (mDAET) scoring rubric

Julie Thomas, Leslie R. Hawley, Beverly DeVore-Wedding

Research output: Contribution to journalArticlepeer-review

Abstract

With this research, we sought to expand the pool of engineering education assessment instruments. We drew upon earlier efforts to define an expanded protocol for prompting children's drawings of engineers; and evaluate the reliability of a new scoring rubric for the modified Draw-an-Engineer Test (mDAET). An earlier paper introduced our ground theory approach in gathering theoretical propositions derived from children's drawings, explanations of engineers (n = 940) to develop the mDAET Scoring Rubric. Here we evaluate the reliability of the mDAET, a component of the larger mDAET rubric validation process. In the current study, eight participants scored twelve sets (including anchor sets) of drawings (n = 36) and, scores across drawings were evaluated by percentage match with the known-score and interrater reliability. Kappa values for exact matches across eight raters met the threshold for “substantial” agreement and matches that were exact or adjacent indicated “almost perfect” agreement. Notably, 92% of the possible matches among common anchor items were either exact or adjacent matches. These results indicate adequate inter-rater reliability evidence of the mDAET scoring rubric and demonstrate the feasibility of largescale administration that may no longer require individual child interviews.

Original languageEnglish (US)
Pages (from-to)391-401
Number of pages11
JournalSchool Science and Mathematics
Volume120
Issue number7
DOIs
StatePublished - Nov 2020
Externally publishedYes

Keywords

  • engineering education
  • STEM

ASJC Scopus subject areas

  • Education
  • History and Philosophy of Science
  • Mathematics (miscellaneous)
  • Physics and Astronomy (miscellaneous)
  • Engineering (miscellaneous)

Fingerprint Dive into the research topics of 'Expanded understanding of student conceptions of engineers: Validation of the modified draw-an-engineer test (mDAET) scoring rubric'. Together they form a unique fingerprint.

Cite this