Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones

Michael S. Beeson, Stanley J. Hamstra, Melissa A. Barton, Kenji Yamazaki, Francis L. Counselman, Philip H. Shayne, Eric S. Holmboe, Robert Leo Muelleman, Earl J. Reisdorff

Research output: Contribution to journalArticle

3 Scopus citations

Abstract

Objective : This study examined the incidence of straight line scoring (SLS) for EM Milestone ratings, defined as a resident being assessed the same score across the milestone subcompetencies.

Background : In 2013, milestone ratings became a reporting requirement for emergency medicine (EM) residency programs. Programs rate each resident in the fall and spring on 23 milestone subcompetencies.

Methods : This descriptive analysis measured the frequencies of SLS for all Accreditation Council for Graduate Medical Education (ACGME)-accredited EM programs during the 2015-2016 academic year. Outcomes were the frequency of SLS in the fall and spring milestone assessments, changes in the number of SLS reports, and reporting trends. Chi-square analysis compared nominal variables.

Results : There were 6257 residents in the fall and 6588 in the spring. Milestone scores were reported for 6173 EM residents in the fall (99% of 6257) and spring (94% of 6588). In the fall, 93% (5753 residents) did not receive SLS ratings and 420 (7%) did, with no significant difference compared with the spring (5776 [94%] versus 397 [6%]). Subgroup analysis showed higher SLS results for residents' first ratings (183 of 2136 versus 237 of 4220, P < .0001) and for their final ratings (200 of 2019 versus 197 of 4354, P < .0001). Twenty percent of programs submitted 10% or more SLS ratings, and a small percentage submitted more than 50% of ratings as SLS.

Conclusions : Most programs did not submit SLS ratings. Because of the statistical improbability of SLS, any SLS ratings reduce the validity assertions of the milestone assessments.

Original languageEnglish (US)
Pages (from-to)716-720
Number of pages5
JournalJournal of graduate medical education
Volume9
Issue number6
DOIs
StatePublished - Dec 1 2017

ASJC Scopus subject areas

  • Medicine(all)

Fingerprint Dive into the research topics of 'Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones'. Together they form a unique fingerprint.

  • Cite this

    Beeson, M. S., Hamstra, S. J., Barton, M. A., Yamazaki, K., Counselman, F. L., Shayne, P. H., Holmboe, E. S., Muelleman, R. L., & Reisdorff, E. J. (2017). Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones. Journal of graduate medical education, 9(6), 716-720. https://doi.org/10.4300/JGME-D-17-00304.1