Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones

Michael S. Beeson; Stanley J. Hamstra; Melissa A. Barton; Kenji Yamazaki; Francis L. Counselman; Philip H. Shayne; Eric S. Holmboe; Robert Leo Muelleman; Earl J. Reisdorff

doi:10.4300/JGME-D-17-00304.1

Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones

Michael S. Beeson, Stanley J. Hamstra, Melissa A. Barton, Kenji Yamazaki, Francis L. Counselman, Philip H. Shayne, Eric S. Holmboe, Robert Leo Muelleman, Earl J. Reisdorff

Research output: Contribution to journal › Article

3 Scopus citations

Abstract

Objective : This study examined the incidence of straight line scoring (SLS) for EM Milestone ratings, defined as a resident being assessed the same score across the milestone subcompetencies.

Background : In 2013, milestone ratings became a reporting requirement for emergency medicine (EM) residency programs. Programs rate each resident in the fall and spring on 23 milestone subcompetencies.

Methods : This descriptive analysis measured the frequencies of SLS for all Accreditation Council for Graduate Medical Education (ACGME)-accredited EM programs during the 2015-2016 academic year. Outcomes were the frequency of SLS in the fall and spring milestone assessments, changes in the number of SLS reports, and reporting trends. Chi-square analysis compared nominal variables.

Results : There were 6257 residents in the fall and 6588 in the spring. Milestone scores were reported for 6173 EM residents in the fall (99% of 6257) and spring (94% of 6588). In the fall, 93% (5753 residents) did not receive SLS ratings and 420 (7%) did, with no significant difference compared with the spring (5776 [94%] versus 397 [6%]). Subgroup analysis showed higher SLS results for residents' first ratings (183 of 2136 versus 237 of 4220, P < .0001) and for their final ratings (200 of 2019 versus 197 of 4354, P < .0001). Twenty percent of programs submitted 10% or more SLS ratings, and a small percentage submitted more than 50% of ratings as SLS.

Conclusions : Most programs did not submit SLS ratings. Because of the statistical improbability of SLS, any SLS ratings reduce the validity assertions of the milestone assessments.

Original language	English (US)
Pages (from-to)	716-720
Number of pages	5
Journal	Journal of graduate medical education
Volume	9
Issue number	6
DOIs	https://doi.org/10.4300/JGME-D-17-00304.1
State	Published - Dec 1 2017

ASJC Scopus subject areas

Medicine(all)

Access to Document

10.4300/JGME-D-17-00304.1

Fingerprint Dive into the research topics of 'Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones'. Together they form a unique fingerprint.

Cite this

Beeson, M. S., Hamstra, S. J., Barton, M. A., Yamazaki, K., Counselman, F. L., Shayne, P. H., Holmboe, E. S., Muelleman, R. L., & Reisdorff, E. J. (2017). Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones. Journal of graduate medical education, 9(6), 716-720. https://doi.org/10.4300/JGME-D-17-00304.1

Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones. / Beeson, Michael S.; Hamstra, Stanley J.; Barton, Melissa A.; Yamazaki, Kenji; Counselman, Francis L.; Shayne, Philip H.; Holmboe, Eric S.; Muelleman, Robert Leo; Reisdorff, Earl J.

In: Journal of graduate medical education, Vol. 9, No. 6, 01.12.2017, p. 716-720.

Research output: Contribution to journal › Article

@article{b5fb2fdc84e5402a81aea6439c5b249d,

title = "Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones",

abstract = "Objective : This study examined the incidence of straight line scoring (SLS) for EM Milestone ratings, defined as a resident being assessed the same score across the milestone subcompetencies.Background : In 2013, milestone ratings became a reporting requirement for emergency medicine (EM) residency programs. Programs rate each resident in the fall and spring on 23 milestone subcompetencies.Methods : This descriptive analysis measured the frequencies of SLS for all Accreditation Council for Graduate Medical Education (ACGME)-accredited EM programs during the 2015-2016 academic year. Outcomes were the frequency of SLS in the fall and spring milestone assessments, changes in the number of SLS reports, and reporting trends. Chi-square analysis compared nominal variables.Results : There were 6257 residents in the fall and 6588 in the spring. Milestone scores were reported for 6173 EM residents in the fall (99% of 6257) and spring (94% of 6588). In the fall, 93% (5753 residents) did not receive SLS ratings and 420 (7%) did, with no significant difference compared with the spring (5776 [94%] versus 397 [6%]). Subgroup analysis showed higher SLS results for residents' first ratings (183 of 2136 versus 237 of 4220, P < .0001) and for their final ratings (200 of 2019 versus 197 of 4354, P < .0001). Twenty percent of programs submitted 10% or more SLS ratings, and a small percentage submitted more than 50% of ratings as SLS.Conclusions : Most programs did not submit SLS ratings. Because of the statistical improbability of SLS, any SLS ratings reduce the validity assertions of the milestone assessments.",

author = "Beeson, {Michael S.} and Hamstra, {Stanley J.} and Barton, {Melissa A.} and Kenji Yamazaki and Counselman, {Francis L.} and Shayne, {Philip H.} and Holmboe, {Eric S.} and Muelleman, {Robert Leo} and Reisdorff, {Earl J.}",

year = "2017",

month = dec,

day = "1",

doi = "10.4300/JGME-D-17-00304.1",

language = "English (US)",

volume = "9",

pages = "716--720",

journal = "Journal of graduate medical education",

issn = "1949-8349",

publisher = "University of Finance and Management",

number = "6",

}

TY - JOUR

T1 - Straight Line Scoring by Clinical Competency Committees Using Emergency Medicine Milestones

AU - Beeson, Michael S.

AU - Hamstra, Stanley J.

AU - Barton, Melissa A.

AU - Yamazaki, Kenji

AU - Counselman, Francis L.

AU - Shayne, Philip H.

AU - Holmboe, Eric S.

AU - Muelleman, Robert Leo

AU - Reisdorff, Earl J.

PY - 2017/12/1

Y1 - 2017/12/1

N2 - Objective : This study examined the incidence of straight line scoring (SLS) for EM Milestone ratings, defined as a resident being assessed the same score across the milestone subcompetencies.Background : In 2013, milestone ratings became a reporting requirement for emergency medicine (EM) residency programs. Programs rate each resident in the fall and spring on 23 milestone subcompetencies.Methods : This descriptive analysis measured the frequencies of SLS for all Accreditation Council for Graduate Medical Education (ACGME)-accredited EM programs during the 2015-2016 academic year. Outcomes were the frequency of SLS in the fall and spring milestone assessments, changes in the number of SLS reports, and reporting trends. Chi-square analysis compared nominal variables.Results : There were 6257 residents in the fall and 6588 in the spring. Milestone scores were reported for 6173 EM residents in the fall (99% of 6257) and spring (94% of 6588). In the fall, 93% (5753 residents) did not receive SLS ratings and 420 (7%) did, with no significant difference compared with the spring (5776 [94%] versus 397 [6%]). Subgroup analysis showed higher SLS results for residents' first ratings (183 of 2136 versus 237 of 4220, P < .0001) and for their final ratings (200 of 2019 versus 197 of 4354, P < .0001). Twenty percent of programs submitted 10% or more SLS ratings, and a small percentage submitted more than 50% of ratings as SLS.Conclusions : Most programs did not submit SLS ratings. Because of the statistical improbability of SLS, any SLS ratings reduce the validity assertions of the milestone assessments.

AB - Objective : This study examined the incidence of straight line scoring (SLS) for EM Milestone ratings, defined as a resident being assessed the same score across the milestone subcompetencies.Background : In 2013, milestone ratings became a reporting requirement for emergency medicine (EM) residency programs. Programs rate each resident in the fall and spring on 23 milestone subcompetencies.Methods : This descriptive analysis measured the frequencies of SLS for all Accreditation Council for Graduate Medical Education (ACGME)-accredited EM programs during the 2015-2016 academic year. Outcomes were the frequency of SLS in the fall and spring milestone assessments, changes in the number of SLS reports, and reporting trends. Chi-square analysis compared nominal variables.Results : There were 6257 residents in the fall and 6588 in the spring. Milestone scores were reported for 6173 EM residents in the fall (99% of 6257) and spring (94% of 6588). In the fall, 93% (5753 residents) did not receive SLS ratings and 420 (7%) did, with no significant difference compared with the spring (5776 [94%] versus 397 [6%]). Subgroup analysis showed higher SLS results for residents' first ratings (183 of 2136 versus 237 of 4220, P < .0001) and for their final ratings (200 of 2019 versus 197 of 4354, P < .0001). Twenty percent of programs submitted 10% or more SLS ratings, and a small percentage submitted more than 50% of ratings as SLS.Conclusions : Most programs did not submit SLS ratings. Because of the statistical improbability of SLS, any SLS ratings reduce the validity assertions of the milestone assessments.

UR - http://www.scopus.com/inward/record.url?scp=85049251142&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049251142&partnerID=8YFLogxK

U2 - 10.4300/JGME-D-17-00304.1

DO - 10.4300/JGME-D-17-00304.1

M3 - Article

C2 - 29270260

AN - SCOPUS:85049251142

VL - 9

SP - 716

EP - 720

JO - Journal of graduate medical education

JF - Journal of graduate medical education

SN - 1949-8349

IS - 6

ER -