A pilot study of the relationship between experts’ ratings and scores generated by the NBME’s computer-based examination system

David J. Solomon; Janet R. Osuch; Kimberly Anderson; James Babel; James Gruenberg; John Kisala; Mary Milroy; Willard Stawski

doi:10.1097/00001888-199202000-00020

A pilot study of the relationship between experts’ ratings and scores generated by the NBME’s computer-based examination system

David J. Solomon, Janet R. Osuch, Kimberly Anderson, James Babel, James Gruenberg, John Kisala, Mary Milroy, Willard Stawski

Research output: Contribution to journal › Article › peer-review

Abstract

This pilot study evaluates the consistency of experts’ ratings of students’ performances on the National Board of Medical Examiners’ Computer Based Examination (CBX) cases and the relationship of those ratings to the CBX’s scoring algorithm. The authors were investigating whether an automated scoring algorithm can adequately assess an examinee’s management of a computer-simulated patient. In 1989-90, at the Michigan State University College of Human Medicine, eight students, completing a surgery clerkship, each managed eight CBX cases and took a computer-administered, multiple-choice examination. Six clerkship coordinators rated the students’ performances in terms of overall management, efficiency, and dangerous actions. The ratings correlated highly with scores produced by the CBX’s scoring system.

Original language	English (US)
Pages (from-to)	130-132
Number of pages	3
Journal	Academic Medicine
Volume	67
Issue number	2
DOIs	https://doi.org/10.1097/00001888-199202000-00020
State	Published - Feb 1992
Externally published	Yes

ASJC Scopus subject areas

Education

Access to Document

10.1097/00001888-199202000-00020

Cite this

@article{e621789099ba45719f55096dfed39d0e,

title = "A pilot study of the relationship between experts{\textquoteright} ratings and scores generated by the NBME{\textquoteright}s computer-based examination system",

abstract = "This pilot study evaluates the consistency of experts{\textquoteright} ratings of students{\textquoteright} performances on the National Board of Medical Examiners{\textquoteright} Computer Based Examination (CBX) cases and the relationship of those ratings to the CBX{\textquoteright}s scoring algorithm. The authors were investigating whether an automated scoring algorithm can adequately assess an examinee{\textquoteright}s management of a computer-simulated patient. In 1989-90, at the Michigan State University College of Human Medicine, eight students, completing a surgery clerkship, each managed eight CBX cases and took a computer-administered, multiple-choice examination. Six clerkship coordinators rated the students{\textquoteright} performances in terms of overall management, efficiency, and dangerous actions. The ratings correlated highly with scores produced by the CBX{\textquoteright}s scoring system.",

author = "Solomon, {David J.} and Osuch, {Janet R.} and Kimberly Anderson and James Babel and James Gruenberg and John Kisala and Mary Milroy and Willard Stawski",

year = "1992",

month = feb,

doi = "10.1097/00001888-199202000-00020",

language = "English (US)",

volume = "67",

pages = "130--132",

journal = "Academic Medicine",

issn = "1040-2446",

publisher = "Lippincott Williams and Wilkins",

number = "2",

}

TY - JOUR

T1 - A pilot study of the relationship between experts’ ratings and scores generated by the NBME’s computer-based examination system

AU - Solomon, David J.

AU - Osuch, Janet R.

AU - Anderson, Kimberly

AU - Babel, James

AU - Gruenberg, James

AU - Kisala, John

AU - Milroy, Mary

AU - Stawski, Willard

PY - 1992/2

Y1 - 1992/2

N2 - This pilot study evaluates the consistency of experts’ ratings of students’ performances on the National Board of Medical Examiners’ Computer Based Examination (CBX) cases and the relationship of those ratings to the CBX’s scoring algorithm. The authors were investigating whether an automated scoring algorithm can adequately assess an examinee’s management of a computer-simulated patient. In 1989-90, at the Michigan State University College of Human Medicine, eight students, completing a surgery clerkship, each managed eight CBX cases and took a computer-administered, multiple-choice examination. Six clerkship coordinators rated the students’ performances in terms of overall management, efficiency, and dangerous actions. The ratings correlated highly with scores produced by the CBX’s scoring system.

AB - This pilot study evaluates the consistency of experts’ ratings of students’ performances on the National Board of Medical Examiners’ Computer Based Examination (CBX) cases and the relationship of those ratings to the CBX’s scoring algorithm. The authors were investigating whether an automated scoring algorithm can adequately assess an examinee’s management of a computer-simulated patient. In 1989-90, at the Michigan State University College of Human Medicine, eight students, completing a surgery clerkship, each managed eight CBX cases and took a computer-administered, multiple-choice examination. Six clerkship coordinators rated the students’ performances in terms of overall management, efficiency, and dangerous actions. The ratings correlated highly with scores produced by the CBX’s scoring system.

UR - http://www.scopus.com/inward/record.url?scp=0026590199&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0026590199&partnerID=8YFLogxK

U2 - 10.1097/00001888-199202000-00020

DO - 10.1097/00001888-199202000-00020

M3 - Article

C2 - 1546992

AN - SCOPUS:0026590199

SN - 1040-2446

VL - 67

SP - 130

EP - 132

JO - Academic Medicine

JF - Academic Medicine

IS - 2

ER -

A pilot study of the relationship between experts’ ratings and scores generated by the NBME’s computer-based examination system

Abstract

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this