Psychometric testing of the pecka grading rubric for evaluating higher-order thinking in distance learning

Shannon Pecka, Kendra Schmid, Bunny Pozehl

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


This article describes development of the Pecka Grading Rubric (PGR) as a strategy to facilitate and evaluate students' higher-order thinking in discussion boards. The purpose of this study was to describe psychometric properties of the PGR. Rubric reliability was pilot tested on a discussion board assignment used by 15 senior student registered nurse anesthetist enrolled in an Advanced Principles of Anesthesia course. Interrater and intrarater reliabilities were tested using an interclass correlation coefficient (ICC) to evaluate absolute agreement of scoring. Raters gave each category a score, scores of the categories were summed, and a total score was calculated for the entire rubric. Interrater (ICC = 0.939, P < .001) and intrarater (ICC = 0.902 to 0.994, P < .001) reliabilities were excellent for total point scores. A content validity index was used to evaluate content validity. Raters evaluated content validity of each cell of the PGR. The content validity index (0.8-1.0) was acceptable. Known-group validity was evaluated by comparing graduate student registered nurse anesthetists (N = 7) with undergraduate senior nursing students (N = 13). Beginning evidence indicates a valid and reliable instrument that measures higher-order thinking in the student registered nurse anesthetist.

Original languageEnglish (US)
Pages (from-to)449-456
Number of pages8
JournalAANA Journal
Issue number6
StatePublished - 2014


  • Distance learning
  • Grading rubric
  • Higherorder thinking
  • Reliability
  • Validity

ASJC Scopus subject areas

  • Medical–Surgical
  • Advanced and Specialized Nursing
  • Anesthesiology and Pain Medicine


Dive into the research topics of 'Psychometric testing of the pecka grading rubric for evaluating higher-order thinking in distance learning'. Together they form a unique fingerprint.

Cite this