Reliability and accuracy of delirium assessments among investigators at multiple international centres

Hannah R. Maybrier, Angela M. Mickle, Krisztina E. Escallier, Nan Lin, Eva M. Schmitt, Ravi T. Upadhyayula, Troy S. Wildes, George A. Mashour, Kerry Palihnich, Sharon K. Inouye, Michael Simon Avidan

Research output: Contribution to journalArticlepeer-review

28 Scopus citations


Introduction Delirium is a common, serious postoperative complication. For clinical studies to generate valid findings, delirium assessments must be standardised and administered accurately by independent researchers. The Confusion Assessment Method (CAM) is a widely used delirium assessment tool. The objective was to determine whether implementing a standardised CAM training protocol for researchers at multiple international sites yields reliable inter-rater assessment and accurate delirium diagnosis. Methods Patients consented to video recordings of CAM delirium assessments for research purposes. Raters underwent structured training in CAM administration. Training entailed didactic education, role-playing with intensive feedback, apprenticeship with experienced researchers and group discussions of complex cases. Raters independently viewed and scored nine video-recorded CAM interviews. Inter-rater reliability was determined using Fleiss kappa. Accuracy was judged by comparing raters' scores with those of an expert delirium researcher. Results Twenty-seven raters from eight international research centres completed the study and achieved almost perfect agreement for overall delirium diagnosis, kappa=0.88 (95% CI 0.85 to 0.92). Agreement of the four core CAM features ranged from fair to substantial. The sensitivity and specificity for identifying delirium were 72% (95% CI 60% to 81%) and 99% (95% CI 96% to 100%), considering an expert rater's scores as the reference standard (delirious, n=3; non-delirious, n=6). Delirium severity ratings were tightly clustered, with most scores within 5% of the median. Conclusion Our results demonstrate that, with appropriate training and ongoing scoring discussions, researchers at multiple sites can reliably detect delirium in postsurgical patients. These results support the premise that methodologically rigorous multi-centre studies can yield standardised and accurate determinations of delirium.

Original languageEnglish (US)
Article numbere023137
JournalBMJ open
Issue number11
StatePublished - Nov 1 2018
Externally publishedYes


  • adult anaesthesia
  • anesthesia
  • confusion
  • delirium
  • surgery

ASJC Scopus subject areas

  • General Medicine


Dive into the research topics of 'Reliability and accuracy of delirium assessments among investigators at multiple international centres'. Together they form a unique fingerprint.

Cite this