Assessing diagnostic arthroscopy performance in the operating room using the Arthroscopic Surgery Skill Evaluation Tool (ASSET)

Purpose To determine the validity and reliability of using the Arthroscopic Surgery Skill Evaluation Tool (ASSET) to assess arthroscopic skill in the operating room. Methods Eight orthopaedic residents, 1 sports medicine fellow, and 3 sports medicine faculty members recorded the diagnostic portion of 3 shoulder and 3 knee procedures in the operating room. Two blinded raters used the ASSET to assess each recorded procedure video. Criterion for a passing score on a procedure was attaining a 3 or greater in all 8 domains assessed. Results In total 70 videos (36 shoulder, 34 knee) were evaluated by each rater. The attending/fellow group was assigned significantly higher mean ASSET scores compared with resident groups for both procedures (P =.01). The attending/fellow group also had the highest passing percentage (95.8%). Raters were in agreement for total ASSET scores assigned to both diagnostic arthroscopy of the shoulder (intraclass correlation coefficient [ICC] = 0.84) and knee (ICC = 0.81). Agreement on individual ASSET domains was moderate (ICC = 0.61 to 0.80) for all domains except safety and difficulty of procedure. Raters concurred on the pass-fail evaluation in 62 of 70 (88.6%) of procedures evaluated. Conclusions Using the ASSET to assess surgical skills in the operating room is feasible, reliable, and valid. Clinical Relevance When combined with previously published results using the ASSET, the ASSET may provide a validated and reliable method for evaluating arthroscopic surgical skills in the surgical simulation lab and operating room.