The ASSET Global Rating Scale is a Valid and Reliable Adjunct Measure of Performance on a Virtual Reality Simulator for Hip Arthroscopy
| Author | Bishop ME, Ode GE, Hurwit DJ, Zmugg S, Rauck RC, Nguyen JT, Ranawat AS. |
|---|
Abstract
The purpose of this study is to further evaluate the construct validity and interobserver reliability of a hip arthroscopy virtual simulator using the Arthroscopic Surgery Skill Evaluation Tool (ASSET) global rating scale.
Method
Thirty participants (23 male/7 female) completed a diagnostic arthroscopy and a loose body retrieval simulation on the VirtaMed ArthroS™ Hip Simulator (Zurich, Switzerland) twice at a minimum of one week apart. Subjects consisted of 12 novices (medical students, postgraduate year - PGY1-2), 5 intermediate trainees (PGY3-4), 9 senior trainees (PGY5 and fellows) and 4 attending faculty. Simulator metrics were recorded and then compiled to generate a total simulator score (TSS). The loose body retrieval was graded using the ASSET scoring tool. Inter and intra-rater reliability for the ASSET for two blinded raters and construct validity of the ASSET and the TSS were calculated. Correlation between the TSS, ASSET and individual simulator metrics was determined.
Results
Prior simulation experience (p <0.01) correlated with higher TSS and higher ASSET, while video game experience correlated with higher TSS on the diagnostic module only (p=0.004). There was a significant difference in ASSET score between all experience groups (p<0.04). Novices had the lowest mean ASSET while Experts had the highest mean ASSET with a difference of 17.4 points. Overall performance on the surgical module significantly correlated with the ASSET score (r=0.444, p=0.016). There was a significant positive correlation between higher ASSET and number of loose bodies retrieved, operation time, camera path and grasper path length, and percentage of cartilage injury. ASSET demonstrated excellent intra-rater reliability and showed substantial or better inter-reliability in 8 of 9 domains.
Conclusions
The VirtaMed hip arthroscopy simulator demonstrated good construct validity and excellent reliability for simulator-based metrics and ASSET score. Utilization of both simulator metrics and ASSET offers a more comprehensive performance assessment on hip arthroscopy simulation than either measure alone.