Inter-rater reliability in the radiological classification of renal injuries

Introduction: Although many radiologists invoke the surgical classification of renal injury proposed by the American Association for Surgery in Trauma (AAST), there has been only limited work on the role of the AAST system as an imaging stratification. The aim was to determine the inter-rater reliability (IRR) amongst radiologists and urologists using the AAST system. Methods: A 1-year retrospective study of consecutive patients with computed tomography (CT) evidence of renal trauma managed at a Level 1 trauma center. Three radiologists and three urologists independently stratified the presentation CT findings according to the AAST renal trauma classification. Agreement between independent raters and mutually exclusive groups was determined utilizing weighted kappa coefficients. Results: One hundred and one patients were included. Individual inter-observer agreements ranged from 54/101 (53.4%) to 62/101 (61.4%), with corresponding weighted kappa values from 0.61 to 0.69, constituting substantial agreement. Urologists achieved intra-disciplinary agreement in 49 cases (48.5%) and radiologists in 36 cases (35.6%). Six-reader agreement was achieved in 24 cases (23.7%). The AAST grade I injuries had the highest level of agreement, overall. Conclusion: The finding of substantial IRR amongst radiologists and urologists utilizing the AAST system supports continued use of the broad parameters of the AAST system, with some modification in specific categories with lower agreement.