Skip to content
Home
About Us
Resources
Profiles Metrics
Authors Directory
Institutions Directory
Top Authors
Top Institutions
Top Sponsors
AI Digest
Contact Us
Menu
Home
About Us
Resources
Profiles Metrics
Authors Directory
Institutions Directory
Top Authors
Top Institutions
Top Sponsors
AI Digest
Contact Us
Home
About Us
Resources
Profiles Metrics
Authors Directory
Institutions Directory
Top Authors
Top Institutions
Top Sponsors
AI Digest
Contact Us
Menu
Home
About Us
Resources
Profiles Metrics
Authors Directory
Institutions Directory
Top Authors
Top Institutions
Top Sponsors
AI Digest
Contact Us
Publication Details
AFRICAN RESEARCH NEXUS
SHINING A SPOTLIGHT ON AFRICAN RESEARCH
medicine
Reliability of Trachoma Clinical Grading-Assessing Grading of Marginal Cases
PLoS Neglected Tropical Diseases, Volume 8, No. 5, Article e2840, Year 2014
Notification
URL copied to clipboard!
Description
Background:Clinical examination of trachoma is used to justify intervention in trachoma-endemic regions. Currently, field graders are certified by determining their concordance with experienced graders using the kappa statistic. Unfortunately, trachoma grading can be highly variable and there are cases where even expert graders disagree (borderline/marginal cases). Prior work has shown that inclusion of borderline cases tends to reduce apparent agreement, as measured by kappa. Here, we confirm those results and assess performance of trainees on these borderline cases by calculating their reliability error, a measure derived from the decomposition of the Brier score.Methods and Findings:We trained 18 field graders using 200 conjunctival photographs from a community-randomized trial in Niger and assessed inter-grader agreement using kappa as well as reliability error. Three experienced graders scored each case for the presence or absence of trachomatous inflammation - follicular (TF) and trachomatous inflammation - intense (TI). A consensus grade for each case was defined as the one given by a majority of experienced graders. We classified cases into a unanimous subset if all 3 experienced graders gave the same grade. For both TF and TI grades, the mean kappa for trainees was higher on the unanimous subset; inclusion of borderline cases reduced apparent agreement by 15.7% for TF and 12.4% for TI. When we assessed the breakdown of the reliability error, we found that our trainees tended to over-call TF grades and under-call TI grades, especially in borderline cases.Conclusions:The kappa statistic is widely used for certifying trachoma field graders. Exclusion of borderline cases, which even experienced graders disagree on, increases apparent agreement with the kappa statistic. Graders may agree less when exposed to the full spectrum of disease. Reliability error allows for the assessment of these borderline cases and can be used to refine an individual trainee's grading. © 2014 Rahman et al.
Authors & Co-Authors
Rahman, Salman A.
United States, San Francisco
F.i. Proctor Foundation
Yu, Sun N.
United States, San Francisco
F.i. Proctor Foundation
Amza, Abdou
Niger
Programme Nationale Des Soins Oculaire
Gebreselassie, Sintayehu
United States, Atlanta
The Carter Center
Kadri, Boubacar
Niger
Programme Nationale Des Soins Oculaire
Baido, Nassirou
Niger
Programme Nationale Des Soins Oculaire
Stoller, Nicole E.
United States, San Francisco
F.i. Proctor Foundation
Sheehan, Joseph P.
United States, San Francisco
F.i. Proctor Foundation
United States, San Francisco
University of California, San Francisco
Porco, Travis C.
United States, San Francisco
F.i. Proctor Foundation
United States, San Francisco
University of California, San Francisco
Gaynor, Bruce D.
United States, San Francisco
F.i. Proctor Foundation
United States, San Francisco
University of California, San Francisco
Keenan, Jeremy David
United States, San Francisco
F.i. Proctor Foundation
United States, San Francisco
University of California, San Francisco
Lietman, Thomas M.
United States, San Francisco
F.i. Proctor Foundation
United States, San Francisco
University of California, San Francisco
Statistics
Citations: 12
Authors: 12
Affiliations: 4
Identifiers
Doi:
10.1371/journal.pntd.0002840
ISSN:
19352727
e-ISSN:
19352735
Study Design
Randomised Control Trial
Study Locations
Niger