This effort requires an aggressive program of training and testing for the IR that starts with the selection of the correct review team and lead reviewer: subspecialty, board-certified experts with significant clinical experience.
WCC develops clinical and statistical criteria to demonstrate proficiency of all proposed readers prior to their assignment to the trial, and once the full team is established, sets goals for performance. Consistent monitoring of reading results includes:
To maintain our focus on consistency and precision throughout the trial, our inter- and intra-reader agreement testing allows us to assess reproducibility on randomly selected cases. Readers participating in a study are sometimes asked to complete a blinded re-read to check for consistency, and multiple images may even be tested by multiple readers to confirm consistency between IRs. By conducting both tests, WCC promises active monitoring of readers’ potential variability.
Complementing our inter- and intra-reader agreement is score frequency monitoring. By evaluating reader-specific scoring frequencies and comparing it to both expected frequencies and the mean frequency of the group, WCC can obtain an indication of reader accuracy. For this reader evaluation, we statistically assess heterogeneity across readers using all reads from the trial. The end result is an identification of readers that may be under-calling or over-calling a disease.
Retraining is most often done for the whole group of readers when WCC finds high levels of variability in the diagnosis of results. Here, tough and important cases that have already been viewed are reintroduced into the reader queue to provide more confident estimates of reader variability than randomly selected images.
© 2012 WorldCare Clinical
Contact Us | Internet Security & Terms of Use
