When it comes to reader review and data management, precision and control are of the utmost importance. Our scientific approach to imaging provides sponsors with the most accurate and reproducible datasets available. We provide access to operational metrics and clinical statistics every step of the way to improve the accuracy of the trial. Our advanced understanding of diagnostic statistics and variability testing metrics allow WCC to provide an unparalleled depth of understanding of trial data.
Training and Testing
WCC's intensive training program starts with the selection of the review team and lead reviewer. Criteria are specified to measure the statistical proficiency of all readers prior to assignment and to set goals for performance.
We look at adjudication rates, apply our statistical approach and use tests such as kappa analysis with confidence intervals to test the performance of the team. To help maintain focus, we provide this data to the lead reviewer and the sponsor so potential issues can be recognized and corrected.
Our approach to retraining is adaptive with retraining being done periodically and on an "as needed" basis. When retraining is required, we select tough to interpret cases rather than randomly selected cases that allow us to better understand reader performance and hone reader skills.
Blinded Independent Central Reviews (BICRs) have the potential to minimize bias and increase the precision of trial data by utilizing multiple independent readers (in a multi-reader BICR). Currently, most multi-reader BICRs are performed primarily for radiological assessments. This is in part attributed to both an understanding of the variability associated with image interpretations and the growth of teleradiology, which allows multiple readers to make rapid assessments of imaging data with software that can automatically send discrepant reads to an adjudicator.
Reader Performance can be Improved
A carefully conducted multi-reader BICR may result in a decrease in inter-reader variability and thus increase the accuracy of the assessment. The inter-reader variability of a small team of independent reviewers can be honed through an iterative process of team training after testing with a focus on difficult cases (i.e., adjudicated cases).
Readers are only the sum of their precision and accuracy
Performance of readers can be explained by their precision and accuracy. When a BICR includes a cohesive team of board-certified, subspecialty-trained practicing physicians, you have the best chance of maximizing the accuracy of an assessment. When coupled with WCC's adaptive training and testing methodology, differences between reviewers are minimized to such a degree that the smallest discrepancies between treatments can be detected.
Precision Where You Need It
The BICR process allows you to add precision to different components of the assessment. For example, we usually recommend multi-reader BICR paradigms for primary endpoints or for complex assessments, whereas a single reader BICR paradigm should be sufficient for secondary endpoints. Regardless of the read paradigm, WCC can help titrate the cost value proposition for your BICR.
Simple Fact: BICRs can be Cost-Effective
Assuming both local evaluation and BICR readers have the same precision and accuracy, an increase in sample size (multi-reader BICR) will decrease variability and result in an increase in precision over the single local evaluation. A simulation model using a cancer study, in collaboration with Dr. Duan from Brown University, demonstrated that compared to local evaluations, BICR paradigms decreased overall trial costs and sample sizes. The effect was even greater if the precision and/or accuracy of BICR reviews were better than the site reviewers.
If data validity is not being actively monitored as it is collected, submissions are at risk. Discrepancies build on one another and the data drifts further and further off course if uncorrected. We proactively assess imaging data so potential issues can be identified and corrected as early as possible. Our regular status updates on every trial and intra- and inter- reader reliability testing allows us to report on the performance of readers and even help reduce the rate that errors occur.