MoreTrials

MoreTrials: A public campaign to make it much easier to do randomised trials

You are here: Home / News Post / News: Rob Califf (Duke) GCP’s mistaken focus on precision at the expense of reliability

Oct 06 2017

News: Rob Califf (Duke) GCP’s mistaken focus on precision at the expense of reliability

Professor Rob Califf, in a recent talk at the National Academies of Sciences, Engineering, and Medicine workshop on “real-world evidence”  highlighted the failures of the current healthcare evidence generation system and presented a new approach to conducting randomised trials. You can watch the talk here.

The current system of evidence generation is based upon a mistaken focus on the need for precision in clinical trials which has been at the expense of getting reliable results from trials. This has resulted in the massively increasing cost of doing trials without a corresponding increase in quality. Following SOPs has become more important than the science of clinical investigation and one of the most damaging effects is the effort to record each data item more precisely in the mistaken notion that more reliable estimates of treatment effect will be derived for outcomes that matter.

This old system was built at a time when automation was not possible because data in an electronic format largely did not exist. Doctors wrote things on paper and then as time evolved this information was transferred to a computer. Then nurses and others flew around on airplanes at great expense to check that the information matched. This resulted in bizarre practices where nurses were printing out electronic health records to produce a written record so that a monitor can then check and make sure it matches what is in the trial’s electronic record. The cost of this monitoring is some hundreds of millions of dollars for some trials. The regulators devised a doctrine called GCP to enable these things to be done in a consistent fashion, but consistency doesn’t necessarily mean better, especially if one set of rules are applied to an entire array of different questions that need to be asked. Things were made even worse by “local experts” taking the doctrine and turning it into voluminous SOPs with multiple interpretations.

In a mistaken understanding of the theory and purpose of clinical trials, the regulated clinical trial industry has diverted enormous resources to an effort to increase precision and the academic trials community have adopted some of this thinking through the proliferation of GCP. Tearing down the entire structure would be counterproductive as people need structure to conduct these complex human experiments, but there is a need to change the focus from precision to reliability. For most clinical research, a focus on precision can actually detract from reliability because it limits, for example, the size of the study you can do or the number of endpoints that you have.

The result of all of this is that it is estimated that less than 15% of treatment guideline recommendations are supported by high-quality evidence.

In the second half of the talk, Califf went on to describe the four critical elements to build a new effective evidence generation system. These are:

  1. Create a reusable system for research embedded in practice.
  2. Apply the principle of Quality-by-Design.
  3. Use technology to automate repetitive tasks.
  4. Operate from basic principles.

Create a reusable system for research embedded in practice

One of the major inefficiencies of current trials is the need to put in place the required infrastructure for each individual trial and which then gets discarded at the end of the study. Instead, networks of investigational sites linked to co-ordinating centres and utilising routine electronic data are being created for the purpose of doing trials much more efficiently. An example of such a reusable system is the NIH Research Collaboratory project, which has undertaken a number of demonstration trials at about 10% of the cost of doing traditional randomised trials.

Apply the principle of Quality-by-Design

 

The second concept is to apply the principle of Quality-by-Design, which I recently covered and can be found here. Califf pointed out that identifying the errors that matter for an individual trial requires careful thought instead of following a rigid set of SOPs. QbD can’t be implemented by recipe, but requires a need to understand the question the trial is asking, the clinical context and the needs of potential participants and future patients. The overall objective is to eliminate those errors that matter to the reliability of the result of the trial and the safety of trial participants.

Use technology to automate repetitive tasks

Califf used the example of Google Maps where the question is no longer, “do you have a map?”, but rather, “given that you have a map, how can you use that information?” In healthcare, continuous learning from the analysis of routine medical data is providing the platform for the more efficient conduct of research. The NIH Research Collaboratory is an example of such a learning healthcare system being used as a platform for research.

Operate from basic principles

Califf closed by pointing out that the new system won’t be built upon a rigid set of SOPs. New guidance will need to be developed, but instead of focusing on operational detail it would highlight those key principles in order to do a randomised trial well. This is exactly what we are planning to do at MoreTrials.

Print Friendly

Written by Tim Sprosen · Categorized: News Post

Search Form

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

A public campaign for more, better, randomised trials

Pages

  • The Problem
    • ICH
    • ICH-GCP guideline
    • Response from ICH
    • ICH’s GCP Renovation Project
    • Stories to Highlight the Problems
  • The Solution
    • Developing the New-GCP
    • The Plan
  • News
  • The unique value of randomised trials
    • Illustration
    • The wider context
    • CTTI: Making things better
    • Stories of Trials That Matter
  • Have your say
  • Supporters
    • AllTrials
  • Videos
  • Join us
  • Contact us

Join over 200 trialists from 30 countries

Search Form

  • Google+
  • Twitter

© 2018 Moretrials.net · Rainmaker Platform