Design

EHRs and RCTs: Outcome Prediction vs. Optimal Treatment Selection

Frank Harrell Professor of Biostatistics Vanderbilt University School of Medicine Laura Lazzeroni Professor of Psychiatry and, by courtesy, of Medicine (Cardiovascular Medicine) and of Biomedical Data Science Stanford University School of Medicine Revised July 17, 2017 It is often said that randomized clinical trials (RCTs) are the gold standard for learning about therapeutic effectiveness. This is because the treatment is assigned at random so no variables, measured or unmeasured, will be truly related to treatment assignment.

Randomized Clinical Trials Do Not Mimic Clinical Practice, Thank Goodness

Randomized clinical trials (RCT) have long been held as the gold standard for generating evidence about the effectiveness of medical and surgical treatments, and for good reason. But I commonly hear clinicians lament that the results of RCTs are not generalizable to medical practice, primarily for two reasons: Patients in clinical practice are different from those enrolled in RCTs Drug adherence in clinical practice is likely to be lower than that achieved in RCTs, resulting in lower efficacy.

Fundamental Principles of Statistics

There are many principles involved in the theory and practice of statistics, but here are the ones that guide my practice the most. Use methods grounded in theory or extensive simulation Understand uncertainty Design experiments to maximize information Understand the measurements you are analyzing and don’t hesitate to question how the underlying information was captured Be more interested in questions than in null hypotheses, and be more interested in estimation than in answering narrow questions Use all information in data during analysis Use discovery and estimation procedures not likely to claim that noise is signal Strive for optimal quantification of evidence about effects Give decision makers the inputs (other than the utility function) that optimize decisions Present information in ways that are intuitive, maximize information content, and are correctly perceived Give the client what she needs, not what she wants Teach the client to want what she needs … the statistician must be instinctively and primarily a logician and a scientist in the broader sense, and only secondarily a user of the specialized statistical techniques.