# New Year Goals

Here are some goals related to scientific research and clinical medicine that I’d like to see accomplished in 2018. Here are some updates for 2019.

- Physicians come to know that precision/personalized medicine for the most part is based on a false premise
- Machine learning/deep learning is understood to not find previously unknown information in data in the majority of cases, and tends to work better than traditional statistical models only when dominant non-additive effects are present and the signal:noise ratio is decently high
- Practitioners will make more progress in correctly using “old” statistical tools such as regression models
- Medical diagnosis is finally understood as a task in probabilistic thinking, and sensitivity and specificity (which are characteristics not only of tests but also of patients) are seldom used
- Practitioners using cutpoints/thresholds for inherently continuous measurements will finally go back to primary references and find that the thresholds were never supported by data
- Dichotomania is seen as a failure to understand utility/loss/cost functions and as a tragic loss of information
- Clinical quality improvement initiatives will rely on randomized trial evidence and de-emphasize purely observational evidence; learning health systems will learn things that are actually true
- Clinicians will give up on the idea that randomized clinical trials do not generalize to real-world settings
- Fewer pre-post studies will be done
- More research will be reproducible with sounder sample size calculations, all data manipulation and analysis fully scripted, and data available for others to analyze in different ways
- Fewer sample size calculations will be based on a ‘miracle’ effect size
- Non-inferiority studies will no longer use non-inferiority margins that are far beyond clinically significant
- Fewer sample size calculations will be undertaken and more sequential experimentation done
- More Bayesian studies will be designed and executed
- Classification accuracy will be mistrusted as a measure of predictive accuracy
- More researchers will realize that estimation rather than hypothesis testing is the goal
- Change from baseline will seldom be
*computed,*not to mention not used in an analysis - Percents will begin to be replaced with fractions and ratios
- Fewer researchers will draw
**any**conclusion from large p-values other than “the money was spent” - Fewer researchers will draw conclusions from small p-values

Some wishes expressed by others on Twitter:

- No more ROC curves
- No more bar plots
- Ban the term ‘statistical significance’ and ‘statistically insignificant’

# Updates for 2019

My goals for 2018 were lofty so it’s not surprising that I’m disappointed overall with how little progress has been made on many of the fronts. But I am heartened by seven things:

- Clinicians are getting noticeably more dubious about personalized/precision medicine
- Researchers and clinicians are more dubious about benefits of machine learning
- Researchers are more enlightened about problems with p-values and dichotomous thinking that usually comes with them, and are especially starting to understand what’s wrong with “significant”
- Researchers are more enlightened about harm caused by dichotomania in general
- We successfully launched datamethods.org and have created in-depth discussion in the community about many of the issues listed under goals for 2018
- More researchers are seeing what a waste of ink ROC curves are
- More high-profile Bayesian analysis of clinical trials are being published

Areas that remain particularly frustrating are:

- Too many clinicians still believe that randomized clinical trials do not provide valuable efficacy data outside of the types of patients enrolled in the trials
- Clinical researchers are still computing change from baseline
- Sequential clinical trials are not being done (trials in which the sample size is not pretended to be known)
- A failure to understand conditioning (as in what is assumed when computing a conditional probability)

If I had to make just one plea for 2019, a general one is this: Recognize that actionable statistical information comes from thinking in a predictive mode. Condition on what you already know to predict what you don’t. Use forward-time, complete, conditioning. As opposed to type-I errors, p-values, sensitivity, specificity, and marginal (sample averaged) estimates.