New Year Goals

Here are some goals related to scientific research and clinical medicine that I’d like to see accomplished in 2018.

  • Physicians come to know that precision/personalized medicine for the most part is based on a false premise
  • Machine learning/deep learning is understood to not find previously unknown information in data in the majority of cases, and tends to work better than traditional statistical models only when dominant non-additive effects are present and the signal:noise ratio is decently high
  • Practitioners will make more progress in correctly using “old” statistical tools such as regression models
  • Medical diagnosis is finally understood as a task in probabilistic thinking, and sensitivity and specificity (which are characteristics not only of tests but also of patients) are seldom used
  • Practitioners using cutpoints/thresholds for inherently continuous measurements will finally go back to primary references and find that the thresholds were never supported by data
  • Dichotomania is seen as a failure to understand utility/loss/cost functions and as a tragic loss of information
  • Clinical quality improvement initiatives will rely on randomized trial evidence and de-emphasize purely observational evidence; learning health systems will learn things that are actually true
  • Clinicians will give up on the idea that randomized clinical trials do not generalize to real-world settings
  • Fewer pre-post studies will be done
  • More research will be reproducible with sounder sample size calculations, all data manipulation and analysis fully scripted, and data available for others to analyze in different ways
  • Fewer sample size calculations will be based on a ‘miracle’ effect size
  • Non-inferiority studies will no longer use non-inferiority margins that are far beyond clinically significant
  • Fewer sample size calculations will be undertaken and more sequential experimentation done
  • More Bayesian studies will be designed and executed
  • Classification accuracy will be mistrusted as a measure of predictive accuracy
  • More researchers will realize that estimation rather than hypothesis testing is the goal
  • Change from baseline will seldom be *computed,* not to mention not used in an analysis
  • Percents will begin to be replaced with fractions and ratios
  • Fewer researchers will draw any conclusion from large p-values other than “the money was spent”
  • Fewer researchers will draw conclusions from small p-values

Some wishes expressed by others on Twitter:

  • No more ROC curves
  • No more bar plots
  • Ban the term ‘statistical significance’ and ‘statistically insignificant’
comments powered by Disqus