There are many principles involved in the theory and practice of statistics, but here are the ones that guide my practice the most.
- Use methods grounded in theory or extensive simulation
- Understand uncertainty, and realize that the most honest approach to inference is a Bayesian model that takes into account what you don’t know (e.g., Are variances equal? Is the distribution normal? Should an interaction term be in the model?)
- Design experiments to maximize information
- Understand the measurements you are analyzing and don’t hesitate to question how the underlying information was captured
- Be more interested in questions than in null hypotheses, and be more interested in estimation than in answering narrow questions
- Use all information in data during analysis
- Use discovery and estimation procedures not likely to claim that noise is signal
- Strive for optimal quantification of evidence about effects
- Give decision makers the inputs (other than the utility function) that optimize decisions
- Present information in ways that are intuitive, maximize information content, and are correctly perceived
- Give the client what she needs, not what she wants
- Teach the client to want what she needs
... the statistician must be instinctively and primarily a logician and a scientist in the broader sense, and only secondarily a user of the specialized statistical techniques.
In considering the refinements and modifications of the scientific method which particularly apply to the work of the statistician, the first point to be emphasized is that the statistician is always dealing with probabilities and degrees of uncertainty. He is, in effect, a Sherlock Holmes of figures, who must work mainly, or wholly, from circumstantial evidence.
Malcolm C Rorty: Statistics and the Scientific Method
JASA 26:1-10, 1931
See this post for related thoughts.