The difference between Bayesian and frequentist inference in a nutshell:
With Bayes you start with a prior distribution for θ and given your data make an inference about the θ-driven process generating your data (whatever that process happened to be), to quantify evidence for every possible value of θ. With frequentism, you make assumptions about the process that generated your data and infinitely many replications of them, and try to build evidence for what θ is not.
There are many principles involved in the theory and practice of statistics, but here are the ones that guide my practice the most.
Use methods grounded in theory or extensive simulation Understand uncertainty, and realize that the most honest approach to inference is a Bayesian model that takes into account what you don’t know (e.g., Are variances equal? Is the distribution normal? Should an interaction term be in the model?