1 General statistics and statistical modelling
1.1 Books
Following are some great books that focus on a more practical introduction to statistics and statistical modelling (e.g. using R as a compute environment). The order of books loosely reflects the increase in complexity of the material covered.
Many introductory statistics textbooks exist. I am listing here Stats - Data and models by de Veaux as one of the more modern introductory statistics book. It has received (and continues to receive) very positive reviews, covers the basics and more, and is targeting undergraduate students across different disciplines (biomedical sciences, social sciences, political sciences, physics, etc.). Unfortunately it is not cheap.
Dalgaard and Casella & Berger are two introductory and more advanced statistics textbooks. McElreath offers a fantastic introduction to Bayesian statistics and modelling, with a minimum dependence on existing stats and maths knowledge. Gelman et al. is a very comprehensive treatise on Bayesian data analysis containing many practical examples.
1.2 Tutorial and notes
Bååth, Beginners Exercise: Bayesian computation with Stan and Farmer Jöns
Bååth, Answers to Bayesian Computation with Stan and Farmer Jöns
Bååth gives a set of tutorial questions involving Bayesian statistics using R and Stan.
Betancourt, A Conceptual Introduction to Hamiltonian Monte Carlo: Betancourt gives a comprehensive account of Hamiltonian Monte Carlo (HMC) while avoiding (most of) the mathematical details. HMC has become a key element in statistical computing, and is used in software implementations such as Stan.
1.3 Free ebooks
I have not used the OpenIntro Statistics book but reviews have been very favourable. It’s also free. OpenIntro also offers additional reading and course material.
1.4 Web sources, blogs and online forums
CrossValidated is an online forum for people interested in statistics, machine learning, and data analysis. CrossValidated focusses on the theory and methodology of analysing data rather than on programming language-specific issues. Make sure to search through the CrossValidated records before posting a new question.
The xkcd.