# Statistics, for the mathematically rigorous

I don't know where I can find a rigorous statistics course or textbook. The closest thing I can think of is measure-theoretic probability theory, but I wouldn't really call that "statistics". By 'close to statistics', I mean something that, although as mathematically rigorous as probability theory, can be reasonably substituted for a statistics course with minimal study.

Does such a thing exist? Perhaps not, as the point of many statistics courses is in their applications.

• Maybe Introduction to Mathematical Statistics by Robert Vincent Hogg and Allen Thornton Craig [Joseph McKean added beginning with 2005 6th edition] (1959, 1965, 1970, 1978, 1995, 2005, 2013, 2019). The 1978 4th edition was used for a popular 2-semester "elective" course by math graduate students when I was at Indiana University (1982-84), although I didn't take it myself (I did buy a copy of the text however), and it seems to be fairly widely used and well known. Only Riemann integration is used, no measure theory or Lebesgue integration is needed. Dec 11, 2020 at 19:18
• You might see if Statistical Inference by Casella and Berger meets your needs. Dec 11, 2020 at 20:41
• See stats.stackexchange.com/questions/56385/… (and further links there) and mathoverflow.net/questions/31655/statistics-for-mathematicians Dec 23, 2020 at 19:10
• I seem to have run into the same issue as you. All the texts appear to be either just re-hashes of the same thing over and over again (many books which will state what various different distributions are, etc) or they are about things like measure theory and only really suitable reading for someone who has a masters in mathematics. (I, unfortunatly, don't.) In my personal situation, I wanted more detail about parameter estimation in the context of likelihood and log likelihood estimation methods. (Specifically relating to applications of numerical methods.) Nov 4 at 21:29
• I don't think this is a particularly niche area - we live in a world where big data has been a thing for well over a decade, and there are plenty of scientific disciplines and areas of industry where these kind of techniques have application. The other issue appears to be with machine learning techniques, which are essentially something else in their own right, and usually just considered to be a magic black box which produces sensible answers provided a bit of work to prevent over-fitting is done. Nov 4 at 21:31

I am not familiar with this book, but the title alone suggests it might be worth examining for your purposes.

Statistics for Mathematicians: A Rigorous First Course. Victor M. Panaretos. Compact Textbook in Mathematics. Birkhäuser/Springer 142 (2016). ISBN-10 : 9783319283395. Springer link.

"Intended for students of Mathematics taking their first course in Statistics, the focus is on Statistics for Mathematicians rather than on Mathematical Statistics." • How rigorous is it? Does it correctly and precisely delineate the meaning of the various hypothesis tests? For example, that rejecting the null hypothesis at 95% confidence level does not imply that one should have 95% confidence in the non-null hypothesis? Dec 23, 2020 at 14:41
• "This chapter considers the problem of hypothesis testing. Starting from first principles, it develops the Neyman-Pearson framework, and discusses different forms of hypothesis pairs. It then uses the question of optimality of tests as a means for motivating specific test functions. This is done in a general setup for simple-vs-simple hypotheses, and in an exponential family context for one-sided alternatives. The likelihood ratio method and Wald’s method are then introduced, ... The chapter concludes with the introduction of p-values,.. and their relation to the Neyman-Pearson paradigm." Dec 24, 2020 at 0:49
• I don't doubt the mathematics. What I was asking about was the use of the stated mathematical theorems applied to real-world statistical analysis. Especially things like I brought up in the (second) question in my above comment. (Related cartoon: green jelly beans.) Dec 24, 2020 at 3:20
• If you don't believe me that there is a significant issue to be addressed here, I would like to hear your own attempt at a precise statement of what can be deduced from a 1-tailed hypothesis test that has been actually performed on some sample data resulting in a p-value less than 0.05, and we can go from there. =) Dec 24, 2020 at 3:24
• @user21820 These concerns don't have anything to do with mathematical, but with statistical rigor. (Your particular concern is, to be clear, addressed in any halfway-decent statistics book, not that many students remember this.) If that's the kind of issue you have then you should avoid a book written for mathematicians, which is likely to have virtually no material on connecting the mathematics to real-world statistical analysis. Jan 11, 2021 at 14:47

I would suggest "All of Statistics" by Wasserman. It is reasonably concise and moderate in its demands on background, but much more mathematically serious, also covering a much wider range of material, than a typical first course in statistics.

• I have to disagree strongly with this suggestion. This book is nothing more than a short series of notes which at best touch on a (broad) range of areas in Statistics. It is fine if you want to quickly look something up as a reference, but it barely scratches the surface in terms of detail. That also makes it hard to understand if you are not already familiar with the concepts. It's a short book for a reason, I don't personally think it has much value to offer. Nov 4 at 21:25

As a student of mathematics taking a statistics course next semester, I have been plagued with this problem. I have found a solution in "Lectures on Probability Theory and Mathematical Statistics" by Marco Taboga, which is a collection of many lectures and exercises from the website https://www.statlect.com/.

Kevin Arlin already gave my "best" answer in All of Statistics. That said, statistics is a large topic. If you're looking specifically at inference and statistical learning and want a rigorous exposition of most of the different methodologies used I would recommend Elements of Statistical Learning, although it assumes a heavier mathematics background than you might expect (in particular, matrix differentiation and probability).

Harald Cramer, Mathematical Methods of Statistics. Princeton

First Part: MATHEMATICAL INTRODUCTION

Chapter 1. General properties of sets Chapter 2. Linear point sets Chapter 3. Point sets in η dimensions Chapter 4. The Lebesgue measure of a linear point set Chapter 5. The Lebesgue integral for functions of one variable.
Chapter 6. Non-negative additive set functions in R Chapter 7. The Lebesgue-Stieltjes integral for functions of one variable Chapter 8. Lebeegue measure and other additive set functions in R Chapter 9. The Lebesgue-Stieltjes integral for functions of n variables Chapter 10. Fourier integrals
Chapter 11. Matrices, determinants and quadratic forms Chapter 12. Miscellaneous complements

Second Part : RANDOM VARIABLES AND PROBABILITY DISTRIBUTIONS.

Chapter 13. Statistics and probability Chapter 14. Fundamental definitions and axioms Chapter 15. General properties Chapter 16. Various discrete distributions Chapter 17. The normal distribution Chapter 18. Various distributions related to the normal Chapter 19. Further continuous distributions Chapter 20. Some convergence theorems Chapter 21. The two-dimensional case Chapter 22. General properties of distributions in R¬n Chapter 23. Regression and correlation in η variables
Chapter 24. The normal distribution

Third Part. STATISTICAL INFERENCE.

Chapter 25. Preliminary notions on sampling Chapter 26. Statistical inference Chapter 27. Characteristics of sampling distributions Chapter 28. Asymptotic properties of Sampling distributions Chapter 29. Exact sampling distributions Chapter 30. Tests of goodness of fit and allied tests Chapter 31. Tests of significance for parameters Chapter 32. Classification of estimates Chapter 33. Methods of estimation Chapter 34. Confidence regions Chapter 35. General theory of testing statistical hypotheses Chapter 36. Analysis of variance Chapter 37. Some regression problems

Late answer, but I feel that the suggestion from Steve in a comment (a few hours after the question was posted) deserves to be promoted to an answer.

Casella, George, and Roger L. Berger. Statistical inference. Cengage Learning, 2021.

Arguably, this may be the standard text for a mathematically rigorous treatment of statistics. It's what I used in graduate school; and I keep it close by on my bookshelf and refer to it frequently when I teach undergraduate statistics. It's sufficiently prominent to have spawned several prior questions on SE Mathematics.