Statistics, for the mathematically rigorous

I don't know where I can find a rigorous statistics course or textbook. The closest thing I can think of is measure-theoretic probability theory, but I wouldn't really call that "statistics". By 'close to statistics', I mean something that, although as mathematically rigorous as probability theory, can be reasonably substituted for a statistics course with minimal study.

Does such a thing exist? Perhaps not, as the point of many statistics courses is in their applications.

• Maybe Introduction to Mathematical Statistics by Robert Vincent Hogg and Allen Thornton Craig [Joseph McKean added beginning with 2005 6th edition] (1959, 1965, 1970, 1978, 1995, 2005, 2013, 2019). The 1978 4th edition was used for a popular 2-semester "elective" course by math graduate students when I was at Indiana University (1982-84), although I didn't take it myself (I did buy a copy of the text however), and it seems to be fairly widely used and well known. Only Riemann integration is used, no measure theory or Lebesgue integration is needed. Dec 11 '20 at 19:18
• You might see if Statistical Inference by Casella and Berger meets your needs. Dec 11 '20 at 20:41
• See stats.stackexchange.com/questions/56385/… (and further links there) and mathoverflow.net/questions/31655/statistics-for-mathematicians Dec 23 '20 at 19:10

I am not familiar with this book, but the title alone suggests it might be worth examining for your purposes.

Statistics for Mathematicians: A Rigorous First Course. Victor M. Panaretos. Compact Textbook in Mathematics. Birkhäuser/Springer 142 (2016). ISBN-10 : 9783319283395. Springer link.

"Intended for students of Mathematics taking their first course in Statistics, the focus is on Statistics for Mathematicians rather than on Mathematical Statistics." • How rigorous is it? Does it correctly and precisely delineate the meaning of the various hypothesis tests? For example, that rejecting the null hypothesis at 95% confidence level does not imply that one should have 95% confidence in the non-null hypothesis? Dec 23 '20 at 14:41
• "This chapter considers the problem of hypothesis testing. Starting from first principles, it develops the Neyman-Pearson framework, and discusses different forms of hypothesis pairs. It then uses the question of optimality of tests as a means for motivating specific test functions. This is done in a general setup for simple-vs-simple hypotheses, and in an exponential family context for one-sided alternatives. The likelihood ratio method and Wald’s method are then introduced, ... The chapter concludes with the introduction of p-values,.. and their relation to the Neyman-Pearson paradigm." Dec 24 '20 at 0:49
• I don't doubt the mathematics. What I was asking about was the use of the stated mathematical theorems applied to real-world statistical analysis. Especially things like I brought up in the (second) question in my above comment. (Related cartoon: green jelly beans.) Dec 24 '20 at 3:20
• If you don't believe me that there is a significant issue to be addressed here, I would like to hear your own attempt at a precise statement of what can be deduced from a 1-tailed hypothesis test that has been actually performed on some sample data resulting in a p-value less than 0.05, and we can go from there. =) Dec 24 '20 at 3:24
• @user21820 These concerns don't have anything to do with mathematical, but with statistical rigor. (Your particular concern is, to be clear, addressed in any halfway-decent statistics book, not that many students remember this.) If that's the kind of issue you have then you should avoid a book written for mathematicians, which is likely to have virtually no material on connecting the mathematics to real-world statistical analysis. Jan 11 '21 at 14:47

I would suggest "All of Statistics" by Wasserman. It is reasonably concise and moderate in its demands on background, but much more mathematically serious, also covering a much wider range of material, than a typical first course in statistics.

As a student of mathematics taking a statistics course next semester, I have been plagued with this problem. I have found a solution in "Lectures on Probability Theory and Mathematical Statistics" by Marco Taboga, which is a collection of many lectures and exercises from the website https://www.statlect.com/.

Kevin Arlin already gave my "best" answer in All of Statistics. That said, statistics is a large topic. If you're looking specifically at inference and statistical learning and want a rigorous exposition of most of the different methodologies used I would recommend Elements of Statistical Learning, although it assumes a heavier mathematics background than you might expect (in particular, matrix differentiation and probability).