2006 AllofNonparametricStatistics

• (Wasserman, 2006) ⇒ Larry Wasserman. (2006). “All of Nonparametric Statistics.” Springer.

Subject Headings: Non-Parametric Statistics; Non-Parametric Inference; Non-Parametric Learning Method.

Cited By

• http://scholar.google.com/scholar?q=%222006%22+All+of+Nonparametric+Statistics

Quotes

Book Overview

The goal of this text is to provide the reader with a single book where they can find a brief account of many, modern topics in nonparametric inference. The book is aimed at Master's level or Ph.D. level students in statistics, computer science, and engineering. It is also suitable for researchers who want to get up to speed quickly on modern nonparametric methods.

This text covers a wide range of topics including: the bootstrap, the nonparametric delta method, nonparametric regression, density estimation, orthogonal function methods, minimax estimation, nonparametric confidence sets, and wavelets. The book has a mixture of methods and theory.

Larry Wasserman is Professor of Statistics at Carnegie Mellon University and a member of the Center for Automated Learning and Discovery in the School of Computer Science. His research areas include nonparametric inference, asymptotic theory, multiple testing, and applications to astrophysics, bioinformatics and genetics. He is the 1999 winner of the Committee of Presidents of Statistical Societies Presidents' Award and the 2002 winner of the Centre de recherches mathematiques de Montreal-Statistical Society of Canada Prize in Statistics. He is Associate Editor of The Journal of the American Statistical Association and The Annals of Statistics. He is a fellow of the American Statistical Association and of the Institute of Mathematical Statistics. He is the author of All of Statistics: A Concise Course in Statistical Inference (Springer, 2003).

Table of Contents

1 Introduction 1
1.1 What is Nonparametric Inference? . . . . . . . . . . . . . . . . 1
1.2 Notation and Background . . . . . . . . . . . . . . . . . . . . . 2
1.3 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Useful Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 10
1.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

2 Estimating the cdf and Statistical Functionals 13
2.1 The cdf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Estimating Statistical Functionals . . . . . . . . . . . . . . . . 15
2.3 Influence Functions . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Empirical Probability Distributions . . . . . . . . . . . . . . . . 21
2.5 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 23
2.6 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3 The Bootstrap and The Jackknife 27
3.1 The Jackknife . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 The Bootstrap . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3.3 Parametric Bootstrap . . . . . . . . . . . . . . . . . . . . . . . 31
3.4 Bootstrap Confidence Intervals . . . . . . . . . . . . . . . . . . 32
3.5 Some Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.6 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 37
3.7 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

4 Smoothing: General Concepts 43
4.1 The Bias-Variance Tradeoff . . . . . . . . . . . . . . . . . . . . 50
4.2 Kernels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.3 Which Loss Function? . . . . . . . . . . . . . . . . . . . . . . . 57
4.4 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
4.5 The Curse of Dimensionality . . . . . . . . . . . . . . . . . . . 58
4.6 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 59
4.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5 Nonparametric Regression 61
5.1 Review of Linear and Logistic Regression . . . . . . . . . . . . 62
5.2 Linear Smoothers . . . . . . . . . . . . . . . . . . . . . . . . . . 66
5.3 Choosing the Smoothing Parameter . . . . . . . . . . . . . . . 68
5.4 Local Regression . . . . . . . . . . . . . . . . . . . . . . . . . . 71
5.5 Penalized Regression, Regularization and Splines . . . . . . . . 80
5.6 Variance Estimation . . . . . . . . . . . . . . . . . . . . . . . . 85
5.7 Confidence Bands . . . . . . . . . . . . . . . . . . . . . . . . . . 88
5.8 Average Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . 93
5.9 Summary of Linear Smoothing . . . . . . . . . . . . . . . . . . 94
5.10 Local Likelihood and Exponential Families . . . . . . . . . . . . 95
5.11 Scale-Space Smoothing . . . . . . . . . . . . . . . . . . . . . . . 99
5.12 Multiple Regression . . . . . . . . . . . . . . . . . . . . . . . . 100
5.13 Other Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
5.14 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 119
5.15 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.16 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

6 Density Estimation 125
6.1 Cross-Validation . . . . . . . . . . . . . . . . . . . . . . . . . . 126
6.2 Histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.3 Kernel Density Estimation . . . . . . . . . . . . . . . . . . . . . 131
6.4 Local Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . 137
6.5 Multivariate Problems . . . . . . . . . . . . . . . . . . . . . . . 138
6.6 Converting Density Estimation Into Regression . . . . . . . . . 139
6.7 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 140
6.8 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
6.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

7 Normal Means and Minimax Theory 145
7.1 The Normal Means Model . . . . . . . . . . . . . . . . . . . . . 145
7.2 Function Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
7.3 Connection to Regression and Density Estimation . . . . . . . 149
7.4 Stein's Unbiased Risk Estimator (sure) . . . . . . . . . . . . . 150
7.5 Minimax Risk and Pinsker's Theorem . . . . . . . . . . . . . . 153
7.6 Linear Shrinkage and the James-Stein Estimator . . . . . . . . 155
7.7 Adaptive Estimation Over Sobolev Spaces . . . . . . . . . . . . 158
7.8 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 159
7.9 Optimality of Confidence Sets . . . . . . . . . . . . . . . . . . . 167
7.10 Random Radius Bands? . . . . . . . . . . . . . . . . . . . . . . 170
7.11 Penalization, Oracles and Sparsity . . . . . . . . . . . . . . . . 171
7.12 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 173
7.13 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
7.13.1 The White Noise Model . . . . . . . . . . . . . . . . . . 173
7.13.2 Weak Differentiability . . . . . . . . . . . . . . . . . . . 174
7.13.3 Proof of Pinsker's Theorem (Theorem 7.28). . . . . . . . 174
7.13.4 Proof of Theorem 7.74 . . . . . . . . . . . . . . . . . . . 177
7.13.5 Proof of Theorem 7.77 . . . . . . . . . . . . . . . . . . . 179
7.14 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

8 Nonparametric Inference Using Orthogonal Functions 183
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
8.2 Nonparametric Regression . . . . . . . . . . . . . . . . . . . . . 183
8.3 Irregular Designs . . . . . . . . . . . . . . . . . . . . . . . . . . 191
8.4 Density Estimation . . . . . . . . . . . . . . . . . . . . . . . . . 192
8.5 Comparison of Methods . . . . . . . . . . . . . . . . . . . . . . 193
8.6 Tensor Product Models . . . . . . . . . . . . . . . . . . . . . . 193
8.7 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 194
8.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194

9 Wavelets and Other Adaptive Methods 197
9.1 Haar Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
9.2 Constructing Wavelets . . . . . . . . . . . . . . . . . . . . . . . 203
9.3 Wavelet Regression . . . . . . . . . . . . . . . . . . . . . . . . . 206
9.4 Wavelet Thresholding . . . . . . . . . . . . . . . . . . . . . . . 208
9.5 Besov Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
9.6 Confidence Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
9.7 Boundary Corrections and Unequally Spaced Data . . . . . . . 215
9.8 Overcomplete Dictionaries . . . . . . . . . . . . . . . . . . . . . 216
9.9 Other Adaptive Methods . . . . . . . . . . . . . . . . . . . . . 216
9.10 Do Adaptive Methods Work? . . . . . . . . . . . . . . . . . . . 220
9.11 Bibliographic Remarks . . . . . . . . . . . . . . . . . . . . . . . 221
9.12 Appendix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
9.12.1 Localization of Wavelets . . . . . . . . . . . . . . . . . . 222
9.12.2 Fast Computations for Wavelets . . . . . . . . . . . . . 222
9.13 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

10 Other Topics 227
10.1 Measurement Error . . . . . . . . . . . . . . . . . . . . . . . . . 227
10.2 Inverse Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 233
10.3 Nonparametric Bayes . . . . . . . . . . . . . . . . . . . . . . . . 235
10.4 Semiparametric Inference . . . . . . . . . . . . . . . . . . . . . 235
10.5 Correlated Errors . . . . . . . . . . . . . . . . . . . . . . . . . . 236
10.6 Classification . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
10.7 Sieves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 237
10.8 Shape Restricted Inference . . . . . . . . . . . . . . . . . . . . . 237
10.9 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
10.10 Computational Issues . . . . . . . . . . . . . . . . . . . . . . . 240
10.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240

…

References

;

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2006 AllofNonparametricStatistics	Larry Wasserman			All of Nonparametric Statistics						2006