High-dimensional Statistics: Prediction, Association and Causal Inference

This tutorial surveys methodology and theory for high-dimensional statistical inference when the number of variables or features greatly exceeds sample size. Particular emphasis will be placed on problems of model and feature selection. This includes variable selection in regression models or estimation of the edge set in graphical modeling. While the former is concerned with association, the latter can be used for causal analysis. In the high-dimensional setting, major challenges include designing computational algorithms that are feasible for large-scale problems, assigning statistical error rates (e.g., p-values), and developing theoretical insights about the limits of what is possible. We will present some of the most important recent developments and discuss their implications for prediction, association analysis and some exciting new directions in causal inference.