Faster Rates via Active Learning

Traditional sampling and statistical learning theories deal with data collection processes that are completely independent of the target function to be estimated, aside from possible a priori specifications reflective of assumed properties of the target. We refer to such processes as passive learning methods. Alternatively, one can envision sequential, adaptive data collection procedures that use information gleaned from previous observations to guide the process. We refer to such feedback-driven processes as active learning methods. While there have been many successful practical applications of active learning, there is scant theoretical evidence to support the effectiveness of active over passive learning. This talk covers some of the most encouraging theoretical results to date, and focuses on new results regarding the capabilities of active methods for learning (nonparametric) smooth and piecewise smooth functions. Significantly faster rates of error convergence are achieved by active learning compared to passive learning in cases involving functions whose complexity is highly concentrated within small regions its domain (e.g., functions that are smoothly varying apart from highly localized abrupt changes such as jumps or edges). This is joint work with Rui Castro and Rebecca Willett. Please see our on-line technical report for further details: http://www.ece.wisc.edu/~nowak/ECE-05-03.pdf