Machine Learning

We provide a general introduction to machine learning, aimed to put all participants on the same page in terms of definitions and basic background. After a brief overview of different machine learning problems, we discuss linear regression, its objective function and closed-form solution. We discuss the bias-variance trade-off and the issue of overfitting (and the proper use of cross-validation to measure performance objectively). We discuss the probabilistic view of the sum-squared error as maximizing likelihood under specific assumptions on the data generation process, and present L2 and L1 regularization methods as priors from a Bayesian perspective. We briefly discuss Bayesian methodology for learning. Finally, we present logistic regression, the cross-entropy optimization criterion and its solution through first- and second-order methods.