Probability and Mathematical Needs

This lectures covers basics in linear algebra and probabilities as well as a brief introduction to optimization. In linear algebra, the lecture starts with the definition of vectors spaces, dimension, basis, span of vectors and so forth. Norms and dot products as well as Hilbert spaces are introduced. Then the problem of solving linear system is tackled, introducing matrices, eigenvalues... and some common factorization (SVD, LU, Choleski, QR). In probabilities, we start from the definition of discrete and continuous random variables, give common examples, introduce the concepts of independence and conditional probabilities. We tackle estimation through Bayes framework, give the basic definitions in information theory (entropy, Kullback-Leibler divergence) and introduce error bounds (Hoeffding bounds). Optimization is briefly introduced defining extrema and convex functions. An example of constrained minimization is demonstrated.