Discriminative and Generative Views of Binary Experiments

We consider Binary experiments (supervised learning problems where there are two different labels) and explore formal relationships between two views of them, which we call “generative” and “discriminative”. The discriminative perspective involves an expected loss. The generative perspective (in our sense) involves the distances between class-conditional distributions. We extend known results to the class of all proper losses (scoring rules) and all f-divergences as distances between distributions. We also sketch how one can derive the SVM and MMD algorithms from the generative perspective.