Application of expectation consistent approximate inference

I will discuss two types of applications of an approximate inference technique (EC = expectation consistent) recently developed together with Ole Winther. The EC method is an extension of the TAP (Thouless, Anderson & Palmer) approach which originated in the field of disordered materials and which has been further developed to become applicable to a variety of scenarios in probabilistic modelling & machine laerning. My first application (joint work with Doerthe Malzahn) deals with an approximation to resampling methods (such as the bootstrap) which allows to estimate eg generalization errors in supervised learning. While the exact resampling approach requires the drawing of many samples from the training data and a costly repeated retraining of the model, the approximation attempts an analytic average which combines the replica trick and an inference method which can be performed much faster. In the second application (ongoing work) I discuss the scenario of many solutions to the EC framework and the possibility of averaging them using Parisi's hierarchical scheme.