NIPS Workshop on Dynamical Systems, Stochastic Processes and Bayesian Inference, Whistler 2006

The modelling of continuous-time dynamical systems from uncertain observations is an important task that comes up in a wide range of applications ranging from numerical weather prediction over finance to genetic networks and motion capture in video. Often, we may assume that the dynamical models are formulated by systems of differential equations. In a Bayesian approach, we may then incorporate a priori knowledge about the dynamics by providing probability distributions on the unknown functions, which correspond for example to driving forces and appear as coefficients or parameters in the differential equations. Hence, such functions become stochastic processes in a probabilistic Bayesian framework.

Gaussian processes (GPs) provide a natural and flexible framework in such circumstances. The use of GPs in the learning of functions from data is now a well-established technique in Machine Learning. Nevertheless, their application to dynamical systems becomes highly nontrivial when the dynamics is nonlinear in the (Gaussian) parameter functions. This happens naturally for nonlinear systems which are driven by a Gaussian noise process, or when the nonlinearity is needed to provide necessary constraints (e.g., positivity) for the parameter functions. In such a case, the prior process over the system's dynamics is non-Gaussian right from the start. This means, that closed form analytical posterior predictions (even in the case of Gaussian observation noise) are no longer possible. Moreover, their computation requires the entire underlying Gaussian latent process at all times (not just at the discrete observation times). Hence, inference of the dynamics would require nontrivial sampling methods or approximation techniques.

Detailed information can be found at Dynamical Systems, Stochastic Processes and Bayesian Inference website.