From linearly-solvable optimal control to trajectory optimization, and (hopefully) back

We have identified a general class of stochastic optimal control problems which are inherently linear, in the sense that the exponentiated optimal value function satisfies a linear equation. These problems have a number of unique properties which enable more efficient numerical methods than generic formulations. However, after several attempts to go beyond the simple numerical examples characteristic of this literature and scale to real-world problems (particularly in robotics), we realized that the curse of dimensionality is still a curse. We then took a detour, and developed trajectory optimization methods that can synthesize remarkably complex behaviors fully automatically. Thanks to the parallel processing capabilities of modern computers, some of these methods work in real time in model-predictive-control (MPC) mode, giving rise to implicitly defined feedback control laws. But not all problems can be solved in this way, and furthermore it would be nice to somehow re-use the local solutions that MPC generates. The next step is to combine the strengths of these two approaches: using trajectory optimization to identify the regions of state space where the optimally-controlled stochastic system is likely to spend its time, and then applying linearly-solvable optimal control restricted to these regions.