From activity to language: learning to recognise the meaning of motion

Whether the task is recognising an atomic action of an individual or their implied activity, the continuous multichannel nature of sign language recognition or the appearance of words on the lips, all approaches can be categorised at the most basic level as the learning and recognition of spatio-temporal patterns. The complexity of the problem to some extend dictates the approaches used e.g. for simple gestures and actions, holistic descriptors can work well but are less suitable for the subtitles and complexities of sign. However, in all cases, inaccuracies in labelling and the curse of dimensionality lead us to explore new learning approaches that can operate in a weakly supervised setting. This talk will concentrate on 3 areas covering the spectrum of spatiotemporal recognition from whole body through to subtle motions of body parts covering activity and gesture, sign and expression and finally lip reading. In each case, feature and learning approaches will be presented which are suited to both the inherent variability as well as the complexity of the problem. **//Disclaimer:// There may be mistakes or omissions in the interpretation as the interpreters are not experts in the field of interest and performed a simultaneous translation without comprehensive preparation.**