Deep Natural Language Understanding

In this lecture, I start with a claim that natural language understanding can largely be approached as building a better language model and explain three widely-adopted approaches to language modelling. They are n-gram language modelling, feedforward neural language modelling and recurrent language modelling. As I develop from the traditional n-gram language model toward recurrent language model, I discuss the concepts of data sparsity and generalization via continuous space representations. I then continue on to the recent development of a novel paradigm in machine translation based on recurrent language modelling, often called neural machine translation. The lecture concludes with three new opportunities in natural language processing/understanding made possible by the introduction of continuous space representations in deep neural networks.