Internals of an Aggregated Web News Feed

We present a pipeline for acquiring a clean, continuous, real-time aggregated stream of publically available news articles from web sites across the world. The articles are stripped of the web page chrome and semantically enriched to include e.g. a list of entities appearing in each article. The results are cached and distributed in an efficient manner.