Analyzing and Linking Big Data with Stratosphere

Linking and Analyzing Big Data Summary of the presentation: In this talk, I will provide an overview of two projects at TU Berlin, and the research and innovation challenges in their intersection. Stratosphere (www.stratosphere.eu, funded by the German Research Foundation) is an open platform for Big Data Analytics. It features a cloud-enabled execution engine with flexible fault tolerance schemes, a novel programming model centered around second-order functions that extends MapReduce, and a cost-based query optimizer. Stratosphere is validated by several use-case scenarios, including climate data analysis, text mining in the Bioinformatics, and data cleansing on Linked Open Data. DOPA (an FP7 STREP project) focuses on linking large Data Pools of both structured and unstructured data using data supply chains. The goal is to multiply the utility of each individual service while simultaneously sharing the costs between them. This way DOPA lowers the barrier of entry for SMEs that need to perform advanced analytics across multiple data pools since the required input data as well as the processing environment do not have to be provided by the SME itself.