Codalab - A new service for data exchange, code execution, benchmarks and reproducible research

CodaLab is open source platform which lets communities explore Experiments together and create Competitions designed to advance the state of the art. The first two communities helping build CodaLab are the Machine Learning and the Medical Imaging communities. CodaLab Experiments enable collaborative research and computational research to be done in an efficient and reproducible manner. By providing modularity, live execution, and inline annotation of code with rich explanations, CodaLab enables you to quickly sketch ideas and collaborate with fellow community members CodaLab competitions provide an opportunity for researchers and developers to create solutions for problems across a wide range of domains, and advance the state of the art for their respective areas of interest; once the challenge is over, further work can be pursued by the broader community as Experiments on CodaLab. CodaLab is a community-driven effort led by Percy Liang from Stanford University who built the precursor of CodaLab, namely, MLComp. We invite the NIPS community to participate in CodaLab by creating experiments as executable papers and by sharing them with the rest of the community at http://codalab.org These “executable papers” can then be freely reproduced, appended, and otherwise modified to improve productivity and accelerate the pace of discovery and learning among machine learning research professionals.