RL-Glue is a protocol and software implementation for evaluating reinforcement learning algorithms. Our system facilitates the comparison of alternative algorithms and can greatly accelerate research