We present a novel cross-dataset action recognition framework that utilizes relevant actions from other visual domains as auxiliary knowledge for enhancing the learning system in the target domain.