Distributed reinforcement learning (DRL) has been studied as an approach to learn control policies thorough interactions between distributed agents and environments. The main emphasis of DRL has been