A modular approach to multi-agent reinforcement learning

Abstract

Several attempts have been reported to let multiple monolithic reinforcement-learning agents synthesize coordinated decision policies needed to accomplish their common goal effectively. Most of these straightforward reinforcement-learning approaches, however, scale poorly to more complex multi-agent learning problems, because the state space for each learning agent grows exponentially in the number of its partner agents engaged in the joint task. To remedy the exponentially large state space in multi-agent reinforcement learning, we previously proposed a modular approach and demonstrated its effectiveness through the application to a modified version of the pursuit problem. In this paper, the effectiveness of the proposed idea is further demonstrated using several variants of the pursuit problem. Just as in the previous case, our modular Q-learning hunters can successfully capture a randomly-evading prey agent, by synthesizing and taking advantage of effective coordinated behavior.