Competitive Multi-Agent Reinforcement Learning with Self-Supervised Representation

We present MASRL: Competitive Multi-Agent Self-supervised representations for Reinforcement Learning in the multi-agent competitive environment. MASRL introduces a simple but effective self-supervised task: predicting a learning agent’s opponent’s future move. In doing this, the agent learns a stronger representation from this additional signal, focusing not only on itself but also on its opponent. By understanding and anticipating the opponent’s future moves, MASRL allows the learning agent to develop effective strategies for opponent exploitation. Our method stabilizes training, improves sample efficiency, and allows the agent to generalize and adapt its playing strategy to other unseen expert opponents. On the Multi-Agent Atari benchmark, MASRL achieves remarkable performance, outperforming other strong baselines. Examples of demo videos can be found at: https://sites.google.com/view/compmarl