Peer incentivization (PI) is a recent approach, where all agents learn to reward or to penalize each other in a distributed fashion which often leads to emergent cooperation. Current PI mechanisms implicitly assume a flawless communication channel in order to exchange rewards. These rewards are directly integrated into the learning process without any chance to respond with feedback. Furthermore, most PI approaches rely on global information which limits scalability and applicability to real-world scenarios, where only local information is accessible. In this paper, we propose Mutual Acknowledgment Token Exchange (MATE), a PI approach defined by a two-phase communication protocol to mutually exchange acknowledgment tokens to shape individual rewards. Each agent evaluates the monotonic improvement of its individual situation in order to accept or reject acknowledgment requests from other agents. MATE is completely decentralized and only requires local communication and information. We evaluate MATE in three social dilemma domains. Our results show that MATE is able to achieve and maintain significantly higher levels of cooperation than previous PI approaches. In addition, we evaluate the robustness of MATE in more realistic scenarios, where agents can defect from the protocol and where communication failures can occur.
@inproceedings{ phanAAMAS22,
author = "Thomy Phan and Felix Sommer and Philipp Altmann and Fabian Ritz and Lenz Belzner and Claudia Linnhoff-Popien",
title = "Emergent Cooperation from Mutual Acknowledgment Exchange",
year = "2022",
abstract = "Peer incentivization (PI) is a recent approach, where all agents learn to reward or to penalize each other in a distributed fashion which often leads to emergent cooperation. Current PI mechanisms implicitly assume a flawless communication channel in order to exchange rewards. These rewards are directly integrated into the learning process without any chance to respond with feedback. Furthermore, most PI approaches rely on global information which limits scalability and applicability to real-world scenarios, where only local information is accessible. In this paper, we propose Mutual Acknowledgment Token Exchange (MATE), a PI approach defined by a two-phase communication protocol to mutually exchange acknowledgment tokens to shape individual rewards. Each agent evaluates the monotonic improvement of its individual situation in order to accept or reject acknowledgment requests from other agents. MATE is completely decentralized and only requires local communication and information. We evaluate MATE in three social dilemma domains. Our results show that MATE is able to achieve and maintain significantly higher levels of cooperation than previous PI approaches. In addition, we evaluate the robustness of MATE in more realistic scenarios, where agents can defect from the protocol and where communication failures can occur.",
url = "https://www.ifaamas.org/Proceedings/aamas2022/pdfs/p1047.pdf",
eprint = "https://thomyphan.github.io/files/2022-aamas.pdf",
location = "Virtual Event, New Zealand",
publisher = "International Foundation for Autonomous Agents and Multiagent Systems",
booktitle = "Proceedings of the 21st International Conference on Autonomous Agents and MultiAgent Systems",
pages = "1047--1055",
keywords = "mutual acknowledgments, emergent cooperation, peer incentivization, reinforcement learning, multi-agent learning",
doi = "https://dl.acm.org/doi/abs/10.5555/3535850.3535967"
}
Related Articles
- T. Phan et al., “Emergent Cooperation from Mutual Acknowledgment Exchange in Multi-Agent Reinforcement Learning”, JAAMAS 2024 (journal version)
- M. Koelle et al., “Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimization, ICAART 2024
- L. Belzner et al., “The Sharer’s Dilemma in Collective Adaptive Systems of Self-Interested Agents”, ISoLA 2018
- K. Schmid et al., “Action Markets in Deep Multi-Agent Reinforcement Learning”, ICANN 2018
- T. Phan, “Emergence and Resilience in Multi-Agent Reinforcement Learning”, PhD Thesis
Relevant Research Areas
