Recent work on learning in multi-agent systems (MAS) is concerned with the ability of self-interested agents to learn cooperative behavior. In many settings such as resource allocation tasks the lack of cooperative behavior can be seen as a consequence of wrong incentives. I.e., when agents can not freely exchange their resources then greediness is not uncooperative but only a consequence of reward maximization. In this work, we show how the introduction of markets helps to reduce the negative effects of individual reward maximization. To study the emergence of trading behavior in MAS we use Deep Reinforcement Learning (RL) where agents are self-interested, independent learners represented through Deep Q-Networks (DQNs). Specifically, we propose Action Traders, referring to agents that can trade their atomic actions in exchange for environmental reward. For empirical evaluation we implemented action trading in the Coin Game -- and find that trading significantly increases social efficiency in terms of overall reward compared to agents without action trading.
@inproceedings{ schmidICANN18,
author = "Kyrill Schmid and Lenz Belzner and Thomas Gabor and Thomy Phan",
title = "Action Markets in Deep Multi-Agent Reinforcement Learning",
year = "2018",
abstract = "Recent work on learning in multi-agent systems (MAS) is concerned with the ability of self-interested agents to learn cooperative behavior. In many settings such as resource allocation tasks the lack of cooperative behavior can be seen as a consequence of wrong incentives. I.e., when agents can not freely exchange their resources then greediness is not uncooperative but only a consequence of reward maximization. In this work, we show how the introduction of markets helps to reduce the negative effects of individual reward maximization. To study the emergence of trading behavior in MAS we use Deep Reinforcement Learning (RL) where agents are self-interested, independent learners represented through Deep Q-Networks (DQNs). Specifically, we propose Action Traders, referring to agents that can trade their atomic actions in exchange for environmental reward. For empirical evaluation we implemented action trading in the Coin Game -- and find that trading significantly increases social efficiency in terms of overall reward compared to agents without action trading.",
url = "https://link.springer.com/chapter/10.1007/978-3-030-01421-6_24",
eprint = "https://thomyphan.github.io/files/2018-icann.pdf",
publisher = "Springer International Publishing",
booktitle = "Artificial Neural Networks and Machine Learning",
pages = "240--249",
doi = "https://doi.org/10.1007/978-3-030-01421-6_24"
}
Related Articles
- T. Phan et al., “Emergent Cooperation from Mutual Acknowledgment Exchange in Multi-Agent Reinforcement Learning”, JAAMAS 2024
- M. Koelle et al., “Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimization, ICAART 2024
- T. Phan et al., “Emergent Cooperation from Mutual Acknowledgment Exchange”, AAMAS 2022
- L. Belzner et al., “The Sharer’s Dilemma in Collective Adaptive Systems of Self-Interested Agents”, ISoLA 2018
- T. Phan, “Emergence and Resilience in Multi-Agent Reinforcement Learning”, PhD Thesis
Relevant Research Areas
