In collective adaptive systems (CAS), adaptation can be implemented by optimization wrt. utility. Agents in a CAS may be self-interested, while their utilities may depend on other agents’ choices. Independent optimization of agent utilities may yield poor individual and global reward due to locally interfering individual preferences. Joint optimization may scale poorly, and is impossible if agents cannot expose their preferences due to privacy or security issues.In this paper, we study utility sharing for mitigating this issue. Sharing utility with others may incentivize individuals to consider choices that are locally suboptimal but increase global reward. We illustrate our approach with a utility sharing variant of distributed cross entropy optimization. Empirical results show that utility sharing increases expected individual and global payoff in comparison to optimization without utility sharing.We also investigate the effect of greedy defectors in a CAS of sharing, self-interested agents. We observe that defection increases the mean expected individual payoff at the expense of sharing individuals’ payoff. We empirically show that the choice between defection and sharing yields a fundamental dilemma for self-interested agents in a CAS.
@inproceedings{ belznerISoLA18,
author = "Lenz Belzner and Kyrill Schmid and Thomy Phan and Thomas Gabor and Martin Wirsing",
title = "The Sharer's Dilemma in Collective Adaptive Systems of Self-Interested Agents",
year = "2018",
abstract = "In collective adaptive systems (CAS), adaptation can be implemented by optimization wrt. utility. Agents in a CAS may be self-interested, while their utilities may depend on other agents’ choices. Independent optimization of agent utilities may yield poor individual and global reward due to locally interfering individual preferences. Joint optimization may scale poorly, and is impossible if agents cannot expose their preferences due to privacy or security issues.In this paper, we study utility sharing for mitigating this issue. Sharing utility with others may incentivize individuals to consider choices that are locally suboptimal but increase global reward. We illustrate our approach with a utility sharing variant of distributed cross entropy optimization. Empirical results show that utility sharing increases expected individual and global payoff in comparison to optimization without utility sharing.We also investigate the effect of greedy defectors in a CAS of sharing, self-interested agents. We observe that defection increases the mean expected individual payoff at the expense of sharing individuals’ payoff. We empirically show that the choice between defection and sharing yields a fundamental dilemma for self-interested agents in a CAS.",
url = "https://link.springer.com/chapter/10.1007/978-3-030-03424-5_16",
eprint = "https://arxiv.org/pdf/1804.10781.pdf",
location = "Limassol, Cyprus",
publisher = "Springer-Verlag",
booktitle = "International Symposium on Leveraging Applications of Formal Methods, Verification and Validation",
pages = "241--256",
doi = "https://doi.org/10.1007/978-3-030-03424-5_16"
}
Related Articles
- T. Phan et al., “Emergent Cooperation from Mutual Acknowledgment Exchange in Multi-Agent Reinforcement Learning”, JAAMAS 2024
- M. Koelle et al., “Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimization, ICAART 2024
- T. Phan et al., “Emergent Cooperation from Mutual Acknowledgment Exchange”, AAMAS 2022
- R. Roch et al., “Cross Entropy Hyperparameter Optimization for Constrained Problem Hamiltonians Applied to QAOA”, ICRC 2020
- R. Roch et al., “A Quantum Annealing Algorithm for Finding Pure Nash Equilibria in Graphical Games”, ICCS 2020
- K. Schmid et al., “Action Markets in Deep Multi-Agent Reinforcement Learning”, ICANN 2018
- T. Phan, “Emergence and Resilience in Multi-Agent Reinforcement Learning”, PhD Thesis
Relevant Research Areas
