Multi-Agent Reinforcement Learning is becoming increasingly more important in times of autonomous driving and other smart industrial applications. Simultaneously a promising new approach to Reinforcement Learning arises using the inherent properties of quantum mechanics, reducing the trainable parameters of a model significantly. However, gradient-based Multi-Agent Quantum Reinforcement Learning methods often have to struggle with barren plateaus, holding them back from matching the performance of classical approaches. We build upon an existing approach for gradient free Quantum Reinforcement Learning and propose three genetic variations with Variational Quantum Circuits for Multi-Agent Reinforcement Learning using evolutionary optimization. We evaluate our genetic variations in the Coin Game environment and also compare them to classical approaches. We showed that our Variational Quantum Circuit approaches perform significantly better compared to a neural network with a similar amount of trainable parameters. Compared to the larger neural network, our approaches archive similar results using 97.88% less parameters.
@inproceedings{ koelleICAART24,
author = "Michael Kölle and Felix Topp and Thomy Phan and Philipp Altmann and Jonas Nüßlein and Claudia Linnhoff-Popien",
title = "Multi-Agent Quantum Reinforcement Learning using Evolutionary Optimization",
year = "2024",
abstract = "Multi-Agent Reinforcement Learning is becoming increasingly more important in times of autonomous driving and other smart industrial applications. Simultaneously a promising new approach to Reinforcement Learning arises using the inherent properties of quantum mechanics, reducing the trainable parameters of a model significantly. However, gradient-based Multi-Agent Quantum Reinforcement Learning methods often have to struggle with barren plateaus, holding them back from matching the performance of classical approaches. We build upon an existing approach for gradient free Quantum Reinforcement Learning and propose three genetic variations with Variational Quantum Circuits for Multi-Agent Reinforcement Learning using evolutionary optimization. We evaluate our genetic variations in the Coin Game environment and also compare them to classical approaches. We showed that our Variational Quantum Circuit approaches perform significantly better compared to a neural network with a similar amount of trainable parameters. Compared to the larger neural network, our approaches archive similar results using 97.88% less parameters.",
url = "https://www.scitepress.org/Papers/2024/123828/123828.pdf",
eprint = "https://thomyphan.github.io/files/2024-icaart-preprint.pdf",
publisher = "SciTePress",
booktitle = "Proceedings of the 16th International Conference on Agents and Artificial Intelligence",
pages = "71--82",
doi = "https://doi.org/10.5220/0012382800003636"
}
Related Articles
- T. Phan et al., “Emergent Cooperation from Mutual Acknowledgment Exchange in Multi-Agent Reinforcement Learning”, JAAMAS 2024
- T. Phan et al., “Emergent Cooperation from Mutual Acknowledgment Exchange”, AAMAS 2022
- R. Roch et al., “A Quantum Annealing Algorithm for Finding Pure Nash Equilibria in Graphical Games”, ICCS 2020
- L. Belzner et al., “The Sharer’s Dilemma in Collective Adaptive Systems of Self-Interested Agents”, ISoLA 2018
- K. Schmid et al., “Action Markets in Deep Multi-Agent Reinforcement Learning”, ICANN 2018
Relevant Research Areas