Policy Gradient With Value Function Approximation For Collective Multiagent Planning
Decentralized (PO)MDPs provide an expressive framework for sequential decision making in a multiagent system. Given their computational complexity, recent research has focused on tractable yet practical subclasses of Dec-POMDPs. We address such a subclass called CDEC-POMDP where the collective behav...
Guardat en:
| Autors principals: | , , |
|---|---|
| Format: | Article |
| Publicat: |
2018
|
| Matèries: | |
| Accés en línia: | https://arxiv.org/abs/1804.02884 http://arxiv.org/abs/1804.02884 |
| Etiquetes: |
Sense etiquetes, Sigues el primer a etiquetar aquest registre!
|
| _version_ | 1879179340577505280 |
|---|---|
| author | Nguyen, Duc Thien Kumar, Akshat Lau, Hoong Chuin |
| author_browse | Kumar, Akshat Lau, Hoong Chuin Nguyen, Duc Thien |
| author_facet | Nguyen, Duc Thien Kumar, Akshat Lau, Hoong Chuin |
| author_sort | Nguyen, Duc Thien |
| building | arXiv |
| collection | Acceso abierto |
| description | Decentralized (PO)MDPs provide an expressive framework for sequential decision making in a multiagent system. Given their computational complexity, recent research has focused on tractable yet practical subclasses of Dec-POMDPs. We address such a subclass called CDEC-POMDP where the collective behavior of a population of agents affects the joint-reward and environment dynamics. Our main contribution is an actor-critic (AC) reinforcement learning method for optimizing CDEC-POMDP policies. Vanilla AC has slow convergence for larger problems. To address this, we show how a particular decomposition of the approximate action-value function over agents leads to effective updates, and also derive a new way to train the critic based on local reward signals. Comparisons on a synthetic benchmark and a real-world taxi fleet optimization problem show that our new AC approach provides better quality solutions than previous best approaches. |
| first_indexed | 2026-10-16T01:31:06Z |
| format | Article |
| fullrecord | <oai_dc:dc xmlns:oai_dc="http://www.openarchives.org/OAI/2.0/oai_dc/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/oai_dc/ https://www.openarchives.org/OAI/2.0/oai_dc.xsd"><identifier>arxiv-1804.02884</identifier><datestamp>2026-06-04</datestamp>
<dc:title>Policy Gradient With Value Function Approximation For Collective Multiagent Planning</dc:title>
<dc:creator>Nguyen, Duc Thien</dc:creator>
<dc:creator>Kumar, Akshat</dc:creator>
<dc:creator>Lau, Hoong Chuin</dc:creator>
<dc:subject>Artificial Intelligence</dc:subject>
<dc:subject>Machine Learning</dc:subject>
<dc:subject>Multiagent Systems</dc:subject>
<dc:subject>Neural and Evolutionary Computing</dc:subject>
<dc:subject>Systems and Control</dc:subject>
<dc:description>Decentralized (PO)MDPs provide an expressive framework for sequential decision making in a multiagent system. Given their computational complexity, recent research has focused on tractable yet practical subclasses of Dec-POMDPs. We address such a subclass called CDEC-POMDP where the collective behavior of a population of agents affects the joint-reward and environment dynamics. Our main contribution is an actor-critic (AC) reinforcement learning method for optimizing CDEC-POMDP policies. Vanilla AC has slow convergence for larger problems. To address this, we show how a particular decomposition of the approximate action-value function over agents leads to effective updates, and also derive a new way to train the critic based on local reward signals. Comparisons on a synthetic benchmark and a real-world taxi fleet optimization problem show that our new AC approach provides better quality solutions than previous best approaches. </dc:description>
<dc:date>2018-04-09</dc:date>
<dc:type>text</dc:type>
<dc:identifier>https://arxiv.org/abs/1804.02884</dc:identifier>
<dc:identifier>http://arxiv.org/abs/1804.02884</dc:identifier>
</oai_dc:dc> |
| id | arxiv-1804.02884 |
| institution | arXiv |
| last_indexed | 2026-10-16T01:31:06Z |
| publishDate | 2018 |
| publishDateRange | 2018 |
| publishDateSort | 2018 |
| record_format | oai_dc |
| spelling | arxiv-1804.028842026-06-04 Policy Gradient With Value Function Approximation For Collective Multiagent Planning Nguyen, Duc Thien Kumar, Akshat Lau, Hoong Chuin Artificial Intelligence Machine Learning Multiagent Systems Neural and Evolutionary Computing Systems and Control Decentralized (PO)MDPs provide an expressive framework for sequential decision making in a multiagent system. Given their computational complexity, recent research has focused on tractable yet practical subclasses of Dec-POMDPs. We address such a subclass called CDEC-POMDP where the collective behavior of a population of agents affects the joint-reward and environment dynamics. Our main contribution is an actor-critic (AC) reinforcement learning method for optimizing CDEC-POMDP policies. Vanilla AC has slow convergence for larger problems. To address this, we show how a particular decomposition of the approximate action-value function over agents leads to effective updates, and also derive a new way to train the critic based on local reward signals. Comparisons on a synthetic benchmark and a real-world taxi fleet optimization problem show that our new AC approach provides better quality solutions than previous best approaches. 2018-04-09 text https://arxiv.org/abs/1804.02884 http://arxiv.org/abs/1804.02884 |
| spellingShingle | Artificial Intelligence Machine Learning Multiagent Systems Neural and Evolutionary Computing Systems and Control Nguyen, Duc Thien Kumar, Akshat Lau, Hoong Chuin Policy Gradient With Value Function Approximation For Collective Multiagent Planning |
| title | Policy Gradient With Value Function Approximation For Collective Multiagent Planning |
| title_full | Policy Gradient With Value Function Approximation For Collective Multiagent Planning |
| title_fullStr | Policy Gradient With Value Function Approximation For Collective Multiagent Planning |
| title_full_unstemmed | Policy Gradient With Value Function Approximation For Collective Multiagent Planning |
| title_short | Policy Gradient With Value Function Approximation For Collective Multiagent Planning |
| title_sort | policy gradient with value function approximation for collective multiagent planning |
| topic | Artificial Intelligence Machine Learning Multiagent Systems Neural and Evolutionary Computing Systems and Control |
| url | https://arxiv.org/abs/1804.02884 http://arxiv.org/abs/1804.02884 |
| work_keys_str_mv | AT nguyenducthien policygradientwithvaluefunctionapproximationforcollectivemultiagentplanning AT kumarakshat policygradientwithvaluefunctionapproximationforcollectivemultiagentplanning AT lauhoongchuin policygradientwithvaluefunctionapproximationforcollectivemultiagentplanning |