Reinforcement Learning From Hierarchical Critics
Files
(Published version)
Date
2023
Authors
Cao, Z.
Lin, C.T.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Journal article
Citation
IEEE Transactions on Neural Networks and Learning Systems, 2023; 34(2):1066-1073
Statement of Responsibility
Conference Name
Abstract
In this study, we investigate the use of global information to speed up the learning process and increase the cumulative rewards of reinforcement learning (RL) in competition tasks. Within the framework of actor-critic RL, we introduce multiple cooperative critics from two levels of a hierarchy and propose an RL from the hierarchical critics (RLHC) algorithm. In our approach, each agent receives value information from local and global critics regarding a competition task and accesses multiple cooperative critics in a top-down hierarchy. Thus, each agent not only receives low-level details, but also considers coordination from higher levels, thereby obtaining global information to improve the training performance. Then, we test the proposed RLHC algorithm against a benchmark algorithm, that is, proximal policy optimization (PPO), under four experimental scenarios consisting of tennis, soccer, banana collection, and crawler competitions within the Unity environment. The results show that RLHC outperforms the benchmark on these four competitive tasks.
School/Discipline
Dissertation Note
Provenance
Description
Access Status
Rights
Copyright 2021 IEEE
Access Condition Notes: Accepted manuscript available on Open Access