Qiancheng Wu

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2022-129

May 15, 2022

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-129.pdf

The success of Reinforcement Learning (RL) methods relies heavily on the diversity and quality of learning scenarios generated by the environment. However, while RL methods are applied to increasingly complex environments, the support for environment modeling is lagging behind. This work introduces the Scenic4RL interface that enables researchers to use a probabilistic scenario specification language, Scenic, to intuitively model, specify, and generate complex environments. We interface Scenic with a real-time-strategy game environment, Google Research Football (GRF), to demonstrate the benefits of adopting a formal scenario specification language to assist RL researchers in training, debugging, and evaluating RL policies. Our interface allows researchers to easily model players' initial positions and dynamic behaviors as Scenic scenarios, and with options to customize the environment's reward and termination conditions. In addition, we release a benchmark of mini-game scenarios encoded in Scenic for both the single-agent and multi-agent settings to train and test the agents' generalization abilities. Lastly, we demonstrate that researchers can use the interface to facilitate automated curriculum learning.

Advisors: Sanjit A. Seshia

---

BibTeX citation:

@mastersthesis{Wu:EECS-2022-129,
    Author= {Wu, Qiancheng},
    Title= {Assisting Reinforcement Learning in Real-time Strategy Environments with SCENIC},
    School= {EECS Department, University of California, Berkeley},
    Year= {2022},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-129.html},
    Number= {UCB/EECS-2022-129},
    Abstract= {The success of Reinforcement Learning (RL) methods relies heavily on the diversity and quality of learning scenarios generated by the environment. However, while RL methods are applied to increasingly complex environments, the support for environment modeling is lagging behind. This work introduces the Scenic4RL interface that enables researchers to use a probabilistic scenario specification language, Scenic, to intuitively model, specify, and generate complex environments. We interface Scenic with a real-time-strategy game environment, Google Research Football (GRF), to demonstrate the benefits of adopting a formal scenario specification language to assist RL researchers in training, debugging, and evaluating RL policies. Our interface allows researchers to easily model players' initial positions and dynamic behaviors as Scenic scenarios, and with options to customize the environment's reward and termination conditions. In addition, we release a benchmark of mini-game scenarios encoded in Scenic for both the single-agent and multi-agent settings to train and test the agents' generalization abilities. Lastly, we demonstrate that researchers can use the interface to facilitate automated curriculum learning.},
}

EndNote citation:

%0 Thesis
%A Wu, Qiancheng 
%T Assisting Reinforcement Learning in Real-time Strategy Environments with SCENIC
%I EECS Department, University of California, Berkeley
%D 2022
%8 May 15
%@ UCB/EECS-2022-129
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-129.html
%F Wu:EECS-2022-129