Kai Xu

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2025-168

August 15, 2025

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2025/EECS-2025-168.pdf

There has been significant recent interest in using reinforcement learning for control in cyber- physical-systems (CPS). Domains affected include autonomous driving, robotics, and drone control. Much of these applications should be considered as safety-critical, where system failure can cause significant damage and injury to humans. Even in non-safety critical applications, such system failures could also be expensive. It is therefore important to be able to add assurance to the process of reinforcement learning, which is a challenge due to the statistical nature of modern learning algorithms. In this thesis, we present ScenicGym, which is an RL training tool based on the probabilistic programming language Scenic and its related toolkit VerifAI. The new tools allow RL researchers to train agents completely using data generated by concise Scenic programs with sampling conducted by VerifAI to incorporate edge cases. We demonstrate the use of ScenicGym and the influence of incorporating VerifAI’s sampler with experiments in autonomous driving.

Advisors: Sanjit A. Seshia

---

BibTeX citation:

@mastersthesis{Xu:EECS-2025-168,
    Author= {Xu, Kai},
    Title= {ScenicGym: Reinforcement Learning with Data Generation Using Scenic},
    School= {EECS Department, University of California, Berkeley},
    Year= {2025},
    Month= {Aug},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2025/EECS-2025-168.html},
    Number= {UCB/EECS-2025-168},
    Abstract= {There has been significant recent interest in using reinforcement learning for control in cyber- physical-systems (CPS). Domains affected include autonomous driving, robotics, and drone control. Much of these applications should be considered as safety-critical, where system failure can cause significant damage and injury to humans. Even in non-safety critical applications, such system failures could also be expensive. It is therefore important to be able to add assurance to the process of reinforcement learning, which is a challenge due to the statistical nature of modern learning algorithms. In this thesis, we present ScenicGym, which is an RL training tool based on the probabilistic programming language Scenic and its related toolkit VerifAI. The new tools allow RL researchers to train agents completely using data generated by concise Scenic programs with sampling conducted by VerifAI to incorporate edge cases. We demonstrate the use of ScenicGym and the influence of incorporating VerifAI’s sampler with experiments in autonomous driving.},
}

EndNote citation:

%0 Thesis
%A Xu, Kai 
%T ScenicGym: Reinforcement Learning with Data Generation Using Scenic
%I EECS Department, University of California, Berkeley
%D 2025
%8 August 15
%@ UCB/EECS-2025-168
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2025/EECS-2025-168.html
%F Xu:EECS-2025-168