Ashwat Chidambaram

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2024-90

May 10, 2024

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-90.pdf

Autonomous vehicles are one of the most exciting technologies at the forefront of innovation in the current era we live in. In particular, accurate perception of the world around the vehicle is a key component for safe and reliable autonomy. As a result, improving autonomous vehicle perception models to achieve peak performance is an ideal goal that has been an active area of research, oftentimes bottlenecked by the necessity for high quality training data. The use of simulation engines to generate data typically aims to account for this data scarcity, but unfortunately the outputs often look visually dissimilar to the real-world domain of images, a problem that is commonly referred to as the "reality gap". Through the work presented in this thesis, we aim to bridge this reality gap through the use of zero-shot sim2real learning approaches, in order to generate realistic training data for perception models with limited real-world baseline images to begin with. In our specific context for this project, we demonstrate significantly improved performance for an autonomous race car in high-speed races against other cars on a professional track, with an enhanced ability to detect and segment these opponent vehicles in diverse scenarios compared to before. Furthermore, our work ultimately demonstrates a versatile approach to pipeline simulation data into sim2real outputs for training real-world models, unlocking a new level of data generation that allows us to create practically infinite scenarios, and ultimately improve the robustness of autonomous vehicle perception in novel unseen domains.

Advisors: Murat Arcak

---

BibTeX citation:

@mastersthesis{Chidambaram:EECS-2024-90,
    Author= {Chidambaram, Ashwat},
    Title= {Leveraging Zero-Shot Sim2Real Learning to Improve Autonomous Vehicle Perception},
    School= {EECS Department, University of California, Berkeley},
    Year= {2024},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-90.html},
    Number= {UCB/EECS-2024-90},
    Abstract= {Autonomous vehicles are one of the most exciting technologies at the forefront of innovation in the current era we live in. In particular, accurate perception of the world around the vehicle is a key component for safe and reliable autonomy. As a result, improving autonomous vehicle perception models to achieve peak performance is an ideal goal that has been an active area of research, oftentimes bottlenecked by the necessity for high quality training data. The use of simulation engines to generate data typically aims to account for this data scarcity, but unfortunately the outputs often look visually dissimilar to the real-world domain of images, a problem that is commonly referred to as the "reality gap". Through the work presented in this thesis, we aim to bridge this reality gap through the use of zero-shot sim2real learning approaches, in order to generate realistic training data for perception models with limited real-world baseline images to begin with. In our specific context for this project, we demonstrate significantly improved performance for an autonomous race car in high-speed races against other cars on a professional track, with an enhanced ability to detect and segment these opponent vehicles in diverse scenarios compared to before. Furthermore, our work ultimately demonstrates a versatile approach to pipeline simulation data into sim2real outputs for training real-world models, unlocking a new level of data generation that allows us to create practically infinite scenarios, and ultimately improve the robustness of autonomous vehicle perception in novel unseen domains.},
}

EndNote citation:

%0 Thesis
%A Chidambaram, Ashwat 
%T Leveraging Zero-Shot Sim2Real Learning to Improve Autonomous Vehicle Perception
%I EECS Department, University of California, Berkeley
%D 2024
%8 May 10
%@ UCB/EECS-2024-90
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-90.html
%F Chidambaram:EECS-2024-90