David Shen

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2024-204

December 1, 2024

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-204.pdf

We present the framework of uncertainty-aware modularized autonomy stack to describe modern robotic systems that utilize uncertainty quantification (UQ). In the first part of the thesis, we introduce a realization of the framework in navigation. We present a novel pipeline to obtain probabilistically safe and dynamically feasible reachable sets from a trajectory forecasting model using conformal prediction, as well as a planning method that leverages the safety guarantees of those sets. We showcase the efficacy of our pipeline in simulation with real autonomous driving data and in an experiment with Boeing vehicles. In the second part, we present an analysis of the framework through studying the system-wide impact of using UQ. We use level set estimation tools to efficiently quantify system robustness and calibration, even when the evaluation process is costly. We apply our analysis to two realistic industry-grade systems. We discover that UQ improves overall system robustness in the presence of input error, and that UQ enables a downstream module to give calibrated outputs despite erroneous outputs from upstream.

Advisors: Claire Tomlin

---

BibTeX citation:

@mastersthesis{Shen:EECS-2024-204,
    Author= {Shen, David},
    Title= {Design and Analysis of Uncertainty-Aware Modularized Autonomy Stacks},
    School= {EECS Department, University of California, Berkeley},
    Year= {2024},
    Month= {Dec},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-204.html},
    Number= {UCB/EECS-2024-204},
    Abstract= {We present the framework of uncertainty-aware modularized autonomy stack to describe modern robotic systems that utilize uncertainty quantification (UQ). In the first part of the thesis, we introduce a realization of the framework in navigation. We present a novel pipeline to obtain probabilistically safe and dynamically feasible reachable sets from a trajectory forecasting model using conformal prediction, as well as a planning method that leverages the safety guarantees of those sets. We showcase the efficacy of our pipeline in simulation with real autonomous driving data and in an experiment with Boeing vehicles. In the second part, we present an analysis of the framework through studying the system-wide impact of using UQ. We use level set estimation tools to efficiently quantify system robustness and calibration, even when the evaluation process is costly. We apply our analysis to two realistic industry-grade systems. We discover that UQ improves overall system robustness in the presence of input error, and that UQ enables a downstream module to give calibrated outputs despite erroneous outputs from upstream.},
}

EndNote citation:

%0 Thesis
%A Shen, David 
%T Design and Analysis of Uncertainty-Aware Modularized Autonomy Stacks
%I EECS Department, University of California, Berkeley
%D 2024
%8 December 1
%@ UCB/EECS-2024-204
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-204.html
%F Shen:EECS-2024-204