YuXuan Liu and Pieter Abbeel

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2023-122

May 12, 2023

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-122.pdf

Recent advances in artificial intelligence, particularly deep learning and large foundation models, have demonstrated remarkable progress. However, when applying AI to real-world robotics applications, we still face many challenges due to the diverse scenarios and objects encountered, as well as the need for high throughput and accuracy. Off-the-shelf models often fail to meet the stringent requirements for high-performing robotic applications, because they do not adequately model uncertainty that arises in the real world. Moreover, training such models require large datasets which can be expensive to annotate or not immediately applicable for robotic applications.

We address these challenges by introducing a novel class of models that explicitly model and handle ambiguity in 2D and 3D perception. These models offer improved adaptability and decision-making capabilities by incorporating uncertainty estimation, better equipping robots for the dynamic nature of real-world environments. Furthermore, we explore methods of leveraging diverse data collected in robotic applications without requiring costly human annotation. We propose a self-supervised learning method that enables robots to autonomously learn from the rich information available in the diverse images they encounter during operation. This approach leads to enhanced performance and adaptability, allowing robotic systems to continuously refine their perception capabilities. We hope these contributions pave the way for more robust, adaptable, and high-performing robotic systems that excel in complex and dynamic environments, addressing the unique challenges posed by real-world robotics and bridging the gap between AI research and practical robotic applications.

Advisors: Pieter Abbeel

---

BibTeX citation:

@phdthesis{Liu:EECS-2023-122,
    Author= {Liu, YuXuan and Abbeel, Pieter},
    Title= {Perception for Real-World Robotic Applications},
    School= {EECS Department, University of California, Berkeley},
    Year= {2023},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-122.html},
    Number= {UCB/EECS-2023-122},
    Abstract= {Recent advances in artificial intelligence, particularly deep learning and large foundation models, have demonstrated remarkable progress. However, when applying AI to real-world robotics applications, we still face many challenges due to the diverse scenarios and objects encountered, as well as the need for high throughput and accuracy. Off-the-shelf models often fail to meet the stringent requirements for high-performing robotic applications, because they do not adequately model uncertainty that arises in the real world. Moreover, training such models require large datasets which can be expensive to annotate or not immediately applicable for robotic applications.

We address these challenges by introducing a novel class of models that explicitly model and handle ambiguity in 2D and 3D perception. These models offer improved adaptability and decision-making capabilities by incorporating uncertainty estimation, better equipping robots for the dynamic nature of real-world environments. Furthermore, we explore methods of leveraging diverse data collected in robotic applications without requiring costly human annotation. We propose a self-supervised learning method that enables robots to autonomously learn from the rich information available in the diverse images they encounter during operation. This approach leads to enhanced performance and adaptability, allowing robotic systems to continuously refine their perception capabilities. We hope these contributions pave the way for more robust, adaptable, and high-performing robotic systems that excel in complex and dynamic environments, addressing the unique challenges posed by real-world robotics and bridging the gap between AI research and practical robotic applications.},
}

EndNote citation:

%0 Thesis
%A Liu, YuXuan 
%A Abbeel, Pieter 
%T Perception for Real-World Robotic Applications
%I EECS Department, University of California, Berkeley
%D 2023
%8 May 12
%@ UCB/EECS-2023-122
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2023/EECS-2023-122.html
%F Liu:EECS-2023-122