Learning Efficiently with Trajectory Data for Real World Robotics

Philipp Wu

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2024-209
December 10, 2024

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-209.pdf

With the technological advancements enabled by AI, the vision of generally capable robots is now within reach. In this dissertation, I discuss my work on leveraging data-driven learning approaches for real-world robotic systems, centering on trajectory data—the complex, multi-modal, time-series information that serves as the core unit of data in robotics. The data sources in robotics are complex, potentially coming from multiple sources with varying quality. Additionally, collecting real-world robot data can be expensive and time-consuming, making efficient use of each data point essential.

I aim to address these challenges through three key research directions: trajectory representation learning, high-quality data collection, and sample-efficient policy learning. First, we explore how to learn effective representations from trajectory data, using both reconstruction-based and contrastive learning methods, and demonstrate how these representations enhance a variety of downstream robotics tasks. Next, we examine practical methods that can be used with real-world robotic systems to collect high-quality trajectory data and subsequently utilize that data to learn new skills. Finally, we investigate how to compose these robot skills with high-level language models to imbue robots with stronger reasoning and planning capabilities.

Together, these contributions advance the development of general-purpose robots capable of operating in complex, unstructured environments.

Advisor: Pieter Abbeel

"; ?>

BibTeX citation:

@phdthesis{Wu:EECS-2024-209,
    Author = {Wu, Philipp},
    Title = {Learning Efficiently with Trajectory Data for Real World Robotics},
    School = {EECS Department, University of California, Berkeley},
    Year = {2024},
    Month = {Dec},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-209.html},
    Number = {UCB/EECS-2024-209},
    Abstract = {With the technological advancements enabled by AI, the vision of generally capable robots is now within reach. In this dissertation, I discuss my work on leveraging data-driven learning approaches for real-world robotic systems, centering on trajectory data—the complex, multi-modal, time-series information that serves as the core unit of data in robotics. The data sources in robotics are complex, potentially coming from multiple sources with varying quality. Additionally, collecting real-world robot data can be expensive and time-consuming, making efficient use of each data point essential.

I aim to address these challenges through three key research directions: trajectory representation learning, high-quality data collection, and sample-efficient policy learning. First, we explore how to learn effective representations from trajectory data, using both reconstruction-based and contrastive learning methods, and demonstrate how these representations enhance a variety of downstream robotics tasks. Next, we examine practical methods that can be used with real-world robotic systems to collect high-quality trajectory data and subsequently utilize that data to learn new skills. Finally, we investigate how to compose these robot skills with high-level language models to imbue robots with stronger reasoning and planning capabilities.

Together, these contributions advance the development of general-purpose robots capable of operating in complex, unstructured environments.}
}

EndNote citation:

%0 Thesis
%A Wu, Philipp
%T Learning Efficiently with Trajectory Data for Real World Robotics
%I EECS Department, University of California, Berkeley
%D 2024
%8 December 10
%@ UCB/EECS-2024-209
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-209.html
%F Wu:EECS-2024-209