Hrish Leen

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2024-73

May 9, 2024

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-73.pdf

Lab-trained robot policies for manipulators often suffer performance drops when deployed in real-world unstructured environments. This happens from encountering data that’s out of distribution from their training data that are usually gathered in structured lab environments. To overcome this challenge and help the robot continually cope in such scenarios, we introduce Continuous Autonomous Improvement for Mobile Manipulation or CAMo. CAMo is a robot learning system that builds on top of existing foundation models for navigation and manipulation by collecting data directly from these real-world environments and asynchronously compiling them to a server for further fine-tuning. Through its mobile base, CAMo is able to incorporate a lot of diverse scenes and real-world perturbations in its ever-increasing data set, better enabling itself to adapt to the difficulty of being in an unstructured environment. By leveraging the multi-modal capacity and stochastic nature of the diffusion head of its manipulation policy, CAMo can reinforce good manipulation behaviors through autonomously collected rollouts for similar but unseen tasks. Along with a LIDAR sensor onboard to enact fail-safe mechanisms and human intervention data for further navigation improvement, CAMo is able to continuously improve in the real world with decreasing human involvement as time goes on.

Advisors: Sergey Levine

---

BibTeX citation:

@mastersthesis{Leen:EECS-2024-73,
    Author= {Leen, Hrish},
    Title= {Continuous Autonomous Improvement for Mobile  Manipulation},
    School= {EECS Department, University of California, Berkeley},
    Year= {2024},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-73.html},
    Number= {UCB/EECS-2024-73},
    Abstract= {Lab-trained robot policies for manipulators often suffer performance drops when deployed in real-world unstructured environments. This happens from encountering data that’s out of distribution from their training data that are usually
gathered in structured lab environments. To overcome this challenge and help the robot continually cope in such scenarios, we introduce Continuous Autonomous Improvement for Mobile Manipulation or CAMo. CAMo is a robot learning system
that builds on top of existing foundation models for navigation and manipulation by collecting data directly from these real-world environments and asynchronously compiling them to a server for further fine-tuning. Through its mobile base, CAMo is able to incorporate a lot of diverse scenes and real-world perturbations in its ever-increasing data set, better enabling itself to adapt to the difficulty of being in an unstructured environment. By leveraging the multi-modal capacity and stochastic nature of the diffusion head of its manipulation policy, CAMo can reinforce good manipulation behaviors through autonomously collected rollouts for similar but unseen tasks. Along with a LIDAR sensor onboard to enact fail-safe mechanisms and human intervention data for further navigation improvement, CAMo is able to continuously improve in the real world with decreasing human involvement as
time goes on.},
}

EndNote citation:

%0 Thesis
%A Leen, Hrish 
%T Continuous Autonomous Improvement for Mobile  Manipulation
%I EECS Department, University of California, Berkeley
%D 2024
%8 May 9
%@ UCB/EECS-2024-73
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-73.html
%F Leen:EECS-2024-73