Avi Singh

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2021-179

August 11, 2021

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-179.pdf

Robots that can operate in an open, unstructured environment and perform a wide range of tasks have been a long-standing goal of artificial intelligence. For such robots to operate effectively, they need the ability to a) perceive the world around them through general-purpose on-board sensors like cameras b) generalize to new situations c) improve their performance as they collect more data. In this thesis, we posit that deep reinforcement learning (deep RL) methods are well-positioned to overcome the aforementioned challenges, but are difficult to apply to real world domains like robotics. The central conjecture that we study in this work is the following: while dominant robot learning pipelines often rely on hand-engineering certain components (such as reward functions and physics simulators), we can overcome many bottlenecks of these pipelines via the adoption of a more data-driven perspective. We argue that, instead of hand-engineering reward functions, we should instead learn reward functions from data. Instead of learning mostly in a hand-designed simulation and then transferring learned policies to the real world, we should learn using real data, and re-use all past experience (as much as possible) to maintain sample efficiency. We show how this perspective change greatly simplifies robot learning, and demonstrate results on a variety of real world object manipulation tasks.

Advisors: Sergey Levine

---

BibTeX citation:

@phdthesis{Singh:EECS-2021-179,
    Author= {Singh, Avi},
    Title= {Real World Robot Learning: Learned Rewards, Offline Datasets and Skill Re-Use},
    School= {EECS Department, University of California, Berkeley},
    Year= {2021},
    Month= {Aug},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-179.html},
    Number= {UCB/EECS-2021-179},
    Abstract= {Robots that can operate in an open, unstructured environment and perform a wide range of tasks have been a long-standing goal of artificial intelligence. For such robots to operate effectively, they need the ability to a) perceive the world around them through general-purpose on-board sensors like cameras b) generalize to new situations c) improve their performance as they collect more data. In this thesis, we posit that deep reinforcement learning (deep RL) methods are well-positioned to overcome the aforementioned challenges, but are difficult to apply to real world domains like robotics. The central conjecture that we study in this work is the following: while dominant robot learning pipelines often rely on hand-engineering certain components (such as reward functions and physics simulators), we can overcome many bottlenecks of these pipelines via the adoption of a more data-driven perspective. We argue that, instead of hand-engineering reward functions, we should instead learn reward functions from data. Instead of learning mostly in a hand-designed simulation and then transferring learned policies to the real world, we should learn using real data, and re-use all past experience (as much as possible) to maintain sample efficiency. We show how this perspective change greatly simplifies robot learning, and demonstrate results on a variety of real world object manipulation tasks.},
}

EndNote citation:

%0 Thesis
%A Singh, Avi 
%T Real World Robot Learning: Learned Rewards, Offline Datasets and Skill Re-Use
%I EECS Department, University of California, Berkeley
%D 2021
%8 August 11
%@ UCB/EECS-2021-179
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-179.html
%F Singh:EECS-2021-179