Gregory Kahn and Adam Villaflor and Bosen Ding and Pieter Abbeel and Sergey Levine

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2018-47

May 10, 2018

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-47.pdf

Enabling robots to autonomously navigate complex environments is essential for real-world deployment. Prior methods approach this problem by having the robot maintain an internal map of the world, and then use a localization and planning method to navigate through the internal map. However, these approaches often include a variety of assumptions, are computationally intensive, and do not learn from failures. In contrast, learning-based methods improve as the robot acts in the environment, but are difficult to deploy in the real-world due to their high sample complexity. To address the need to learn complex policies with few samples, we propose a generalized computation graph that subsumes value-based model-free methods and model-based methods, with specific instantiations interpolating between model-free and model-based. We then instantiate this graph to form a navigation model that learns from raw images and is sample efficient. Our simulated car experiments explore the design decisions of our navigation model, and show our approach outperforms single-step and N-step double Q-learning. We also evaluate our approach on a real-world RC car and show it can learn to navigate through a complex indoor environment with a few hours of fully autonomous, self-supervised training.

Advisors: Pieter Abbeel and Sergey Levine

---

BibTeX citation:

@mastersthesis{Kahn:EECS-2018-47,
    Author= {Kahn, Gregory and Villaflor, Adam and Ding, Bosen and Abbeel, Pieter and Levine, Sergey},
    Title= {Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation},
    School= {EECS Department, University of California, Berkeley},
    Year= {2018},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-47.html},
    Number= {UCB/EECS-2018-47},
    Abstract= {Enabling robots to autonomously navigate complex environments is essential for real-world deployment. Prior methods approach this problem by having the robot maintain an internal map of the world, and then use a localization and planning method to navigate through the internal map. However, these approaches often include a variety of assumptions, are computationally intensive, and do not learn from failures. In contrast, learning-based methods improve as the robot acts in the environment, but are difficult to deploy in the real-world due to their high sample complexity. To address the need to learn complex policies with few samples, we propose a generalized computation graph that subsumes value-based model-free methods and model-based methods, with specific instantiations interpolating between model-free and model-based. We then instantiate this graph to form a navigation model that learns from raw images and is sample efficient. Our simulated car experiments explore the design decisions of our navigation model, and show our approach outperforms single-step and N-step double Q-learning. We also evaluate our approach on a real-world RC car and show it can learn to navigate through a complex indoor environment with a few hours of fully autonomous, self-supervised training.},
}

EndNote citation:

%0 Thesis
%A Kahn, Gregory 
%A Villaflor, Adam 
%A Ding, Bosen 
%A Abbeel, Pieter 
%A Levine, Sergey 
%T Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation
%I EECS Department, University of California, Berkeley
%D 2018
%8 May 10
%@ UCB/EECS-2018-47
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-47.html
%F Kahn:EECS-2018-47