Jesus Navarro

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2021-254

December 14, 2021

Bipedal locomotion presents a challenging set of control tasks that can be de- composed as (1) primitive and (2) task-specific high-level tasks. There exists a diverse set of data-driven and model-based controller optimization methods that enable legged robots to perform well on primitive (low-level) tasks such as walk- ing, standing, and jumping. Learning high-level legged locomotion tasks is gen- erally modeled as a two-layer optimization problem with a high-level and low- level control policy. We propose a similar and simple framework that uses Deep Reinforcement Learning (RL) to learn a high-level task given a fixed and high- performing low-level policy. State-of-the-art architectures are developed as end- to-end frameworks where both the parameters of low-level and high-level policies are optimized jointly during training. Our method decouples the learning proce- dure of the high and low-level tasks as disjoint optimization problems, and uses a curriculum learning based approach to optimize the high-level task. We demon- strate the learning framework by teaching Cassie, a bipedal robot, to dodge a rolling ball using a jumping and standing primitive controller.

Advisors: Murat Arcak

---

BibTeX citation:

@mastersthesis{Navarro:EECS-2021-254,
    Author= {Navarro, Jesus},
    Editor= {Sreenath, Koushil},
    Title= {Cassie Learning Dodging Skills: A Hierarchical Reinforcement Learning Based Approach},
    School= {EECS Department, University of California, Berkeley},
    Year= {2021},
    Month= {Dec},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-254.html},
    Number= {UCB/EECS-2021-254},
    Abstract= {Bipedal locomotion presents a challenging set of control tasks that can be de- composed as (1) primitive and (2) task-specific high-level tasks. There exists a diverse set of data-driven and model-based controller optimization methods that enable legged robots to perform well on primitive (low-level) tasks such as walk- ing, standing, and jumping. Learning high-level legged locomotion tasks is gen- erally modeled as a two-layer optimization problem with a high-level and low- level control policy. We propose a similar and simple framework that uses Deep Reinforcement Learning (RL) to learn a high-level task given a fixed and high- performing low-level policy. State-of-the-art architectures are developed as end- to-end frameworks where both the parameters of low-level and high-level policies are optimized jointly during training. Our method decouples the learning proce- dure of the high and low-level tasks as disjoint optimization problems, and uses a curriculum learning based approach to optimize the high-level task. We demon- strate the learning framework by teaching Cassie, a bipedal robot, to dodge a rolling ball using a jumping and standing primitive controller.},
}

EndNote citation:

%0 Thesis
%A Navarro, Jesus 
%E Sreenath, Koushil 
%T Cassie Learning Dodging Skills: A Hierarchical Reinforcement Learning Based Approach
%I EECS Department, University of California, Berkeley
%D 2021
%8 December 14
%@ UCB/EECS-2021-254
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-254.html
%F Navarro:EECS-2021-254