Maxime Kawawa-Beaudan

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2022-146
May 19, 2022

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-146.pdf

This thesis presents methods for training intelligent robotic systems to navigate challenging, diverse environments. In particular it focuses on legged locomotion for hexapods rather than well-studied bipedal or quadrupedal robots. We employ hierarchical reinforcement learning to integrate proprioception with a low-level gait controller, and train on-policy algorithms entirely in simulation before transferring to a real robot. We develop command-conditioned policies in PyBullet that learn to walk at commanded target velocities, and switch to Isaac Gym to train policies with multi-objective rewards both in rough and flat terrains. A comparison is established between gaits with motion priors and prior-free gaits. Both methods successfully surmount joist obstacles typically seen in attics in the real world. We propose novel approaches to the sim-to-real problem designed to address limitations of affordable hardware. We demonstrate methods in the Isaac Gym simulation for integrating vision.

Advisor: Avideh Zakhor

BibTeX citation:

@mastersthesis{Kawawa-Beaudan:EECS-2022-146,
    Author = {Kawawa-Beaudan, Maxime},
    Editor = {Zakhor, Avideh},
    Title = {Learning to Walk: Legged Hexapod Locomotion from Simulation to the Real World},
    School = {EECS Department, University of California, Berkeley},
    Year = {2022},
    Month = {May},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-146.html},
    Number = {UCB/EECS-2022-146},
    Abstract = {This thesis presents methods for training intelligent robotic systems to navigate challenging,
diverse environments. In particular it focuses on legged locomotion for hexapods rather
than well-studied bipedal or quadrupedal robots. We employ hierarchical reinforcement
learning to integrate proprioception with a low-level gait controller, and train on-policy
algorithms entirely in simulation before transferring to a real robot. We develop command-conditioned
policies in PyBullet that learn to walk at commanded target velocities, and
switch to Isaac Gym to train policies with multi-objective rewards both in rough and flat
terrains. A comparison is established between gaits with motion priors and prior-free gaits.
Both methods successfully surmount joist obstacles typically seen in attics in the real world.
We propose novel approaches to the sim-to-real problem designed to address limitations of
affordable hardware. We demonstrate methods in the Isaac Gym simulation for integrating
vision.}
}

EndNote citation:

%0 Thesis
%A Kawawa-Beaudan, Maxime
%E Zakhor, Avideh
%T Learning to Walk: Legged Hexapod Locomotion from Simulation to the Real World
%I EECS Department, University of California, Berkeley
%D 2022
%8 May 19
%@ UCB/EECS-2022-146
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-146.html
%F Kawawa-Beaudan:EECS-2022-146