Maxime Kawawa-Beaudan

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2022-146

May 19, 2022

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-146.pdf

This thesis presents methods for training intelligent robotic systems to navigate challenging, diverse environments. In particular it focuses on legged locomotion for hexapods rather than well-studied bipedal or quadrupedal robots. We employ hierarchical reinforcement learning to integrate proprioception with a low-level gait controller, and train on-policy algorithms entirely in simulation before transferring to a real robot. We develop command-conditioned policies in PyBullet that learn to walk at commanded target velocities, and switch to Isaac Gym to train policies with multi-objective rewards both in rough and flat terrains. A comparison is established between gaits with motion priors and prior-free gaits. Both methods successfully surmount joist obstacles typically seen in attics in the real world. We propose novel approaches to the sim-to-real problem designed to address limitations of affordable hardware. We demonstrate methods in the Isaac Gym simulation for integrating vision.

Advisors: Avideh Zakhor

---

BibTeX citation:

@mastersthesis{Kawawa-Beaudan:EECS-2022-146,
    Author= {Kawawa-Beaudan, Maxime},
    Editor= {Zakhor, Avideh},
    Title= {Learning to Walk: Legged Hexapod Locomotion from Simulation to the Real World},
    School= {EECS Department, University of California, Berkeley},
    Year= {2022},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-146.html},
    Number= {UCB/EECS-2022-146},
    Abstract= {This thesis presents methods for training intelligent robotic systems to navigate challenging,
diverse environments. In particular it focuses on legged locomotion for hexapods rather
than well-studied bipedal or quadrupedal robots. We employ hierarchical reinforcement
learning to integrate proprioception with a low-level gait controller, and train on-policy
algorithms entirely in simulation before transferring to a real robot. We develop command-conditioned
policies in PyBullet that learn to walk at commanded target velocities, and
switch to Isaac Gym to train policies with multi-objective rewards both in rough and flat
terrains. A comparison is established between gaits with motion priors and prior-free gaits.
Both methods successfully surmount joist obstacles typically seen in attics in the real world.
We propose novel approaches to the sim-to-real problem designed to address limitations of
affordable hardware. We demonstrate methods in the Isaac Gym simulation for integrating
vision.},
}

EndNote citation:

%0 Thesis
%A Kawawa-Beaudan, Maxime 
%E Zakhor, Avideh 
%T Learning to Walk: Legged Hexapod Locomotion from Simulation to the Real World
%I EECS Department, University of California, Berkeley
%D 2022
%8 May 19
%@ UCB/EECS-2022-146
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-146.html
%F Kawawa-Beaudan:EECS-2022-146