Evolving Robotic Leg Shapes via Deep Reinforcement Learning

Hayden Sheung

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2020-103

May 29, 2020

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-103.pdf

In the current state of the art for deep reinforcement learning in robotics, the primary focus is on maximizing an objective function by controlling an unchanged agent. On the other hand, there is a subfield called evolutionary robotics. As implied by the name, it involves using evolutionary algorithms to develop better hardware for robots' performances. However, the field is a little outdated. The core of the aged evolutionary algorithm is that given a set number of shapes, a subset of them (called "parents") is chosen, based on their performances in the environment. They are the foundations for the next generation's shapes, and the program mutates the "parents" set to acquire the next set of shapes to examine. The experiment is then run for a fixed number of generations, and the best shapes are picked from the last generation.

An exciting crossover between the two fields is using the new advanced deep reinforcement learning algorithms in evolutionary robotics. Namely, instead of merely picking the best N shapes in each generation based on their performances in the environment, deep reinforcement learning methods are used to identify the most optimal shape. Then, some mutations are applied to this shape, and it serves as the starting position for the next generation.

The design of the agent is parameterized and allows the agent to learn its body parts during the simulation. The primary objective of the robot is to crawl on the desired path on a chosen terrain. REINFORCE and actor-critic are the deep reinforcement learning optimization algorithms applied. The results indicate that, with the new proposed approach, agents can learn body parts to facilitate their movements in the given environments.

Advisors: Ronald S. Fearing

BibTeX citation:

@mastersthesis{Sheung:EECS-2020-103,
    Author= {Sheung, Hayden},
    Editor= {Fearing, Ronald S.},
    Title= {Evolving Robotic Leg Shapes via Deep Reinforcement Learning},
    School= {EECS Department, University of California, Berkeley},
    Year= {2020},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-103.html},
    Number= {UCB/EECS-2020-103},
    Abstract= {In the current state of the art for deep reinforcement learning in robotics, the primary focus is on maximizing an objective function by controlling an unchanged agent. On the other hand, there is a subfield called evolutionary robotics. As implied by the name, it involves using evolutionary algorithms to develop better hardware for robots' performances. However, the field is a little outdated. The core of the aged evolutionary algorithm is that given a set number of shapes, a subset of them (called "parents") is chosen, based on their performances in the environment. They are the foundations for the next generation's shapes, and the program mutates the "parents" set to acquire the next set of shapes to examine. The experiment is then run for a fixed number of generations, and the best shapes are picked from the last generation. 

An exciting crossover between the two fields is using the new advanced deep reinforcement learning algorithms in evolutionary robotics. Namely, instead of merely picking the best N shapes in each generation based on their performances in the environment, deep reinforcement learning methods are used to identify the most optimal shape. Then, some mutations are applied to this shape, and it serves as the starting position for the next generation. 

The design of the agent is parameterized and allows the agent to learn its body parts during the simulation. The primary objective of the robot is to crawl on the desired path on a chosen terrain. REINFORCE and actor-critic are the deep reinforcement learning optimization algorithms applied. The results indicate that, with the new proposed approach, agents can learn body parts to facilitate their movements in the given environments.},
}

EndNote citation:

%0 Thesis
%A Sheung, Hayden 
%E Fearing, Ronald S. 
%T Evolving Robotic Leg Shapes via Deep Reinforcement Learning
%I EECS Department, University of California, Berkeley
%D 2020
%8 May 29
%@ UCB/EECS-2020-103
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-103.html
%F Sheung:EECS-2020-103