Somil Bansal

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2020-186

November 24, 2020

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-186.pdf

For successful integration of autonomous systems such as drones and self-driving cars in our day-to-day life, they must be able to quickly adapt to ever changing environments, and actively reason about their safety and that of other users and autonomous systems around them. Even though control theoretic approaches have been used for decades now for the control and safety analysis of autonomous systems, these approaches typically operate under the assumption of a known system dynamics model and the environment in which the system is operating. To overcome these challenges, machine learning approaches have been explored to operate autonomous systems intelligently and reliably in unpredictable environments based on prior data. However, learning techniques widely used today are extremely data inefficient, making it challenging to apply them to real-world physical systems. Moreover, unlike control theoretic approaches, these techniques lack the necessary mathematical framework to provide guarantees on correctness, causing safety concerns as data-driven physical systems are integrated in our society.   This dissertation aims to combine the control theoretic perspective with the modern learning approaches to enable autonomous systems to safely adapt to unknown environments. It first introduces a suite of core tools from dynamical system theory and robust control that permits the safety analysis of single and multi-agent autonomous systems under the assumption of known system model and environment. In the remainder of the dissertation, we discuss how we can go past these assumptions with the help of machine learning, while minimizing the data requirement for learning and maintaining the safety guarantees for the autonomous system.

To that end, we first discuss how we can learn inaccuracies in the dynamics model of the system, such as unknown ground effects for quadrotors, and use the learned model along with optimal control tools to improve the control performance. Our particular focus is on learning task-specific models that allow for a quick adaptation for the task at hand; these techniques are showcased on physical quadrotors and robotic arms. We next present modular architectures that use a learning-based perception module for the environment level reasoning and a dynamics model-based module for system level reasoning to solve challenging perception and control problems in a priori unknown environments in a data-efficient fashion. Moreover, due to their modularity, these architectures are amenable to simulation-to-real transfer, and can be used for different robotic systems without any retraining. These approaches are demonstrated on a variety of ground robots navigating in unknown buildings around humans based only on onboard visual sensors.

Next, we discuss how we can use dynamics models not only for data-efficient learning, but also to monitor and recognize the learning system’s failures, and to provide online corrective safe actions when necessary. This allows us to provide safety assurances for learning-enabled systems in unknown and human-centric environments, which has remained a challenge to date. Together these techniques enable autonomous systems to learn to operate in unknown environments, but do so in a data-efficient and safe fashion. The dissertation ends with a discussion of future challenges and opportunities at the intersection of learning and control, including the safety analysis in online learning settings and the need to close the loop between the design of learning systems and their safety analysis for developing resilient and continually improving autonomous systems.

Advisors: Claire Tomlin

---

BibTeX citation:

@phdthesis{Bansal:EECS-2020-186,
    Author= {Bansal, Somil},
    Title= {Safe and Data-Efficient Learning for Robotics: A Control Theoretic Approach},
    School= {EECS Department, University of California, Berkeley},
    Year= {2020},
    Month= {Nov},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-186.html},
    Number= {UCB/EECS-2020-186},
    Abstract= {For successful integration of autonomous systems such as drones and self-driving cars in our day-to-day life, they must be able to quickly adapt to ever changing environments, and actively reason about their safety and that of other users and autonomous systems around them. Even though control theoretic approaches have been used for decades now for the control and safety analysis of autonomous systems, these approaches typically operate under the assumption of a known system dynamics model and the environment in which the system is operating. To overcome these challenges, machine learning approaches have been explored to operate autonomous systems intelligently and reliably in unpredictable environments based on prior data. However, learning techniques widely used today are extremely data inefficient, making it challenging to apply them to real-world physical systems. Moreover, unlike control theoretic approaches, these techniques lack the necessary mathematical framework to provide guarantees on correctness, causing safety concerns as data-driven physical systems are integrated in our society.
 
This dissertation aims to combine the control theoretic perspective with the modern learning approaches to enable autonomous systems to safely adapt to unknown environments. It first introduces a suite of core tools from dynamical system theory and robust control that permits the safety analysis of single and multi-agent autonomous systems under the assumption of known system model and environment. In the remainder of the dissertation, we discuss how we can go past these assumptions with the help of machine learning, while minimizing the data requirement for learning and maintaining the safety guarantees for the autonomous system.

To that end, we first discuss how we can learn inaccuracies in the dynamics model of the system, such as unknown ground effects for quadrotors, and use the learned model along with optimal control tools to improve the control performance. Our particular focus is on learning task-specific models that allow for a quick adaptation for the task at hand; these techniques are showcased on physical quadrotors and robotic arms. We next present modular architectures that use a learning-based perception module for the environment level reasoning and a dynamics model-based module for system level reasoning to solve challenging perception and control problems in a priori unknown environments in a data-efficient fashion. Moreover, due to their modularity, these architectures are amenable to simulation-to-real transfer, and can be used for different robotic systems without any retraining. These approaches are demonstrated on a variety of ground robots navigating in unknown buildings around humans based only on onboard visual sensors.

Next, we discuss how we can use dynamics models not only for data-efficient learning, but also to monitor and recognize the learning system’s failures, and to provide online corrective safe actions when necessary. This allows us to provide safety assurances for learning-enabled systems in unknown and human-centric environments, which has remained a challenge to date. 
Together these techniques enable autonomous systems to learn to operate in unknown environments, but do so in a data-efficient and safe fashion. The dissertation ends with a discussion of future challenges and opportunities at the intersection of learning and control, including the safety analysis in online learning settings and the need to close the loop between the design of learning systems and their safety analysis for developing resilient and continually improving autonomous systems.},
}

EndNote citation:

%0 Thesis
%A Bansal, Somil 
%T Safe and Data-Efficient Learning for Robotics: A Control Theoretic Approach
%I EECS Department, University of California, Berkeley
%D 2020
%8 November 24
%@ UCB/EECS-2020-186
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-186.html
%F Bansal:EECS-2020-186