Sherry Yang

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2024-152

August 2, 2024

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-152.pdf

AlphaGo and ChatGPT are perhaps two most significant breakthroughs in artificial intelligence in the past decade. These technologies were empowered by research in sequential decision making (e.g., planning, search, and reinforcement learning) and foundation models (e.g., language and video generation model trained on internet data). This thesis proposes new techniques, algorithms, and frameworks of leveraging foundation models with broad knowledge in the context of real-world decision making tasks, impacting applications such as building dialogue agent, controlling robots, and making scientific discoveries. This thesis starts with traditional decision making in offline settings and progressively incorporating broader, internet-scale data through representation learning and generative modeling. Emphasis is placed on both theoretical foundations and practical implications. Key contributions of this thesis include algorithmic advancements of offline reinforcement learning, improved representation learning for decision making, novel generative modeling techniques as an alternative to reinforcement learning, and generative agents and generative simulators at internet scale, all aimed at equipping foundation models with enhanced decision-making capabilities and vice versa. Through extensive empirical and theoretical analysis, this thesis demonstrates that foundation models, when properly leveraged, can significantly improve decision-making tasks. The findings offer new directions for integrating machine learning models with real-world applications, paving the way for more intelligent, adaptable, and efficient systems.

Advisors: Pieter Abbeel

---

BibTeX citation:

@phdthesis{Yang:EECS-2024-152,
    Author= {Yang, Sherry},
    Title= {Foundation Models for Decision Making: Algorithms, Frameworks, and Applications},
    School= {EECS Department, University of California, Berkeley},
    Year= {2024},
    Month= {Aug},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-152.html},
    Number= {UCB/EECS-2024-152},
    Abstract= {AlphaGo and ChatGPT are perhaps two most significant breakthroughs in artificial intelligence in the past decade. These technologies were empowered by research in sequential decision making (e.g., planning, search, and reinforcement learning) and foundation models (e.g., language and video generation model trained on internet data). This thesis proposes new techniques, algorithms, and frameworks of leveraging foundation models with broad knowledge in the context of real-world decision making tasks, impacting applications such as building dialogue agent, controlling robots, and making scientific discoveries. This thesis starts with traditional decision making in offline settings and progressively incorporating broader, internet-scale data through representation learning and generative modeling. Emphasis is placed on both theoretical foundations and practical implications. Key contributions of this thesis include algorithmic advancements of offline reinforcement learning, improved representation learning for decision making, novel generative modeling techniques as an alternative to reinforcement learning, and generative agents and generative simulators at internet scale, all aimed at equipping foundation models with enhanced decision-making capabilities and vice versa. Through extensive empirical and theoretical analysis, this thesis demonstrates that foundation models, when properly leveraged, can significantly improve decision-making tasks. The findings offer new directions for integrating machine learning models with real-world applications, paving the way for more intelligent, adaptable, and efficient systems.},
}

EndNote citation:

%0 Thesis
%A Yang, Sherry 
%T Foundation Models for Decision Making: Algorithms, Frameworks, and Applications
%I EECS Department, University of California, Berkeley
%D 2024
%8 August 2
%@ UCB/EECS-2024-152
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/EECS-2024-152.html
%F Yang:EECS-2024-152