PhD Candidate
Cornell University
Areas of Interest
- Artificial Intelligence
Poster
Off-Policy Evaluation and Learning for Interactive Systems
Abstract
A key challenge in building robust intelligent systems is to make the agents reason, plan and act effectively in uncertain environments. A path to meeting this challenge is Reinforcement Learning (RL), which is the task of learning from interactions with the environment. It offers a powerful formalization of how agents can learn to act from experience. Most existing algorithms in RL rely on known environments or the existence of a good simulator, where it is cheap to explore and collect the training data. However, this is not the case for human-centered tasks, such as personalized medicine, product recommendation, and self-driving cars, in which online sampling/experimentation is costly, dangerous, even illegal. Instead of relying on online experimentation, my research focuses on how to evaluate and improve the performance of intelligent systems by only using the logged data from prior systems (a.k.a. off-policy evaluation and learning) in the contextual-bandit setting, which is a state-less form of RL that is highly relevant to many applications. While such data is collected in large quantity, reasoning counterfactually is difficult since the data is biased and partial in nature. My current research aims at providing efficient counterfactual estimators that give accurate offline estimates of online performance, which also opens the door for robust learning.
Bio
Yi Su is a PhD student in the Department of Statistics and Data Science at Cornell University, advised by Professor Thorsten Joachims. Her research interests lie in learning from user behavioral data and implicit feedback in search engines, recommender systems and market platforms. She currently works on off-policy evaluation and learning in contextual bandits and reinforcement learning. She has interned at Microsoft Research and Bloomberg AI. Before joining Cornell, Yi received BSc (Honors) in Mathematics from Nanyang Technological University in Singapore. She is the recipient of Lee Kwan Yew Gold Medal (2016) and Bloomberg Data Science Fellowship (2019-2021).
