Doris Xin

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2021-59

May 12, 2021

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-59.pdf

Machine learning has become a key driver for technological advancement in the last decade on the back of major progress in programming interfaces and scalable systems. Libraries such as Scikit-learn and Keras have made it easier to implement machine learning algorithms and applications, while innovations in distributed systems have enabled model training at an unprecedented scale. However, machine learning tooling is far from perfect today; practitioners still face many challenges when developing applications powered by machine learning.

This dissertation aims to improve the usability and resource efficiency of systems for developing and productionizing machine learning applications by investigating multiple directions identified through extensive empirical evidence gathering and analysis. First, we study the applied machine learning literature and execution traces of publicly available machine learning workflows to understand common practices and shed light on the highly iterative process of model development. Using our insights, we present a solution to accelerate the iterative model development process. Next, we analyze the provenance graph of thousands of production pipelines to uncover latent inefficiencies when running these pipelines. Using these insights, we propose a solution to significantly reduce wasted computation in such systems. Our solutions harness classic techniques from systems, databases, and programming languages to automate data management and optimize computation in machine learning application development. Finally, we synthesize findings from interviews with current users of automated machine learning tools to examine the role of automation in model development, as we look ahead to the future of machine learning developer tools.

Advisors: Aditya Parameswaran

---

BibTeX citation:

@phdthesis{Xin:EECS-2021-59,
    Author= {Xin, Doris},
    Title= {Usable and Efficient Systems for Machine Learning},
    School= {EECS Department, University of California, Berkeley},
    Year= {2021},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-59.html},
    Number= {UCB/EECS-2021-59},
    Abstract= {Machine learning has become a key driver for technological advancement in the last decade on the back of major progress in programming interfaces and scalable systems. Libraries such as Scikit-learn and Keras have made it easier to implement machine learning algorithms and applications, while innovations in distributed systems have enabled model training at an unprecedented scale. However, machine learning tooling is far from perfect today; practitioners still face many challenges when developing applications powered by machine learning.

This dissertation aims to improve the usability and resource efficiency of systems for developing and productionizing machine learning applications by investigating multiple directions identified through extensive empirical evidence gathering and analysis. First, we study the applied machine learning literature and execution traces of publicly available machine learning workflows to understand common practices and shed light on the highly iterative process of model development. Using our insights, we present a solution to accelerate the iterative model development process. Next, we analyze the provenance graph of thousands of production pipelines to uncover latent inefficiencies when running these pipelines. Using these insights, we propose a solution to significantly reduce wasted computation in such systems. Our solutions harness classic techniques from systems, databases, and programming languages to automate data management and optimize computation in machine learning application development. Finally, we synthesize findings from interviews with current users of automated machine learning tools to examine the role of automation in model development, as we look ahead to the future of machine learning developer tools.},
}

EndNote citation:

%0 Thesis
%A Xin, Doris 
%T Usable and Efficient Systems for Machine Learning
%I EECS Department, University of California, Berkeley
%D 2021
%8 May 12
%@ UCB/EECS-2021-59
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-59.html
%F Xin:EECS-2021-59