Using Dataflow for Machine Learning Inference

Harikaran Subbaraj

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2020-75

May 28, 2020

This publication is archived. It is kept only for reference purposes, so it is no longer being updated and may not meet accessibility standards. If you need this content in a different format, please email webteam@eecs.berkeley.edu.

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/Archive/EECS-2020-75.pdf

The need for systems to support machine learning inference has grown as the im- portance of machine learning in production systems has increased. Serving pipelines of machine learning models comes with challenges of scaling, low-latency require- ments for requests, and high computation costs for different stages of the pipeline. In this paper we propose that by taking a dataflow abstraction we can simplify and increase performance of serving these machine learning pipelines. The proposed system FLOWSERVEcombines this dataflow paradigm with Cloudburst, a stateful function as a service (FaaS) system to provide a framework to deploy and serve machine learning pipelines at scale. We provide several logical and physical opti- mizations that make FLOWSERVEoutperform currently used research and industry systems.

Advisors: Joseph Gonzalez

BibTeX citation:

@mastersthesis{Subbaraj:EECS-2020-75,
    Author= {Subbaraj, Harikaran},
    Title= {Using Dataflow for Machine Learning Inference},
    School= {EECS Department, University of California, Berkeley},
    Year= {2020},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-75.html},
    Number= {UCB/EECS-2020-75},
    Abstract= {The need for systems to support machine learning inference has grown as the im- portance of machine learning in production systems has increased. Serving pipelines of machine learning models comes with challenges of scaling, low-latency require- ments for requests, and high computation costs for different stages of the pipeline. In this paper we propose that by taking a dataflow abstraction we can simplify and increase performance of serving these machine learning pipelines. The proposed system FLOWSERVEcombines this dataflow paradigm with Cloudburst, a stateful function as a service (FaaS) system to provide a framework to deploy and serve machine learning pipelines at scale. We provide several logical and physical opti- mizations that make FLOWSERVEoutperform currently used research and industry systems.},
}

EndNote citation:

%0 Thesis
%A Subbaraj, Harikaran 
%T Using Dataflow for Machine Learning Inference
%I EECS Department, University of California, Berkeley
%D 2020
%8 May 28
%@ UCB/EECS-2020-75
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2020/EECS-2020-75.html
%F Subbaraj:EECS-2020-75