Mingwei Samuel

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2021-201

August 16, 2021

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-201.pdf

The cloud gives everyone the power of infinite computing resources, but programming distributed systems is hard. Current programming models require very precise and error-prone reasoning about network reliability and message handling, and frequent expensive coordination stages in which all nodes have to communicate to reach consensus.

In this paper we present our ongoing work on Hydroflow, a new cloud programming model used to create constructively correct distributed systems. The model is a refinement and unification of the existing dataflow and reactive programming models. Like dataflow, Hydroflow is based on an algebra of operators which execute in streaming fashion across multiple nodes. However in Hydroflow data passed between operators can be of any lattice type, represented using a compositional lattice sub-language. These features allow us to construct provably monotonic distributed programs which can always make forward progress without incurring the high cost of coordination.

Hydroflow is primarily a low-level compilation target for future declarative cloud programming languages, but developers can use it directly to precisely control program execution or fine-tune and debug compiled programs.

Advisors: Alvin Cheung

---

BibTeX citation:

@mastersthesis{Samuel:EECS-2021-201,
    Author= {Samuel, Mingwei},
    Editor= {Hellerstein, Joseph M. and Cheung, Alvin},
    Title= {Hydroflow: A Model and Runtime for Distributed Systems Programming},
    School= {EECS Department, University of California, Berkeley},
    Year= {2021},
    Month= {Aug},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-201.html},
    Number= {UCB/EECS-2021-201},
    Abstract= {The cloud gives everyone the power of infinite computing resources, but programming distributed systems is hard.
Current programming models require very precise and error-prone reasoning about network reliability and message handling, and frequent expensive coordination stages in which all nodes have to communicate to reach consensus.

In this paper we present our ongoing work on Hydroflow, a new cloud programming model used to create constructively correct distributed systems.
The model is a refinement and unification of the existing dataflow and reactive programming models.
Like dataflow, Hydroflow is based on an algebra of operators which execute in streaming fashion across multiple nodes.
However in Hydroflow data passed between operators can be of any lattice type, represented using a compositional lattice sub-language.
These features allow us to construct provably monotonic distributed programs which can always make forward progress without incurring the high cost of coordination.

Hydroflow is primarily a low-level compilation target for future declarative cloud programming languages, but developers can use it directly to precisely control program execution or fine-tune and debug compiled programs.},
}

EndNote citation:

%0 Thesis
%A Samuel, Mingwei 
%E Hellerstein, Joseph M. 
%E Cheung, Alvin 
%T Hydroflow: A Model and Runtime for Distributed Systems Programming
%I EECS Department, University of California, Berkeley
%D 2021
%8 August 16
%@ UCB/EECS-2021-201
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2021/EECS-2021-201.html
%F Samuel:EECS-2021-201