Haoyuan Li and Ali Ghodsi and Matei Zaharia and Scott Shenker and Ion Stoica

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2014-135

June 16, 2014

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-135.pdf

Tachyon is a distributed file system enabling reliable data sharing at memory speed across cluster computing frameworks. While caching today improves read workloads, writes are either network or disk bound, as replication is used for fault-tolerance. Tachyon eliminates this bottleneck by pushing lineage, a well-known technique borrowed from application frameworks, into the storage layer. The key challenge in making a long-lived lineage-based storage system is timely data recovery in case of failures. Tachyon addresses this issue by introducing a checkpointing algorithm that guarantees bounded recovery cost and resource allocation strategies for recomputation under common resource schedulers. Our evaluation shows that Tachyon outperforms in-memory HDFS by 110x for writes. It also improves the end-to-end latency of a realistic workflow by 4x. Tachyon is open source and is deployed at multiple companies.

---

BibTeX citation:

@techreport{Li:EECS-2014-135,
    Author= {Li, Haoyuan and Ghodsi, Ali and Zaharia, Matei and Shenker, Scott and Stoica, Ion},
    Title= {Reliable, Memory Speed Storage for Cluster Computing Frameworks},
    Year= {2014},
    Month= {Jun},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-135.html},
    Number= {UCB/EECS-2014-135},
    Abstract= {Tachyon is a distributed file system enabling reliable data sharing at memory speed across cluster computing frameworks. While caching today improves read workloads, writes are either network or disk bound, as replication is used for fault-tolerance. Tachyon eliminates this bottleneck by pushing lineage, a well-known technique borrowed from application frameworks, into the storage layer.  The key challenge in making a long-lived lineage-based storage system is timely data recovery in case of failures. Tachyon addresses this issue by introducing a checkpointing algorithm that guarantees bounded recovery cost and resource allocation strategies for recomputation under common resource schedulers. Our evaluation shows that Tachyon outperforms in-memory HDFS by 110x for writes. It also improves the end-to-end latency of a realistic workflow by 4x. Tachyon is open source and is deployed at multiple companies.},
}

EndNote citation:

%0 Report
%A Li, Haoyuan 
%A Ghodsi, Ali 
%A Zaharia, Matei 
%A Shenker, Scott 
%A Stoica, Ion 
%T Reliable, Memory Speed Storage for Cluster Computing Frameworks
%I EECS Department, University of California, Berkeley
%D 2014
%8 June 16
%@ UCB/EECS-2014-135
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2014/EECS-2014-135.html
%F Li:EECS-2014-135