To Compress or Not To Compress - Compute vs. IO tradeoffs for MapReduce Energy Efficiency

Yanpei Chen, Archana Sulochana Ganapathi and Randy H. Katz

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2010-36
March 29, 2010

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-36.pdf

Compression enables us to shift the computation load from IO to CPU. In modern datacenters where energy efficiency is a growing concern, the benefits of using compression have not been completely exploited. We develop a decision algorithm that helps MapReduce users identify when and where to use compression. For some jobs, using compression gives energy savings of up to 60%. As MapReduce represents a common computation framework for Internet datacenters, we believe our findings will provide signficant impact on improving datacenter energy efficiency.


BibTeX citation:

@techreport{Chen:EECS-2010-36,
    Author = {Chen, Yanpei and Ganapathi, Archana Sulochana and Katz, Randy H.},
    Title = {To Compress or Not To Compress - Compute vs. IO tradeoffs for MapReduce Energy Efficiency},
    Institution = {EECS Department, University of California, Berkeley},
    Year = {2010},
    Month = {Mar},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-36.html},
    Number = {UCB/EECS-2010-36},
    Abstract = {Compression enables us to shift the computation load from IO to CPU. In modern datacenters where energy efficiency is a growing concern, the benefits of using compression have not been completely exploited. We develop a decision algorithm that helps MapReduce users identify when and where to use compression. For some jobs, using compression gives energy savings of up to 60%. As MapReduce represents a common computation framework for Internet datacenters, we believe our findings will provide signficant impact on improving datacenter energy efficiency.}
}

EndNote citation:

%0 Report
%A Chen, Yanpei
%A Ganapathi, Archana Sulochana
%A Katz, Randy H.
%T To Compress or Not To Compress - Compute vs. IO tradeoffs for MapReduce Energy Efficiency
%I EECS Department, University of California, Berkeley
%D 2010
%8 March 29
%@ UCB/EECS-2010-36
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-36.html
%F Chen:EECS-2010-36