To Compress or Not To Compress - Compute vs. IO tradeoffs for MapReduce Energy Efficiency

Yanpei Chen and Archana Sulochana Ganapathi and Randy H. Katz

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2010-36

March 29, 2010

This publication is archived. It is kept only for reference purposes, so it is no longer being updated and may not meet accessibility standards. If you need this content in a different format, please email webteam@eecs.berkeley.edu.

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/Archive/EECS-2010-36.pdf

Compression enables us to shift the computation load from IO to CPU. In modern datacenters where energy efficiency is a growing concern, the benefits of using compression have not been completely exploited. We develop a decision algorithm that helps MapReduce users identify when and where to use compression. For some jobs, using compression gives energy savings of up to 60%. As MapReduce represents a common computation framework for Internet datacenters, we believe our findings will provide signficant impact on improving datacenter energy efficiency.

BibTeX citation:

@techreport{Chen:EECS-2010-36,
    Author= {Chen, Yanpei and Ganapathi, Archana Sulochana and Katz, Randy H.},
    Title= {To Compress or Not To Compress - Compute vs. IO tradeoffs for MapReduce Energy Efficiency},
    Year= {2010},
    Month= {Mar},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-36.html},
    Number= {UCB/EECS-2010-36},
    Abstract= {Compression enables us to shift the computation load from IO to CPU. In modern datacenters where energy efficiency is a growing concern, the benefits of using compression have not been completely exploited. We develop a decision algorithm that helps MapReduce users identify when and where to use compression. For some jobs, using compression gives energy savings of up to 60%. As MapReduce represents a common computation framework for Internet datacenters, we believe our findings will provide signficant impact on improving datacenter energy efficiency.},
}

EndNote citation:

%0 Report
%A Chen, Yanpei 
%A Ganapathi, Archana Sulochana 
%A Katz, Randy H. 
%T To Compress or Not To Compress - Compute vs. IO tradeoffs for MapReduce Energy Efficiency
%I EECS Department, University of California, Berkeley
%D 2010
%8 March 29
%@ UCB/EECS-2010-36
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2010/EECS-2010-36.html
%F Chen:EECS-2010-36