Mr. Plotter: Unifying Data Reduction Techniques in Storage and Visualization Systems

Sam Kumar, Michael P Andersen and David E. Culler

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2018-85
May 22, 2018

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-85.pdf

As the rate of data collection continues to grow rapidly, developing visualization tools that scale to immense data sets is a serious and ever-increasing challenge. Existing approaches generally seek to decouple storage and visualization systems, performing just-in-time data reduction to transparently avoid overloading the visualizer. We present a new architecture in which the visualizer and data store are tightly coupled. Unlike systems that read raw data from storage, the performance of our system scales linearly with the size of the final visualization, essentially independent of the size of the data. Thus, it scales to massive data sets while supporting interactive performance (sub-100 ms query latency). This enables a new class of visualization clients that automatically manage data, quickly and transparently requesting data from the underlying database without requiring the user to explicitly initiate queries. It lays a groundwork for supporting truly interactive exploration of big data and opens new directions for research on scalable information visualization systems.


BibTeX citation:

@techreport{Kumar:EECS-2018-85,
    Author = {Kumar, Sam and Andersen, Michael P and Culler, David E.},
    Title = {Mr. Plotter: Unifying Data Reduction Techniques in Storage and Visualization Systems},
    Institution = {EECS Department, University of California, Berkeley},
    Year = {2018},
    Month = {May},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-85.html},
    Number = {UCB/EECS-2018-85},
    Abstract = {As the rate of data collection continues to grow rapidly, developing visualization tools that scale to immense data sets is a serious and ever-increasing challenge.
Existing approaches generally seek to decouple storage and visualization systems, performing just-in-time data reduction to transparently avoid overloading the visualizer.
We present a new architecture in which the visualizer and data store are tightly coupled. Unlike systems that read raw data from storage, the performance of our system scales linearly with the size of the final visualization, essentially independent of the size of the data.
Thus, it scales to massive data sets while supporting interactive performance (sub-100 ms query latency).
This enables a new class of visualization clients that automatically manage data, quickly and transparently requesting data from the underlying database without requiring the user to explicitly initiate queries.
It lays a groundwork for supporting truly interactive exploration of big data and opens new directions for research on scalable information visualization systems.}
}

EndNote citation:

%0 Report
%A Kumar, Sam
%A Andersen, Michael P
%A Culler, David E.
%T Mr. Plotter: Unifying Data Reduction Techniques in Storage and Visualization Systems
%I EECS Department, University of California, Berkeley
%D 2018
%8 May 22
%@ UCB/EECS-2018-85
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2018/EECS-2018-85.html
%F Kumar:EECS-2018-85