Joseph M. Hellerstein and Christopher Ré and Florian Schoppmann and Zhe Daisy Wang and Eugene Fratkin and Aleksander Gorajek and Kee Siong Ng and Caleb Welton and Xixuan Feng and Kun Li and Arun Kumar

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2012-38

April 3, 2012

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-38.pdf

MADlib is a free, open source library of in-database analytic methods. It provides an evolving suite of SQL-based algorithms for machine learning, data mining and statistics that run at scale within a database engine, with no need for data import/export to other tools. The goal is for MADlib to eventually serve a role for scalable database systems that is similar to the CRAN library for R: a community repository of statistical methods, this time written with scale and parallelism in mind.

In this paper we introduce the MADlib project, including the background that led to its beginnings, and the motivation for its open source nature. We provide an overview of the library's architecture and design patterns, and provide a description of various statistical methods in that context. We include performance and speedup results of a core design pattern from one of those methods over the Greenplum parallel DBMS on a modest-sized test cluster. We then report on two initial efforts at incorporating academic research into MADlib, which is one of the project's goals.

MADlib is freely available at http://madlib.net, and the project is open for contributions of both new methods, and ports to additional database platforms.

---

BibTeX citation:

@techreport{Hellerstein:EECS-2012-38,
    Author= {Hellerstein, Joseph M. and Ré, Christopher and Schoppmann, Florian and Wang, Zhe Daisy and Fratkin, Eugene and Gorajek, Aleksander and Ng, Kee Siong and Welton, Caleb and Feng, Xixuan and Li, Kun and Kumar, Arun},
    Title= {The MADlib Analytics Library or MAD Skills, the SQL},
    Year= {2012},
    Month= {Apr},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-38.html},
    Number= {UCB/EECS-2012-38},
    Abstract= {MADlib is a free, open source library of in-database analytic methods. It
provides an evolving suite of SQL-based algorithms for machine learning, data
mining and statistics that run at scale within a database engine, with no need
for data import/export to other tools. The goal is for MADlib to eventually
serve a role for scalable database systems that is similar to the CRAN library
for R: a community repository of statistical methods, this time written with
scale and parallelism in mind.

In this paper we introduce the MADlib project, including the background that
led to its beginnings, and the motivation for its open source nature. We
provide an overview of the library's architecture and design patterns, and
provide a description of various statistical methods in that context. We
include performance and speedup results of a core design pattern from one of
those methods over the Greenplum parallel DBMS on a modest-sized test cluster.
We then report on two initial efforts at incorporating academic research into
MADlib, which is one of the project's goals.

MADlib is freely available at http://madlib.net, and the project is open
for contributions of both new methods, and ports to additional database
platforms.},
}

EndNote citation:

%0 Report
%A Hellerstein, Joseph M. 
%A Ré, Christopher 
%A Schoppmann, Florian 
%A Wang, Zhe Daisy 
%A Fratkin, Eugene 
%A Gorajek, Aleksander 
%A Ng, Kee Siong 
%A Welton, Caleb 
%A Feng, Xixuan 
%A Li, Kun 
%A Kumar, Arun 
%T The MADlib Analytics Library or MAD Skills, the SQL
%I EECS Department, University of California, Berkeley
%D 2012
%8 April 3
%@ UCB/EECS-2012-38
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-38.html
%F Hellerstein:EECS-2012-38