Omer Spillinger and David Eliahu and Armando Fox and James Demmel

EECS Department, University of California, Berkeley

Technical Report No. UCB/EECS-2015-29

May 1, 2015

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-29.pdf

We present a machine learning technique for the algorithm selection problem, specifically focusing on algorithms for dense matrix multiplication. Dense matrix multiplication is a core component of many high-performance computing and machine learning algorithms, but the performance of matrix multiplication algorithms can vary significantly based on input parameters and hardware architecture. We build performance models for multiple machines using support vector machines (SVMs) and show that only a sparse exploration of the input space is sufficient to accurately predict the best choice of algorithm over a wide range of possible inputs. We find that by using this classifier-based approach to choose the best algorithm to use at runtime, we are able to achieve as much as a 26% increase in average performance over choosing a single algorithm a priori. This is within 1.5% of the performance possible with a perfect algorithm selector.

Advisors: Armando Fox

---

BibTeX citation:

@mastersthesis{Spillinger:EECS-2015-29,
    Author= {Spillinger, Omer and Eliahu, David and Fox, Armando and Demmel, James},
    Title= {Matrix Multiplication Algorithm Selection with Support Vector Machines},
    School= {EECS Department, University of California, Berkeley},
    Year= {2015},
    Month= {May},
    Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-29.html},
    Number= {UCB/EECS-2015-29},
    Abstract= {We present a machine learning technique for the algorithm selection problem, specifically focusing on algorithms for dense matrix multiplication. Dense matrix multiplication is a core component of many high-performance computing and machine learning algorithms, but the performance of matrix multiplication algorithms can vary significantly based on input parameters and hardware architecture. We build performance models for multiple machines using support vector machines (SVMs) and show that only a sparse exploration of the input space is sufficient to accurately predict the best choice of algorithm over a wide range of possible inputs. We find that by using this classifier-based approach to choose the best algorithm to use at runtime, we are able to achieve as much as a 26% increase in average performance over choosing a single algorithm a priori. This is within 1.5% of the performance possible with a perfect algorithm selector.},
}

EndNote citation:

%0 Thesis
%A Spillinger, Omer 
%A Eliahu, David 
%A Fox, Armando 
%A Demmel, James 
%T Matrix Multiplication Algorithm Selection with Support Vector Machines
%I EECS Department, University of California, Berkeley
%D 2015
%8 May 1
%@ UCB/EECS-2015-29
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-29.html
%F Spillinger:EECS-2015-29