Matrix Multiplication Algorithm Selection with Support Vector Machines
Omer Spillinger and David Eliahu and Armando Fox and James Demmel
EECS Department, University of California, Berkeley
Technical Report No. UCB/EECS-2015-29
May 1, 2015
http://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-29.pdf
We present a machine learning technique for the algorithm selection problem, specifically focusing on algorithms for dense matrix multiplication. Dense matrix multiplication is a core component of many high-performance computing and machine learning algorithms, but the performance of matrix multiplication algorithms can vary significantly based on input parameters and hardware architecture. We build performance models for multiple machines using support vector machines (SVMs) and show that only a sparse exploration of the input space is sufficient to accurately predict the best choice of algorithm over a wide range of possible inputs. We find that by using this classifier-based approach to choose the best algorithm to use at runtime, we are able to achieve as much as a 26% increase in average performance over choosing a single algorithm a priori. This is within 1.5% of the performance possible with a perfect algorithm selector.
Advisors: Armando Fox
BibTeX citation:
@mastersthesis{Spillinger:EECS-2015-29, Author= {Spillinger, Omer and Eliahu, David and Fox, Armando and Demmel, James}, Title= {Matrix Multiplication Algorithm Selection with Support Vector Machines}, School= {EECS Department, University of California, Berkeley}, Year= {2015}, Month= {May}, Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-29.html}, Number= {UCB/EECS-2015-29}, Abstract= {We present a machine learning technique for the algorithm selection problem, specifically focusing on algorithms for dense matrix multiplication. Dense matrix multiplication is a core component of many high-performance computing and machine learning algorithms, but the performance of matrix multiplication algorithms can vary significantly based on input parameters and hardware architecture. We build performance models for multiple machines using support vector machines (SVMs) and show that only a sparse exploration of the input space is sufficient to accurately predict the best choice of algorithm over a wide range of possible inputs. We find that by using this classifier-based approach to choose the best algorithm to use at runtime, we are able to achieve as much as a 26% increase in average performance over choosing a single algorithm a priori. This is within 1.5% of the performance possible with a perfect algorithm selector.}, }
EndNote citation:
%0 Thesis %A Spillinger, Omer %A Eliahu, David %A Fox, Armando %A Demmel, James %T Matrix Multiplication Algorithm Selection with Support Vector Machines %I EECS Department, University of California, Berkeley %D 2015 %8 May 1 %@ UCB/EECS-2015-29 %U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2015/EECS-2015-29.html %F Spillinger:EECS-2015-29