Rafael H. Saavedra-Barrera, Alan J. Smith and Eugene Miya

EECS Department
University of California, Berkeley
Technical Report No. UCB/CSD-89-494
March 1989

http://www2.eecs.berkeley.edu/Pubs/TechRpts/1989/CSD-89-494.pdf

Runs of a benchmark or a suite of benchmarks are inadequate to either characterize a given machine or to predict the running time of some benchmark not included in the suite. Further, the observed results are quite sensitive to the nature of the benchmarks, and the relative performance of two machines can vary greatly depending on the benchmarks used. In this paper, we report on a new approach to benchmarking and machine characterization. The idea is to create and use a machine characterizer, which measures the performance of a given system in terms of a Fortran abstract machine. Fortran is used because of its relative simplicity and its wide use for scientific computation. The analyzer yields a set of parameters which characterize the system and spotlight its strong and weak points; each parameter provides the execution time for some primitive operation in Fortran.

We present measurements for a large number of machines ranging from small workstations to supercomputers. We then combine these measurements into groups of parameters which relate to specific aspects of the machine implementation, and use these groups to provide overall machine characterizations. We also define the concept of pershapes, which represent the level of performance of a machine for different types of computation. We introduce a metric based on pershapes that provides a quantitative way of measuring how similar two machines are in terms of their performance distributions. This metric is related to the extent to which pairs of machines have varying relative performance levels depending on which benchmark is used.

BibTeX citation:

@techreport{Saavedra-Barrera:CSD-89-494,
    Author = {Saavedra-Barrera, Rafael H. and Smith, Alan J. and Miya, Eugene},
    Title = {Machine Characterization Based on an Abstract High Level Language Machine},
    Institution = {EECS Department, University of California, Berkeley},
    Year = {1989},
    Month = {Mar},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/1989/6151.html},
    Number = {UCB/CSD-89-494},
    Abstract = {Runs of a benchmark or a suite of benchmarks are inadequate to either characterize a given machine or to predict the running time of some benchmark not included in the suite. Further, the observed results are quite sensitive to the nature of the benchmarks, and the relative performance of two machines can vary greatly depending on the benchmarks used. In this paper, we report on a new approach to benchmarking and machine characterization. The idea is to create and use a machine characterizer, which measures the performance of a given system in terms of a Fortran abstract machine. Fortran is used because of its relative simplicity and its wide use for scientific computation. The analyzer yields a set of parameters which characterize the system and spotlight its strong and weak points; each parameter provides the execution time for some primitive operation in Fortran. <p>We present measurements for a large number of machines ranging from small workstations to supercomputers. We then combine these measurements into groups of parameters which relate to specific aspects of the machine implementation, and use these groups to provide overall machine characterizations. We also define the concept of pershapes, which represent the level of performance of a machine for different types of computation. We introduce a metric based on pershapes that provides a quantitative way of measuring how similar two machines are in terms of their performance distributions. This metric is related to the extent to which pairs of machines have varying relative performance levels depending on which benchmark is used.}
}

EndNote citation:

%0 Report
%A Saavedra-Barrera, Rafael H.
%A Smith, Alan J.
%A Miya, Eugene
%T Machine Characterization Based on an Abstract High Level Language Machine
%I EECS Department, University of California, Berkeley
%D 1989
%@ UCB/CSD-89-494
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/1989/6151.html
%F Saavedra-Barrera:CSD-89-494