Benchmarking High Performance Computers: Fundamentals and Practice

One of the properties of High Performance Computers is that their level of performance generally will differ by one or even several orders of magnitude depending on the ability to parallelise and/or vectorise the programs at hand. Therefore are benchmarks to identify the performance spectrum of High Performance Computers of the highest importance. Various benchmarks are available, but only part of them is relevant in providing insight in the performance characteristics of High Performance Computers.

In this tutorial we discuss the general principles of properly setting up benchmarks and how to assess the results. Also an overview of some existing benchmarks is given and their value and limitations are discussed. Furthermore, possible benchmarking procedures and their relation with other techniques like simulation and machine modellingwill be treated.