Benchmarks

Benchmark programs are used by compiler vendors as a standard for compiler assessment, and are selected to cover interesting aspects of the compiler and target computer. We use Ackermann, Sieve and Whetstone.

Ackermann

The Ackermann benchmark program focuses on function calls. Using a clever yet simple formula, the Ackermann benchmark is able to execute millions of non-redundant calls, while using a modest amount of stack space. We always time a call of ‘ackermann (3,6)’. Years ago this would execute in a time we could measure with a stopwatch. Modern computers are much faster, and in most cases the time will be much less than one second. For comparison, a VAX 780 (circa 1980) programmed in VAX Pascal, scores 6 seconds.

Sieve

The sieve benchmark appeared in Byte Magazine in the early 1980s, and has been a popular benchmark ever since. It focuses on simple integer operations and is therefore relevant for many embedded applications. On the original IBM PC (4.7 MHz Intel 8088), sieve written in C would take about 10 seconds.

Whetstone

The Whetstone benchmark program was originally written in Algol 60 to measure the performance of an early computer built at the Whetstone laboratory in the UK. The result is a number of Whetstone instructions per second, and is weighted to represent a typical contemporary scientific program. We have Whetstone results for many languages, compilers and computers. Modern personal computers and workstations do well, and many score more than 100,000 KWIPS (that is 100 million Whetstone instructions per second). A typical embedded computer, such as a Motorola 68020 with floating point coprocessor scores about 2000 KWIPS at a clock speed of 25MHz.

How to Compile and Run the Benchmarks

The Makefile is ready to compile and run all three benchmark programs for the ERC32. Just type ‘make’ to compile only. Then type ‘make run’ to run all two/three.

Benchmark Sources

You can download the source for the benchmark programs, with IO files and Makefile as follows:

• For Ada 95 sources, click here.
• For C sources, click here.
• For Coral66 sources, click here.

Benchmark Results (ERC32 simulator and ERC32 Ada)

The results of running the three benchmark programs Ackermann, Sieve and Whetstone are given in the following table. These programs were run on the ERC32 simulator, with a 10 MHz clock, zero wait states on data read and write, and one wait state on instruction fetch.

Benchmark Program	ANSI C Result	Ada 95 Result
Ackermann	1.1 Sec	1.13 Sec
Sieve	290 mSec	289 mSec
Whetstone	3702 KWIPS	3533 KWIPS