1============================================================================= 2Notes about performance benchmarks 3============================================================================= 4For each benchmark, here is a brief description and notes about its 5strengths and weaknesses. 6 7----------------------------------------------------------------------------- 8Artificial stress tests 9----------------------------------------------------------------------------- 10bigcode1, bigcode2: 11- Description: Executes a lot of (nonsensical) code. 12- Strengths: Demonstrates the cost of translation which is a large part 13 of runtime, particularly on larger programs. 14- Weaknesses: Highly artificial. 15 16heap: 17- Description: Does a lot of heap allocation and deallocation, and has a lot 18 of heap blocks live while doing so. 19- Strengths: Stress test for an important sub-system; bug #105039 showed 20 that inefficiencies in heap allocation can make a big 21 difference to programs that allocate a lot. 22- Weaknesses: Highly artificial -- allocation pattern is not real, and only 23 a few different size allocations are used. 24 25sarp: 26- Description: Does a lot of stack allocation and deallocation. 27- Strengths: Tests for a specific performance bug that existed in 3.1.0 and 28 all earlier versions. 29- Weaknesses: Highly artificial. 30 31----------------------------------------------------------------------------- 32Real programs 33----------------------------------------------------------------------------- 34bz2: 35- Description: Burrows-Wheeler compression and decompression. 36- Strengths: A real, widely used program, very similar to the 256.bzip2 37 SPEC2000 benchmark. Not dominated by any code, the hottest 38 55 blocks account for only 90% of execution. Has lots of 39 short blocks and stresses the memory system hard. 40- Weaknesses: None, really, it's a good benchmark. 41 42fbench: 43- Description: Does some ray-tracing. 44- Strengths: Moderately realistic program. 45- Weaknesses: Dominated by sin and cos, which are not widely used, and are 46 hardware-supported on x86 but not on other platforms such as 47 PPC. 48 49ffbench: 50- Description: Does a Fast Fourier Transform (FFT). 51- Strengths: Tests common FP ops (mostly adding and multiplying array 52 elements), FFT is a very important operation. 53- Weaknesses: Dominated by the inner loop, which is quite long and flatters 54 Valgrind due to the small dispatcher overhead. 55 56tinycc: 57- Description: A very small and fast C compiler. A munged version of 58 Fabrice Bellard's TinyCC compiling itself multiple times. 59- Strengths: A real program, lots of code (top 100 blocks only account for 60 47% of execution), involves large irregular data structures 61 (presumably, since it's a compiler). Does lots of 62 malloc/free calls and so changes that make a big improvement 63 to perf/heap typically cause a small improvement. 64- Weaknesses None, really, it's a good benchmark. 65 66