Part 1 of this two-part article guides you around the many pitfalls associated with benchmarking Java code. This second installment covers two distinct areas. First, it describes some elementary statistics that are useful for coping with the measurement variations inevitably seen in benchmarking. Second, it introduces a software framework for doing benchmarking and uses it in a series of worked examples to illustrate important points.
Benchmarking is extremely difficult. Many factors, both obvious and subtle, can affect your results. To obtain accurate results, you need a thorough command of these issues, possibly by using a benchmarking framework that addresses some of them. Go to Part 2 to learn about just such a robust Java benchmarking framework.