Administration Guide

Benchmark Testing Methodology

This benchmarking technique is based on the scientific method. A repeatable environment will be created in which the same test, run under the same conditions, will yield comparable results.

Benchmarking can also begin by running the test application in a normal environment. As a performance problem is narrowed down, specialized test cases can be developed to limit the scope of the function that is being tested and observed. The specialized test cases need not emulate an entire application in order to obtain valuable information. Start with simple measurements, and increase the complexity only when warranted.

Characteristics of good benchmarks (or measurements) include:

• Each test is repeatable.
• Each iteration of a test is started in the same system state.
• There are no functions or applications active in the system other than those being measured (unless the scenario includes some amount of other activity going on in the system).

  Note:

  Applications that are started use memory even when they are minimized or idle. This increases the likelihood of paging skewing the results of the benchmark and violating the repeatability rule.

• The hardware and software used for benchmarking matches your production environment.

As with any benchmarking, a scenario must be devised and then executed. The following information applies these concepts to the DB2 environment.

• "Preparing for Benchmark Testing"
• "Creating a Benchmark Program"
• "Executing the Benchmark Tests".

[ DB2 List of Books | Search the DB2 Books ]