Testing methodology
This topic describes the testing methodology that was used in this paper.
Testing was performed by running uperf
with the 20 unique tests defined in the
topic Workload tests
of Network workload, that scaled the
simulated users for each of the five workloads. The execution of each of the 20 workload tests
invoked sequentially, one after the other, are called a run
. Runs were performed for 1, 2, 4
and 8 uperf
pairs. The KVM host LPARs were freshly rebooted at the
start of each run.
A separate run was conducted for each LPAR, network configuration, uperf
pair
count and when a different value was used for one of the many Linux® operating system, network, KVM, or
uperf
settings / tunables.
Running uperf
uperf
, the server must first be started with the following
command:[root@kvmguest ~] # uperf -s
When the server starts, it enters listening mode waiting for one or more clients to connect.
[root@kvmguest ~] # uperf -m profile.xml -a -i 30
The -m parameter specifies the XML which contains the workload test definition or profile
that uperf
will use for this invocation. Two additional command line options were
used. First, the -a option to tells uperf
to collect all statistics
which is useful for analysis purposes. Second, the -i option specifies the interval (in
seconds) until the next real-time update of throughput and operations per second statistics are
reported by uperf
to the user.
There are a variety of additional command line parameters available. For the complete list of options, refer to the www.uperf.org Web page.
uperf
Output
# uperf -m profile.xml -a
Starting 5 processes running profile:tcp_rr ... 0.00 seconds
Txn1 0 / 1.00(s) = 0 5op/s
Txn2 18.49MB / 300.50(s) = 516.11Kb/s 64513op/s
Txn3 0 / 0.00(s) = 0 0op/s
------------------------------------------------------------------------
Total 18.49MB / 302.60(s) = 512.52Kb/s 64065op/s
Group Details
------------------------------------------------------------------------
Group0 0 / 302.50(s) = 0 0op/s
Strand Details
------------------------------------------------------------------------
Thr0 3.70MB / 302.50(s) = 102.55Kb/s 12818op/s
Thr0 3.70MB / 302.50(s) = 102.55Kb/s 12818op/s
Thr0 3.70MB / 302.50(s) = 102.55Kb/s 12818op/s
Thr0 3.70MB / 302.50(s) = 102.55Kb/s 12818op/s
Thr0 3.70MB / 302.50(s) = 102.55Kb/s 12818op/s
Txn Count avg cpu max min
------------------------------------------------------------------------
Txn0 5 453.42us 0.00ns 513.58us 383.42us
Txn1 9693177 154.87us 0.00ns 200.06ms 32.28us
Txn2 5 10.65us 0.00ns 12.38us 8.63us
Flowop Count avg cpu max min
------------------------------------------------------------------------
accept 5 453.10us 0.00ns 513.18us 383.18us
write 9693177 3.53us 0.00ns 232.93us 1.49us
read 9693172 151.25us 0.00ns 200.06ms 867.00ns
disconnect 5 10.50us 0.00ns 12.30us 8.47us
Netstat statistics for this run
------------------------------------------------------------------------
Nic opkts/s ipkts/s obits/s ibits/s
10gb2 0 0 0 24.75b/s
1gb 1 1 2.42Kb/s 617.18b/s
10gb1 32033 32033 17.17Mb/s 13.58Mb/s
-------------------------------------------------------------------------------------------------
Run Statistics
Hostname Time Data Throughput Operations Errors xferbytes nsec
-------------------------------------------------------------------------------------------------
10.12.38.254 302.60s 18.49MB 512.52Kb/s 19386369 0.00 19386354 302603373999
master 302.60s 18.49MB 512.52Kb/s 19386364 0.00 19386349 302603147820
-------------------------------------------------------------------------------------------------
Difference(%) -0.00% -0.00% 0.00% -0.00% 0.00%
The uperf
output data is separated into three sections.
- The first transaction was to connect each process from the client to the server.
- The second transaction sends a 1 byte Request and reads a 1 byte Response. The throughput rate and operations per second for each transaction is updated based on the specified interval (-i) parameter (which defaults to 1 second).
- The third transaction disconnects each client process from the server. When the transaction has completed, a final update is displayed.1
If the -a parameter is specified, four additional detail sections (group, strand, transaction and flowop) will be displayed. Each section will show totals for each element type relevant to that section.
The next section reports the Netstat statistics for the run, showing the packets and bits sent and received by each network interface in the system. Typically only one or two interfaces will be relevant to the test that was run.
The final section reports the Run statistics which is a high level summary of the completed test, reporting the total test duration, data transferred, operations and errors across the hostname or IP of all the members involved in the test. The data in this section will be a primary source of the results used to conduct our comparisons.
For the measurement results used in this paper, the workload test throughputs and
transaction times were extracted from the uperf
output report.