Disk Testing using iperf3
The latest version of iperf3 now supports disk to disk testing.
For Disk to Disk testing, you would do this:
1st test memory to memory:
server: iperf3 -s
client: iperf3 -c testhost
Then disk to memory:
server: iperf3 -s
client: iperf3 -c testhost -F filename -t 60
Then memory to disk:
server: iperf3 -s -F filename
client: iperf3 -c testhost -t 60
The results of the slowest test will indicate the bottleneck. You can also run disk-to-disk tests using a combination of the above commands.
Note that if you run the test a second time the file will be cached, so you need to use a different file, or do this (as root):
sync; echo 3 > /proc/sys/vm/drop_caches
The tests will run till the end of the file, or till the end of the test duration, whichever comes first.
Sample Tests:
First try a memory to memory test:
server: iperf3 -s
client: iperf3 -c hostname -i1
[ ID] Interval Transfer Bandwidth Retransmits
[ 4] 0.00-1.00 sec 460 MBytes 3.86 Gbits/sec 0
[ 4] 1.00-2.00 sec 694 MBytes 5.82 Gbits/sec 0
[ 4] 2.00-3.00 sec 698 MBytes 5.85 Gbits/sec 0
[ 4] 3.00-4.00 sec 700 MBytes 5.87 Gbits/sec 0
[ 4] 4.00-5.00 sec 695 MBytes 5.83 Gbits/sec 0
[ 4] 5.00-6.00 sec 695 MBytes 5.83 Gbits/sec 0
[ 4] 6.00-7.00 sec 696 MBytes 5.84 Gbits/sec 0
[ 4] 7.00-8.00 sec 699 MBytes 5.86 Gbits/sec 0
[ 4] 8.00-9.00 sec 694 MBytes 5.82 Gbits/sec 0
[ 4] 9.00-10.00 sec 695 MBytes 5.83 Gbits/sec 0
next do a disk read test:
server: iperf3 -s
client: iperf3 -c hostname -i1 -F /storage/data/filename
[ ID] Interval Transfer Bandwidth Retransmits
[ 4] 0.00-1.16 sec 92.5 MBytes 669 Mbits/sec 0
[ 4] 1.16-2.00 sec 142 MBytes 1.42 Gbits/sec 0
[ 4] 2.00-3.00 sec 235 MBytes 1.97 Gbits/sec 0
[ 4] 3.00-4.00 sec 188 MBytes 1.57 Gbits/sec 0
[ 4] 4.00-5.02 sec 202 MBytes 1.66 Gbits/sec 0
[ 4] 5.02-6.21 sec 256 MBytes 1.81 Gbits/sec 0
[ 4] 6.21-7.00 sec 204 MBytes 2.15 Gbits/sec 0
[ 4] 7.00-8.00 sec 209 MBytes 1.75 Gbits/sec 0
[ 4] 8.00-9.14 sec 228 MBytes 1.67 Gbits/sec 0
[ 4] 9.14-10.15 sec 256 MBytes 2.13 Gbits/sec 0
This is slower than the above test, so we are disk limited, not network limited.
Next try memory to disk. Note that for disk write tests, you need to run a longer test to factor out network buffering issues.
server: iperf3 -s -F /storage/data/test.out
client: iperf3 -c hostname -i1 -t 40
[ ID] Interval Transfer Bandwidth Retransmits
[ 4] 0.00-1.00 sec 379 MBytes 3.18 Gbits/sec 0
[ 4] 1.00-2.00 sec 659 MBytes 5.53 Gbits/sec 0
[ 4] 2.00-3.00 sec 692 MBytes 5.81 Gbits/sec 0
[ 4] 3.00-4.00 sec 634 MBytes 5.32 Gbits/sec 0
[ 4] 4.00-5.00 sec 615 MBytes 5.16 Gbits/sec 0
[ 4] 5.00-6.00 sec 650 MBytes 5.45 Gbits/sec 0
[ 4] 6.00-7.00 sec 392 MBytes 3.29 Gbits/sec 0
[ 4] 7.00-8.00 sec 622 MBytes 5.22 Gbits/sec 0
[ 4] 8.00-9.00 sec 585 MBytes 4.91 Gbits/sec 0
[ 4] 9.00-10.00 sec 519 MBytes 4.35 Gbits/sec 0
[ 4] 10.00-11.00 sec 484 MBytes 4.06 Gbits/sec 0
...
[ 4] 30.00-31.00 sec 399 MBytes 3.34 Gbits/sec 0
[ 4] 31.00-32.00 sec 396 MBytes 3.32 Gbits/sec 0
[ 4] 32.00-33.00 sec 229 MBytes 1.92 Gbits/sec 0
[ 4] 33.00-34.00 sec 474 MBytes 3.97 Gbits/sec 0
[ 4] 34.00-35.00 sec 456 MBytes 3.83 Gbits/sec 0
[ 4] 35.00-36.00 sec 421 MBytes 3.53 Gbits/sec 0
[ 4] 36.00-37.00 sec 382 MBytes 3.21 Gbits/sec 0
[ 4] 37.00-38.00 sec 395 MBytes 3.31 Gbits/sec 0
[ 4] 38.00-39.00 sec 412 MBytes 3.46 Gbits/sec 0
[ 4] 39.00-40.00 sec 372 MBytes 3.12 Gbits/sec 0
Disk writes in this case is slightly faster than reading, but still slower than the network.
