iozone examples for file system benchmark on linux

Thank you for visiting this page.
I made some update on my another link. Please visit iozone examples for file system benchmark on Linux, for the update

iozone is a file system benchmark tool, which is a very good measurement tool for file system based applications, servers. etc.. Even database applications. Togegher with network performance tools, CPU benchmark tools, they basically give you a baseline of how well you server can do, or how well your server tunned.
From the link below, you can download it's full documentation, as well as source and examples
http://www.iozone.org/

Here I'm going to just show most popular functions to get you easy start.

Before get started, one thing has to mentioned is that it tests filesystem, each filesystem has buffer cache, so if you want to measure sustenable rate, then probably you need to specify a enough big file to eliminate cache effect.

For example: In the example, you can see there are two main sections in the output, same for each test you run.

#/usr/bin/iozone -s24m -r64 -i 0 -i 1 -t 1 -b 24m.xls
    Iozone: Performance Test of File I/O
            Version $Revision: 3.327 $
        Compiled for 64 bit mode.

        Build: linux 

    Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins
                 Al Slater, Scott Rhine, Mike Wisner, Ken Goss
                 Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR,
                 Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner,
                 Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy,
                 Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root.

To here is the section one, it tells you iozone tool info, same everytime you run it.

From here to bottom, is the section two, it has two subsessions.

    File size set to 24576 KB
    Record Size 64 KB
    Command line used: /usr/bin/iozone -s24m -r64 -i 0 -i 1 -t 1 -b 24m.xls
    Output is in Kbytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 Kbytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
    Throughput test with 1 process
    Each process writes a 24576 Kbyte file in 64 Kbyte records

To here is subsession one, which shows brief test environment, includs command line, mode, file and record size etc..

From here to bottom is the subsession two, shows detailed results.

...

"Throughput report Y-axis is type of test X-axis is number of processes"
"Record size = 64 Kbytes "
"Output is in Kbytes/sec"

" Initial write " 422137.78

"        Rewrite " 1028377.50

"           Read " 2072368.75

"        Re-read " 2137207.00

In the example, I just tested one of my server's /root filesystem, which is sitting on a mirrored SATA disks. iozone tells that it got 2GB/sec readiing on it.

Also, as you can see, there are two main sections in the output,

#1. -a option

       -a     Used  to  select  full automatic mode. Produces output that covers all tested file operations for record sizes of 4k to 16M for

file sizes of 64k to 512M.

Auto Mode output looks like below, like I mentioned in the early of this article, it doesn't mean too much, but you can combine it with -g or -s option to specify larger files.

    Command line used: iozone -a
    Output is in Kbytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 Kbytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.

                                                            random  random    bkwd   record   stride

              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread

              64       4  557144  866463  2444640  3541098 2689580  752329 2203800  2203800  2892445   615952   889431 2662899  3203069

              64       8  608965  942521  3791156  4274062 3541098  942521 3057153  2379626  3958892   673098  1002351 3363612  3958892

              64      16  627470  913649  4018152  4564786 3791156  987600 3363612  1828508  3958892   789967  1188789 3541098  2772930

              64      32  754443  926260  3958892  4564786 3958892 1013707 3541098  1518251  3958892   969761  1734015 3791156  4564786

              64      64  811459  942521  4274062  4897948 4564786 1029254 3738358   913649  4274062   710511   983980 4018152  4897948

             128       4  656477  836500  3199360  2843510 2714132  864797 2571150  2409592  2571150   576161   826202 2969325  2727923

             128       8  667081  889145  3867787  4407601 3657016  955616 3380677  2985839  4267461   673778   889145 2985839  4135958

    1st column KB: Indicates the file size that was used for the testing.
    2nd column reclen: Indicates the record length that was used for the testing.
    3rd column until the last column: Indicates the various tests that are performed and its output values in per second.

#2 -s, -n and -g option

Auto Mode
Command line used: iozone -s 2g -a

                                                             random  random    bkwd   record   stride

              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread

         2097152       4  860348 1201573  2081567  2173982 1826295 1132584 1960767  3962311  1936269   843183  1176150 2125247  2182763

         2097152       8  933899 1252630  2297032  2373218 2134208 1257601 2216773  5434625  2204942   906740  1254996 2224330  2363552

Command line used: iozone -n 1g -g 2g -a

                                                            random  random    bkwd   record   stride

              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread

         1048576      64 1008942 1428607  2263735  2513172 2471750 1516633 2342705  5436863  2472584  1011489  1322385 2291308  2510891

As you can see, -s specify only one file size, while -n,-g give a range of file size for test. -n defualt is 64k. Note, -n and -g have to work with -a option, while -s is not the case.

#3 -i option, used to specify which tests to run

Here are list of tests you run, you can run one or more in one test, 0 must be specified for it creates a file, auto mode does them all.

    0=write/rewrite
    1=read/re-read
    2=random-read/write
    3=Read-backwards
    4=Re-write-record
    5=stride-read
    6=fwrite/re-fwrite
    7=fread/Re-fread,
    8=random mix
    9=pwrite/Re-pwrite
    10=pread/Re-pread
    11=pwritev/Re-pwritev
    12=preadv/Re-preadv

Here is one example

Command line used: iozone -a -i 0 -i 1 -s 2g

                                                            random  random    bkwd   record   stride

              KB  reclen   write rewrite    read    reread    read   write    read  rewrite     read   fwrite frewrite   fread  freread

         2097152       4  867286 1212607  2085904  2200394

..


         2097152   16384  802502 1021890  1235077  1252524

#4-r option, to specify the record size, in Kbytes, to test. One may also specify -r #k (size in Kbytes) or -r #m (size in Mbytes) or -r #g (size in Gbytes).

This option is particular important, especially you are testing a filesystem for database, then you need to specify the record size equal to db block size, mostly 4k. Also, it also helps when you test a storage throughput. For example.

If you have an array, raid6 8+2, segment size 64k size, then you probably want to get the best throughput by using -r 512. Here is a test output from one of my old SAN storage. 8+2 raid6, dual 4Gb FC connection

    Command line used: /usr/bin/iozone -s24g -r512 -i 0 -i 1 -t 1 -b iozone_lun1_2013-11-01-23-49.xls
    Output is in Kbytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 1024 Kbytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
    Throughput test with 1 process
    Each process writes a 25165824 Kbyte file in 512 Kbyte records

    Children see throughput for 1 initial writers     = 464439.69 KB/sec
    Parent sees throughput for 1 initial writers     = 371337.60 KB/sec
    Min throughput per process             = 464439.69 KB/sec
    Max throughput per process             = 464439.69 KB/sec
    Avg throughput per process             = 464439.69 KB/sec

    Min xfer                     = 25165824.00 KB

    Children see throughput for  1 rewriters     =  469936.88 KB/sec
    Parent sees throughput for  1 rewriters     =  375833.57 KB/sec
    Min throughput per process             =  469936.88 KB/sec 
    Max throughput per process             =  469936.88 KB/sec
    Avg throughput per process             =  469936.88 KB/sec
    Min xfer                     = 25165824.00 KB

    Children see throughput for  1 readers         =  381264.19 KB/sec
    Parent sees throughput for  1 readers         =  381205.64 KB/sec
    Min throughput per process             =  381264.19 KB/sec 
    Max throughput per process             =  381264.19 KB/sec
    Avg throughput per process             =  381264.19 KB/sec
    Min xfer                     = 25165824.00 KB

    Children see throughput for 1 re-readers     =  378713.69 KB/sec
    Parent sees throughput for 1 re-readers     =  378627.10 KB/sec
    Min throughput per process             =  378713.69 KB/sec 
    Max throughput per process             =  378713.69 KB/sec
    Avg throughput per process             =  378713.69 KB/sec
    Min xfer                     = 25165824.00 KB


"Throughput report Y-axis is type of test X-axis is number of processes"
"Record size = 512 Kbytes "
"Output is in Kbytes/sec"

"  Initial write "  464439.69 

"        Rewrite "  469936.88 

"           Read "  381264.19 

"        Re-read "  378713.69 

iozone test complete.

#5 -t Run Iozone in a throughput mode. This option allows the user to specify how many threads or processes to have active during the measurement.

Already showed the test output in previous test throughput, I use it almost all the time. To simulate multiple threads test, specify -t 10 for example.

#6 -F Specify each of the temporary file names to be used in the throughput testing.

This option allows you to combine multiple tests into one, very useful if you want to know a host i/o capability on multiple LUNs.

Here is an example:

    Command line used: /usr/bin/iozone -s96g -i 0 -i 1 -r2048 -S2048 -t 2 -F /lun0/test1 /lun1/test1
    Output is in Kbytes/sec
    Time Resolution = 0.000001 seconds.
    Processor cache size set to 2048 Kbytes.
    Processor cache line size set to 32 bytes.
    File stride size set to 17 * record size.
    Throughput test with 2 processes
    Each process writes a 100663296 Kbyte file in 2048 Kbyte records

    Children see throughput for 2 initial writers     = 1514665.19 KB/sec
    Parent sees throughput for 2 initial writers     = 1404630.06 KB/sec
    Min throughput per process             = 739716.50 KB/sec
    Max throughput per process             = 774948.69 KB/sec
    Avg throughput per process             = 757332.59 KB/sec
    Min xfer                     = 96083968.00 KB

Note: The number of names should be equal to the number of processes or threads that are specified.

#7 -l -u options Set the lower and upper limit on number of processes to run, similar with -t, but with flexible range.

When running throughput tests this option allows the user to specify the greatest number of processes or threads to start

#8 Set minimum record size and max record size -y and -g

#9 -+m and -+t Enable network performance test. Use with -+m

              Used  to  specify  a filename that will be used to specify the clients in a distributed measurement. The file contains one line

              for each client. The fields are space delimited. Field 1 is the client name. Field 2 is the working directory, on  the  client,

where Iozone will run. Field 3 is the path to the executable Iozone on the client.

#10 Report CPU utilization time

Command line used: /usr/bin/iozone -s24g -r512 -i 0 -i 1 -t 1 -+u

Children see throughput for 1 initial writers = 470748.72 KB/sec

...

CPU Utilization: Wall time 53.459 CPU time 42.402 CPU utilization 79.32 %

Children see throughput for 1 rewriters = 470614.66 KB/sec

...

CPU utilization: Wall time 53.474 CPU time 51.333 CPU utilization 96.00 %

...

"Output is in CPU%" " Initial write " 79.32 " Rewrite " 96.00 " Read " 25.46 " Re-read " 25.70

Comments