ディスク性能の計測

シーケンシャル読み出し

read: IOPS=14.8k, BW=57.7MiB/s (60.5MB/s)(578MiB/10016msec)

koyama@koyama-log1:~/distlog$ fio -filename=/tmp/test2g -direct=1 -rw=read -bs=4k -size=2G -numjobs=64 -runtime=10 -group_reporting -name=file1

file1: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 64 processes
file1: Laying out IO file (1 file / 2048MiB)
Jobs: 64 (f=64): [R(64)][100.0%][r=49.1MiB/s,w=0KiB/s][r=12.6k,w=0 IOPS][eta 00m:00s]
file1: (groupid=0, jobs=64): err= 0: pid=9319: Sat Mar 13 06:26:52 2021
   read: IOPS=14.8k, BW=57.7MiB/s (60.5MB/s)(578MiB/10016msec)
    clat (usec): min=167, max=2248.8k, avg=4326.41, stdev=39906.94
     lat (usec): min=167, max=2248.8k, avg=4326.50, stdev=39906.94
    clat percentiles (usec):
     |  1.00th=[    182],  5.00th=[    190], 10.00th=[    196],
     | 20.00th=[    215], 30.00th=[    243], 40.00th=[    269],
     | 50.00th=[    285], 60.00th=[    310], 70.00th=[    338],
     | 80.00th=[    392], 90.00th=[    529], 95.00th=[    627],
     | 99.00th=[ 162530], 99.50th=[ 246416], 99.90th=[ 434111],
     | 99.95th=[ 742392], 99.99th=[1451230]
   bw (  KiB/s): min=    8, max= 8305, per=1.73%, avg=1020.64, stdev=1312.70, samples=1164
   iops        : min=    2, max= 2076, avg=254.98, stdev=328.07, samples=1164
  lat (usec)   : 250=33.35%, 500=55.28%, 750=7.23%, 1000=0.63%
  lat (msec)   : 2=0.46%, 4=0.13%, 10=0.10%, 20=0.67%, 50=0.55%
  lat (msec)   : 100=0.31%, 250=0.82%, 500=0.41%, 750=0.02%, 1000=0.01%
  lat (msec)   : 2000=0.04%, >=2000=0.01%
  cpu          : usr=0.06%, sys=0.10%, ctx=148093, majf=1, minf=856
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=148029,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=57.7MiB/s (60.5MB/s), 57.7MiB/s-57.7MiB/s (60.5MB/s-60.5MB/s), io=578MiB (606MB), run=10016-10016msec

Disk stats (read/write):
  sda: ios=123152/2, merge=21044/1, ticks=538060/8, in_queue=548044, util=98.94%

シーケンシャル書き込み

write: IOPS=3679, BW=14.4MiB/s (15.1MB/s)(144MiB/10037msec)

koyama@koyama-log1:~/distlog$ fio -filename=/tmp/test2g -direct=1 -rw=write -bs=4k -size=2G -numjobs=64 -runtime=10 -group_reporting -name=file1

file1: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 64 processes
Jobs: 64 (f=64): [W(64)][100.0%][r=0KiB/s,w=14.2MiB/s][r=0,w=3645 IOPS][eta 00m:00s]
file1: (groupid=0, jobs=64): err= 0: pid=9390: Sat Mar 13 06:31:33 2021
  write: IOPS=3679, BW=14.4MiB/s (15.1MB/s)(144MiB/10037msec)
    clat (usec): min=196, max=3576.3k, avg=17369.35, stdev=223347.16
     lat (usec): min=197, max=3576.3k, avg=17369.46, stdev=223347.16
    clat percentiles (usec):
     |  1.00th=[    204],  5.00th=[    215], 10.00th=[    239],
     | 20.00th=[    243], 30.00th=[    247], 40.00th=[    253],
     | 50.00th=[    269], 60.00th=[    281], 70.00th=[    289],
     | 80.00th=[    306], 90.00th=[    314], 95.00th=[    318],
     | 99.00th=[    363], 99.50th=[1619002], 99.90th=[3539993],
     | 99.95th=[3539993], 99.99th=[3573548]
   bw (  KiB/s): min=    8, max= 1755, per=8.93%, avg=1314.19, stdev=469.47, samples=225
   iops        : min=    2, max=  438, avg=328.15, stdev=117.25, samples=225
  lat (usec)   : 250=36.07%, 500=63.14%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.03%, 4=0.01%, 20=0.01%, 50=0.01%, 100=0.01%
  lat (msec)   : 250=0.02%, 500=0.04%, 750=0.03%, 1000=0.04%, 2000=0.12%
  lat (msec)   : >=2000=0.46%
  cpu          : usr=0.02%, sys=0.02%, ctx=74066, majf=0, minf=745
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,36927,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=14.4MiB/s (15.1MB/s), 14.4MiB/s-14.4MiB/s (15.1MB/s-15.1MB/s), io=144MiB (151MB), run=10037-10037msec

Disk stats (read/write):
  sda: ios=0/36450, merge=0/1, ticks=0/9672, in_queue=9672, util=96.73%

ランダム読み出し

read: IOPS=27.6k, BW=108MiB/s (113MB/s)(1078MiB/10009msec)

koyama@koyama-log1:~/distlog$ fio -filename=/tmp/test2g -direct=1 -rw=randread -bs=4k -size=2G -numjobs=64 -runtime=10 -group_reporting -name=file1

file1: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 64 processes
Jobs: 64 (f=64): [r(64)][100.0%][r=108MiB/s,w=0KiB/s][r=27.8k,w=0 IOPS][eta 00m:00s]
file1: (groupid=0, jobs=64): err= 0: pid=9457: Sat Mar 13 06:33:39 2021
   read: IOPS=27.6k, BW=108MiB/s (113MB/s)(1078MiB/10009msec)
    clat (usec): min=171, max=89857, avg=2319.86, stdev=989.82
     lat (usec): min=171, max=89857, avg=2319.92, stdev=989.82
    clat percentiles (usec):
     |  1.00th=[ 2040],  5.00th=[ 2114], 10.00th=[ 2114], 20.00th=[ 2147],
     | 30.00th=[ 2180], 40.00th=[ 2245], 50.00th=[ 2278], 60.00th=[ 2343],
     | 70.00th=[ 2409], 80.00th=[ 2442], 90.00th=[ 2474], 95.00th=[ 2540],
     | 99.00th=[ 2671], 99.50th=[ 2835], 99.90th=[ 4555], 99.95th=[13042],
     | 99.99th=[57410]
   bw (  KiB/s): min= 1384, max= 1971, per=1.57%, avg=1732.05, stdev=60.27, samples=1280
   iops        : min=  346, max=  492, avg=432.52, stdev=15.03, samples=1280
  lat (usec)   : 250=0.13%, 500=0.02%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.38%, 4=99.31%, 10=0.07%, 20=0.03%, 50=0.03%
  lat (msec)   : 100=0.01%
  cpu          : usr=0.06%, sys=0.20%, ctx=275932, majf=0, minf=673
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=275889,0,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
   READ: bw=108MiB/s (113MB/s), 108MiB/s-108MiB/s (113MB/s-113MB/s), io=1078MiB (1130MB), run=10009-10009msec

Disk stats (read/write):
  sda: ios=272738/2, merge=0/1, ticks=620952/0, in_queue=621012, util=98.93%

ランダム書き込み

write: IOPS=4003, BW=15.6MiB/s (16.4MB/s)(157MiB/10032msec)

koyama@koyama-log1:~/distlog$ fio -filename=/tmp/test2g -direct=1 -rw=randwrite -bs=4k -size=2G -numjobs=64 -runtime=10 -group_reporting -name=file1
file1: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.1
Starting 64 processes
Jobs: 64 (f=64): [w(64)][100.0%][r=0KiB/s,w=15.9MiB/s][r=0,w=4081 IOPS][eta 00m:00s]
file1: (groupid=0, jobs=64): err= 0: pid=9527: Sat Mar 13 06:36:43 2021
  write: IOPS=4003, BW=15.6MiB/s (16.4MB/s)(157MiB/10032msec)
    clat (usec): min=189, max=2882.7k, avg=15964.55, stdev=189852.73
     lat (usec): min=189, max=2882.7k, avg=15964.64, stdev=189852.73
    clat percentiles (usec):
     |  1.00th=[    200],  5.00th=[    235], 10.00th=[    237],
     | 20.00th=[    239], 30.00th=[    239], 40.00th=[    241],
     | 50.00th=[    241], 60.00th=[    243], 70.00th=[    247],
     | 80.00th=[    265], 90.00th=[    273], 95.00th=[    293],
     | 99.00th=[    322], 99.50th=[2264925], 99.90th=[2768241],
     | 99.95th=[2835350], 99.99th=[2868904]
   bw (  KiB/s): min=   96, max= 1521, per=7.42%, avg=1188.88, stdev=301.71, samples=271
   iops        : min=   24, max=  380, avg=296.90, stdev=75.41, samples=271
  lat (usec)   : 250=74.09%, 500=25.08%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.03%, 4=0.01%, 10=0.01%, 50=0.01%, 100=0.01%
  lat (msec)   : 250=0.02%, 500=0.03%, 750=0.03%, 1000=0.03%, 2000=0.12%
  lat (msec)   : >=2000=0.54%
  cpu          : usr=0.01%, sys=0.03%, ctx=80590, majf=0, minf=617
  IO depths    : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     issued rwt: total=0,40165,0, short=0,0,0, dropped=0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=1

Run status group 0 (all jobs):
  WRITE: bw=15.6MiB/s (16.4MB/s), 15.6MiB/s-15.6MiB/s (16.4MB/s-16.4MB/s), io=157MiB (165MB), run=10032-10032msec

Disk stats (read/write):
  sda: ios=0/39651, merge=0/1, ticks=0/9592, in_queue=9584, util=95.89%

結果

  • bs(ブロックサイズ)=4k
  • size(書き込みデータ量)=2G
  • numjobs(ジョブ数)=64

結果は以下の表となる.IOPSはInput/Output Pre Secondを表す.BWは帯域幅(bandwidth)を表す.

モードIOPSBW
ランダム書き込み400315.6MiB/s (16.4MB/s)
ランダム読み出し27.6k108MiB/s (113MB/s)
シーケンシャル書き込み367914.4MiB/s (15.1MB/s)
シーケンシャル読み出し14.8k57.7MiB/s (60.5MB/s)

参考資料

sudo apt install fio