Multiple Sequential Reader vs Random Reader =============================================== Vanilla CFQ ----------------------------------- [Multiple Sequential Reader] [Random Reader] nr Max-bandw Min-bandw Agg-bandw Max-latency nr Agg-bandw Max-latency 1 13978KB/s 13978KB/s 13650KB/s 27614 usec 1 22KB/s 227 msec 2 6225KB/s 6166KB/s 12101KB/s 568K usec 1 10KB/s 457 msec 4 4052KB/s 2462KB/s 13107KB/s 322K usec 1 6KB/s 841 msec 8 1899KB/s 557KB/s 12960KB/s 829K usec 1 13KB/s 1628 msec 16 1007KB/s 279KB/s 13833KB/s 1629K usec 1 10KB/s 3236 msec 32 506KB/s 98KB/s 13704KB/s 3389K usec 1 6KB/s 3238 msec IO scheduler controller + CFQ ----------------------------------- [Multiple Sequential Reader] [Random Reader] nr Max-bandw Min-bandw Agg-bandw Max-latency nr Agg-bandw Max-latency 1 5721KB/s 5721KB/s 5587KB/s 126K usec 1 223KB/s 126K usec 2 3216KB/s 1442KB/s 4549KB/s 349K usec 1 224KB/s 176K usec 4 1895KB/s 640KB/s 5121KB/s 775K usec 1 222KB/s 189K usec 8 957KB/s 285KB/s 6368KB/s 1680K usec 1 223KB/s 142K usec 16 458KB/s 132KB/s 6455KB/s 3343K usec 1 219KB/s 165K usec 32 248KB/s 55KB/s 6001KB/s 6957K usec 1 220KB/s 504K usec - Stable BW and latencies for random readers in group 2. dm-ioaband + CFQ ----------------------------------- [Multiple Sequential Reader] [Random Reader] nr Max-bandw Min-bandw Agg-bandw Max-latency nr Agg-bandw Max-latency 1 12511KB/s 12511KB/s 12218KB/s 34608 usec 1 39KB/s 218K usec 2 6434KB/s 5833KB/s 11980KB/s 135K usec 1 12KB/s 424 msec 4 3770KB/s 2368KB/s 12258KB/s 604K usec 1 7KB/s 811 msec 8 1941KB/s 432KB/s 10111KB/s 835K usec 1 66KB/s 692K usec 16 939KB/s 319KB/s 11426KB/s 1387K usec 1 30KB/s 2670K usec 32 481KB/s 61KB/s 9318KB/s 2729K usec 1 80KB/s 2413K usec - Variable BW and increasing latencies for random reader. BW first drops to 7KB/s and then picks up to 80KB/s. - Latencies are on the rise for random reader. io-throttle + CFQ ----------------------------------- BW limit group1=10 MB/s BW limit group2=10 MB/s [Multiple Sequential Reader] [Random Reader] nr Max-bandw Min-bandw Agg-bandw Max-latency nr Agg-bandw Max-latency 1 8200KB/s 8200KB/s 8007KB/s 20275 usec 1 37KB/s 217K usec 2 3926KB/s 3919KB/s 7661KB/s 122K usec 1 16KB/s 441 msec 4 2271KB/s 1497KB/s 7672KB/s 611K usec 1 9KB/s 927 msec 8 1113KB/s 513KB/s 7507KB/s 849K usec 1 21KB/s 1020 msec 16 661KB/s 236KB/s 7959KB/s 1679K usec 1 13KB/s 2926 msec 32 292KB/s 109KB/s 7864KB/s 3446K usec 1 8KB/s 3439 msec BW limit group1=5 MB/s BW limit group2=5 MB/s [Multiple Sequential Reader] [Random Reader] nr Max-bandw Min-bandw Agg-bandw Max-latency nr Agg-bandw Max-latency 1 4686KB/s 4686KB/s 4576KB/s 21095 usec 1 57KB/s 219K usec 2 2298KB/s 2179KB/s 4372KB/s 132K usec 1 37KB/s 431K usec 4 1245KB/s 1019KB/s 4449KB/s 324K usec 1 26KB/s 835 msec 8 584KB/s 403KB/s 4109KB/s 833K usec 1 30KB/s 1625K usec 16 346KB/s 252KB/s 4605KB/s 1641K usec 1 129KB/s 3236K usec 32 175KB/s 56KB/s 4269KB/s 3236K usec 1 8KB/s 3235 msec