pNFS SCSI testing
Testing with a 4.6 server on gfs-a24c-02
client is 3.10.0-427.el7.scsi_layout.x86_64
gfs-a24c-01, /mnt/gfs-a24c-02/scsi_pnfs_lun_path, NFS IO only:
[root@gfs-a24c-01 scsi_pnfs_lun_path]# bonnie++ -u bonnie -r 1G -d bonnie_work
Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
gfs-a24c-01.mp 300M 42546 56 87135 11 51915 11 77779 99 +++++ +++ +++++ +++
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 739 4 +++++ +++ 715 7 710 4 6226 14 772 6
gfs-a24c-01.mpc.lab.eng.bos.redhat.com,300M,42546,56,87135,11,51915,11,77779,99,+++++,+++,+++++,+++,16,739,4,+++++,+++,715,7,710,4,6226,14,772,6
gfs-a24c-01, /mnt/gfs-a24c-02/scsi_pnfs_lun_path, SCSI layout:
[root@gfs-a24c-01 scsi_pnfs_lun_path]# bonnie++ -u bonnie -r 1G -d bonnie_work
Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
gfs-a24c-01.mp 300M 42522 56 87042 11 51913 11 77655 99 +++++ +++ +++++ +++
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 709 4 +++++ +++ 718 7 712 4 6232 13 774 6
gfs-a24c-01.mpc.lab.eng.bos.redhat.com,300M,42522,56,87042,11,51913,11,77655,99,+++++,+++,+++++,+++,16,709,4,+++++,+++,718,7,712,4,6232,13,774,6
Looks like a slight performance drop there. Let’s time an allmodconfig kernel build:
gfs-a24c-01, /mnt/gfs-a24c-02/scsi_pnfs_lun_path, NFS IO only:
real 13m36.537s
user 187m19.714s
sys 41m54.048s
gfs-a24c-01, /mnt/gfs-a24c-02/scsi_pnfs_lun_path, SCSI layout:
real 14m9.965s
user 189m9.571s
sys 43m33.948s
real 14m9.305s
user 188m29.822s
sys 43m58.021s
So, same deal. Drop in performance.
Testing with a RHEL7.3 server on gfs-a24c-01: client is same machine - 3.10.0-460.el7.scsi_layout_server_v3.x86_64
gfs-a24c-01, /mnt/gfs-a24c-01/scsi_pnfs_lun_path, NFS IO only:
[root@gfs-a24c-01 scsi_pnfs_lun_path]# bonnie++ -u bonnie -r 1G -d bonnie_work
Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
gfs-a24c-01.mp 300M 62710 84 263895 37 217642 48 76327 99 +++++ +++ +++++ +++
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 1097 7 +++++ +++ 1141 16 1120 7 16480 50 1256 12
gfs-a24c-01.mpc.lab.eng.bos.redhat.com,300M,62710,84,263895,37,217642,48,76327,99,+++++,+++,+++++,+++,16,1097,7,+++++,+++,1141,16,1120,7,16480,50,1256,12
gfs-a24c-01, /mnt/gfs-a24c-01/scsi_pnfs_lun_path, SCSI layout:
[root@gfs-a24c-01 scsi_pnfs_lun_path]# bonnie++ -u bonnie -r 1G -d bonnie_work
Version 1.03e ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
gfs-a24c-01.mp 300M 69876 91 421622 50 421576 49 61536 99 +++++ +++ +++++ +++
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 1127 7 +++++ +++ 1152 14 1119 7 18565 50 1255 12
gfs-a24c-01.mpc.lab.eng.bos.redhat.com,300M,69876,91,421622,50,421576,49,61536,99,+++++,+++,+++++,+++,16,1127,7,+++++,+++,1152,14,1119,7,18565,50,1255,12
This is faster than the two-host test, and shows a marked performance gain from native NFS IO to SCSI layout.
Let’s now time an allmodconfig kernel build
gfs-a24c-01, /mnt/gfs-a24c-01/scsi_pnfs_lun_path, NFS IO only:
real 13m28.380s
user 191m40.546s
sys 44m24.298s
gfs-a24c-01, /mnt/gfs-a24c-01/scsi_pnfs_lun_path, SCSI layout:
real 14m19.311s
user 191m25.000s
sys 45m32.918s
real 14m15.681s
user 191m25.779s
sys 45m44.646s
( .. editing in progress .. )