[ovirt-users] Poor iSCSI Performance

[John Florian]

Hi all,

I've been trying to resolve a storage performance issue but have had no
luck in identifying the exact cause.  I have my storage domain on iSCSI
and I can get the expected performance (limited by the Gbit Ethernet)
when running bonnie++ on:

  * a regular physical machine configured with the iSCSI initiator
    connected to a dedicated iSCSI test target -- thus oVirt and VM
    technology are completely out of the picture
  * my oVirt host with the initiator connected to that same dedicated
    target -- thus I have an iSCSI connection on the oVirt host but I'm
    not using the iSCSI connection provided by oVirt's storage domain
  * a VM (hosted by oVirt) with the initiator (inside the VM) connected
    to that target -- thus bypassing oVirt's storage domain and the
    virtual disk it provides this VM

However, if I just use a regular virtual disk via oVirt's storage domain
the performance is much worse.  I've tried both VirtIO and VirtIO-SCSI
and have found no appreciable difference.

Here's a typical example of the poor performance I get (as tested with
bonnie++) with the normal virtual disk setup:

# bonnie++ -d . -r 2048 -u root:root
<snip>
Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 
/sec %CP
narvi-f21.double 4G   806  91 18507   1 15675   1  3174  56 33175   1
176.4   3
Latency             15533us    8142ms    2440ms     262ms    1289ms    
780ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
narvi-f21.doubledog -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
/sec %CP
                 16 13641  24 +++++ +++ 22702  17 18919  31 +++++ +++
+++++ +++
Latency             27724us     247us     292us      71us      30us    
172us

For comparison, here's what I see if I run the same test, same VM, same
host but this time the file system is mounted from a device obtained
using iscsi-initiator-utils within the VM, i.e., the 3rd bullet config
above:

bonnie++ -d . -r 2048 -u root:root
<snip>
Version  1.96       ------Sequential Output------ --Sequential Input-
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP 
/sec %CP
narvi-f21.double 4G  2051  89 103877   4 36286   3  4803  88 88166   4
163.6   3
Latency              7724us     191ms     396ms   48734us   73004us   
1645ms
Version  1.96       ------Sequential Create------ --------Random
Create--------
narvi-f21.doubledog -Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP 
/sec %CP
                 16  6531  18 +++++ +++ 16388  20  5924  15 +++++ +++
17906  23
Latency             15623us      64us      92us    1281us      14us    
256us

My host is Fedora 20 running oVirt 3.5 (hosted-engine).  VM is running
Fedora Server 21.  Tonight I tried updating the host with the Fedora
virt preview repo and I didn't see any significant change in the
performance.  Where should I look next?


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150217/85ab2a4a/attachment.html>