------=_Part_10123618_874496395.1390595641747 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Adding Bob Sibley to this thread. ----- Original Message -----
From: "Steve Dainard" <sdainard@miovision.com> To: "Andrew Cathrow" <acathrow@redhat.com> Cc: "Ronen Hod" <rhod@redhat.com>, "users" <users@ovirt.org>, "Sanjay Rao" <srao@redhat.com>, "Itamar Heim" <iheim@redhat.com> Sent: Friday, January 24, 2014 3:01:25 PM Subject: Re: [Users] Extremely poor disk access speeds in Windows guest
Not sure what a good method to bench this would be, but:
An NFS mount point on virt host: [root@ovirt001 iso-store]# dd if=/dev/zero of=test1 bs=4k count=100000 100000+0 records in 100000+0 records out 409600000 bytes (410 MB) copied, 3.95399 s, 104 MB/s
Raw brick performance on gluster server (yes, I know I shouldn't write directly to the brick): [root@gluster1 iso-store]# dd if=/dev/zero of=test bs=4k count=100000 100000+0 records in 100000+0 records out 409600000 bytes (410 MB) copied, 3.06743 s, 134 MB/s
Gluster mount point on gluster server: [root@gluster1 iso-store]# dd if=/dev/zero of=test bs=4k count=100000 100000+0 records in 100000+0 records out 409600000 bytes (410 MB) copied, 19.5766 s, 20.9 MB/s
The storage servers are a bit older, but are both dual socket quad core opterons with 4x 7200rpm drives.
I'm in the process of setting up a share from my desktop and I'll see if I can bench between the two systems. Not sure if my ssd will impact the tests, I've heard there isn't an advantage using ssd storage for glusterfs.
Does anyone have a hardware reference design for glusterfs as a backend for virt? Or is there a benchmark utility?
Steve Dainard IT Infrastructure Manager Miovision | Rethink Traffic 519-513-2407 ex.250 877-646-8476 (toll-free)
Blog | LinkedIn | Twitter | Facebook Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON, Canada | N2C 1L3 This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
On Thu, Jan 23, 2014 at 7:18 PM, Andrew Cathrow < acathrow@redhat.com > wrote:
Are we sure that the issue is the guest I/O - what's the raw performance on the host accessing the gluster storage?
From: "Steve Dainard" < sdainard@miovision.com >
To: "Itamar Heim" < iheim@redhat.com >
Cc: "Ronen Hod" < rhod@redhat.com >, "users" < users@ovirt.org >, "Sanjay Rao" < srao@redhat.com >
Sent: Thursday, January 23, 2014 4:56:58 PM
Subject: Re: [Users] Extremely poor disk access speeds in Windows guest
I have two options, virtio and virtio-scsi.
I was using virtio, and have also attempted virtio-scsi on another Windows guest with the same results.
Using the newest drivers, virtio-win-0.1-74.iso.
Steve Dainard
IT Infrastructure Manager
Miovision | Rethink Traffic
519-513-2407 ex.250
877-646-8476 (toll-free)
Blog | LinkedIn | Twitter | Facebook
Miovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON, Canada | N2C 1L3
This e-mail may contain information that is privileged or confidential. If you are not the intended recipient, please delete the e-mail and any attachments and notify us immediately.
On Thu, Jan 23, 2014 at 4:24 PM, Itamar Heim < iheim@redhat.com > wrote:
On 01/23/2014 07:46 PM, Steve Dainard wrote:
Backing Storage: Gluster Replica
Storage Domain: NFS
Ovirt Hosts: CentOS 6.5
Ovirt version: 3.3.2
Network: GigE
# of VM's: 3 - two Linux guests are idle, one Windows guest is
installing updates.
I've installed a Windows 2008 R2 guest with virtio disk, and all the
drivers from the latest virtio iso. I've also installed the spice agent
drivers.
Guest disk access is horribly slow, Resource monitor during Windows
updates shows Disk peaking at 1MB/sec (scale never increases) and Disk
Queue Length Peaking at 5 and looks to be sitting at that level 99% of
the time. 113 updates in Windows has been running solidly for about 2.5
hours and is at 89/113 updates complete.
virtio-block or virtio-scsi?
which windows guest driver version for that?
I can't say my Linux guests are blisteringly fast, but updating a guest
from RHEL 6.3 fresh install to 6.5 took about 25 minutes.
If anyone has any ideas, please let me know - I haven't found any tuning
docs for Windows guests that could explain this issue.
Thanks,
*Steve Dainard *
______________________________ _________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/ mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org
<b>Subject: </b>Re: [Users] Extremely poor disk access speeds in Windows g= uest<br><div><br></div><div dir=3D"ltr">Not sure what a good method to benc= h this would be, but:<div><br></div><div>An NFS mount point on virt host:</=
<div class=3D"h5"><br><div><br></div><div dir=3D"ltr">I have two options, = virtio and virtio-scsi.<div><br></div><div>I was using virtio, and have als= o attempted virtio-scsi on another Windows guest with the same results.</di= v><div><br></div><div>Using the newest drivers, virtio-win-0.1-74.iso.</div= </div><div class=3D"gmail_extra"><br clear=3D"all"><div><div dir=3D"ltr"><= span style=3D"font-family:arial,sans-serif;font-size:16px" data-mce-style= =3D"font-family: arial,sans-serif; font-size: 16px;"><strong>Steve Dai= nard </strong></span><span style=3D"font-size:12px" data-mce-style=3D"= font-size: 12px;"></span><br> <span style=3D"font-family:arial,sans-serif;f= ont-size:12px" data-mce-style=3D"font-family: arial,sans-serif; font-size: = 12px;">IT Infrastructure Manager<br> <a href=3D"http://miovision.com/" targ= et=3D"_blank" data-mce-href=3D"http://miovision.com/">Miovision</a> | <em>R= ethink Traffic</em><br> <a href=3D"tel:519-513-2407" target=3D"_blank" data= -mce-href=3D"tel:519-513-2407">519-513-2407</a> ex.250<br> <a href=3D"tel:8= 77-646-8476" target=3D"_blank" data-mce-href=3D"tel:877-646-8476">877-646-8= 476</a> (toll-free)<br> <br> <strong style=3D"font-family:arial,sans-serif;= font-size:13px;color:#999999" data-mce-style=3D"font-family: arial,sans-ser= if; font-size: 13px; color: #999999;"><a href=3D"http://miovision.com/blog"= target=3D"_blank" data-mce-href=3D"http://miovision.com/blog">Blog</a>&nbs=
-- Sanjay Rao Principal Performance Engineer Phone: 978-392-2479 Red Hat, Inc. FAX: 978-392-1001 314 Littleton Road Email: srao@redhat.com Westford, MA 01886 ------=_Part_10123618_874496395.1390595641747 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable <html><body><div style=3D"font-family: times new roman, new york, times, se= rif; font-size: 12pt; color: #000000"><div>Adding Bob Sibley to this thread= . <br></div><div><br></div><div><br></div><hr id=3D"zwchr"><blockquote styl= e=3D"border-left:2px solid #1010FF;margin-left:5px;padding-left:5px;color:#= 000;font-weight:normal;font-style:normal;text-decoration:none;font-family:H= elvetica,Arial,sans-serif;font-size:12pt;" data-mce-style=3D"border-left: 2= px solid #1010FF; margin-left: 5px; padding-left: 5px; color: #000; font-we= ight: normal; font-style: normal; text-decoration: none; font-family: Helve= tica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"Steve Dainard" <s= dainard@miovision.com><br><b>To: </b>"Andrew Cathrow" <acathrow@redha= t.com><br><b>Cc: </b>"Ronen Hod" <rhod@redhat.com>, "users" <us= ers@ovirt.org>, "Sanjay Rao" <srao@redhat.com>, "Itamar Heim" <= iheim@redhat.com><br><b>Sent: </b>Friday, January 24, 2014 3:01:25 PM<br= div><div><div>[root@ovirt001 iso-store]# dd if=3D/dev/zero of=3Dtest1 bs=3D= 4k count=3D100000</div><div>100000+0 records in</div><div>100000+0 records = out</div><div>409600000 bytes (410 MB) copied, 3.95399 s, 104 MB/s</div></d= iv><div><br></div><div>Raw brick performance on gluster server (yes, I know= I shouldn't write directly to the brick):</div><div><div>[root@gluster1 is= o-store]# dd if=3D/dev/zero of=3Dtest bs=3D4k count=3D100000</div><div>1000= 00+0 records in</div><div>100000+0 records out</div><div>409600000 bytes (4= 10 MB) copied, 3.06743 s, 134 MB/s</div></div><div><br></div><div>Gluster m= ount point on gluster server:</div><div><div>[root@gluster1 iso-store]# dd = if=3D/dev/zero of=3Dtest bs=3D4k count=3D100000</div><div>100000+0 records = in</div><div>100000+0 records out</div><div>409600000 bytes (410 MB) copied= , 19.5766 s, 20.9 MB/s</div></div><div><br></div><div>The storage servers a= re a bit older, but are both dual socket quad core opterons with 4x 7200rpm= drives. </div><div><br></div><div>I'm in the process of setting up a = share from my desktop and I'll see if I can bench between the two systems. = Not sure if my ssd will impact the tests, I've heard there isn't an advanta= ge using ssd storage for glusterfs.</div><div><br></div><div>Does anyone ha= ve a hardware reference design for glusterfs as a backend for virt? Or is t= here a benchmark utility?</div></div><div class=3D"gmail_extra"><br clear= =3D"all"><div><div dir=3D"ltr"><span style=3D"font-family:arial,sans-serif;= font-size:16px" data-mce-style=3D"font-family: arial,sans-serif; font-size:= 16px;"><strong>Steve Dainard </strong></span><span style=3D"font= -size:12px" data-mce-style=3D"font-size: 12px;"></span><br> <span style=3D"= font-family:arial,sans-serif;font-size:12px" data-mce-style=3D"font-family:= arial,sans-serif; font-size: 12px;">IT Infrastructure Manager<br> <a href= =3D"http://miovision.com/" target=3D"_blank" data-mce-href=3D"http://miovis= ion.com/">Miovision</a> | <em>Rethink Traffic</em><br> 519-513-2407 ex.250<= br> 877-646-8476 (toll-free)<br> <br> <strong style=3D"font-family:arial,sa= ns-serif;font-size:13px;color:#999999" data-mce-style=3D"font-family: arial= ,sans-serif; font-size: 13px; color: #999999;"><a href=3D"http://miovision.= com/blog" target=3D"_blank" data-mce-href=3D"http://miovision.com/blog">Blo= g</a> | </strong><span style=3D"font-family: arial,sans-serif; = font-size: 13px; color: #999999;" data-mce-style=3D"font-family: arial,sans= -serif; font-size: 13px; color: #999999;" color=3D"#999999"><strong><a href= =3D"https://www.linkedin.com/company/miovision-technologies" target=3D"_bla= nk" data-mce-href=3D"https://www.linkedin.com/company/miovision-technologie= s">LinkedIn</a> | <a href=3D"https://twitter.com/miovision= " target=3D"_blank" data-mce-href=3D"https://twitter.com/miovision">Twitter= </a> | <a href=3D"https://www.facebook.com/miovision" targ= et=3D"_blank" data-mce-href=3D"https://www.facebook.com/miovision">Facebook= </a></strong></span> </span><hr style=3D"font-family:arial,sans-serif;font-= size:13px;color:#333333;clear:both" data-mce-style=3D"font-family: arial,sa= ns-serif; font-size: 13px; color: #333333; clear: both;"><div style=3D"colo= r:#999999;font-family:arial,sans-serif;font-size:13px;padding-top:5px" data= -mce-style=3D"color: #999999; font-family: arial,sans-serif; font-size: 13p= x; padding-top: 5px;"><span style=3D"font-family:arial,sans-serif;font-size= :12px" data-mce-style=3D"font-family: arial,sans-serif; font-size: 12px;">M= iovision Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON, C= anada | N2C 1L3</span><br> <span style=3D"font-family:arial,sans-serif= ;font-size:12px" data-mce-style=3D"font-family: arial,sans-serif; font-size= : 12px;">This e-mail may contain information that is privileged or confiden= tial. If you are not the intended recipient, please delete the e-mail and a= ny attachments and notify us immediately.</span></div></div></div><br><div>= <br></div><div class=3D"gmail_quote">On Thu, Jan 23, 2014 at 7:18 PM, Andre= w Cathrow <span dir=3D"ltr"><<a href=3D"mailto:acathrow@redhat.com" targ= et=3D"_blank" data-mce-href=3D"mailto:acathrow@redhat.com">acathrow@redhat.= com</a>></span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"mar= gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style= =3D"margin: 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;"><d= iv><div style=3D"font-size:10pt;font-family:arial,helvetica,sans-serif" dat= a-mce-style=3D"font-size: 10pt; font-family: arial,helvetica,sans-serif;">A= re we sure that the issue is the guest I/O - what's the raw performance on = the host accessing the gluster storage?<br><div><br></div><hr><blockquote s= tyle=3D"padding-left:5px;font-size:12pt;font-style:normal;margin-left:5px;f= ont-family:Helvetica,Arial,sans-serif;text-decoration:none;font-weight:norm= al;border-left:2px solid #1010ff" data-mce-style=3D"padding-left: 5px; font= -size: 12pt; font-style: normal; margin-left: 5px; font-family: Helvetica,A= rial,sans-serif; text-decoration: none; font-weight: normal; border-left: 2= px solid #1010ff;"><b>From: </b>"Steve Dainard" <<a href=3D"mailto:sdain= ard@miovision.com" target=3D"_blank" data-mce-href=3D"mailto:sdainard@miovi= sion.com">sdainard@miovision.com</a>><br><b>To: </b>"Itamar Heim" <<a= href=3D"mailto:iheim@redhat.com" target=3D"_blank" data-mce-href=3D"mailto= :iheim@redhat.com">iheim@redhat.com</a>><br> <b>Cc: </b>"Ronen Hod" <= <a href=3D"mailto:rhod@redhat.com" target=3D"_blank" data-mce-href=3D"mailt= o:rhod@redhat.com">rhod@redhat.com</a>>, "users" <<a href=3D"mailto:u= sers@ovirt.org" target=3D"_blank" data-mce-href=3D"mailto:users@ovirt.org">= users@ovirt.org</a>>, "Sanjay Rao" <<a href=3D"mailto:srao@redhat.com= " target=3D"_blank" data-mce-href=3D"mailto:srao@redhat.com">srao@redhat.co= m</a>><br> <b>Sent: </b>Thursday, January 23, 2014 4:56:58 PM<br><b>Subj= ect: </b>Re: [Users] Extremely poor disk access speeds in Windows guest<div= p; | </strong><span style=3D"font-family: arial,sans-serif; font-size= : 13px; color: #999999;" data-mce-style=3D"font-family: arial,sans-serif; f= ont-size: 13px; color: #999999;" color=3D"#999999"><strong><a href=3D"https= ://www.linkedin.com/company/miovision-technologies" target=3D"_blank" data-= mce-href=3D"https://www.linkedin.com/company/miovision-technologies">Linked= In</a> | <a href=3D"https://twitter.com/miovision" target= =3D"_blank" data-mce-href=3D"https://twitter.com/miovision">Twitter</a>&nbs= p; | <a href=3D"https://www.facebook.com/miovision" target=3D"_b= lank" data-mce-href=3D"https://www.facebook.com/miovision">Facebook</a></st= rong></span> </span><hr style=3D"font-family:arial,sans-serif;font-size:13p= x;color:#333333;clear:both" data-mce-style=3D"font-family: arial,sans-serif= ; font-size: 13px; color: #333333; clear: both;"><div style=3D"color:#99999= 9;font-family:arial,sans-serif;font-size:13px;padding-top:5px" data-mce-sty= le=3D"color: #999999; font-family: arial,sans-serif; font-size: 13px; paddi= ng-top: 5px;"><span style=3D"font-family:arial,sans-serif;font-size:12px" d= ata-mce-style=3D"font-family: arial,sans-serif; font-size: 12px;">Miovision= Technologies Inc. | 148 Manitou Drive, Suite 101, Kitchener, ON, Canada |&= nbsp;N2C 1L3</span><br> <span style=3D"font-family:arial,sans-serif;font-si= ze:12px" data-mce-style=3D"font-family: arial,sans-serif; font-size: 12px;"=
This e-mail may contain information that is privileged or confidential. If= you are not the intended recipient, please delete the e-mail and any attac= hments and notify us immediately.</span></div></div></div><br><div><br></di= v><div class=3D"gmail_quote">On Thu, Jan 23, 2014 at 4:24 PM, Itamar Heim <= span dir=3D"ltr"><<a href=3D"mailto:iheim@redhat.com" target=3D"_blank" = data-mce-href=3D"mailto:iheim@redhat.com">iheim@redhat.com</a>></span> w= rote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;borde= r-left:1px #ccc solid;padding-left:1ex" data-mce-style=3D"margin: 0 0 0 .8e= x; border-left: 1px #ccc solid; padding-left: 1ex;"><div>On 01/23/2014 07:4= 6 PM, Steve Dainard wrote:<br><blockquote class=3D"gmail_quote" style=3D"ma= rgin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style= =3D"margin: 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;">Ba= cking Storage: Gluster Replica<br> Storage Domain: NFS<br> Ovirt Hosts: Cen= tOS 6.5<br> Ovirt version: 3.3.2<br> Network: GigE<br> # of VM's: 3 - two L= inux guests are idle, one Windows guest is<br> installing updates.<br> <br>= I've installed a Windows 2008 R2 guest with virtio disk, and all the<br> d= rivers from the latest virtio iso. I've also installed the spice agent<br> = drivers.<br> <br> Guest disk access is horribly slow, Resource monitor duri= ng Windows<br> updates shows Disk peaking at 1MB/sec (scale never increases= ) and Disk<br> Queue Length Peaking at 5 and looks to be sitting at that le= vel 99% of<br> the time. 113 updates in Windows has been running solidly fo= r about 2.5<br> hours and is at 89/113 updates complete.<br></blockquote><b= r></div>virtio-block or virtio-scsi?<br> which windows guest driver version= for that?<br> <br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex" data-mce-style=3D"margin:= 0 0 0 .8ex; border-left: 1px #ccc solid; padding-left: 1ex;"><div><br> I c= an't say my Linux guests are blisteringly fast, but updating a guest<br> fr= om RHEL 6.3 fresh install to 6.5 took about 25 minutes.<br> <br> If anyone = has any ideas, please let me know - I haven't found any tuning<br> docs for= Windows guests that could explain this issue.<br> <br> Thanks,<br> <br> <b= r></div>*Steve Dainard *<br> <br> <br> <br> ______________________________<= span style=3D"text-decoration: underline;" data-mce-style=3D"text-decoratio= n: underline;"></span>_________________<br> Users mailing list<br> <a href= =3D"mailto:Users@ovirt.org" target=3D"_blank" data-mce-href=3D"mailto:Users= @ovirt.org">Users@ovirt.org</a><br> <a href=3D"http://lists.ovirt.org/mailm= an/listinfo/users" target=3D"_blank" data-mce-href=3D"http://lists.ovirt.or= g/mailman/listinfo/users">http://lists.ovirt.org/<span style=3D"text-decora= tion: underline;" data-mce-style=3D"text-decoration: underline;"></span>mai= lman/listinfo/users</a><br> <br></blockquote><br></blockquote></div><br></d= iv><br>_______________________________________________<br>Users mailing lis= t<br><a href=3D"mailto:Users@ovirt.org" target=3D"_blank" data-mce-href=3D"= mailto:Users@ovirt.org">Users@ovirt.org</a><br><a href=3D"http://lists.ovir= t.org/mailman/listinfo/users" target=3D"_blank" data-mce-href=3D"http://lis= ts.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinf= o/users</a><br></div></div></blockquote><br></div></div></blockquote></div>= <br></div></blockquote><div><br><br></div><div><br></div><div>-- <br></div>= <div><span name=3D"x"></span>Sanjay Rao <br>Principal Performance Engineer = Phone: 978-3= 92-2479 <br>Red Hat, Inc. = = ;FAX: 978-392-1001 <br>314 Littleton Road = ; Ema= il: srao@redhat.com <br>Westford, MA 01886 <span name=3D"x"></span><br></di= v></div></body></html> ------=_Part_10123618_874496395.1390595641747--