From Ernest.Beinrohr at axonpro.sk Fri Nov 29 05:49:07 2013 Content-Type: multipart/mixed; boundary="===============0122443233063818967==" MIME-Version: 1.0 From: Ernest Beinrohr To: users at ovirt.org Subject: [Users] Is it possible to limit migration speed and number of concurrent migrations? Date: Fri, 29 Nov 2013 11:49:05 +0100 Message-ID: <52987121.8050905@axonpro.sk> --===============0122443233063818967== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable This is a multi-part message in MIME format. --------------080809060902070208070503 Content-Type: text/plain; charset=3Dwindows-1250; format=3Dflowed Content-Transfer-Encoding: 8bit I want to put some of my hvs to maintenance, but that causes a migration = storm, which causes an temporary unavailability of the hv and ovirt = fences it, while migrations are still running. So I have to migrate one-by-one manually and then put the hv to maintenance. Is it possible to limit migration speed and number of concurrent migrations? -- = Ernest Beinrohr, AXON PRO DevOps, Ing , RHCE = , RHCVA = , LPIC = , VCA , = +421-2--6241-0360 , +421-903--482-603 = icq:28153343, skype:oernii-work , = jabber:oernii(a)jabber.org ------------------------------------------------------------------------ =C2=93For a successful technology, reality must take precedence over public = relations, for Nature cannot be fooled.=C2=94 Richard Feynman --------------080809060902070208070503 Content-Type: text/html; charset=3Dwindows-1250 Content-Transfer-Encoding: 8bit I want to put some of my hvs to maintenance, but that causes a migration storm, which causes an temporary unavailability of the hv and ovirt fences it, while migrations are still running.

So I have to migrate one-by-one manually and then put the hv to maintenance.

Is it possible to limit migration speed and number of concurrent migrations?

--
Ernest Beinrohr, AXON PRO
DevOps, Ing, RHCE, RHCVA, LPIC, VCA, +421-2--6241-0360, +421-903--482-603
icq:28153343, skype:oernii-work, jabber:oernii(a)jabber.org
=C2=93For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled.=C2=94 Richard Feynman

--------------080809060902070208070503-- --===============0122443233063818967== Content-Type: multipart/alternative MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="attachment.bin" VGhpcyBpcyBhIG11bHRpLXBhcnQgbWVzc2FnZSBpbiBNSU1FIGZvcm1hdC4KLS0tLS0tLS0tLS0t LS0wODA4MDkwNjA5MDIwNzAyMDgwNzA1MDMKQ29udGVudC1UeXBlOiB0ZXh0L3BsYWluOyBjaGFy c2V0PXdpbmRvd3MtMTI1MDsgZm9ybWF0PWZsb3dlZApDb250ZW50LVRyYW5zZmVyLUVuY29kaW5n OiA4Yml0CgpJIHdhbnQgdG8gcHV0IHNvbWUgb2YgbXkgaHZzIHRvIG1haW50ZW5hbmNlLCBidXQg dGhhdCBjYXVzZXMgYSBtaWdyYXRpb24gCnN0b3JtLCB3aGljaCBjYXVzZXMgYW4gdGVtcG9yYXJ5 IHVuYXZhaWxhYmlsaXR5IG9mIHRoZSBodiBhbmQgb3ZpcnQgCmZlbmNlcyBpdCwgd2hpbGUgbWln cmF0aW9ucyBhcmUgc3RpbGwgcnVubmluZy4KClNvIEkgaGF2ZSB0byBtaWdyYXRlIG9uZS1ieS1v bmUgbWFudWFsbHkgYW5kIHRoZW4gcHV0IHRoZSBodiB0byBtYWludGVuYW5jZS4KCklzIGl0IHBv c3NpYmxlIHRvIGxpbWl0IG1pZ3JhdGlvbiBzcGVlZCBhbmQgbnVtYmVyIG9mIGNvbmN1cnJlbnQg bWlncmF0aW9ucz8KCi0tIApFcm5lc3QgQmVpbnJvaHIsIEFYT04gUFJPCkRldk9wcywgSW5nIDxo dHRwOi8vd3d3LmJlaW5yb2hyLnNrL2luZy5waHA+LCBSSENFIAo8aHR0cDovL3d3dy5iZWlucm9o ci5zay9yaGNlLnBocD4sIFJIQ1ZBIAo8aHR0cDovL3d3dy5iZWlucm9oci5zay9yaGNlLnBocD4s IExQSUMgCjxodHRwOi8vd3d3LmJlaW5yb2hyLnNrL2xwaWMucGhwPiwgVkNBIDxodHRwOi8vd3d3 LmJlaW5yb2hyLnNrL3ZjYS5waHA+LCAKKzQyMS0yLS02MjQxLTAzNjAgPGNhbGx0bzovLys0MjEt Mi0tNjI0MS0wMzYwPiwgKzQyMS05MDMtLTQ4Mi02MDMgCjxjYWxsdG86Ly8rNDIxLTkwMy0tNDgy LTYwMz4KaWNxOjI4MTUzMzQzLCBza3lwZTpvZXJuaWktd29yayA8Y2FsbHRvOi8vb2VybmlpLXdv cms+LCAKamFiYmVyOm9lcm5paUBqYWJiZXIub3JnCi0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLQqTRm9yIGEgc3Vj Y2Vzc2Z1bCB0ZWNobm9sb2d5LCByZWFsaXR5IG11c3QgdGFrZSBwcmVjZWRlbmNlIG92ZXIgcHVi bGljIApyZWxhdGlvbnMsIGZvciBOYXR1cmUgY2Fubm90IGJlIGZvb2xlZC6UIFJpY2hhcmQgRmV5 bm1hbgoKCi0tLS0tLS0tLS0tLS0tMDgwODA5MDYwOTAyMDcwMjA4MDcwNTAzCkNvbnRlbnQtVHlw ZTogdGV4dC9odG1sOyBjaGFyc2V0PXdpbmRvd3MtMTI1MApDb250ZW50LVRyYW5zZmVyLUVuY29k aW5nOiA4Yml0Cgo8aHRtbD4KICA8aGVhZD4KCiAgICA8bWV0YSBodHRwLWVxdWl2PSJjb250ZW50 LXR5cGUiIGNvbnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD13aW5kb3dzLTEyNTAiPgogIDwvaGVh ZD4KICA8Ym9keSBiZ2NvbG9yPSIjRkZGRkZGIiB0ZXh0PSIjMDAwMDAwIj4KICAgIEkgd2FudCB0 byBwdXQgc29tZSBvZiBteSBodnMgdG8gbWFpbnRlbmFuY2UsIGJ1dCB0aGF0IGNhdXNlcyBhCiAg ICBtaWdyYXRpb24gc3Rvcm0sIHdoaWNoIGNhdXNlcyBhbiB0ZW1wb3JhcnkgdW5hdmFpbGFiaWxp dHkgb2YgdGhlIGh2CiAgICBhbmQgb3ZpcnQgZmVuY2VzIGl0LCB3aGlsZSBtaWdyYXRpb25zIGFy ZSBzdGlsbCBydW5uaW5nLjxicj4KICAgIDxicj4KICAgIFNvIEkgaGF2ZSB0byBtaWdyYXRlIG9u ZS1ieS1vbmUgbWFudWFsbHkgYW5kIHRoZW4gcHV0IHRoZSBodiB0bwogICAgbWFpbnRlbmFuY2Uu PGJyPgogICAgPGJyPgogICAgSXMgaXQgcG9zc2libGUgdG8gbGltaXQgbWlncmF0aW9uIHNwZWVk IGFuZCBudW1iZXIgb2YgY29uY3VycmVudAogICAgbWlncmF0aW9ucz88YnI+CiAgICA8YnI+CiAg ICA8ZGl2IGNsYXNzPSJtb3otc2lnbmF0dXJlIj4tLSA8YnI+CiAgICAgIDxkaXYgaWQ9Im9lcm5p aV9mb290ZXIiIHN0eWxlPSJjb2xvcjogZ3JheTsiPgogICAgICAgIDxzcGFuIHN0eWxlPSJmb250 LWZhbWlseTogTHVjaWRhIENvbnNvbGUsIEx1eGkgTW9ubywgQ291cmllciwKICAgICAgICAgIG1v bm9zcGFjZTsgZm9udC1zaXplOiA5MCU7Ij4KICAgICAgICAgIEVybmVzdCBCZWlucm9ociwgQVhP TiBQUk88YnI+CiAgICAgICAgICBEZXZPcHMsCiAgICAgICAgICA8YSBzdHlsZT0idGV4dC1kZWNv cmF0aW9uOiBub25lOyBjb2xvcjogZ3JheTsiCiAgICAgICAgICAgIGhyZWY9Imh0dHA6Ly93d3cu YmVpbnJvaHIuc2svaW5nLnBocCI+SW5nPC9hPiwgPGEKICAgICAgICAgICAgc3R5bGU9InRleHQt ZGVjb3JhdGlvbjogbm9uZTsgY29sb3I6IGdyYXk7IgogICAgICAgICAgICBocmVmPSJodHRwOi8v d3d3LmJlaW5yb2hyLnNrL3JoY2UucGhwIj5SSENFPC9hPiwgPGEKICAgICAgICAgICAgc3R5bGU9 InRleHQtZGVjb3JhdGlvbjogbm9uZTsgY29sb3I6IGdyYXk7IgogICAgICAgICAgICBocmVmPSJo dHRwOi8vd3d3LmJlaW5yb2hyLnNrL3JoY2UucGhwIj5SSENWQTwvYT4sIDxhCiAgICAgICAgICAg IHN0eWxlPSJ0ZXh0LWRlY29yYXRpb246IG5vbmU7IGNvbG9yOiBncmF5OyIKICAgICAgICAgICAg aHJlZj0iaHR0cDovL3d3dy5iZWlucm9oci5zay9scGljLnBocCI+TFBJQzwvYT4sIDxhCiAgICAg ICAgICAgIHN0eWxlPSJ0ZXh0LWRlY29yYXRpb246IG5vbmU7IGNvbG9yOiBncmF5OyIKICAgICAg ICAgICAgaHJlZj0iaHR0cDovL3d3dy5iZWlucm9oci5zay92Y2EucGhwIj5WQ0E8L2E+LCA8YQog ICAgICAgICAgICBzdHlsZT0idGV4dC1kZWNvcmF0aW9uOiBub25lOyBjb2xvcjogZ3JheTsiCiAg ICAgICAgICAgIGhyZWY9ImNhbGx0bzovLys0MjEtMi0tNjI0MS0wMzYwIj4rNDIxLTItLTYyNDEt MDM2MDwvYT4sIDxhCiAgICAgICAgICAgIHN0eWxlPSJ0ZXh0LWRlY29yYXRpb246IG5vbmU7IGNv bG9yOiBncmF5OyIKICAgICAgICAgICAgaHJlZj0iY2FsbHRvOi8vKzQyMS05MDMtLTQ4Mi02MDMi Pis0MjEtOTAzLS00ODItNjAzPC9hPjxicj4KICAgICAgICAgIGljcToyODE1MzM0MywgPGEgc3R5 bGU9InRleHQtZGVjb3JhdGlvbjogbm9uZTsgY29sb3I6IGdyYXk7IgogICAgICAgICAgICBocmVm PSJjYWxsdG86Ly9vZXJuaWktd29yayI+c2t5cGU6b2VybmlpLXdvcms8L2E+LAogICAgICAgICAg PGEgY2xhc3M9Im1vei10eHQtbGluay1hYmJyZXZpYXRlZCIgaHJlZj0ibWFpbHRvOmphYmJlcjpv ZXJuaWlAamFiYmVyLm9yZyI+amFiYmVyOm9lcm5paUBqYWJiZXIub3JnPC9hPgogICAgICAgICAg PGJyPgogICAgICAgIDwvc3Bhbj4KICAgICAgICA8aHIgc3R5bGU9ImhlaWdodDogMXB4OyB3aWR0 aDogOTUlIj4gPHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToKICAgICAgICAgIDcwJTsiPgogICAgICAg ICAgk0ZvciBhIHN1Y2Nlc3NmdWwgdGVjaG5vbG9neSwgcmVhbGl0eSBtdXN0IHRha2UgcHJlY2Vk ZW5jZQogICAgICAgICAgb3ZlciBwdWJsaWMgcmVsYXRpb25zLCBmb3IgTmF0dXJlIGNhbm5vdCBi ZSBmb29sZWQulCBSaWNoYXJkCiAgICAgICAgICBGZXlubWFuIDwvc3Bhbj4gPC9kaXY+CiAgICA8 L2Rpdj4KICAgIDxpbWcKICAgICAgc3JjPSJodHRwOi8vbm9qc3N0YXRzLmFwcHNwb3QuY29tL1VB LTQ0NDk3MDk2LTEvZW1haWwuYmVpbnJvaHIuc2siCiAgICAgIG1vei1kby1ub3Qtc2VuZD0idHJ1 ZSIgaGVpZ2h0PSIxIiBib3JkZXI9IjAiIHdpZHRoPSIxIj48YnI+CiAgPC9ib2R5Pgo8L2h0bWw+ CgotLS0tLS0tLS0tLS0tLTA4MDgwOTA2MDkwMjA3MDIwODA3MDUwMy0tCg== --===============0122443233063818967==-- From danken at redhat.com Fri Nov 29 07:09:26 2013 Content-Type: multipart/mixed; boundary="===============5788852283658405406==" MIME-Version: 1.0 From: Dan Kenigsberg To: users at ovirt.org Subject: Re: [Users] Is it possible to limit migration speed and number of concurrent migrations? Date: Fri, 29 Nov 2013 12:08:27 +0000 Message-ID: <20131129120827.GG9681@redhat.com> In-Reply-To: 52987121.8050905@axonpro.sk --===============5788852283658405406== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Fri, Nov 29, 2013 at 11:49:05AM +0100, Ernest Beinrohr wrote: > I want to put some of my hvs to maintenance, but that causes a > migration storm, which causes an temporary unavailability of the hv > and ovirt fences it, while migrations are still running. > = > So I have to migrate one-by-one manually and then put the hv to maintenan= ce. > = > Is it possible to limit migration speed and number of concurrent migratio= ns? You could set max_outgoing_migrations to 1 (in each /etc/vdsm/vdsm.conf), but even a single VM migrating may choke your connection (depends which wins, the CPU running qemu, or your bandwidth). Currently, your only option is to define a migration network for your cluster (say, over a different nic, or over a vlan), and use tools external to ovirt to throtle the bandwidth on it (virsh net-edit and http://libvirt.org/formatnetwork.html#elementQoS can come up handy). ovirt-3.4 should expose the ability to set QoS limits on migration networks. Dan. --===============5788852283658405406==-- From iheim at redhat.com Fri Nov 29 17:26:12 2013 Content-Type: multipart/mixed; boundary="===============5776899128031258746==" MIME-Version: 1.0 From: Itamar Heim To: users at ovirt.org Subject: Re: [Users] Is it possible to limit migration speed and number of concurrent migrations? Date: Sat, 30 Nov 2013 00:25:31 +0200 Message-ID: <5299145B.6080602@redhat.com> In-Reply-To: 20131129120827.GG9681@redhat.com --===============5776899128031258746== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On 11/29/2013 02:08 PM, Dan Kenigsberg wrote: > On Fri, Nov 29, 2013 at 11:49:05AM +0100, Ernest Beinrohr wrote: >> I want to put some of my hvs to maintenance, but that causes a >> migration storm, which causes an temporary unavailability of the hv >> and ovirt fences it, while migrations are still running. >> >> So I have to migrate one-by-one manually and then put the hv to maintena= nce. >> >> Is it possible to limit migration speed and number of concurrent migrati= ons? > > You could set max_outgoing_migrations to 1 (in each > /etc/vdsm/vdsm.conf), but even a single VM migrating may choke your > connection (depends which wins, the CPU running qemu, or your > bandwidth). > > Currently, your only option is to define a migration network for your > cluster (say, over a different nic, or over a vlan), and use tools > external to ovirt to throtle the bandwidth on it (virsh net-edit > and http://libvirt.org/formatnetwork.html#elementQoS can > come up handy). ovirt-3.4 should expose the ability to set QoS limits on > migration networks. > > Dan. > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users > I thought we can control both the number of concurrent migrations and = the bandwidth per migration which defaults to 30MB/s? --===============5776899128031258746==-- From danken at redhat.com Sun Dec 1 17:55:08 2013 Content-Type: multipart/mixed; boundary="===============5051735161989245860==" MIME-Version: 1.0 From: Dan Kenigsberg To: users at ovirt.org Subject: Re: [Users] Is it possible to limit migration speed and number of concurrent migrations? Date: Sun, 01 Dec 2013 22:54:08 +0000 Message-ID: <20131201225408.GA18153@redhat.com> In-Reply-To: 5299145B.6080602@redhat.com --===============5051735161989245860== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable On Sat, Nov 30, 2013 at 12:25:31AM +0200, Itamar Heim wrote: > On 11/29/2013 02:08 PM, Dan Kenigsberg wrote: > >On Fri, Nov 29, 2013 at 11:49:05AM +0100, Ernest Beinrohr wrote: > >>I want to put some of my hvs to maintenance, but that causes a > >>migration storm, which causes an temporary unavailability of the hv > >>and ovirt fences it, while migrations are still running. > >> > >>So I have to migrate one-by-one manually and then put the hv to mainten= ance. > >> > >>Is it possible to limit migration speed and number of concurrent migrat= ions? > > > >You could set max_outgoing_migrations to 1 (in each > >/etc/vdsm/vdsm.conf), but even a single VM migrating may choke your > >connection (depends which wins, the CPU running qemu, or your > >bandwidth). > > > >Currently, your only option is to define a migration network for your > >cluster (say, over a different nic, or over a vlan), and use tools > >external to ovirt to throtle the bandwidth on it (virsh net-edit > > and http://libvirt.org/formatnetwork.html#elementQoS can > >come up handy). ovirt-3.4 should expose the ability to set QoS limits on > >migration networks. > > > = > I thought we can control both the number of concurrent migrations > and the bandwidth per migration which defaults to 30MB/s? You are perfectly right: I forgot about 'migration_max_bandwidth', which vdsm sets to 32MiBps (per single migration). Sorry for misleading the list and its archives. --===============5051735161989245860==--