5:38 a.m.
Hi
The update notifier in the webadmin was originally designed to alert for
new vdsm* packages. Now, I noticed that available update of virt
packages and more are notified. I know that hot updating qemu-kvm
package does break vms that are running on concerned hosts, but what
about other one like libvirt-client? I know it is recommended to put in
maintenance while updating, but can we update some minor packages
without waiting for migration?
--
Nathanaël Blanchet
Supervision réseau
Pôle Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
Tél. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr
6:08 a.m.
On Mon, Jan 23, 2017 at 1:38 PM, Nathanaël Blanchet <blanchet(a)abes.fr>
wrote:
Hi
The update notifier in the webadmin was originally designed to alert for
new vdsm* packages. Now, I noticed that available update of virt packages
and more are notified. I know that hot updating qemu-kvm package does break
vms that are running on concerned hosts, but what about other one like
libvirt-client? I know it is recommended to put in maintenance while
updating, but can we update some minor packages without waiting for
migration?
Hot-updating any package should not break any running VMs. If it does, it's
a bug.
I personally agree it's not the best habit to do it nevertheless, and I
expect users to put hosts to maintenance before performing any upgrade.
We cannot tell which update requires maintenance and which doesn't (or for
that matter - requires a reboot or a service restart) - there's no metadata
available to do attached to the packages that can tell us that.
Y.
--
Nathanaël Blanchet
Supervision réseau
Pôle Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
Tél. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
7:20 a.m.
This is a multi-part message in MIME format.
--------------E83A5F83A0804BCBACCC816A
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable
Le 23/01/2017 =C3=A0 13:08, Yaniv Kaul a =C3=A9crit :
On Mon, Jan 23, 2017 at 1:38 PM, Nathana=C3=ABl Blanchet <blanchet@abes=
.fr=20
<mailto:blanchet@abes.fr>> wrote:
Hi
The update notifier in the webadmin was originally designed to
alert for new vdsm* packages. Now, I noticed that available update
of virt packages and more are notified. I know that hot updating
qemu-kvm package does break vms that are running on concerned
hosts, but what about other one like libvirt-client? I know it is
recommended to put in maintenance while updating, but can we
update some minor packages without waiting for migration?
Hot-updating any package should not break any running VMs. If it does,=20
it's a bug.
The last time I did a qemu-kvm upgrade, my host became down and the
vms=20
on it with a question mark and no possibility to interact with them. My=20
only solution was to use the "confirm host has rebooted" to fence the=20
vms, and then the nightmare began : the high availaible vms rebooted on=20
an other host while they were still active on the first one, so they=20
were up on two hosts at the same time. Their disk began to be written by=20
two vms at the same time and I had to fscsk them to make them up on the=20
next boot. Some database vms were completely unusabled!
So I'm very surprised to hear that it is possible to do hot-updating on=20
these kind of upgrade, and I won't do it anymore!
I personally agree it's not the best habit to do it nevertheless,
and=20
I expect users to put hosts to maintenance before performing any upgrad=
e.
We cannot tell which update requires maintenance and which doesn't (or=20
for that matter - requires a reboot or a service restart) - there's no=20
metadata available to do attached to the packages that can tell us that=
.
Y.
--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr <mailto:blanchet@abes.fr>
_______________________________________________
Users mailing list
Users(a)ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
<http://lists.ovirt.org/mailman/listinfo/users>
--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr
--------------E83A5F83A0804BCBACCC816A
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable
<html>
<head>
<meta content=3D"text/html; charset=3Dutf-8"
http-equiv=3D"Content-Ty=
pe">
</head>
<body text=3D"#000000" bgcolor=3D"#FFFFFF">
<p><br>
</p>
<br>
<div class=3D"moz-cite-prefix">Le 23/01/2017 =C3=A0 13:08, Yaniv
Kaul=
a
=C3=A9crit=C2=A0:<br>
</div>
<blockquote
cite=3D"mid:CAJgorsZscAF9uQ=3DWcjwFEg0EB1XNOEV2Abnrr0NxNiKZrpHwaA@mail.gm=
ail.com"
type=3D"cite">
<div dir=3D"ltr"><br>
<div class=3D"gmail_extra"><br>
<div class=3D"gmail_quote">On Mon, Jan 23, 2017 at 1:38 PM,
Nathana=C3=ABl Blanchet <span dir=3D"ltr"><<a
moz-do-not-send=3D"true"
href=3D"mailto:blanchet@abes.fr"
target=3D"_blank"><a
class=3D"moz-txt-link-abbreviated" h=
ref=3D"mailto:blanchet@abes.fr">blanchet@abes.fr</a></a>></span>
wrote=
:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">Hi<br>
<br>
The update notifier in the webadmin was originally
designed to alert for new vdsm* packages. Now, I noticed
that available update of virt packages and more are
notified. I know that hot updating qemu-kvm package does
break vms that are running on concerned hosts, but what
about other one like libvirt-client? I know it is
recommended to put in maintenance while updating, but can
we update some minor packages without waiting for
migration?<br>
</blockquote>
<div><br>
</div>
<div>Hot-updating any package should not break any running
VMs. If it does, it's a bug.</div>
</div>
</div>
</div>
</blockquote>
The last time I did a qemu-kvm upgrade, my host became down and the
vms on it with a question mark and no possibility to interact with
them. My only solution was to use the "confirm host has rebooted" to
fence the vms, and then the nightmare began : the high availaible
vms rebooted on an other host while they were still active on the
first one, so they were up on two hosts at the same time. Their disk
began to be written by two vms at the same time and I had to fscsk
them to make them up on the next boot. Some database vms were
completely unusabled!<br>
<br>
So I'm very surprised to hear that it is possible to do=C2=A0
hot-updating on these kind of upgrade, and I won't do it anymore!<br>
<blockquote
cite=3D"mid:CAJgorsZscAF9uQ=3DWcjwFEg0EB1XNOEV2Abnrr0NxNiKZrpHwaA@mail.gm=
ail.com"
type=3D"cite">
<div dir=3D"ltr">
<div class=3D"gmail_extra">
<div class=3D"gmail_quote">
<div>I personally agree it's not the best habit to do it
nevertheless, and I expect users to put hosts to
maintenance before performing any upgrade.</div>
<div><br>
</div>
<div>We cannot tell which update requires maintenance and
which doesn't (or for that matter - requires a reboot or a
service restart) - there's no metadata available to do
attached to the packages that can tell us that.</div>
<div>Y.</div>
<div>=C2=A0</div>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
-- <br>
Nathana=C3=ABl Blanchet<br>
<br>
Supervision r=C3=A9seau<br>
P=C3=B4le Infrastrutures Informatiques<br>
227 avenue Professeur-Jean-Louis-Viala<br>
34193 MONTPELLIER CEDEX 5=C2=A0 =C2=A0 =C2=A0 =C2=A0<br>
T=C3=A9l. 33 (0)4 67 54 84 55<br>
Fax=C2=A0 33 (0)4 67 54 84 14<br>
<a moz-do-not-send=3D"true"
href=3D"mailto:blanchet@abes.fr=
"
target=3D"_blank">blanchet(a)abes.fr</a><br>
<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a moz-do-not-send=3D"true"
href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br>
<a moz-do-not-send=3D"true"
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
rel=3D"noreferrer"
target=3D"_blank">http://lists.ovirt.o=
rg/mailman<wbr>/listinfo/users</a><br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<pre class=3D"moz-signature" cols=3D"72">--=20
Nathana=C3=ABl Blanchet
Supervision r=C3=A9seau
P=C3=B4le Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5 =09
T=C3=A9l. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
<a class=3D"moz-txt-link-abbreviated"
href=3D"mailto:blanchet@abes.fr">bl=
anchet(a)abes.fr</a> </pre>
</body>
</html>
--------------E83A5F83A0804BCBACCC816A--
7:27 a.m.
On Jan 23, 2017 3:20 PM, "Nathanaël Blanchet" <blanchet(a)abes.fr> wrote:
Le 23/01/2017 à 13:08, Yaniv Kaul a écrit :
On Mon, Jan 23, 2017 at 1:38 PM, Nathanaël Blanchet < <blanchet(a)abes.fr>
blanchet(a)abes.fr> wrote:
Hi
The update notifier in the webadmin was originally designed to alert for
new vdsm* packages. Now, I noticed that available update of virt packages
and more are notified. I know that hot updating qemu-kvm package does break
vms that are running on concerned hosts, but what about other one like
libvirt-client? I know it is recommended to put in maintenance while
updating, but can we update some minor packages without waiting for
migration?
Hot-updating any package should not break any running VMs. If it does, it's
a bug.
The last time I did a qemu-kvm upgrade, my host became down and the vms on
it with a question mark and no possibility to interact with them. My only
solution was to use the "confirm host has rebooted" to fence the vms, and
then the nightmare began : the high
Was the host indeed rebooted?
availaible vms rebooted on an other host while they were still active on
the first one, so they were up on two hosts at the same time. Their disk
began to be written by two vms at the same time and I had to fscsk them to
make them up on the next boot. Some database vms were completely unusabled!
On 4.1 we are going to introduce a feature that will protect against this
situation, by taking a lock on the storage.
Y.
So I'm very surprised to hear that it is possible to do hot-updating on
these kind of upgrade, and I won't do it anymore!
I personally agree it's not the best habit to do it nevertheless, and I
expect users to put hosts to maintenance before performing any upgrade.
We cannot tell which update requires maintenance and which doesn't (or for
that matter - requires a reboot or a service restart) - there's no metadata
available to do attached to the packages that can tell us that.
Y.
--
Nathanaël Blanchet
Supervision réseau
Pôle Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
Tél. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
Nathanaël Blanchet
Supervision réseau
Pôle Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
Tél. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14blanchet(a)abes.fr
11:07 a.m.
On Mon, Jan 23, 2017 at 3:27 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
On Jan 23, 2017 3:20 PM, "Nathanaël Blanchet" <blanchet(a)abes.fr> wrote:
Le 23/01/2017 à 13:08, Yaniv Kaul a écrit :
On Mon, Jan 23, 2017 at 1:38 PM, Nathanaël Blanchet <blanchet(a)abes.fr>
wrote:
>
> Hi
>
> The update notifier in the webadmin was originally designed to alert for
> new vdsm* packages. Now, I noticed that available update of virt packages
> and more are notified. I know that hot updating qemu-kvm package does break
> vms that are running on concerned hosts, but what about other one like
> libvirt-client? I know it is recommended to put in maintenance while
> updating, but can we update some minor packages without waiting for
> migration?
Hot-updating any package should not break any running VMs. If it does, it's
a bug.
Updating vdsm is not supported when the host is not in maintenance.
The major issue is sanlock, if it is maintaining a lease on storage, updating
sanlock will cause the host to reboot. Sanlock is not petting the host watchdog
because you killed sanlock during the upgrade, the watchdog will
reboot the host.
Updating vdms while file based storage domain are mounted is also not
supported, since the local mount path may change between versions, for
example because of fixed bugs. If the local mount path changed, the domain
will not be considered mounted, and some flows may fail.
Nir
The last time I did a qemu-kvm upgrade, my host became down and the
vms on
it with a question mark and no possibility to interact with them. My only
solution was to use the "confirm host has rebooted" to fence the vms, and
then the nightmare began : the high
Was the host indeed rebooted?
availaible vms rebooted on an other host while they were still active on the
first one, so they were up on two hosts at the same time. Their disk began
to be written by two vms at the same time and I had to fscsk them to make
them up on the next boot. Some database vms were completely unusabled!
On 4.1 we are going to introduce a feature that will protect against this
situation, by taking a lock on the storage.
Y.
So I'm very surprised to hear that it is possible to do hot-updating on
these kind of upgrade, and I won't do it anymore!
I personally agree it's not the best habit to do it nevertheless, and I
expect users to put hosts to maintenance before performing any upgrade.
We cannot tell which update requires maintenance and which doesn't (or for
that matter - requires a reboot or a service restart) - there's no metadata
available to do attached to the packages that can tell us that.
Y.
>
>
>
> --
> Nathanaël Blanchet
>
> Supervision réseau
> Pôle Infrastrutures Informatiques
> 227 avenue Professeur-Jean-Louis-Viala
> 34193 MONTPELLIER CEDEX 5
> Tél. 33 (0)4 67 54 84 55
> Fax 33 (0)4 67 54 84 14
> blanchet(a)abes.fr
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
--
Nathanaël Blanchet
Supervision réseau
Pôle Infrastrutures Informatiques
227 avenue Professeur-Jean-Louis-Viala
34193 MONTPELLIER CEDEX 5
Tél. 33 (0)4 67 54 84 55
Fax 33 (0)4 67 54 84 14
blanchet(a)abes.fr
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
12:31 p.m.
On Mon, Jan 23, 2017 at 7:07 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
On Mon, Jan 23, 2017 at 3:27 PM, Yaniv Kaul <ykaul(a)redhat.com>
wrote:
>
>
> On Jan 23, 2017 3:20 PM, "Nathanaël Blanchet" <blanchet(a)abes.fr>
wrote:
>
>
>
> Le 23/01/2017 à 13:08, Yaniv Kaul a écrit :
>
>
>
> On Mon, Jan 23, 2017 at 1:38 PM, Nathanaël Blanchet <blanchet(a)abes.fr>
> wrote:
>>
>> Hi
>>
>> The update notifier in the webadmin was originally designed to alert for
>> new vdsm* packages. Now, I noticed that available update of virt
packages
>> and more are notified. I know that hot updating qemu-kvm package does
break
>> vms that are running on concerned hosts, but what about other one like
>> libvirt-client? I know it is recommended to put in maintenance while
>> updating, but can we update some minor packages without waiting for
>> migration?
>
>
> Hot-updating any package should not break any running VMs. If it does,
it's
> a bug.
Updating vdsm is not supported when the host is not in maintenance.
The major issue is sanlock, if it is maintaining a lease on storage,
updating
sanlock will cause the host to reboot. Sanlock is not petting the host
watchdog
because you killed sanlock during the upgrade, the watchdog will
reboot the host.
Is the sanlock RPM preventing an upgrade (in the pre-upgrade script) if it
has a lock?
Y.
Updating vdms while file based storage domain are mounted is also
not
supported, since the local mount path may change between versions, for
example because of fixed bugs. If the local mount path changed, the domain
will not be considered mounted, and some flows may fail.
Nir
> The last time I did a qemu-kvm upgrade, my host became down and the vms
on
> it with a question mark and no possibility to interact with them. My only
> solution was to use the "confirm host has rebooted" to fence the vms, and
> then the nightmare began : the high
>
>
> Was the host indeed rebooted?
>
> availaible vms rebooted on an other host while they were still active on
the
> first one, so they were up on two hosts at the same time. Their disk
began
> to be written by two vms at the same time and I had to fscsk them to make
> them up on the next boot. Some database vms were completely unusabled!
>
>
> On 4.1 we are going to introduce a feature that will protect against this
> situation, by taking a lock on the storage.
> Y.
>
>
> So I'm very surprised to hear that it is possible to do hot-updating on
> these kind of upgrade, and I won't do it anymore!
>
> I personally agree it's not the best habit to do it nevertheless, and I
> expect users to put hosts to maintenance before performing any upgrade.
>
> We cannot tell which update requires maintenance and which doesn't (or
for
> that matter - requires a reboot or a service restart) - there's no
metadata
> available to do attached to the packages that can tell us that.
> Y.
>
>>
>>
>>
>> --
>> Nathanaël Blanchet
>>
>> Supervision réseau
>> Pôle Infrastrutures Informatiques
>> 227 avenue Professeur-Jean-Louis-Viala
>> 34193 MONTPELLIER CEDEX 5
>> Tél. 33 (0)4 67 54 84 55
>> Fax 33 (0)4 67 54 84 14
>> blanchet(a)abes.fr
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> --
> Nathanaël Blanchet
>
> Supervision réseau
> Pôle Infrastrutures Informatiques
> 227 avenue Professeur-Jean-Louis-Viala
> 34193 MONTPELLIER CEDEX 5
> Tél. 33 (0)4 67 54 84 55
> Fax 33 (0)4 67 54 84 14
> blanchet(a)abes.fr
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
1:50 p.m.
On Mon, Jan 23, 2017 at 8:31 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
On Mon, Jan 23, 2017 at 7:07 PM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>
> On Mon, Jan 23, 2017 at 3:27 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
> >
> >
> > On Jan 23, 2017 3:20 PM, "Nathanaël Blanchet" <blanchet(a)abes.fr>
wrote:
> >
> >
> >
> > Le 23/01/2017 à 13:08, Yaniv Kaul a écrit :
> >
> >
> >
> > On Mon, Jan 23, 2017 at 1:38 PM, Nathanaël Blanchet <blanchet(a)abes.fr>
> > wrote:
> >>
> >> Hi
> >>
> >> The update notifier in the webadmin was originally designed to alert
> >> for
> >> new vdsm* packages. Now, I noticed that available update of virt
> >> packages
> >> and more are notified. I know that hot updating qemu-kvm package does
> >> break
> >> vms that are running on concerned hosts, but what about other one like
> >> libvirt-client? I know it is recommended to put in maintenance while
> >> updating, but can we update some minor packages without waiting for
> >> migration?
> >
> >
> > Hot-updating any package should not break any running VMs. If it does,
> > it's
> > a bug.
>
> Updating vdsm is not supported when the host is not in maintenance.
>
> The major issue is sanlock, if it is maintaining a lease on storage,
> updating
> sanlock will cause the host to reboot. Sanlock is not petting the host
> watchdog
> because you killed sanlock during the upgrade, the watchdog will
> reboot the host.
Is the sanlock RPM preventing an upgrade (in the pre-upgrade script) if it
has a lock?
Adding David.
Y.
>
> Updating vdms while file based storage domain are mounted is also not
> supported, since the local mount path may change between versions, for
> example because of fixed bugs. If the local mount path changed, the domain
> will not be considered mounted, and some flows may fail.
>
> Nir
>
> > The last time I did a qemu-kvm upgrade, my host became down and the vms
> > on
> > it with a question mark and no possibility to interact with them. My
> > only
> > solution was to use the "confirm host has rebooted" to fence the vms,
> > and
> > then the nightmare began : the high
> >
> >
> > Was the host indeed rebooted?
> >
> > availaible vms rebooted on an other host while they were still active on
> > the
> > first one, so they were up on two hosts at the same time. Their disk
> > began
> > to be written by two vms at the same time and I had to fscsk them to
> > make
> > them up on the next boot. Some database vms were completely unusabled!
> >
> >
> > On 4.1 we are going to introduce a feature that will protect against
> > this
> > situation, by taking a lock on the storage.
> > Y.
> >
> >
> > So I'm very surprised to hear that it is possible to do hot-updating on
> > these kind of upgrade, and I won't do it anymore!
> >
> > I personally agree it's not the best habit to do it nevertheless, and I
> > expect users to put hosts to maintenance before performing any upgrade.
> >
> > We cannot tell which update requires maintenance and which doesn't (or
> > for
> > that matter - requires a reboot or a service restart) - there's no
> > metadata
> > available to do attached to the packages that can tell us that.
> > Y.
> >
> >>
> >>
> >>
> >> --
> >> Nathanaël Blanchet
> >>
> >> Supervision réseau
> >> Pôle Infrastrutures Informatiques
> >> 227 avenue Professeur-Jean-Louis-Viala
> >> 34193 MONTPELLIER CEDEX 5
> >> Tél. 33 (0)4 67 54 84 55
> >> Fax 33 (0)4 67 54 84 14
> >> blanchet(a)abes.fr
> >>
> >> _______________________________________________
> >> Users mailing list
> >> Users(a)ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/users
> >
> >
> >
> > --
> > Nathanaël Blanchet
> >
> > Supervision réseau
> > Pôle Infrastrutures Informatiques
> > 227 avenue Professeur-Jean-Louis-Viala
> > 34193 MONTPELLIER CEDEX 5
> > Tél. 33 (0)4 67 54 84 55
> > Fax 33 (0)4 67 54 84 14
> > blanchet(a)abes.fr
> >
> >
> >
> > _______________________________________________
> > Users mailing list
> > Users(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/users
> >
3:31 a.m.
On Mon, Jan 23, 2017 at 09:50:38PM +0200, Nir Soffer wrote:
>> The major issue is sanlock, if it is maintaining a lease on
storage,
>> updating sanlock will cause the host to reboot. Sanlock is not
>> petting the host watchdog because you killed sanlock during the
>> upgrade, the watchdog will reboot the host.
>
> Is the sanlock RPM preventing an upgrade (in the pre-upgrade script) if it
> has a lock?
I think that rpm upgrade is not interacting with the running daemon,
although there was a problem with that at one time.
3015
days inactive
3016
days old
7 comments
4 participants
participants (4)
-
David Teigland -
Nathanaël Blanchet -
Nir Soffer -
Yaniv Kaul