[ovirt-users] ovirt upgrade => vms refusing to (re)boot

[Michal Skrivanek]

> On 31 Mar 2017, at 15:49, Nelson Lameiras <nelson.lameiras at lyra-network.com> wrote:
> 
> Hello,
> 
> We had a rather unpleseant surprise while upgrading our production datacenters to oVirt 4.0 => 4.1 (although I think this problem is not related to oVirt 4.1). 
> Some critical VMs simply refused to reboot without any particular error message other than "VM failed to boot"
> 
> After a stressfull 2h of investigation, it turns out that the problematic VMs shared the same "exotic" condition. They were not rebooted since the latest major upgrade 3.6 => 4.0 (I know, this is not a good practise and we underestimated it's priority, we will change our reboot policies)

it’s indeed not a good practice. You can’t use the features (those related to hw) without updating the guest HW and that will only happen once you shut the VM down. If you can’t do that then just keep the old cluster level, that is fully supported.

> Their "custom compatilibity setting" was set to 3.6 !! When changing this setting to "Empty", The VM booted normaly. Hard to find, easy to fix !! ;)
> 
> Neverthess, there is little to none information/documentation about the usefullness of this setting, and I have a hard time understanding the consequences of changing it manually.
> 
> Our oVirt datacenter (which has  engine 4.1.1, but some hosts on 4.0, so it's still on 4.0 compatibility) hosts currently +- 200 Vms,
> - 10 with "custom compatilibity setting" set to 3.6 - which will not (reb)boot in current state,
> - 30some with "custom compatilibity setting" set to 4.0 - which (re)boot normally 
> - the rest with "custom compatilibity setting" set to "empty" - which (re)boot normally 
> 
> So can a oVirt guru please answer the following questions :
> 
> - What does this setting do? which are the consequences of changing it manually ?

it emulates the corresponding guest hardware and engine feature behavior.

> - Is it "normal" that a VM not reboooted since the 3.6 update does not boot if a new major upgrade is done on hostig host ? (maybe a exotic bug worth correcting ?)

no, it’s a bug https://bugzilla.redhat.com/show_bug.cgi?id=1436325 <https://bugzilla.redhat.com/show_bug.cgi?id=1436325>
will be fixed in 4.1.2

> - Which are the possible consequences of manually setting this setting to "Empty" in all my VMs (running or stopped) ?

then it will use the cluster settings. that is the “normal” state

> - Which events will change this setting automatically ? (cluster major version upgrade, first reboot after upgrade, ...) ?

cluster level upgrade while VM is running. Since the VM is running the changes to hardware cannot be applied and it is temporarily reconfigured to use the previous cluster level. On VM shutdown the settings are all updated and the field set to empty.
there was another bug (hopefully fixed) which kept the value there even on shutdown. Just edit and put “empty”, then it will start as a regular VM in its cluster.

> - Some of my VMs have custom_compatibility_version set to 4.0 (in REST API)  eventough they have been recently rebooted and "custom compatilibity setting" is empty in GUI, who's this possible ?

was it done before the reboot? are there pending changes to be applied perhaps?

Thanks,
michal

> 
> cordialement, regards,
> 
> <element-signature_logo_lyra_115x94.jpg>
>  <https://www.lyra-network.com/>
> Nelson LAMEIRAS
> Ingénieur Systèmes et Réseaux / Systems and Networks engineer
> Tel: +33 5 32 09 09 70
> nelson.lameiras at lyra-network.com <mailto:nelson.lameiras at lyra-network.com>
> www.lyra-network.com <https://www.lyra-network.com/> | www.payzen.eu <https://payzen.eu/>
> <element-signature_logo_YouTube_32x28.jpg> <https://www.youtube.com/channel/UCrVl1CO_Jlu3KbiRH-tQ_vA>
> <element-signature_logo_LinkedIn_41x28.jpg> <https://www.linkedin.com/company/lyra-network_2>
> <element-signature_logo_Twitter_42x28.jpg> <https://twitter.com/LyraNetwork>
> <element-signature_payzen_61x28.jpg> <https://payzen.eu/>
> Lyra Network, 109 rue de l'innovation, 31670 Labège, FRANCE
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170402/f4f30d77/attachment-0001.html>