Self hosted engine and upgrade path to 3.6.1 CentOS 7.2 with single host
Hello, having a self hosted engine environment in 3.6.0 and both host and engine VM in CentOS 7.1, and having only one single host, now that both CentOS 7.2 and 3.6.1 have been released, what would be the correct path/approach? I have a test system with an NFS storage domain and some VMs on it, to try on before and then report, but if I get a confirmation/suggestion in advance is preferable. This is my idea, adapting what described at: http://www.ovirt.org/Hosted_Engine_Howto#Upgrade_Hosted_Engine - set hosted engine maintenance mode to global - upgrade engine VM both to 3.6.1 and CentOS 7.2 with: # yum update "ovirt-engine-setup*" # yum update # engine-setup - reboot engine VM and verify correct access No what if I want to update the host too? - shutdown the VMs in my Datacenter Can I put the only existing host into maintenance? Would this shutdown the engine vm too or is it separate in terms of maintenance of the host? Or should I simply - shutdown the VMs in my Datacenter - shutdown engine VM - stop vdsmd on host - update host to both CentOS 7.2 and oVirt 3.6.1 packages with: # yum update - reboot host - verify vdsmd is ok - exit the global maintenance mode and wait for a few minutes the engine VM should on this fresh upgraded host - hopefully the host will come up again as active and I can start my VMs ? Thanks in advance for any help and clarification Gianluca
On Thu, Dec 17, 2015 at 8:31 AM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
Hello, having a self hosted engine environment in 3.6.0 and both host and engine VM in CentOS 7.1, and having only one single host, now that both CentOS 7.2 and 3.6.1 have been released, what would be the correct path/approach?
I have a test system with an NFS storage domain and some VMs on it, to try on before and then report, but if I get a confirmation/suggestion in advance is preferable.
This is my idea, adapting what described at: http://www.ovirt.org/Hosted_Engine_Howto#Upgrade_Hosted_Engine
- set hosted engine maintenance mode to global - upgrade engine VM both to 3.6.1 and CentOS 7.2 with: # yum update "ovirt-engine-setup*" # yum update # engine-setup - reboot engine VM and verify correct access
No what if I want to update the host too? - shutdown the VMs in my Datacenter Can I put the only existing host into maintenance? Would this shutdown the engine vm too or is it separate in terms of maintenance of the host?
You can't put the only existing host in maintenance since Hosted Engine is running on it and it can't migrate
Or should I simply - shutdown the VMs in my Datacenter - shutdown engine VM - stop vdsmd on host - update host to both CentOS 7.2 and oVirt 3.6.1 packages with: # yum update - reboot host - verify vdsmd is ok - exit the global maintenance mode and wait for a few minutes the engine VM should on this fresh upgraded host - hopefully the host will come up again as active and I can start my VMs ?
- shutdown the VMs in your Datacenter - move the host to local maintenance - shutdown engine VM - yum update - reboot - exit maintenance
Thanks in advance for any help and clarification
Gianluca
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
On Fri, Dec 18, 2015 at 2:26 PM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
You can't put the only existing host in maintenance since Hosted Engine is running on it and it can't migrate
Or should I simply - shutdown the VMs in my Datacenter - shutdown engine VM - stop vdsmd on host - update host to both CentOS 7.2 and oVirt 3.6.1 packages with: # yum update - reboot host - verify vdsmd is ok - exit the global maintenance mode and wait for a few minutes the engine VM should on this fresh upgraded host - hopefully the host will come up again as active and I can start my VMs ?
- shutdown the VMs in your Datacenter - move the host to local maintenance - shutdown engine VM - yum update - reboot - exit maintenance
After solving the engine VM part (see dedicated thread) I continued and follow the steps indicated for the host. After reboot and exiting maintenance I found the hosted_storage domain in state "unattached" and not "maintenance" as in another test I have done (note that in in that test there were not true data domains configured apart the hosted_storage...) I also got some exceptions inside the GUI when going into storage tab on hosted_storage line. See: https://drive.google.com/file/d/0BwoPbcrMv8mvYmdQYVM0dkRJaGM/view?usp=sharin... Uncaught exception occurred. Please try reloading the page. Details: (TypeError) __gwt$exception: <skipped>: c is null I tried to attach the hosted_storage but then the engine VM went down I did wait for about 10 minutes and the VM was not restarted. Is there any reason for ithe environment to "automatically" go back into maintenance? Because as soon as I run again the command hosted-engine --set-maintenance --mode=none the engine VM was started again..... Is there any command to see the current state of maintenance (none/local/global)? Anyway the hosted_storage domain still is in unattached state... BTW: even if the host has been updated: [root@ractor ]# yum update Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: ba.mirror.garr.it * extras: ba.mirror.garr.it * ovirt-3.6: ftp.nluug.nl * ovirt-3.6-epel: epel.besthosting.ua * updates: ba.mirror.garr.it No packages marked for update [root@ractor ]# Inside the webadmin gui in hosts tab I still see the icon indicating "update available" for it And in action items: "a new version is available" message In software subtab the host is detected as: OS Version: RHEL - 7 - 2.1511.el7.centos.2.10 Kernel Version: 3.10.0 - 327.3.1.el7.x86_64 KVM Version: 2.3.0 - 31.el7_2.4.1 LIBVIRT Version: libvirt-1.2.17-13.el7_2.2 VDSM Version: vdsm-4.17.13-0.el7.centos SPICE Version: 0.12.4 - 15.el7 CEPH Version: librbd1-0.80.7-3.el7
On Sun, Dec 20, 2015 at 2:19 AM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Fri, Dec 18, 2015 at 2:26 PM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
You can't put the only existing host in maintenance since Hosted Engine is running on it and it can't migrate
Or should I simply - shutdown the VMs in my Datacenter - shutdown engine VM - stop vdsmd on host - update host to both CentOS 7.2 and oVirt 3.6.1 packages with: # yum update - reboot host - verify vdsmd is ok - exit the global maintenance mode and wait for a few minutes the engine VM should on this fresh upgraded host - hopefully the host will come up again as active and I can start my VMs ?
- shutdown the VMs in your Datacenter - move the host to local maintenance - shutdown engine VM - yum update - reboot - exit maintenance
After solving the engine VM part (see dedicated thread) I continued and follow the steps indicated for the host.
After reboot and exiting maintenance I found the hosted_storage domain in state "unattached" and not "maintenance" as in another test I have done (note that in in that test there were not true data domains configured apart the hosted_storage...) I also got some exceptions inside the GUI when going into storage tab on hosted_storage line. See: https://drive.google.com/file/d/0BwoPbcrMv8mvYmdQYVM0dkRJaGM/view?usp=sharin...
Uncaught exception occurred. Please try reloading the page. Details: (TypeError) __gwt$exception: <skipped>: c is null
I tried to attach the hosted_storage but then the engine VM went down
It must have already been attached, because otherwise the engine vm would not be up. So it might be some inconsistency between the engine and host.
I did wait for about 10 minutes and the VM was not restarted.
Can you check/post ha logs (/var/log/ovirt-hosted-engine-ha)? Adding Martin.
Is there any reason for ithe environment to "automatically" go back into maintenance? Because as soon as I run again the command
hosted-engine --set-maintenance --mode=none
the engine VM was started again.....
No idea, Martin?
Is there any command to see the current state of maintenance (none/local/global)?
hosted-engine --vm-status
Anyway the hosted_storage domain still is in unattached state...
BTW: even if the host has been updated:
[root@ractor ]# yum update Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: ba.mirror.garr.it * extras: ba.mirror.garr.it * ovirt-3.6: ftp.nluug.nl * ovirt-3.6-epel: epel.besthosting.ua * updates: ba.mirror.garr.it No packages marked for update [root@ractor ]#
Inside the webadmin gui in hosts tab I still see the icon indicating "update available" for it And in action items: "a new version is available" message
In software subtab the host is detected as: OS Version: RHEL - 7 - 2.1511.el7.centos.2.10 Kernel Version: 3.10.0 - 327.3.1.el7.x86_64 KVM Version: 2.3.0 - 31.el7_2.4.1 LIBVIRT Version: libvirt-1.2.17-13.el7_2.2 VDSM Version: vdsm-4.17.13-0.el7.centos SPICE Version: 0.12.4 - 15.el7 CEPH Version: librbd1-0.80.7-3.el7
Are you sure the engine even manages to speak to the host? Please check/post also engine and vdsm logs. Adding Oved. -- Didi
On Sun, Dec 20, 2015 at 10:21 AM, Yedidyah Bar David <didi@redhat.com> wrote:
After reboot and exiting maintenance I found the hosted_storage domain in state "unattached" and not "maintenance" as in another test I have done (note that in in that test there were not true data domains configured
apart
the hosted_storage...) I also got some exceptions inside the GUI when going into storage tab on hosted_storage line. See:
https://drive.google.com/file/d/0BwoPbcrMv8mvYmdQYVM0dkRJaGM/view?usp=sharin...
Uncaught exception occurred. Please try reloading the page. Details: (TypeError) __gwt$exception: <skipped>: c is null
I tried to attach the hosted_storage but then the engine VM went down
It must have already been attached, because otherwise the engine vm would not be up. So it might be some inconsistency between the engine and host.
I did wait for about 10 minutes and the VM was not restarted.
Can you check/post ha logs (/var/log/ovirt-hosted-engine-ha)?
Adding Martin.
here they are in tgz format. Contents for all yesterday and up to now: drwxr-xr-x root/root 0 2015-12-20 10:25 logs/ -rw-r--r-- root/root 5557044 2015-12-20 10:25 logs/agent.log -rw-r--r-- root/root 11934533 2015-12-20 10:25 logs/agent.log.2015-12-18 -rw-r--r-- root/root 4510074 2015-12-20 10:25 logs/broker.log -rw-r--r-- root/root 9867696 2015-12-20 10:25 logs/broker.log.2015-12-18 https://drive.google.com/file/d/0BwoPbcrMv8mvTEh2bGplZEw2ZmM/view?usp=sharin... Take in mind that starting from both host and engine in 3.6.0 yesterday at 13:30 the engine was updated to 3.6.1. After solving its problems (see http://lists.ovirt.org/pipermail/users/2015-December/036678.html ) at about 23:50 I also updated host as suggested taking it to CentOS 7.2 and 3.6.1 too.
Is there any reason for ithe environment to "automatically" go back into maintenance? Because as soon as I run again the command
hosted-engine --set-maintenance --mode=none
the engine VM was started again.....
No idea, Martin?
Is there any command to see the current state of maintenance (none/local/global)?
hosted-engine --vm-status
Thanks. BTW: what is the meaning of "score" field, passing from 3400 to 0 whn I put host to maintenance? Is the score related to priority to run engine VM among hosts (in my case only one)?
Inside the webadmin gui in hosts tab I still see the icon indicating "update available" for it And in action items: "a new version is available" message
In software subtab the host is detected as: OS Version: RHEL - 7 - 2.1511.el7.centos.2.10 Kernel Version: 3.10.0 - 327.3.1.el7.x86_64 KVM Version: 2.3.0 - 31.el7_2.4.1 LIBVIRT Version: libvirt-1.2.17-13.el7_2.2 VDSM Version: vdsm-4.17.13-0.el7.centos SPICE Version: 0.12.4 - 15.el7 CEPH Version: librbd1-0.80.7-3.el7
Are you sure the engine even manages to speak to the host? Please check/post also engine and vdsm logs.
Adding Oved. -- Didi
WHat do you mean by "the engine even manages to speak to the host"? After both updated to 3.6.1 I was indeed able to power on 2 machines defined in the DC and connect to their consoles, so I think there is communication... Or do you mean particular communication in terms of what upadates are in place? What kind of check makes the engine when it shows "update available" fr the host? Gianluca
On Sun, Dec 20, 2015 at 11:44 AM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Sun, Dec 20, 2015 at 10:21 AM, Yedidyah Bar David <didi@redhat.com> wrote:
After reboot and exiting maintenance I found the hosted_storage domain in state "unattached" and not "maintenance" as in another test I have done (note that in in that test there were not true data domains configured apart the hosted_storage...) I also got some exceptions inside the GUI when going into storage tab on hosted_storage line. See:
https://drive.google.com/file/d/0BwoPbcrMv8mvYmdQYVM0dkRJaGM/view?usp=sharin...
Uncaught exception occurred. Please try reloading the page. Details: (TypeError) __gwt$exception: <skipped>: c is null
I tried to attach the hosted_storage but then the engine VM went down
It must have already been attached, because otherwise the engine vm would not be up. So it might be some inconsistency between the engine and host.
I did wait for about 10 minutes and the VM was not restarted.
Can you check/post ha logs (/var/log/ovirt-hosted-engine-ha)?
Adding Martin.
here they are in tgz format. Contents for all yesterday and up to now: drwxr-xr-x root/root 0 2015-12-20 10:25 logs/ -rw-r--r-- root/root 5557044 2015-12-20 10:25 logs/agent.log -rw-r--r-- root/root 11934533 2015-12-20 10:25 logs/agent.log.2015-12-18 -rw-r--r-- root/root 4510074 2015-12-20 10:25 logs/broker.log -rw-r--r-- root/root 9867696 2015-12-20 10:25 logs/broker.log.2015-12-18
https://drive.google.com/file/d/0BwoPbcrMv8mvTEh2bGplZEw2ZmM/view?usp=sharin...
Take in mind that starting from both host and engine in 3.6.0 yesterday at 13:30 the engine was updated to 3.6.1. After solving its problems (see http://lists.ovirt.org/pipermail/users/2015-December/036678.html ) at about 23:50 I also updated host as suggested taking it to CentOS 7.2 and 3.6.1 too.
Is there any reason for ithe environment to "automatically" go back into maintenance? Because as soon as I run again the command
hosted-engine --set-maintenance --mode=none
the engine VM was started again.....
No idea, Martin?
Is there any command to see the current state of maintenance (none/local/global)?
hosted-engine --vm-status
Thanks. BTW: what is the meaning of "score" field, passing from 3400 to 0 whn I put host to maintenance? Is the score related to priority to run engine VM among hosts (in my case only one)?
Yes. You might want to have a look at [1], especially page 15. I can only find a link to it at [2], perhaps we should link elsewhere too (e.g. howto page). [1] https://www.ovirt.org/images/d/d5/Fosdem-hosted-engine.pdf [2] http://www.ovirt.org/FOSDEM_2014
Inside the webadmin gui in hosts tab I still see the icon indicating "update available" for it And in action items: "a new version is available" message
In software subtab the host is detected as: OS Version: RHEL - 7 - 2.1511.el7.centos.2.10 Kernel Version: 3.10.0 - 327.3.1.el7.x86_64 KVM Version: 2.3.0 - 31.el7_2.4.1 LIBVIRT Version: libvirt-1.2.17-13.el7_2.2 VDSM Version: vdsm-4.17.13-0.el7.centos SPICE Version: 0.12.4 - 15.el7 CEPH Version: librbd1-0.80.7-3.el7
Are you sure the engine even manages to speak to the host? Please check/post also engine and vdsm logs.
Adding Oved. -- Didi
WHat do you mean by "the engine even manages to speak to the host"? After both updated to 3.6.1 I was indeed able to power on 2 machines defined in the DC and connect to their consoles, so I think there is communication...
Sounds so.
Or do you mean particular communication in terms of what upadates are in place? What kind of check makes the engine when it shows "update available" fr the host?
No, I meant the above. Now really adding Oved... -- Didi
On Sun, Dec 20, 2015 at 10:21 AM, Yedidyah Bar David <didi@redhat.com> wrote:
On Sun, Dec 20, 2015 at 2:19 AM, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
On Fri, Dec 18, 2015 at 2:26 PM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
You can't put the only existing host in maintenance since Hosted Engine
is
running on it and it can't migrate
Or should I simply - shutdown the VMs in my Datacenter - shutdown engine VM - stop vdsmd on host - update host to both CentOS 7.2 and oVirt 3.6.1 packages with: # yum update - reboot host - verify vdsmd is ok - exit the global maintenance mode and wait for a few minutes the
engine
VM should on this fresh upgraded host - hopefully the host will come up again as active and I can start my VMs ?
- shutdown the VMs in your Datacenter - move the host to local maintenance - shutdown engine VM - yum update - reboot - exit maintenance
After solving the engine VM part (see dedicated thread) I continued and follow the steps indicated for the host.
After reboot and exiting maintenance I found the hosted_storage domain in state "unattached" and not "maintenance" as in another test I have done (note that in in that test there were not true data domains configured apart the hosted_storage...) I also got some exceptions inside the GUI when going into storage tab on hosted_storage line. See:
https://drive.google.com/file/d/0BwoPbcrMv8mvYmdQYVM0dkRJaGM/view?usp=sharin...
Uncaught exception occurred. Please try reloading the page. Details: (TypeError) __gwt$exception: <skipped>: c is null
I tried to attach the hosted_storage but then the engine VM went down
It must have already been attached, because otherwise the engine vm would not be up. So it might be some inconsistency between the engine and host.
I think that Gianluca is referring to the hosted-engine storage domain. On 3.6.0 that storage domain is not attached to any storage pool but HA agent is still able to start the engine VM as expected. With 3.6.1 the engine, when up, will auto-import the hosted-engine storage domain attaching it to its datacenter storage pool but it will do that if and only if the datacenter is up but a datacenter to be requires at least one valid ad active storage domain so the import process is not triggered if you don't have any valid storage domain. It's not really an issue cause in real world you have at least one storage domain in your datacenter and so it's perfectly fine to postpone till that. Still seeing the hosted-engine storage domain in maintenance after the import is instead a real issue: https://bugzilla.redhat.com/show_bug.cgi?id=1290518
I did wait for about 10 minutes and the VM was not restarted.
Can you check/post ha logs (/var/log/ovirt-hosted-engine-ha)?
Adding Martin.
Is there any reason for ithe environment to "automatically" go back into maintenance? Because as soon as I run again the command
hosted-engine --set-maintenance --mode=none
the engine VM was started again.....
No idea, Martin?
Is there any command to see the current state of maintenance (none/local/global)?
hosted-engine --vm-status
Anyway the hosted_storage domain still is in unattached state...
BTW: even if the host has been updated:
[root@ractor ]# yum update Loaded plugins: fastestmirror, langpacks Loading mirror speeds from cached hostfile * base: ba.mirror.garr.it * extras: ba.mirror.garr.it * ovirt-3.6: ftp.nluug.nl * ovirt-3.6-epel: epel.besthosting.ua * updates: ba.mirror.garr.it No packages marked for update [root@ractor ]#
Inside the webadmin gui in hosts tab I still see the icon indicating "update available" for it And in action items: "a new version is available" message
In software subtab the host is detected as: OS Version: RHEL - 7 - 2.1511.el7.centos.2.10 Kernel Version: 3.10.0 - 327.3.1.el7.x86_64 KVM Version: 2.3.0 - 31.el7_2.4.1 LIBVIRT Version: libvirt-1.2.17-13.el7_2.2 VDSM Version: vdsm-4.17.13-0.el7.centos SPICE Version: 0.12.4 - 15.el7 CEPH Version: librbd1-0.80.7-3.el7
Are you sure the engine even manages to speak to the host? Please check/post also engine and vdsm logs.
Adding Oved. -- Didi _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (4)
-
Gianluca Cecchi -
Sandro Bonazzola -
Simone Tiraboschi -
Yedidyah Bar David