New subject: Cluster upgrade with ansible only upgrades one host then stops

25 Jan 2019


      I have a three node HCI setup, running 4.2.7 and want to upgrade to 4.2.8.
When I use ansible to perform the host updates for some reason it fully
updates one host then stops without error, it does not continue upgrading
the remaining two hosts.  If I run it again it will proceed to upgrade the
next host.  Is there something wrong with the ansible plays I am using or
perhaps the command I'm using to run ansible needing to specify all hosts
to run against.  I don't understand why it's not upgrading all hosts in one
single run.  Here is the complete ansible output of the last run, in this
example it fully updated and rebooted host0 with no errors but did not
proceed to upgrade host1 or host2:


$ cat ovirt-upgrade
# ansible-playbook --ask-vault-pass upgrade.yml
Vault password:

PLAY [oVirt Cluster Upgrade]
*************************************************************************************************************************************************************************************************************************************************************************************************

TASK [oVirt.cluster-upgrade : set_fact]
**************************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [oVirt.cluster-upgrade : Login to oVirt]
********************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [oVirt.cluster-upgrade : Get hosts]
*************************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [oVirt.cluster-upgrade : Check if there are hosts to be updated]
********************************************************************************************************************************************************************************************************************************************************
skipping: [localhost]

TASK [oVirt.cluster-upgrade : include_tasks]
*********************************************************************************************************************************************************************************************************************************************************************************
included:
/usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/cluster_policy.yml for
localhost

TASK [oVirt.cluster-upgrade : Get cluster facts]
*****************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [oVirt.cluster-upgrade : Get name of the original scheduling policy]
****************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [oVirt.cluster-upgrade : Remember the cluster scheduling policy]
********************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [oVirt.cluster-upgrade : Remember the cluster scheduling policy
properties]
*********************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [oVirt.cluster-upgrade : Get API facts]
*********************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [oVirt.cluster-upgrade : Set in cluster upgrade policy]
*****************************************************************************************************************************************************************************************************************************************************************
changed: [localhost]

TASK [oVirt.cluster-upgrade : Get list of VMs in cluster]
********************************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

[ INFO  ] Configuring WebSocket Proxy
skipping: [localhost] => (item=vm1)
skipping: [localhost] => (item=vm2)
skipping: [localhost] => (item=vm3)

TASK [oVirt.cluster-upgrade : Create list of VM names which was shutted
down]
************************************************************************************************************************************************************************************************************************************************
skipping: [localhost]

TASK [oVirt.cluster-upgrade : include_tasks]
*********************************************************************************************************************************************************************************************************************************************************************************
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 235034116096, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/771c67eb-56e6-4736-8c67-668502d4ecf5', u'spm':
{u'priority': 5, u'status': u'none'}, u'id':
u'771c67eb-56e6-4736-8c67-668502d4ecf5', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host0.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:ac5566ae1e1a'}, u'port':
54321, u'hardware_information': {u'serial_number': u'70833W1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0030-3810-8033-B7C04F335731', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:TiiMiaHi++kqhmurXdlQjtwbb3Q1oIBi5ab2alKujyI'}, u'address': u'
host0.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host0', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host0/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host0/swap
rd.lvm.lv=onn_host0/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 250759610368, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d', u'spm':
{u'priority': 5, u'status': u'none'}, u'id':
u'fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host1.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:2f5ea98fcec4'}, u'port':
54321, u'hardware_information': {u'serial_number': u'FYRTGX1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0059-5210-8054-C6C04F475831', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:HGU2WhIlPfxgXHXHOE8IYT4/NGAQeUBRe0QA34qQ+nc'}, u'address': u'
host1.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host1', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host1/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host1/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rd.lvm.lv=onn_host1/swap rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 232492367872, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/fd0752d8-2d41-45b0-887a-0ffacbb8a237', u'spm':
{u'priority': 5, u'status': u'spm'}, u'id':
u'fd0752d8-2d41-45b0-887a-0ffacbb8a237', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host2.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:369c9881d24e'}, u'port':
54321, u'hardware_information': {u'serial_number': u'D9733W1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0039-3710-8033-C4C04F335731', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:cSgSzFFQaziGHDeKH1CIfXd8YbheGTaft3XBgyr4YgQ'}, u'address': u'
host2.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host2', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host2/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host2/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rd.lvm.lv=onn_host2/swap rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})

TASK [oVirt.cluster-upgrade : Upgrade host]
**********************************************************************************************************************************************************************************************************************************************************************************
skipping: [localhost] => (item=vm1)
skipping: [localhost] => (item=vm2)
skipping: [localhost] => (item=vm3)
skipping: [localhost] => (item=vm4)

TASK [oVirt.cluster-upgrade : Create list of VM names which was shutted
down]
************************************************************************************************************************************************************************************************************************************************
skipping: [localhost]

TASK [oVirt.cluster-upgrade : include_tasks]
*********************************************************************************************************************************************************************************************************************************************************************************
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 235034116096, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/771c67eb-56e6-4736-8c67-668502d4ecf5', u'spm':
{u'priority': 5, u'status': u'none'}, u'id':
u'771c67eb-56e6-4736-8c67-668502d4ecf5', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host0.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:ac5566ae1e1a'}, u'port':
54321, u'hardware_information': {u'serial_number': u'70833W1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0030-3810-8033-B7C04F335731', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:TiiMiaHi++kqhmurXdlQjtwbb3Q1oIBi5ab2alKujyI'}, u'address': u'
host0.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host0', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host0/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host0/swap
rd.lvm.lv=onn_host0/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 250759610368, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d', u'spm':
{u'priority': 5, u'status': u'none'}, u'id':
u'fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host1.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:2f5ea98fcec4'}, u'port':
54321, u'hardware_information': {u'serial_number': u'FYRTGX1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0059-5210-8054-C6C04F475831', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:HGU2WhIlPfxgXHXHOE8IYT4/NGAQeUBRe0QA34qQ+nc'}, u'address': u'
host1.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host1', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host1/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host1/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rd.lvm.lv=onn_host1/swap rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 232492367872, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/fd0752d8-2d41-45b0-887a-0ffacbb8a237', u'spm':
{u'priority': 5, u'status': u'spm'}, u'id':
u'fd0752d8-2d41-45b0-887a-0ffacbb8a237', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host2.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:369c9881d24e'}, u'port':
54321, u'hardware_information': {u'serial_number': u'D9733W1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0039-3710-8033-C4C04F335731', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:cSgSzFFQaziGHDeKH1CIfXd8YbheGTaft3XBgyr4YgQ'}, u'address': u'
host2.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host2', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host2/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host2/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rd.lvm.lv=onn_host2/swap rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})

TASK [oVirt.cluster-upgrade : Upgrade host]
**********************************************************************************************************************************************************************************************************************************************************************************
skipping: [localhost] => (item=vm1)
skipping: [localhost] => (item=vm2)
skipping: [localhost] => (item=vm3)

TASK [oVirt.cluster-upgrade : Create list of VM names which was shutted
down]
************************************************************************************************************************************************************************************************************************************************
skipping: [localhost]

TASK [oVirt.cluster-upgrade : include_tasks]
*********************************************************************************************************************************************************************************************************************************************************************************
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 235034116096, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/771c67eb-56e6-4736-8c67-668502d4ecf5', u'spm':
{u'priority': 5, u'status': u'none'}, u'id':
u'771c67eb-56e6-4736-8c67-668502d4ecf5', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host0.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:ac5566ae1e1a'}, u'port':
54321, u'hardware_information': {u'serial_number': u'70833W1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0030-3810-8033-B7C04F335731', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:TiiMiaHi++kqhmurXdlQjtwbb3Q1oIBi5ab2alKujyI'}, u'address': u'
host0.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host0', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host0/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host0/swap
rd.lvm.lv=onn_host0/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 250759610368, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d', u'spm':
{u'priority': 5, u'status': u'none'}, u'id':
u'fb1e62d5-1dc1-4ccc-8b2b-cf48f7077d0d', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host1.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:2f5ea98fcec4'}, u'port':
54321, u'hardware_information': {u'serial_number': u'FYRTGX1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0059-5210-8054-C6C04F475831', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:HGU2WhIlPfxgXHXHOE8IYT4/NGAQeUBRe0QA34qQ+nc'}, u'address': u'
host1.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host1', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host1/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host1/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rd.lvm.lv=onn_host1/swap rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})
included: /usr/share/ansible/roles/ovirt.cluster-upgrade/tasks/upgrade.yml
for localhost => (item={u'comment': u'', u'update_available': True,
u'protocol': u'stomp', u'affinity_labels': [], u'hooks': [],
u'max_scheduling_memory': 232492367872, u'cluster': {u'href':
u'/ovirt-engine/api/clusters/a45fe964-9989-11e8-b3f7-00163e4bf18a', u'id':
u'a45fe964-9989-11e8-b3f7-00163e4bf18a'}, u'href':
u'/ovirt-engine/api/hosts/fd0752d8-2d41-45b0-887a-0ffacbb8a237', u'spm':
{u'priority': 5, u'status': u'spm'}, u'id':
u'fd0752d8-2d41-45b0-887a-0ffacbb8a237', u'external_status': u'ok',
u'statistics': [], u'certificate': {u'organization': u'xxxxxxxx.com',
u'subject': u'O=xxxxxxxx.com,CN=host2.xxxxxxxx.com'}, u'nics': [],
u'iscsi': {u'initiator': u'iqn.1994-05.com.redhat:369c9881d24e'}, u'port':
54321, u'hardware_information': {u'serial_number': u'D9733W1',
u'product_name': u'PowerEdge R720', u'uuid':
u'4C4C4544-0039-3710-8033-C4C04F335731', u'supported_rng_sources':
[u'hwrng', u'random'], u'manufacturer': u'Dell Inc.'}, u'version':
{u'full_version': u'vdsm-4.20.43-1.el7', u'revision': 0, u'build': 43,
u'minor': 20, u'major': 4}, u'memory': 270258929664, u'ksm': {u'enabled':
False}, u'se_linux': {u'mode': u'permissive'}, u'type': u'ovirt_node',
u'storage_connection_extensions': [], u'status': u'up', u'tags': [],
u'katello_errata': [], u'external_network_provider_configurations': [],
u'ssh': {u'port': 22, u'fingerprint':
u'SHA256:cSgSzFFQaziGHDeKH1CIfXd8YbheGTaft3XBgyr4YgQ'}, u'address': u'
host2.xxxxxxxx.com', u'numa_nodes': [], u'device_passthrough': {u'enabled':
False}, u'unmanaged_networks': [], u'permissions': [], u'numa_supported':
True, u'libvirt_version': {u'full_version': u'libvirt-3.9.0-14.el7_5.8',
u'revision': 0, u'build': 0, u'minor': 9, u'major': 3},
u'power_management': {u'kdump_detection': True, u'enabled': True,
u'pm_proxies': [{u'type': u'cluster'}, {u'type': u'dc'}],
u'automatic_pm_enabled': True}, u'name': u'host2', u'devices': [],
u'summary': {u'active': 4, u'migrating': 0, u'total': 4},
u'auto_numa_status': u'enable', u'transparent_huge_pages': {u'enabled':
True}, u'network_attachments': [], u'os': {u'version': {u'full_version':
u'7 - 5.1804.5.el7.centos', u'major': 7}, u'type': u'RHEL',
u'custom_kernel_cmdline': u'', u'reported_kernel_cmdline':
u'BOOT_IMAGE=/ovirt-node-ng-4.2.7.1-0.20181114.0+1/vmlinuz-3.10.0-862.14.4.el7.x86_64
root=/dev/onn_host2/ovirt-node-ng-4.2.7.1-0.20181114.0+1 ro
crashkernel=auto rd.lvm.lv=onn_host2/ovirt-node-ng-4.2.7.1-0.20181114.0+1
rd.lvm.lv=onn_host2/swap rhgb quiet LANG=en_CA.UTF-8
img.bootid=ovirt-node-ng-4.2.7.1-0.20181114.0+1'}, u'cpu': {u'speed':
1200.0, u'name': u'Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz', u'topology':
{u'cores': 8, u'threads': 1, u'sockets': 2}}, u'kdump_status': u'disabled'})

TASK [oVirt.cluster-upgrade : Upgrade host]
**********************************************************************************************************************************************************************************************************************************************************************************
changed: [localhost]

TASK [oVirt.cluster-upgrade : Upgrade host]
**********************************************************************************************************************************************************************************************************************************************************************************
changed: [localhost]

TASK [oVirt.cluster-upgrade : Upgrade host]
**********************************************************************************************************************************************************************************************************************************************************************************
changed: [localhost]

TASK [oVirt.cluster-upgrade : Set original cluster policy]
*******************************************************************************************************************************************************************************************************************************************************************
changed: [localhost]

TASK [oVirt.cluster-upgrade : Start again stopped VMs]
***********************************************************************************************************************************************************************************************************************************************************************

TASK [oVirt.cluster-upgrade : Start again pin to host VMs]
*******************************************************************************************************************************************************************************************************************************************************************

TASK [oVirt.cluster-upgrade : Logout from oVirt]
*****************************************************************************************************************************************************************************************************************************************************************************
ok: [localhost]

PLAY RECAP
*******************************************************************************************************************************************************************************************************************************************************************************************************************
localhost                  : ok=21   changed=5    unreachable=0    failed=0

Cluster upgrade with ansible only upgrades one host then stops

Jayme

Jayme

Greg Sheremeta

tags

participants (2)