ovirt-host 4.4.0 Task Update system failed to execute - Running QEMU processes found, cannot upgrade Vdsm

On a regular basis, there is "Task Update system failed to execute" error in the engine.log. At the same time, the ovirt-host-mgmt-ansible-check log contains "Running QEMU processes found, cannot upgrade Vdsm" error. Is there some configuration setting that controls it? Why oVirt is trying to update ovirt-host package on the host at all if there are virtual machines running? Note that 2019-12-27 11:57:21,880-06 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedScheduledExecutorService-engineThreadMonitoringThreadPool-Thread-1) [] Thread pool 'hostUpdatesChecker' is using 0 threads out of 5, 3 threads waiting for tasks. 2019-12-27 12:02:30,878-06 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Check for update of host harrier. Gathering Facts. 2019-12-27 12:02:45,990-06 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] EVENT_ID: ANSIBLE_RUNNER_EVENT_NOTIFICATION(559), Check for update of host harrier. Install ovirt-host package if it isn't installed. 2019-12-27 12:02:55,024-06 ERROR [org.ovirt.engine.core.bll.host.HostUpgradeManager] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] Failed to read host packages: Task Update system failed to execute: 2019-12-27 12:02:55,025-06 ERROR [org.ovirt.engine.core.bll.hostdeploy.HostUpdatesChecker] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] Failed to check if updates are available for host 'harrier' with error message 'Task Update system failed to execute: ' 2019-12-27 12:02:55,106-06 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedExecutorService-hostUpdatesChecker-Thread-4) [] EVENT_ID: HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates on host harrier with message 'Task Update system failed to execute: '. 2 # rpm -q ovirt-host ovirt-host-4.4.0-0.2.alpha.el8.x86_64 # dnf update Updating Subscription Management repositories. Last metadata expiration check: 0:11:31 ago on Fri Dec 27 13:16:55 2019. Error: Running QEMU processes found, cannot upgrade Vdsm. # dnf update ovirt-host Updating Subscription Management repositories. Extra Packages for Enterprise Linux 8 - x86_64 7.3 kB/s | 5.3 kB 00:00 GlusterFS 6 packages for x86_64 2.8 kB/s | 3.0 kB 00:01 GlusterFS 6 noarch packages 18 kB/s | 3.0 kB 00:00 virtio-win builds roughly matching what will be shipped in upcoming RHEL 8.2 kB/s | 3.0 kB 00:00 Copr repo for EL8_collection owned by sbonazzo 10 kB/s | 3.6 kB 00:00 Copr repo for gluster-ansible owned by sac 13 kB/s | 3.3 kB 00:00 Copr repo for ovsdbapp owned by mdbarroso 14 kB/s | 3.3 kB 00:00 Copr repo for nmstate-stable owned by nmstate 14 kB/s | 3.3 kB 00:00 Copr repo for NetworkManager-1.20 owned by networkmanager 14 kB/s | 3.3 kB 00:00 Latest oVirt master nightly snapshot 4.0 kB/s | 3.0 kB 00:00 Latest oVirt master additional nightly snapshot 5.0 kB/s | 3.0 kB 00:00 Red Hat CodeReady Linux Builder for RHEL 8 x86_64 (RPMs) 14 kB/s | 4.5 kB 00:00 Red Hat Enterprise Linux 8 for x86_64 - BaseOS (RPMs) 27 kB/s | 4.1 kB 00:00 Red Hat Enterprise Linux 8 for x86_64 - AppStream (RPMs) 36 kB/s | 4.5 kB 00:00 Dependencies resolved. Nothing to do. Complete! ------------------------- ovirt-host-mgmt-ansible-check log -------------------------------------------- 2019-12-27 12:02:45 CST - TASK [ovirt-host-upgrade : Update system] ************************************** 2019-12-27 12:02:55 CST - fatal: [harrier....]: FAILED! => {"changed": false, "failures": [], "msg": "Unknown Error occured: Running QEMU processes found, cannot upgrade Vdsm.", "rc": 1, "results": []} 2019-12-27 12:02:55 CST - { "status" : "OK", "msg" : "", "data" : { "event" : "runner_on_failed", "uuid" : "74b495fa-2534-4850-b905-7255fed1fc56", "stdout" : "fatal: [harrier...]: FAILED! => {\"changed\": false, \"failures\": [], \"msg\": \"Unknown Error occured: Running QEMU processes found, cannot upgrade Vdsm.\", \"rc\": 1, \"results\": []}", "counter" : 26, "pid" : 27445, "created" : "2019-12-27T18:02:53.245099", "end_line" : 25, "runner_ident" : "094f2f72-28d3-11ea-bc88-525400060930", "start_line" : 24, "event_data" : { "play_pattern" : "all", "play" : "all", "event_loop" : null, "task_args" : "", "remote_addr" : "harrier....", "res" : { "_ansible_no_log" : false, "changed" : false, "results" : [ ], "msg" : "Unknown Error occured: Running QEMU processes found, cannot upgrade Vdsm.", "rc" : 1, "invocation" : { "module_args" : { "lock_timeout" : 300, "update_cache" : true, "conf_file" : null, "exclude" : [ ], "allow_downgrade" : false, "disable_gpg_check" : false, "disable_excludes" : null, "validate_certs" : true, "state" : "latest", "disablerepo" : [ ], "releasever" : null, "skip_broken" : false, "autoremove" : false, "download_dir" : null, "installroot" : "/", "install_weak_deps" : true, "name" : [ "*" ], "download_only" : false, "bugfix" : false, "list" : null, "install_repoquery" : true, "update_only" : false, "disable_plugin" : [ ], "enablerepo" : [ ], "security" : false, "enable_plugin" : [ ] } }, "failures" : [ ] }, "pid" : 27445, "play_uuid" : "52540006-0930-56fb-1424-000000000006", "task_uuid" : "52540006-0930-56fb-1424-000000000012", "task" : "Update system", "playbook_uuid" : "758d0326-a52a-4fe9-87b3-6be0d3492784", "playbook" : "ovirt-host-upgrade.yml", "task_action" : "yum", "host" : "harrier....", "ignore_errors" : null, "role" : "ovirt-host-upgrade", "task_path" : "/usr/share/ovirt-engine/ansible-runner-service-project/project/roles/ovirt-host-upgrade/tasks/main.yml:18" }, "parent_uuid" : "52540006-0930-56fb-1424-000000000012" } }

I can reproduce it easily. If no virtual machines are running, Host -> Installation -> Check for upgrade returns "no updates found". If there are virtual machines running, Host -> Installation -> Check for upgrade returns "failed to check for available updates..."
participants (1)
-
alexandermurashkin@msn.com