- Users - oVirt List Archives

Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Marko Vrgotic 08 Mar '21

08 Mar '21

The broker log, these lines are pretty much repeating: MainThread::WARNING::2021-03-03 09:19:12,086::storage_broker::97::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(__init__) Can't connect vdsm storage: 'metadata_image_UUID can't be 'None' MainThread::INFO::2021-03-03 09:19:12,829::broker::47::ovirt_hosted_engine_ha.broker.broker.Broker::(run) ovirt-hosted-engine-ha broker 2.3.6 started MainThread::INFO::2021-03-03 09:19:12,829::monitor::40::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Searching for submonitors in /usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/sub monitors MainThread::INFO::2021-03-03 09:19:12,829::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load MainThread::INFO::2021-03-03 09:19:12,832::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load-no-engine MainThread::INFO::2021-03-03 09:19:12,832::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor engine-health MainThread::INFO::2021-03-03 09:19:12,832::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor mem-free MainThread::INFO::2021-03-03 09:19:12,833::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor mgmt-bridge MainThread::INFO::2021-03-03 09:19:12,833::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor network MainThread::INFO::2021-03-03 09:19:12,833::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor storage-domain MainThread::INFO::2021-03-03 09:19:12,833::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load MainThread::INFO::2021-03-03 09:19:12,834::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load-no-engine MainThread::INFO::2021-03-03 09:19:12,835::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor engine-health MainThread::INFO::2021-03-03 09:19:12,835::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor mem-free MainThread::INFO::2021-03-03 09:19:12,835::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor mgmt-bridge MainThread::INFO::2021-03-03 09:19:12,835::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor network MainThread::INFO::2021-03-03 09:19:12,836::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor storage-domain MainThread::INFO::2021-03-03 09:19:12,836::monitor::50::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Finished loading submonitors MainThread::WARNING::2021-03-03 09:19:12,836::storage_broker::97::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(__init__) Can't connect vdsm storage: 'metadata_image_UUID can't be 'None' MainThread::INFO::2021-03-03 09:19:13,574::broker::47::ovirt_hosted_engine_ha.broker.broker.Broker::(run) ovirt-hosted-engine-ha broker 2.3.6 started MainThread::INFO::2021-03-03 09:19:13,575::monitor::40::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Searching for submonitors in /usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/submonitors MainThread::INFO::2021-03-03 09:19:13,575::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load MainThread::INFO::2021-03-03 09:19:13,577::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load-no-engine MainThread::INFO::2021-03-03 09:19:13,578::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor engine-health ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo o: +31 (35) 6774131 m: +31 (65) 5734174 e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message. From: Marko Vrgotic <M.Vrgotic(a)activevideo.com> Date: Monday, 8 March 2021 at 15:34 To: Yedidyah Bar David <didi(a)redhat.com> Cc: users(a)ovirt.org <users(a)ovirt.org> Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue Hi Didi, Please find the attached logs from Host and Engine. Host ovirt-sj-02 HE Undeploy 2021-03-08 14:15:52 till 2021-03-08 14:18:24 Host ovirt-sj-02 HE Deploy 2021-03-08 14:20:51 till 2021-03-08 14:23:22 I do see errors in the agent and broker and vdsm, but I do not see why it happened. Thank you for helping, let me know if any additional files are needed. ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo o: +31 (35) 6774131 m: +31 (65) 5734174 e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message. From: Yedidyah Bar David <didi(a)redhat.com> Date: Monday, 8 March 2021 at 09:25 To: Marko Vrgotic <M.Vrgotic(a)activevideo.com> Cc: users(a)ovirt.org <users(a)ovirt.org> Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue ***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!*** Hi, On Mon, Mar 8, 2021 at 10:13 AM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote: > > I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before. > > No errors, found int the deployment, but it seems half done, based on messages I sent in previous email. Please check/share all relevant logs. Thanks. Can be all of /var/log from engine and hosts, and at least: /var/log/ovirt-engine/engine.log /var/log/vdsm/* /var/log/ovirt-hosted-engine-ha/* Best regards, -- Didi

1 0

Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Marko Vrgotic 08 Mar '21

08 Mar '21

Hi Didi, Please find the attached logs from Host and Engine. Host ovirt-sj-02 HE Undeploy 2021-03-08 14:15:52 till 2021-03-08 14:18:24 Host ovirt-sj-02 HE Deploy 2021-03-08 14:20:51 till 2021-03-08 14:23:22 I do see errors in the agent and broker and vdsm, but I do not see why it happened. Thank you for helping, let me know if any additional files are needed. I will send the broker.log separately ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo o: +31 (35) 6774131 m: +31 (65) 5734174 e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message. From: Marko Vrgotic <M.Vrgotic(a)activevideo.com> Date: Monday, 8 March 2021 at 15:29 To: Yedidyah Bar David <didi(a)redhat.com> Cc: users(a)ovirt.org <users(a)ovirt.org> Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue Hi Didi, Please find the attached logs from Host and Engine. Host ovirt-sj-02 HE Undeploy 2021-03-08 14:15:52 till 2021-03-08 14:18:24 Host ovirt-sj-02 HE Deploy 2021-03-08 14:20:51 till 2021-03-08 14:23:22 I do see errors in the agent and broker and vdsm, but I do not see why it happened. Thank you for helping, let me know if any additional files are needed. ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo o: +31 (35) 6774131 m: +31 (65) 5734174 e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message. From: Yedidyah Bar David <didi(a)redhat.com> Date: Monday, 8 March 2021 at 09:25 To: Marko Vrgotic <M.Vrgotic(a)activevideo.com> Cc: users(a)ovirt.org <users(a)ovirt.org> Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue ***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!*** Hi, On Mon, Mar 8, 2021 at 10:13 AM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote: > > I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before. > > No errors, found int the deployment, but it seems half done, based on messages I sent in previous email. Please check/share all relevant logs. Thanks. Can be all of /var/log from engine and hosts, and at least: /var/log/ovirt-engine/engine.log /var/log/vdsm/* /var/log/ovirt-hosted-engine-ha/* Best regards, -- Didi

1 0

sanlock error
by kim.kargaard＠noroff.no 08 Mar '21

08 Mar '21

Hi, We are running ovirt 4.3 and whenever we create a new VM, we get the error: HSMGetAllTasksStatusesVDS failed: Error creating a new volume: (u"Volume creation 77be733a-a38a-4125-a3ae-beea960d9e28 failed: (28, 'Sanlock resource write failure', 'No space left on device')",) 3/8/2111:56:57 AM I have checked the engine logs and see the following: 2021-03-08 12:09:18,306+01 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [] BaseAsyncTask::logEndTaskFailure: Task 'd4aa81a1-9adc-45c9-be46-05bc63862f4e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Volume does not exist: (u'46bf0806-8d16-40db-92df-8436f43628a4',), code = 201', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Volume does not exist: (u'46bf0806-8d16-40db-92df-8436f43628a4',), code = 201' Not sure how to solve this issue. I have tried restarting the engine, but that did not solve it. Any tips? Thanks.

3 4

Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Yedidyah Bar David 08 Mar '21

08 Mar '21

Hi, On Mon, Mar 8, 2021 at 10:13 AM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote: > > I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before. > > No errors, found int the deployment, but it seems half done, based on messages I sent in previous email. Please check/share all relevant logs. Thanks. Can be all of /var/log from engine and hosts, and at least: /var/log/ovirt-engine/engine.log /var/log/vdsm/* /var/log/ovirt-hosted-engine-ha/* Best regards, -- Didi

1 0

Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Marko Vrgotic 08 Mar '21

08 Mar '21

I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before. No errors, found int the deployment, but it seems half done, based on messages I sent in previous email. ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo o: +31 (35) 6774131 m: +31 (65) 5734174 e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message. From: Vrgotic, Marko <M.Vrgotic(a)activevideo.com> Date: Wednesday, 3 March 2021 at 13:12 To: users(a)ovirt.org <users(a)ovirt.org> Subject: Upgrade from 4.3.5 to 4.3.10 HE Host issue Hi oVirt, Yesterday I upgraded one of my oVIrt platform from 4.3.5 to 4.3.10. Platform has 10Hosts with 3Hosts as Hosts for HA. Before upgrade, all three Hosts were acknowledged Hosts for HE (little happy crown). Pre upgrade: Host1 – HE available host Host2 – HE current host Host3 – HE available host Upgraded first HE to 4.3.10, than started with HE Hosts – post upgrade of Host2, the crown disappeared and did not show up. This happens sometimes, and I usually reinstall the host, than execute HE UNDEPLOY and DEPLOY, which fixes the state. However, this time, after such actions, the server is still not an acknowledged HE host – no crown. However, if I try to remove it, oVIrt says that looks like HE is deploy on it and need to be removed, but command “hosted-engine –vm-status” says you must deploy first. I removed the Host, reinstalled the OS on it, deployed it again as new, synced all networks, than deploy once more with HE Deploy, still same issue. Any suggestions? ----- kind regards/met vriendelijke groeten Marko Vrgotic Sr. System Engineer @ System Administration ActiveVideo o: +31 (35) 6774131 m: +31 (65) 5734174 e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com> w: www.activevideo.com<http://www.activevideo.com> ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.

1 0

4.4.4 Image Copying / Template Create Fails - No Such File
by Alex McWhirter 08 Mar '21

08 Mar '21

I've been wrestling with this all night, digging through various bits of VDSM code trying to figure why and how this is happening. I need to make some templates, but i simply can't. VDSM <host> command HSMGetAllTasksStatusesVDS failed: value=low level Image copy failed: ("Command ['/usr/bin/qemu-img', 'convert', '-p', '-t', 'none', '-T', 'none', '-f', 'raw', '-O', 'raw', '/rhev/data-center/mnt/glusterSD/<gluster>:_Temp/45740f16-b3c9-4bb5-ba5f-3e64657fb663/images/6f87b073-c4ec-42f2-87da-d1cb6a08a150/f2dfb779-b49c-4ec9-86cd-741f3fe5b781', '/rhev/data-center/mnt/glusterSD/<gluster>:_Temp/45740f16-b3c9-4bb5-ba5f-3e64657fb663/images/84e56da6-8c26-4518-80a4-20bc395214db/039c5ada-ad6d-45c0-8393-bd4db0bbc366'] failed with rc=1 out=b'' err=bytearray(b'qemu-img: error while writing at byte 738197504: No such file or directory\\n')",) abortedcode=261 3/7/21 5:44:44 AM Following the VDSM logs, i can see the new image gets created, permissions set, etc... but as soon qemu-img starts, it fails like this. I updated all hosts and the engine, rebooted the entire stack, to no avail. So i detached the storage domain, and wiped every host and fresh installed both engine and all nodes, imported the storage domain, and still no dice. Storage domain is gluster volume, single node, created in ovirt. It happens when i make a template, copy an image, or make a new vm from a template. I can still create new vms from blank, and upload images via the web ui. Watching the gluster share, i can see the image being created, but its deleted at some point. I appears to not be being deleted by the template / copying process, as immediately after the above error, i get this one. VDSM command DeleteImageGroupVDS failed: Image does not exist in domain: 'image=4f359545-01a8-439b-832b-18c26194b066, domain=b4507449-ac40-4e35-be66-56441bb696ac' 3/7/21 5:44:44 AM I thought maybe garbage collection, but don't see any indication of that in the logs. Any ideas? I redacted host names from the log output.

3 5

Which Operating System to choose
by sinan＠turka.nl 07 Mar '21

07 Mar '21

I want to set up a small cluster consisting of 2 Hypervisor nodes (self-hosted engine) with shared storage (SAN, ISCSI). According to https://www.ovirt.org/download/ the following Operating Systems are supported with the latest oVirt version (4.4.4): - Red Hat Enterprise Linux 8.3 - CentOS Linux 8.3 - CentOS Stream (Tech Preview) My doubts are: - RHEL 8.3 is free to use up tp 16 hosts, but until when? Red Hat can decide at anytime to change their subscription program. - CentOS Linux is no more, support will stop at the end of 2021. - CentOS Stream is in Tech Preview. I don't like running production environments on "tech preview / beta" versions. I would say CentOS 7, it will receive maintenance updates up to 2024, and switch to RHEL in 2024 if the free-to-use program is still active. Unfortunately, CentOS 7 seems not to be supported. On what Operating System are you running oVirt 4.4.4 and why? Any feedback and advice is welcome. Thanks!

2 1

Cold Move VM Storage failing
by dmmailing010203＠gmail.com 07 Mar '21

07 Mar '21

Morning oVirt. I've rebuilt my oVirt environment and I'm currently on 4.4.5.8-1.el8. I have storage presented via NFS. I'm unable to move / copy VM disks when powered off. However I can Live migrate the disks when powered on. I have three storage domains presented and this is repeated across all three no matter where the storage is located. The disks are thin provisioned. I've tried deactivating and detaching the disk and it's the same NullPointerException error. 2021-03-06 15:13:17,010Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [af63add0-0ef3-4049-8461-b564abedba02] Command 'org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand' failed: null 2021-03-06 15:13:17,010Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [af63add0-0ef3-4049-8461-b564abedba02] Exception: java.lang.NullPointerException Engine log with stack trace is at https://textuploader.com/1820f Vdsm log is at https://textuploader.com/1820z I'm just checking I've not missed something obvious before raising a bug if neccessary

2 2

Gluster version upgrades
by Sketch 07 Mar '21

07 Mar '21

Is the gluster version on an oVirt host tied to the oVirt version, or would it be safe to upgrade to newer versions of gluster? I have noticed gluster is often updated to new major versions on oVirt point release upgrades. We have some compute+storage hosts on 4.3.6 which can't be upgraded easily at the moment, but we are having some gluster issues that appear to be due to bugs that I wonder if upgrading might help. Would an in-place upgrade of gluster be a bad idea without also updating oVirt?

2 1

VM ovirt-win10-01 is down with error. Exit message: Unable to get volume size for domain 5cee5082-074e-4a8b-a750-0e8fb0aff63c volume 05063eec-ca54-42c6-92d6-0a6775545eba.
by wangyu13476969128＠126.com 06 Mar '21

06 Mar '21

The version of ovirt-manager is 4.3.6.7。 I have two ovirt-nodes. After I restart one ovirt-node, all the Linux virtual machines on the ovirt-node can startup. But all the Windows virtual machines on the ovirt-node can't startup。 The title show the error: VM ovirt-win10-01 is down with error. Exit message: Unable to get volume size for domain 5cee5082-074e-4a8b-a750-0e8fb0aff63c volume 05063eec-ca54-42c6-92d6-0a6775545eba. How can I solve this problem ?

1 0