Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Marko Vrgotic
Hi Didi,
Please find the attached logs from Host and Engine.
Host ovirt-sj-02 HE Undeploy 2021-03-08 14:15:52 till 2021-03-08 14:18:24
Host ovirt-sj-02 HE Deploy 2021-03-08 14:20:51 till 2021-03-08 14:23:22
I do see errors in the agent and broker and vdsm, but I do not see why it happened.
Thank you for helping, let me know if any additional files are needed.
I will send the broker.log separately
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
From: Marko Vrgotic <M.Vrgotic(a)activevideo.com>
Date: Monday, 8 March 2021 at 15:29
To: Yedidyah Bar David <didi(a)redhat.com>
Cc: users(a)ovirt.org <users(a)ovirt.org>
Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
Hi Didi,
Please find the attached logs from Host and Engine.
Host ovirt-sj-02 HE Undeploy 2021-03-08 14:15:52 till 2021-03-08 14:18:24
Host ovirt-sj-02 HE Deploy 2021-03-08 14:20:51 till 2021-03-08 14:23:22
I do see errors in the agent and broker and vdsm, but I do not see why it happened.
Thank you for helping, let me know if any additional files are needed.
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
From: Yedidyah Bar David <didi(a)redhat.com>
Date: Monday, 8 March 2021 at 09:25
To: Marko Vrgotic <M.Vrgotic(a)activevideo.com>
Cc: users(a)ovirt.org <users(a)ovirt.org>
Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!***
Hi,
On Mon, Mar 8, 2021 at 10:13 AM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote:
>
> I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before.
>
> No errors, found int the deployment, but it seems half done, based on messages I sent in previous email.
Please check/share all relevant logs. Thanks. Can be all of /var/log
from engine and hosts, and at least:
/var/log/ovirt-engine/engine.log
/var/log/vdsm/*
/var/log/ovirt-hosted-engine-ha/*
Best regards,
--
Didi
3 years
sanlock error
by kim.kargaard@noroff.no
Hi,
We are running ovirt 4.3 and whenever we create a new VM, we get the error:
HSMGetAllTasksStatusesVDS failed: Error creating a new volume: (u"Volume creation 77be733a-a38a-4125-a3ae-beea960d9e28 failed: (28, 'Sanlock resource write failure', 'No space left on device')",)
3/8/2111:56:57 AM
I have checked the engine logs and see the following:
2021-03-08 12:09:18,306+01 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [] BaseAsyncTask::logEndTaskFailure: Task 'd4aa81a1-9adc-45c9-be46-05bc63862f4e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure:
-- Result: 'cleanSuccess'
-- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Volume does not exist: (u'46bf0806-8d16-40db-92df-8436f43628a4',), code = 201',
-- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Volume does not exist: (u'46bf0806-8d16-40db-92df-8436f43628a4',), code = 201'
Not sure how to solve this issue. I have tried restarting the engine, but that did not solve it. Any tips?
Thanks.
3 years
Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Yedidyah Bar David
Hi,
On Mon, Mar 8, 2021 at 10:13 AM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote:
>
> I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before.
>
> No errors, found int the deployment, but it seems half done, based on messages I sent in previous email.
Please check/share all relevant logs. Thanks. Can be all of /var/log
from engine and hosts, and at least:
/var/log/ovirt-engine/engine.log
/var/log/vdsm/*
/var/log/ovirt-hosted-engine-ha/*
Best regards,
--
Didi
3 years
Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Marko Vrgotic
I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before.
No errors, found int the deployment, but it seems half done, based on messages I sent in previous email.
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
From: Vrgotic, Marko <M.Vrgotic(a)activevideo.com>
Date: Wednesday, 3 March 2021 at 13:12
To: users(a)ovirt.org <users(a)ovirt.org>
Subject: Upgrade from 4.3.5 to 4.3.10 HE Host issue
Hi oVirt,
Yesterday I upgraded one of my oVIrt platform from 4.3.5 to 4.3.10. Platform has 10Hosts with 3Hosts as Hosts for HA.
Before upgrade, all three Hosts were acknowledged Hosts for HE (little happy crown).
Pre upgrade:
Host1 – HE available host
Host2 – HE current host
Host3 – HE available host
Upgraded first HE to 4.3.10, than started with HE Hosts – post upgrade of Host2, the crown disappeared and did not show up. This happens sometimes, and I usually reinstall the host, than execute HE UNDEPLOY and DEPLOY, which fixes the state.
However, this time, after such actions, the server is still not an acknowledged HE host – no crown. However, if I try to remove it, oVIrt says that looks like HE is deploy on it and need to be removed, but command “hosted-engine –vm-status” says you must deploy first.
I removed the Host, reinstalled the OS on it, deployed it again as new, synced all networks, than deploy once more with HE Deploy, still same issue.
Any suggestions?
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
3 years
4.4.4 Image Copying / Template Create Fails - No Such File
by Alex McWhirter
I've been wrestling with this all night, digging through various bits of
VDSM code trying to figure why and how this is happening. I need to make
some templates, but i simply can't.
VDSM <host> command HSMGetAllTasksStatusesVDS failed: value=low level
Image copy failed: ("Command ['/usr/bin/qemu-img', 'convert', '-p',
'-t', 'none', '-T', 'none', '-f', 'raw', '-O', 'raw',
'/rhev/data-center/mnt/glusterSD/<gluster>:_Temp/45740f16-b3c9-4bb5-ba5f-3e64657fb663/images/6f87b073-c4ec-42f2-87da-d1cb6a08a150/f2dfb779-b49c-4ec9-86cd-741f3fe5b781',
'/rhev/data-center/mnt/glusterSD/<gluster>:_Temp/45740f16-b3c9-4bb5-ba5f-3e64657fb663/images/84e56da6-8c26-4518-80a4-20bc395214db/039c5ada-ad6d-45c0-8393-bd4db0bbc366']
failed with rc=1 out=b'' err=bytearray(b'qemu-img: error while writing
at byte 738197504: No such file or directory\\n')",) abortedcode=261
3/7/21 5:44:44 AM
Following the VDSM logs, i can see the new image gets created,
permissions set, etc... but as soon qemu-img starts, it fails like this.
I updated all hosts and the engine, rebooted the entire stack, to no
avail. So i detached the storage domain, and wiped every host and fresh
installed both engine and all nodes, imported the storage domain, and
still no dice. Storage domain is gluster volume, single node, created in
ovirt.
It happens when i make a template, copy an image, or make a new vm from
a template. I can still create new vms from blank, and upload images via
the web ui. Watching the gluster share, i can see the image being
created, but its deleted at some point. I appears to not be being
deleted by the template / copying process, as immediately after the
above error, i get this one.
VDSM command DeleteImageGroupVDS failed: Image does not exist in domain:
'image=4f359545-01a8-439b-832b-18c26194b066,
domain=b4507449-ac40-4e35-be66-56441bb696ac'
3/7/21 5:44:44 AM
I thought maybe garbage collection, but don't see any indication of that
in the logs.
Any ideas? I redacted host names from the log output.
3 years
Which Operating System to choose
by sinan@turka.nl
I want to set up a small cluster consisting of 2 Hypervisor nodes (self-hosted engine) with shared storage (SAN, ISCSI).
According to https://www.ovirt.org/download/ the following Operating Systems are supported with the latest oVirt version (4.4.4):
- Red Hat Enterprise Linux 8.3
- CentOS Linux 8.3
- CentOS Stream (Tech Preview)
My doubts are:
- RHEL 8.3 is free to use up tp 16 hosts, but until when? Red Hat can decide at anytime to change their subscription program.
- CentOS Linux is no more, support will stop at the end of 2021.
- CentOS Stream is in Tech Preview. I don't like running production environments on "tech preview / beta" versions.
I would say CentOS 7, it will receive maintenance updates up to 2024, and switch to RHEL in 2024 if the free-to-use program is still active. Unfortunately, CentOS 7 seems not to be supported.
On what Operating System are you running oVirt 4.4.4 and why? Any feedback and advice is welcome. Thanks!
3 years
Cold Move VM Storage failing
by dmmailing010203@gmail.com
Morning oVirt.
I've rebuilt my oVirt environment and I'm currently on 4.4.5.8-1.el8. I have storage presented via NFS. I'm unable to move / copy VM disks when powered off. However I can Live migrate the disks when powered on. I have three storage domains presented and this is repeated across all three no matter where the storage is located. The disks are thin provisioned. I've tried deactivating and detaching the disk and it's the same NullPointerException error.
2021-03-06 15:13:17,010Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [af63add0-0ef3-4049-8461-b564abedba02] Command 'org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand' failed: null
2021-03-06 15:13:17,010Z ERROR [org.ovirt.engine.core.bll.storage.disk.image.CopyDataCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-84) [af63add0-0ef3-4049-8461-b564abedba02] Exception: java.lang.NullPointerException
Engine log with stack trace is at https://textuploader.com/1820f
Vdsm log is at https://textuploader.com/1820z
I'm just checking I've not missed something obvious before raising a bug if neccessary
3 years
Gluster version upgrades
by Sketch
Is the gluster version on an oVirt host tied to the oVirt version, or
would it be safe to upgrade to newer versions of gluster?
I have noticed gluster is often updated to new major versions on oVirt
point release upgrades. We have some compute+storage hosts on 4.3.6 which
can't be upgraded easily at the moment, but we are having some gluster
issues that appear to be due to bugs that I wonder if upgrading might
help. Would an in-place upgrade of gluster be a bad idea without also
updating oVirt?
3 years
snapshot solution: Existing snapshots that were taken after this one will be erased.
by dhanaraj.ramesh@yahoo.com
Hi Team,
when I want to commit the older snapshots I'm getting warning stating " Existing snapshots that were taken after this one will be erased.". is there any way we can retain the latest snapshots as is in the chain?
I knew cloning and template export options are there to secure that latest snapshot data but these are will consume additional space in storage and take time.
3 years