HCI Fresh Deploy - Cannot upgrade 4.4. to 4.5 or add more hosts
by penguin pages
I reinstalled OS all nodes CentoOS 8 streams.
installed cockpit engine and ran through HCI deploy wizard with gluster. this deployed 4.4 version of engine.. then said 4.5 version available.
I try to add hosts and get error "Error while executing action: Server thor.penguinpages.local is already part of another cluster." (or other node).
I ssh into ovirt engine host and attempt to ssh from that host to three nodes of HCI cluster and it works fine. I added ssh keys for passwordless login but no change
I change cluster then datacenter to 4.5. Then attempt to run upgrade but get same error.
Seems like chicken / egg issue. I can't get nodes. I can't upgrade 4.4. to 4.5 because I have no nodes to allow maintenance mode. And engine shows "no updates" needed
[root@ovirte01 ~]# yum update
Last metadata expiration check: 1:39:16 ago on Mon 08 Mar 2021 08:36:40 AM EST.
Dependencies resolved.
Nothing to do.
Complete!
Suggestions?
4 years, 1 month
[security-updates] oVirt node
by Thiago Linhares
Hello there,
I wonder whats the right approach to get security updates for ovirt nodes? (installed using ovirt node iso image)
Eg.:
'sudo' package has a know vulnerability until version sudo-1.8.23-9.el7.x86_64.
When trying to update this package on a ovirt node, it would not.
Checking versionlock, I confirmed it is listed there:
# grep sudo /etc/yum/pluginconf.d/versionlock.list
0:libsss_sudo-1.16.4-37.el7_8.3.x86_64
0:sudo-1.8.23-9.el7.x86_64
Regards,
4 years, 1 month
CVE-2021-3156 && ovirt-node-ng 4.3 && 4.4 (sudo)
by Renaud RAKOTOMALALA
Hello everyone,
I operate several oVirt clusters including pre-productions using ovirt-node-ng images.
For our traditional clusters we manage the incident in a unitary way with a dedicated rpm, however for ovirt-node-ng I am not yet up to date with critical package updates process.
Do you have any advice or tips?
Nice day,
Renaud
4 years, 1 month
Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Marko Vrgotic
The broker log, these lines are pretty much repeating:
MainThread::WARNING::2021-03-03 09:19:12,086::storage_broker::97::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(__init__) Can't connect vdsm storage: 'metadata_image_UUID can't be 'None'
MainThread::INFO::2021-03-03 09:19:12,829::broker::47::ovirt_hosted_engine_ha.broker.broker.Broker::(run) ovirt-hosted-engine-ha broker 2.3.6 started
MainThread::INFO::2021-03-03 09:19:12,829::monitor::40::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Searching for submonitors in /usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/sub
monitors
MainThread::INFO::2021-03-03 09:19:12,829::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load
MainThread::INFO::2021-03-03 09:19:12,832::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load-no-engine
MainThread::INFO::2021-03-03 09:19:12,832::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor engine-health
MainThread::INFO::2021-03-03 09:19:12,832::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor mem-free
MainThread::INFO::2021-03-03 09:19:12,833::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor mgmt-bridge
MainThread::INFO::2021-03-03 09:19:12,833::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor network
MainThread::INFO::2021-03-03 09:19:12,833::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor storage-domain
MainThread::INFO::2021-03-03 09:19:12,833::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load
MainThread::INFO::2021-03-03 09:19:12,834::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load-no-engine
MainThread::INFO::2021-03-03 09:19:12,835::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor engine-health
MainThread::INFO::2021-03-03 09:19:12,835::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor mem-free
MainThread::INFO::2021-03-03 09:19:12,835::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor mgmt-bridge
MainThread::INFO::2021-03-03 09:19:12,835::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor network
MainThread::INFO::2021-03-03 09:19:12,836::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor storage-domain
MainThread::INFO::2021-03-03 09:19:12,836::monitor::50::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Finished loading submonitors
MainThread::WARNING::2021-03-03 09:19:12,836::storage_broker::97::ovirt_hosted_engine_ha.broker.storage_broker.StorageBroker::(__init__) Can't connect vdsm storage: 'metadata_image_UUID can't be 'None'
MainThread::INFO::2021-03-03 09:19:13,574::broker::47::ovirt_hosted_engine_ha.broker.broker.Broker::(run) ovirt-hosted-engine-ha broker 2.3.6 started
MainThread::INFO::2021-03-03 09:19:13,575::monitor::40::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Searching for submonitors in /usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/submonitors
MainThread::INFO::2021-03-03 09:19:13,575::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load
MainThread::INFO::2021-03-03 09:19:13,577::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor cpu-load-no-engine
MainThread::INFO::2021-03-03 09:19:13,578::monitor::49::ovirt_hosted_engine_ha.broker.monitor.Monitor::(_discover_submonitors) Loaded submonitor engine-health
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
From: Marko Vrgotic <M.Vrgotic(a)activevideo.com>
Date: Monday, 8 March 2021 at 15:34
To: Yedidyah Bar David <didi(a)redhat.com>
Cc: users(a)ovirt.org <users(a)ovirt.org>
Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
Hi Didi,
Please find the attached logs from Host and Engine.
Host ovirt-sj-02 HE Undeploy 2021-03-08 14:15:52 till 2021-03-08 14:18:24
Host ovirt-sj-02 HE Deploy 2021-03-08 14:20:51 till 2021-03-08 14:23:22
I do see errors in the agent and broker and vdsm, but I do not see why it happened.
Thank you for helping, let me know if any additional files are needed.
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
From: Yedidyah Bar David <didi(a)redhat.com>
Date: Monday, 8 March 2021 at 09:25
To: Marko Vrgotic <M.Vrgotic(a)activevideo.com>
Cc: users(a)ovirt.org <users(a)ovirt.org>
Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!***
Hi,
On Mon, Mar 8, 2021 at 10:13 AM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote:
>
> I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before.
>
> No errors, found int the deployment, but it seems half done, based on messages I sent in previous email.
Please check/share all relevant logs. Thanks. Can be all of /var/log
from engine and hosts, and at least:
/var/log/ovirt-engine/engine.log
/var/log/vdsm/*
/var/log/ovirt-hosted-engine-ha/*
Best regards,
--
Didi
4 years, 1 month
Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Marko Vrgotic
Hi Didi,
Please find the attached logs from Host and Engine.
Host ovirt-sj-02 HE Undeploy 2021-03-08 14:15:52 till 2021-03-08 14:18:24
Host ovirt-sj-02 HE Deploy 2021-03-08 14:20:51 till 2021-03-08 14:23:22
I do see errors in the agent and broker and vdsm, but I do not see why it happened.
Thank you for helping, let me know if any additional files are needed.
I will send the broker.log separately
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
From: Marko Vrgotic <M.Vrgotic(a)activevideo.com>
Date: Monday, 8 March 2021 at 15:29
To: Yedidyah Bar David <didi(a)redhat.com>
Cc: users(a)ovirt.org <users(a)ovirt.org>
Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
Hi Didi,
Please find the attached logs from Host and Engine.
Host ovirt-sj-02 HE Undeploy 2021-03-08 14:15:52 till 2021-03-08 14:18:24
Host ovirt-sj-02 HE Deploy 2021-03-08 14:20:51 till 2021-03-08 14:23:22
I do see errors in the agent and broker and vdsm, but I do not see why it happened.
Thank you for helping, let me know if any additional files are needed.
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
From: Yedidyah Bar David <didi(a)redhat.com>
Date: Monday, 8 March 2021 at 09:25
To: Marko Vrgotic <M.Vrgotic(a)activevideo.com>
Cc: users(a)ovirt.org <users(a)ovirt.org>
Subject: Re: [ovirt-users] Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
***CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender!!!***
Hi,
On Mon, Mar 8, 2021 at 10:13 AM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote:
>
> I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before.
>
> No errors, found int the deployment, but it seems half done, based on messages I sent in previous email.
Please check/share all relevant logs. Thanks. Can be all of /var/log
from engine and hosts, and at least:
/var/log/ovirt-engine/engine.log
/var/log/vdsm/*
/var/log/ovirt-hosted-engine-ha/*
Best regards,
--
Didi
4 years, 1 month
sanlock error
by kim.kargaard＠noroff.no
Hi,
We are running ovirt 4.3 and whenever we create a new VM, we get the error:
HSMGetAllTasksStatusesVDS failed: Error creating a new volume: (u"Volume creation 77be733a-a38a-4125-a3ae-beea960d9e28 failed: (28, 'Sanlock resource write failure', 'No space left on device')",)
3/8/2111:56:57 AM
I have checked the engine logs and see the following:
2021-03-08 12:09:18,306+01 ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [] BaseAsyncTask::logEndTaskFailure: Task 'd4aa81a1-9adc-45c9-be46-05bc63862f4e' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure:
-- Result: 'cleanSuccess'
-- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Volume does not exist: (u'46bf0806-8d16-40db-92df-8436f43628a4',), code = 201',
-- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Volume does not exist: (u'46bf0806-8d16-40db-92df-8436f43628a4',), code = 201'
Not sure how to solve this issue. I have tried restarting the engine, but that did not solve it. Any tips?
Thanks.
4 years, 1 month
Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Yedidyah Bar David
Hi,
On Mon, Mar 8, 2021 at 10:13 AM Marko Vrgotic <M.Vrgotic(a)activevideo.com> wrote:
>
> I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before.
>
> No errors, found int the deployment, but it seems half done, based on messages I sent in previous email.
Please check/share all relevant logs. Thanks. Can be all of /var/log
from engine and hosts, and at least:
/var/log/ovirt-engine/engine.log
/var/log/vdsm/*
/var/log/ovirt-hosted-engine-ha/*
Best regards,
--
Didi
4 years, 1 month
Re: Upgrade from 4.3.5 to 4.3.10 HE Host issue
by Marko Vrgotic
I cannot find the reason why the re-Deployment on this Hosts fails, as it was already deployed on it before.
No errors, found int the deployment, but it seems half done, based on messages I sent in previous email.
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
From: Vrgotic, Marko <M.Vrgotic(a)activevideo.com>
Date: Wednesday, 3 March 2021 at 13:12
To: users(a)ovirt.org <users(a)ovirt.org>
Subject: Upgrade from 4.3.5 to 4.3.10 HE Host issue
Hi oVirt,
Yesterday I upgraded one of my oVIrt platform from 4.3.5 to 4.3.10. Platform has 10Hosts with 3Hosts as Hosts for HA.
Before upgrade, all three Hosts were acknowledged Hosts for HE (little happy crown).
Pre upgrade:
Host1 – HE available host
Host2 – HE current host
Host3 – HE available host
Upgraded first HE to 4.3.10, than started with HE Hosts – post upgrade of Host2, the crown disappeared and did not show up. This happens sometimes, and I usually reinstall the host, than execute HE UNDEPLOY and DEPLOY, which fixes the state.
However, this time, after such actions, the server is still not an acknowledged HE host – no crown. However, if I try to remove it, oVIrt says that looks like HE is deploy on it and need to be removed, but command “hosted-engine –vm-status” says you must deploy first.
I removed the Host, reinstalled the OS on it, deployed it again as new, synced all networks, than deploy once more with HE Deploy, still same issue.
Any suggestions?
-----
kind regards/met vriendelijke groeten
Marko Vrgotic
Sr. System Engineer @ System Administration
ActiveVideo
o: +31 (35) 6774131
m: +31 (65) 5734174
e: m.vrgotic(a)activevideo.com<mailto:m.vrgotic@activevideo.com>
w: www.activevideo.com<http://www.activevideo.com>
ActiveVideo Networks BV. Mediacentrum 3745 Joop van den Endeplein 1.1217 WJ Hilversum, The Netherlands. The information contained in this message may be legally privileged and confidential. It is intended to be read only by the individual or entity to whom it is addressed or by their designee. If the reader of this message is not the intended recipient, you are on notice that any distribution of this message, in any form, is strictly prohibited. If you have received this message in error, please immediately notify the sender and/or ActiveVideo Networks, LLC by telephone at +1 408.931.9200 and delete or destroy any copy of this message.
4 years, 1 month
4.4.4 Image Copying / Template Create Fails - No Such File
by Alex McWhirter
I've been wrestling with this all night, digging through various bits of
VDSM code trying to figure why and how this is happening. I need to make
some templates, but i simply can't.
VDSM <host> command HSMGetAllTasksStatusesVDS failed: value=low level
Image copy failed: ("Command ['/usr/bin/qemu-img', 'convert', '-p',
'-t', 'none', '-T', 'none', '-f', 'raw', '-O', 'raw',
'/rhev/data-center/mnt/glusterSD/<gluster>:_Temp/45740f16-b3c9-4bb5-ba5f-3e64657fb663/images/6f87b073-c4ec-42f2-87da-d1cb6a08a150/f2dfb779-b49c-4ec9-86cd-741f3fe5b781',
'/rhev/data-center/mnt/glusterSD/<gluster>:_Temp/45740f16-b3c9-4bb5-ba5f-3e64657fb663/images/84e56da6-8c26-4518-80a4-20bc395214db/039c5ada-ad6d-45c0-8393-bd4db0bbc366']
failed with rc=1 out=b'' err=bytearray(b'qemu-img: error while writing
at byte 738197504: No such file or directory\\n')",) abortedcode=261
3/7/21 5:44:44 AM
Following the VDSM logs, i can see the new image gets created,
permissions set, etc... but as soon qemu-img starts, it fails like this.
I updated all hosts and the engine, rebooted the entire stack, to no
avail. So i detached the storage domain, and wiped every host and fresh
installed both engine and all nodes, imported the storage domain, and
still no dice. Storage domain is gluster volume, single node, created in
ovirt.
It happens when i make a template, copy an image, or make a new vm from
a template. I can still create new vms from blank, and upload images via
the web ui. Watching the gluster share, i can see the image being
created, but its deleted at some point. I appears to not be being
deleted by the template / copying process, as immediately after the
above error, i get this one.
VDSM command DeleteImageGroupVDS failed: Image does not exist in domain:
'image=4f359545-01a8-439b-832b-18c26194b066,
domain=b4507449-ac40-4e35-be66-56441bb696ac'
3/7/21 5:44:44 AM
I thought maybe garbage collection, but don't see any indication of that
in the logs.
Any ideas? I redacted host names from the log output.
4 years, 1 month
Which Operating System to choose
by sinan＠turka.nl
I want to set up a small cluster consisting of 2 Hypervisor nodes (self-hosted engine) with shared storage (SAN, ISCSI).
According to https://www.ovirt.org/download/ the following Operating Systems are supported with the latest oVirt version (4.4.4):
- Red Hat Enterprise Linux 8.3
- CentOS Linux 8.3
- CentOS Stream (Tech Preview)
My doubts are:
- RHEL 8.3 is free to use up tp 16 hosts, but until when? Red Hat can decide at anytime to change their subscription program.
- CentOS Linux is no more, support will stop at the end of 2021.
- CentOS Stream is in Tech Preview. I don't like running production environments on "tech preview / beta" versions.
I would say CentOS 7, it will receive maintenance updates up to 2024, and switch to RHEL in 2024 if the free-to-use program is still active. Unfortunately, CentOS 7 seems not to be supported.
On what Operating System are you running oVirt 4.4.4 and why? Any feedback and advice is welcome. Thanks!
4 years, 1 month
