- Users - oVirt List Archives

Re: Power failure makes cluster and hosted engine unusable
by Vincent Royer 31 Mar '21

31 Mar '21

Seann, If this happens again, try doing nothing (seriously) Each time I've had a power failure, the engine takes a really long time to come back up. I don't know if it's by design or what. Host logs are flooded with errors, everything seemingly storage related. However, my Gluster setup is on fast SSDs and gets back and running pretty much straight away. It takes maybe 5 minutes for the nodes to re-join and the volumes to show 'UP' with no heals. However, it still takes the hosted-engine a good hour or two to simmer down and finally start up. Sometimes I try to help by restarting ha-broker and ha-agent, or plunking in other random commands from the mess of documentation, but it seems to sort itself out on its own time, regardless of my tinkering. I wish I could get more insight into the process, but definitely, doing nothing and waiting has been the most successful troubleshooting step I have taken. Cheers! On Mon, Mar 29, 2021 at 11:32 AM Seann G. Clark via Users <users(a)ovirt.org> wrote: > All, > > > > After a power failure, and generator failure I lost my cluster, and the > Hosted engine refused to restart after power was restored. I would expect, > once storage comes up that the hosted engine comes back online without too > much of a fight. In practice because the SPM went down as well, there is no > (clearly documented) way to clear any of the stale locks, and no way to > recover both the hosted engine and the cluster. > > > > I have spent the last 12 hours trying to get a functional hosted-engine > back online, on a new node and each attempt hits a new error, from the > installer not understanding that 16384mb of dedicated VM memory out of > 192GB free on the host is indeed bigger than 4096MB, to ansible dying on > an error like this “Error while executing action: Cannot add Storage > Connection. Storage connection already exists.” > > The memory error referenced above shows up as: > > [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": > "Available memory ( {'failed': False, 'changed': False, 'ansible_facts': > {u'max_mem': u'180746'}}MB ) is less then the minimal requirement (4096MB). > Be aware that 512MB is reserved for the host and cannot be allocated to the > engine VM."} > > That is what I typically get when I try the steps outlined in the KB > “CHAPTER 7. RECOVERING A SELF-HOSTED ENGINE FROM AN EXISTING BACKUP” from > the RH Customer portal. I have tried this numerous ways, and the cluster > still remains in a bad state, with the hosted engine being 100% inoperable. > > > > What I do have are the two host that are part of the cluster and can host > the engine, and backups of the original hosted engine, both disk and > engine-backup generated. I am not sure what I can do next, to recover this > cluster, any suggestions would be apricated. > > > > Regards, > > Seann > > > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/JLDIFTKYDPQ6Y… >

1 0

Q: Set host CPU type to kvm64 for a single VM
by Andrei Verovski 30 Mar '21

30 Mar '21

Hi ! Is it possible to set host CPU type to kvm64 for a single VM ? Thanks. Andrei

4 4

Power failure makes cluster and hosted engine unusable
by Seann G. Clark 29 Mar '21

29 Mar '21

All, After a power failure, and generator failure I lost my cluster, and the Hosted engine refused to restart after power was restored. I would expect, once storage comes up that the hosted engine comes back online without too much of a fight. In practice because the SPM went down as well, there is no (clearly documented) way to clear any of the stale locks, and no way to recover both the hosted engine and the cluster. I have spent the last 12 hours trying to get a functional hosted-engine back online, on a new node and each attempt hits a new error, from the installer not understanding that 16384mb of dedicated VM memory out of 192GB free on the host is indeed bigger than 4096MB, to ansible dying on an error like this "Error while executing action: Cannot add Storage Connection. Storage connection already exists." The memory error referenced above shows up as: [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Available memory ( {'failed': False, 'changed': False, 'ansible_facts': {u'max_mem': u'180746'}}MB ) is less then the minimal requirement (4096MB). Be aware that 512MB is reserved for the host and cannot be allocated to the engine VM."} That is what I typically get when I try the steps outlined in the KB "CHAPTER 7. RECOVERING A SELF-HOSTED ENGINE FROM AN EXISTING BACKUP" from the RH Customer portal. I have tried this numerous ways, and the cluster still remains in a bad state, with the hosted engine being 100% inoperable. What I do have are the two host that are part of the cluster and can host the engine, and backups of the original hosted engine, both disk and engine-backup generated. I am not sure what I can do next, to recover this cluster, any suggestions would be apricated. Regards, Seann

1 0

VLAN Trunking to Cluster Nodes
by Jason Alexander Hazen Valliant-Saunders 29 Mar '21

29 Mar '21

Good Day Ovirt Users; I am running a the following Setup: [image: image.png] [image: image.png] [image: image.png] I need to setup the BOND1 connector in the Datacenter with 9 VLAN's they Exist on the Ovirt Node already and the Bond is UP: [image: image.png] (bond0 is 10GB Chelsio Storage Network) The idea is to pass through Vlan's on bond1 (bond1.1, 1.2, 1.3, 1.n ) where n = vlanid. However I'm unsure as to how I go about setting up the Provider in the datacenter, so that bond1 is used to trunk vlans into the DC. From what I have read the VLAN"s need to exist on the cluster node (they do); but the ovirt-engine must then have some kind of provider configured (just like it does for ovirt mgt) then i can map the vNic Profiles to that new provider as per their vlan assignment. Regards, Hazen

1 0

Cinderlib problem after upgrade from 4.3.10 to 4.4.5
by Marc-Christian Schröer 29 Mar '21

29 Mar '21

Hello all, first of all thank you very much for this stable virtualization environment. It has been a pillar for our company’s business for more than 5 years now and after migrating from version 3 to 4 it has been so stable ever since. Anyway, I ran into a problem I cannot fix on my own yesterday: After a lot of consideration and hesitation since this is a production environment I followed the upgrade guide (https://www.ovirt.org/documentation/upgrade_guide/ <https://www.ovirt.org/documentation/upgrade_guide/>), configured a vanilla CentOS 8 server as controller, decommissioned the old 4.3 controller and fired up the new one. It worked like a charm until I tried to migrate VMs, start new ones or even create new disks. We use Ceph as managed storage, providing a SSD only and a HDD only pool. The UI simply told me that there was an error. I started investigating the issue and found corresponding log entries in ovirt-engine.log: 2021-03-22 10:36:37,247+01 ERROR [org.ovirt.engine.core.common.utils.cinderlib.CinderlibExecutor] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-24) [67bf193c] cinderlib execution failed: But that was all the engine had to say about the issue. There was no stack trace or additional information. There is no logfile in /var/log/ovirt-engine/cinderlib/, the directory simply is empty while on the other controller it was frequently filed with annoying „already mounted“ messages. Can anyone help me with that issue? I searched the web for a solution or someone else with the same problem, but came up empty. Is there a way to turn up the log level for cinderlib? Are there any dependencies I have to install besides the ovirt packages? Any help is very much appreciated! Kind regards and stay healthy, Marc -- ________________________________________________________________________ Dipl.-Inform. Marc-Christian Schröer schroeer(a)ingenit.com Geschäftsführer / CEO ---------------------------------------------------------------------- ingenit GmbH & Co. KG Tel. +49 (0)231 58 698-120 Emil-Figge-Strasse 76-80 Fax. +49 (0)231 58 698-121 D-44227 Dortmund www.ingenit.com Registergericht: Amtsgericht Dortmund, HRA 13 914 Gesellschafter : Thomas Klute, Marc-Christian Schröer ________________________________________________________________________

2 3

Unable to add hosts to cluster & possible related routing issue
by David White 27 Mar '21

27 Mar '21

Hi all, I used the oVirt installer via cockpit to setup a hyperconverged cluster on 3 physical hosts running Red Hat 8.3. I used the following two resources to guide my work: - https://www.ovirt.org/documentation/gluster-hyperconverged/chap-Deploying_H… - https://blogs.ovirt.org/2018/02/up-and-running-with-ovirt-4-2-and-gluster-s… As I mentioned in a previous email a week ago, it appears as if the wizard successfully setup gluster on all 3 of the servers, and the wizard also successfully setup the ovirt engine on the first host. However, the oVirt engine only recognizes the first host, and only the first host's physical resources are available to the cluster. So I have Gluster 8 installed on the 3 hosts (RHEL 8), and oVirt 4.4.5 installed on the first host, along with the ovirt engine VM. I run `gluster peer status` from the first node, can confirm that the other two physical hosts are healthy: (example.com is replacing actual domain below) [root@cha1-storage ~]# gluster peer statusNumber of Peers: 2Hostname: cha2-storage.mgt.example.comUuid: 240a7ab1-ab52-4e5b-98ed-d978f848835eState: Peer in Cluster (Connected)Hostname: cha3-storage.mgt.example.comUuid: 0563c3e8-237d-4409-a09a-ec51719b0da6State: Peer in Cluster (Connected) I am now trying to get the other two hosts added to the Engine. I navigate to Compute -> Hosts, and click New, fill in the details (hostname, root password, etc..) and begin the installation on the additional hosts. It keeps failing. Checking the error logs in /var/log/ovirt-engine/host-deploy on the Engine VM, I see the following near the bottom: "msg" : "Failed to download metadata for repo 'ovirt-4.4-epel': Cannot prepare internal mirrorlist: Curl error (7): Couldn't connect to server for https://mirrors.fedoraproject.org/metalink?repo=epel-8&arch=x86_64&infra=$i… [Failed to connect to mirrors.fedoraproject.org port 443: No route to host]", I can confirm that the Engine and each of the hosts are able to get to mirrors.fedoraproject.org: I've run the following on both the Engine, as well as each of the hosts (the first host where everything is installed, as well as the 2nd host where I'm trying to get it installed): [root@ovirt-engine1 host-deploy]# curl -i https://mirrors.fedoraproject.org HTTP/2 302 Note, that this traffic is going out the management network. That may be an important distinction -- keep reading. So this does lead me to another issue that may or may not be related. I've discovered that, from the RHEL 8 host's perspective, the public facing network is unable to get out to the internet. Management (Gluster & the Engine VM) is on: 10.1.0.0/24 Each of the hosts are able to ping each other and communicate with each other. Each of the hosts are able to ping 8.8.8.8 whenever the frontend network interface is disabled (see below). The frontend network is on: 192.168.0.0/24 Each of the hosts are able to ping each other on this network But this network isn't able to get out to the internet (yet). Obviously I need to fix the routing and figure out why the 192.168.0.0/24 network is unable to reach the internet. But shouldn't all the traffic to install the ovirt functionality onto the 2nd host go out the management network? So to summarize, I have a few questions: - Why didn't the wizard properly install the ovirt functionality onto all 3 hosts to begin with when I did the initial installation? - From the physical host's perspective, what should be the default route? The internal management, or the front-end network? - Is the following statement accurate? - The ovirt-engine's default route should be management -- it doesn't even have a front-end IP address. - Why would the ovirt engine fail to install, and indicate that it cannot get to mirrors.fedoraproject.org, when clearly it can? - Again, the ovirt-engine VM is able to curl that URL and gets a valid http response. - Any other tips or suggestions on how to troubleshoot this? Sent with ProtonMail Secure Email.

1 1

hosted engine vm don't boot after minor update
by a.cor＠outlook.com 26 Mar '21

26 Mar '21

Hello! I have rhv 4.4.2 and NFS storage type for HE Domain Storage. I updated to 4.4.4 by guide (https://www.ovirt.org/documentation/upgrade_guide/#Updating_a_self-hosted_e…) and reboot Hosted Engine VM. After this VM is not booting. Connected to console I see next error: BdsDxe: failed to load Boot0001 "UEFI Misc Device"... After "Press any key" I got BIOS screen, go to Device Manager -> Drive Health Manager I see nothing. I suggest that something happened with VM config. Check /etc/ovirt-hosted-engine/hosted-engine.conf All looks correctly (I checked UUID to Domain Storage and etc.) Check /var/run/ovirt-hosted-engine-ha/vm.conf. After encoding base64 I found something like this: <source file="/rhev/data-center/0000-0000-0000-0000-0000/6916940d-b21b-4739-9d6b-4f7ff9e72a9e/images/9a17d31d-30f8-480d-a106-2fa7f0d48c40/a817c10b-0366-4c05-b512-e6beb8308b5d All path is correct only zero after data-center/. I changed it to real path, decode to base64 and try to start VM with hosted-engine --vm-start --vm-conf=PATH_TO_NEW_FILE. Nothing. I see similar error. Could someone help me in this case?

1 0

Hosted engine deployment fail because host version does not match cluster version
by emanuele.breuza＠e4company.com 26 Mar '21

26 Mar '21

Hello, I'm Emanuele. I'm interested in setting up a 3 node HA oVirt cluster on RHEL 8.3. Following the oVirt Gluster Hyperconvergence guide, I managed to setup a GlusterFS storage and then I started the procedure to configure a self-hosted engine via cockpit. Unfortunately, I got stack at the "Prepare VM stage". During this procedure, the automated process paused to let you verify inside the ovirt-engine that your host is OK. Since this is a brand new installation of oVirt, I have only one host configured and it is marked as not-good. The reason reported in the web page is that the host is compatible for v. 4.2, 4.3 and 4.4 but the Cluster is v. 4.5. I tried to execute the upgrade procedure on the host but it was unable to go into maintenance mode. I tried both ovirt-release44 and ovirt-release-master RPM to load the latest repository but without success. Do you have any suggestion about how to fix the self-hosted engine procedure? BTW, if I install my server with the oVirt node ISO the procedure works correctly and my host report also 4.5 into the compatibility version list. Thank you very much for your help!!!

1 0

[ANN] oVirt 4.4.6 First Release Candidate is now available for testing
by Lev Veyde 25 Mar '21

25 Mar '21

oVirt 4.4.6 First Release Candidate is now available for testing The oVirt Project is pleased to announce the availability of oVirt 4.4.6 First Release Candidate for testing, as of March 25th, 2021. This update is the sixth in a series of stabilization updates to the 4.4 series. How to prevent hosts entering emergency mode after upgrade from oVirt 4.4.1 Note: Upgrading from 4.4.2 GA or later should not require re-doing these steps, if already performed while upgrading from 4.4.1 to 4.4.2 GA. These are only required to be done once. Due to Bug 1837864 <https://bugzilla.redhat.com/show_bug.cgi?id=1837864> - Host enter emergency mode after upgrading to latest build If you have your root file system on a multipath device on your hosts you should be aware that after upgrading from 4.4.1 to 4.4.6 you may get your host entering emergency mode. In order to prevent this be sure to upgrade oVirt Engine first, then on your hosts: 1. Remove the current lvm filter while still on 4.4.1, or in emergency mode (if rebooted). 2. Reboot. 3. Upgrade to 4.4.6 (redeploy in case of already being on 4.4.6). 4. Run vdsm-tool config-lvm-filter to confirm there is a new filter in place. 5. Only if not using oVirt Node: - run "dracut --force --add multipath” to rebuild initramfs with the correct filter configuration 6. Reboot. Documentation - If you want to try oVirt as quickly as possible, follow the instructions on the Download <https://ovirt.org/download/> page. - For complete installation, administration, and usage instructions, see the oVirt Documentation <https://ovirt.org/documentation/>. - For upgrading from a previous version, see the oVirt Upgrade Guide <https://ovirt.org/documentation/upgrade_guide/>. - For a general overview of oVirt, see About oVirt <https://ovirt.org/community/about.html>. Important notes before you try it Please note this is a pre-release build. The oVirt Project makes no guarantees as to its suitability or usefulness. This pre-release must not be used in production. Installation instructions For installation instructions and additional information please refer to: https://ovirt.org/documentation/ This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 8.3 or newer * CentOS Linux (or similar) 8.3 or newer This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 8.3 or newer * CentOS Linux (or similar) 8.3 or newer * oVirt Node 4.4 based on CentOS Linux 8.3 (available for x86_64 only) See the release notes [1] for installation instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available for CentOS Linux 8 - oVirt Node NG is already available for CentOS Linux 8 - We found a few issues while testing on CentOS Stream so we are still basing oVirt 4.4.6 Node and Appliance on CentOS Linux. Additional Resources: * Read more about the oVirt 4.4.6 release highlights: http://www.ovirt.org/release/4.4.6/ * Get more oVirt project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.4.6/ [2] http://resources.ovirt.org/pub/ovirt-4.4-pre/iso/ -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

Ovirt Node (iso) - OK to enable Centos-BaseOS/Centos-Appstream repos ?
by morgan cox 25 Mar '21

25 Mar '21

Hi. I have installed Ovirt nodes via the ovirt-node iso (centos8 based) - on a fresh install the standard CentOS repos are disabled (the ovirt 4-4 repo is enabled) As part of our company hardening we need to install a few packages from the Centos repos. Can I enable the CentOS-Linux-AppStream.repo + CentOS-Linux-BaseOS.repo repos or will this cause issues when we update the node ? Thanks

3 4