- Users - oVirt List Archives

[Help] The engine is not stable on HostedEngine with GlusterFS Herperverged deployment
by mengz.you＠outlook.com 20 Apr '21

20 Apr '21

Hi, I deployed a oVirt (4.3.10) cluster with HostedEngine and GlusterFS volumes (engine, vmstore, data), the glusterfs cluster on node1/node2/node3, and the engine vm can be running on those 3 nodes. Then I added a 4th nodes into cluster. But, when I operates on Eninge Web Portal, it's always reports 503 error, then I checked `hsoted-engine --vm-status`, see below: ``` [root@vhost1 ~]# hosted-engine –vm-status –== Host vhost1.yhmk.lan (id: 1) status ==– conf_on_shared_storage : True Status up-to-date : True Hostname : vhost1.<span style=”background-color: rgb(255, 255, 255); color: rgb(51, 51, 51);”>alatest</span>.lan Host ID : 1 Engine status : {“reason”: “bad vm status”, “health”: “bad”, “vm”: “down_unexpected”, “detail”: “Down”} Score : 0 stopped : False Local maintenance : False crc32 : 1f25baff local_conf_timestamp : 1253650 Host timestamp : 1253649 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=1253649 (Thu Apr 8 08:05:48 2021) host-id=1 score=0 vm_conf_refresh_time=1253650 (Thu Apr 8 08:05:48 2021) conf_on_shared_storage=True maintenance=False state=EngineUnexpectedlyDown stopped=False timeout=Thu Jan 15 20:23:29 1970 –== Host vhost2.yhmk.lan (id: 2) status ==– conf_on_shared_storage : True Status up-to-date : True Hostname : vhost2.<span style=”background-color: rgb(255, 255, 255); color: rgb(51, 51, 51);”>alatest</span>.lan Host ID : 2 Engine status : {“reason”: “vm not running on this host”, “health”: “bad”, “vm”: “down_unexpected”, “detail”: “unknown”} Score : 3400 stopped : False Local maintenance : False crc32 : 539fc30c local_conf_timestamp : 1253343 Host timestamp : 1253343 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=1253343 (Thu Apr 8 08:05:46 2021) host-id=2 score=3400 vm_conf_refresh_time=1253343 (Thu Apr 8 08:05:46 2021) conf_on_shared_storage=True maintenance=False state=EngineDown stopped=False –== Host vhost3.yhmk.lan (id: 3) status ==– conf_on_shared_storage : True Status up-to-date : True Hostname : vhost3.alatest.lan Host ID : 3 Engine status : {“reason”: “bad vm status”, “health”: “bad”, “vm”: “up”, “detail”: “Powering up”} Score : 3400 stopped : False Local maintenance : False crc32 : 4072e0b8 local_conf_timestamp : 1252345 Host timestamp : 1252345 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=1252345 (Thu Apr 8 08:05:42 2021) host-id=3 score=3400 vm_conf_refresh_time=1252345 (Thu Apr 8 08:05:42 2021) conf_on_shared_storage=True maintenance=False state=EngineStarting stopped=False ``` Then, wait a moment, can access web portal again, and check the hosts status, alway reports one or more hosts with label `unavaiable as HA score`, but it will dispear later. And I found, sometimes the engine vm will migrated to another nodes whill this problem occur. So, seems the HostedEngine is not stable, always occur this problem, could you please help me with this ? Thanks!

3 8

Re: ovirt-hosted-engine-cleanup
by Thomas Hoberg 20 Apr '21

20 Apr '21

ovirt-hosted-engine-cleanup will only operate on the host you run it on. In a cluster that might have side-effects, but as a rule it will try to undo all configuration settings that had a Linux host become an HCI member or just a host under oVirt management. While the GUI will try to do the same with Ansible only, the script seems to do a little more and be more thorough. My guess is that it's somewhat of a leftover from the older 'all scripts' legacy of oVirt, before they re-engineered much of it to fit Ansible. It's not perfect, however. I have come across things that it would not undo, and that were extremely difficult to find (a really clean re-install would then help). There was an issue around keys and certificates for KVM that I only remember as a nightmare and more recently I had an issue with it's ansible scripts inserting an LVM device filter specific for a UID that the HCI setup wizard had created, which then precluded it ever working again after running the *cleanup scripts. All I got was "device excluded by a filter" and only much later I found the added line in /etc/lvm.conf not undone, which caused this nail-biter. But generally that message "Is hosted engine setup finished?" mostly indicates that the four major daemons for HCI still have issues. It all starts with glusterd: If you just added a host in HCI with storage, that would like to be properly healed. AFAIK that doesn't actually preclude already using the machine to host VMs, but in the interest of keeping your head clear, I'd suggest concentrating on getting the storage all clean, with gluster happily showing connectivity all around (gluster volume status all etc.) and 'gluster volume heal <volume> [info]' giving good results. Then you need to check on the services ovirt-ha-broker and ovirt-ha-agent as well as vdsmd via 'servicectl status <servicename>' to see what they are complaining about. I typically restart them in this order to iron out the bumps. As long as you give them a bit of time, restarting them periodically doesn't seem to do any harm, but helps them recover from hangs. And then you simply need patience: Things don't happen immediately in oVirt, because few commands ever intervene into anything directly. Instead they set flags here and there which then a myriad of interlocking state machine gears will pick up and respond to, to do things in the proper order and by avoiding the race conditions manual intervention can so easily cause. Even the fastest systems won't give results in seconds. So have a coffee, better yet, some herbal tea, and then look again. There is a good chance oVirt will eventually sort it out (but restarting those daemons really can help things along, as will looking at those error messages and their potential cause).

1 0

Introduction & general question about oVirt
by Nicolas Kovacs 20 Apr '21

20 Apr '21

Hi, I'm a 53-year old Austrian living in Montpezat, a small village in South France. I'm an IT professional with a focus on Linux and free software, and I've been a Linux user since Slackware 7.1. I'm doing web & mail hosting for myself and several small structures like our local school and a handful of local companies. Up until recently these hostings have happened on "bare metal" root servers using CentOS 7. One main server is hosting most of the stuff: WordPress sites, one OwnCloud instance, Dolibarr management software, GEPI learning platform, Postfix/Dovecot mail server, Roundcube webmail, etc. This setup has become increasingly problematic to manage, since applications have more and more specific requirements, like different versions of PHP and corresponding modules. So I decided to split everything up nicely into a series of virtual machines, each one with a nicely tailored setup. I have a couple of sandbox servers, one public and one local, running Oracle Linux 7 (a RHEL clone like CentOS). I played around with it, and KVM-based virtualization already works quite nicely. While looking for documentation, I stumbled over oVirt, which I didn't even know existed until last week. Before I dive head first into it, I'd be curious to know a few general things. 1. Would it be overkill for a small structure like mine? 2. Will I be able to do HA on a series of modest KVM-capable root servers even if they are located in different datacenters across different countries? 3. One problem I couldn't resolve using a bone-headed keep-it-simple KVM setup is backup. For my bare-metal servers I've been using incremental backups using Rsnapshot for years. Here's a blog article I wrote on the subject: https://blog.microlinux.fr/rsnapshot-centos-7/ Unfortunately I can't use this approach with huge QCOW images, at least not without jumping through burning loops. Is there an easy way to perform remote incremental backups with oVirt? BTW, I took a peek at Proxmox and Ceph, but I admit I'm a die-hard RHEL-clone user. Cheers from the sunny South of France, Niki -- Microlinux - Solutions informatiques durables 7, place de l'église - 30730 Montpezat Site : https://www.microlinux.fr Blog : https://blog.microlinux.fr Mail : info(a)microlinux.fr Tél. : 04 66 63 10 32 Mob. : 06 51 80 12 12

4 3

Upgrading single host ovirt "clusters" to v4.5 via command line?
by Gilboa Davara 19 Apr '21

19 Apr '21

Hello all, As the title suggests I have a couple of single-host hyper converged setups, mostly used for testing. As I cannot use the GUI cluster upgrade method on a single host (it requires me to reboot the host, which will require me to shutdown the hosted engine), is there any method to upgrade the cluster / data-center via command line in maintenance=global mode? Beyond that, can I use the same method on my production setups? (multi-node, gluster based setups). Thanks, Gilboa

1 0

Re: FreeBSD 13 and virtio
by Nir Soffer 19 Apr '21

19 Apr '21

On Mon, Apr 19, 2021 at 5:11 PM Nur Imam Febrianto <nur_imam(a)outlook.com> wrote: > I’m having an issue, if I’m using q35 and using virtio disk + virtio > network adapter, the guest OS wont detect any disk. I440fx work fine. This > issue happened on FBSD 12.2 too. On FBSD 13 release note, it was said q35 > should working okay now, a lot of people using pc-q35-4.2 machine type > confirm it works now on vanilla KVM. But seems it still doesn’t work on > oVirt. > It will help if you can file a bug about this, and attach a vm xml that works outside of ovirt (e.g using virt-manger, or virsh), so we can compare to our vm xml. Nir > > > *From: *Sandro Bonazzola <sbonazzo(a)redhat.com> > *Sent: *19 April 2021 19:35 > *To: *Nur Imam Febrianto <nur_imam(a)outlook.com> > *Cc: *oVirt Users <users(a)ovirt.org> > *Subject: *Re: [ovirt-users] FreeBSD 13 and virtio > > > > Hi, can you please provide some more details on the issues you're seeing? > > > > Il giorno sab 17 apr 2021 alle ore 06:35 Nur Imam Febrianto < > nur_imam(a)outlook.com> ha scritto: > > Hi, > > > > Any body already try FBSD 13 sing q35 + virtio disk + virtio net ? > > I cant get it working, it should be fine in FBSD 13, lastly on FBSD 12 it > was the q35 spec issue with the kernel. Lot of people now it works nicely > on FBSD 13, but I cant get it working on oVirt. > > > > Thanks before. > > > > Regards, > > Nur Imam Febrianto > > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovir…> > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovir…> > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/FCPJBK3AV4BSB… > <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ov…> > > > > > -- > > *Sandro Bonazzola* > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA > <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redh…> > > sbonazzo(a)redhat.com > > > <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redh…> > > > *Red Hat respects your work life balance. Therefore there is no need to > answer this email out of your office hours. > <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmojo.red…>* > > > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/ADIC2ERITKAMC… >

1 0

How to Upgrade Node with Local Storage ?
by Nur Imam Febrianto 19 Apr '21

19 Apr '21

Hi, How we can upgrade oVirt Node that using Local Storage ? Seems I cant find any good documentation about this. Planning to upgrade one 4.4.4 node with local storage to 4.4.5. Thanks before. Regards, Nur Imam Febrianto

1 0

Re: FreeBSD 13 and virtio
by Sandro Bonazzola 19 Apr '21

19 Apr '21

Hi, can you please provide some more details on the issues you're seeing? Il giorno sab 17 apr 2021 alle ore 06:35 Nur Imam Febrianto < nur_imam(a)outlook.com> ha scritto: > Hi, > > > > Any body already try FBSD 13 sing q35 + virtio disk + virtio net ? > > I cant get it working, it should be fine in FBSD 13, lastly on FBSD 12 it > was the q35 spec issue with the kernel. Lot of people now it works nicely > on FBSD 13, but I cant get it working on oVirt. > > > > Thanks before. > > > > Regards, > > Nur Imam Febrianto > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/FCPJBK3AV4BSB… > -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

2 1

Why and when does oVirt make a compute host part of the HCI Gluster?
by Thomas Hoberg 19 Apr '21

19 Apr '21

My understanding is that in a HCI environment, the storage nodes should be rather static, but that the pure compute nodes, can be much more dynamic or opportunistic: actually those should/could even be switched off and restarted as part of oVirt's resource optimization. The 'pure compute' nodes fall into two major categories, a) those who can run the management engine, b) those who can't. What I don't quite understand is why both types seem to be made 'gluster peers', when they don't contribute storage bricks: shouldn't they just be able to mount Gluster volumes? The reason for my concern is that I actually want to manage these computing hosts with much more freedom. I may have them join and take on workloads, or not and I may want to shut them down. To my irritation I sometimes even see these 'missing' hosts being considered for quorum decisions or being listed e.g. when I do a 'gluster volume engine status'. I find hosts there, that definitely are not contributing bricks to 'engine' (or any other volume). Then I'm not even sure I have consistent behavior when I remove hosts from oVirt: I'd swear that quite a few remain as Gluster peers, even if they are completely invisible from the oVirt GUI (while hosted-engine --vm-status will still list them). So what's the theory here and would it be a bug if removed hosts remain gluster members?

2 3

Expand gluster volumes
by David White 19 Apr '21

19 Apr '21

Is it possible to expand an existing gluster volume? I have a hyperconverged environment, and have enough space right now, but I'm going to have to significantly over-provision my environment. The vast majority of our customers are using a small fraction of the amount of space that they are technically allocated, but we still need to make that space available to them. As a result, I'd like to plan ahead, and go ahead and have a plan to add storage later down the road. I'd like to plan to shut down each server in the cluster (individually, not at once), add storage, and then bring them back online. Once all 3 are back online with the additional storage, I'd like to expand the gluster volume to use the additional space. Is that possible? How? Sent with ProtonMail Secure Email.

3 3

After upgrade to 4.4.5: cockpit login error
by ovirt＠r5d.de 18 Apr '21

18 Apr '21

Hello, 1. Installation ovirt node iso 4.4.4 2. dnf upgrade -> 4.4.5 -> error login cockpit: Message: pr 18 13:02:53 h2 systemd[1]: cockpit-wsinstance-https-factory(a)2-7789-995.service: Succeeded. Apr 18 13:02:53 h2 cockpit-ws[7796]: cockpit-ws: Failed to open certificate file /run/cockpit/tls/e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855: No such file or directory Apr 18 13:02:55 h2 cockpit-ws[7796]: cockpit-session: open(/var/log/btmp) failed: Permission denied What is todo? Thx.

1 0