- Users - oVirt List Archives

Re: HostedEngine VM Paused after power failure
by Ian Easter 09 Feb '21

09 Feb '21

Robert, I understand the sentiment of the difficulty here. The recovery feels brutal but the monolithic nature and the dense ecosystem is understandable for the purpose it serves. I am able to mount the raw disk image for the HostedEngine VM cleanly without any errors and it seems to check out, so I don't believe there is any corruption. Everything looks to operate as expected and then it just seems to snag somewhere through the startup. I suppose I'm just trying to trace down the hiccup to clear it out of the way and let the VM boot up. My knowledge is a bit limited digging in and troubleshooting the components here. Additional snippet: MainThread::INFO::2021-02-09 21:00:07,357::hosted_engine::863::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) stderr: Command VM.getStats with args {'vmID': '74b3c839-c89c-4857-ada0-95715672348a'} failed: (code=1, message=Virtual machine does not exist: {'vmId': '74b3c839-c89c-4857-ada0-95715672348a'}) MainThread::INFO::2021-02-09 21:00:07,357::hosted_engine::875::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) Engine VM started on localhost MainThread::INFO::2021-02-09 21:00:07,389::brokerlink::73::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) Success, was notification of state_transition (EngineStart-EngineStarting) sent? ignored MainThread::INFO::2021-02-09 21:00:07,406::hosted_engine::517::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) Current state EngineStarting (score: 3400) MainThread::INFO::2021-02-09 21:00:17,427::states::740::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) Another host already took over.. *Thank you,* *Ian Easter* On Tue, Feb 9, 2021 at 6:31 PM Robert Tongue <phunyguy(a)neverserio.us> wrote: > I've seen this happen with the VM disk itself becoming corrupt. If you > try to read the contents of the file, and it gives you "Input/Output > Error", then it is not good news. I've been testing oVirt recently, and > these issues alone are preventing me from using it full time. I cannot > help further, unfortunately, as I have no idea how to fix it. So best I > can say is, hopefully someone else chimes in and helps both of us. > > -phunyguy > ------------------------------ > *From:* ieaster(a)telvue.com <ieaster(a)telvue.com> > *Sent:* Tuesday, February 9, 2021 6:25 PM > *To:* users(a)ovirt.org <users(a)ovirt.org> > *Subject:* [ovirt-users] Re: HostedEngine VM Paused after power failure > > Attempting to resume or start the VM doesn't yield any results. > > Here is the status of the VM: > Host ID : 1 > Host timestamp : 115601 > Score : 3400 > Engine status : {"vm": "up", "health": "bad", > "detail": "Paused", "reason": "bad vm status"} > Hostname : > Local maintenance : False > stopped : False > crc32 : 68efbf40 > conf_on_shared_storage : True > local_conf_timestamp : 115601 > Status up-to-date : True > Extra metadata (valid at timestamp): > metadata_parse_version=1 > metadata_feature_version=1 > timestamp=115601 (Tue Feb 9 18:25:48 2021) > host-id=1 > score=3400 > vm_conf_refresh_time=115601 (Tue Feb 9 18:25:48 2021) > conf_on_shared_storage=True > maintenance=False > state=EngineStarting > stopped=False > > > Here is a chunk in agent.log that is a bit perplexing. I'm not too sure > what it means that the VM doesn't exist. Storage is correctly mounted, > everything looks fully operational. I can see the HostedEngine disk > available to the Host. > > MainThread::INFO::2021-02-09 > 18:08:13,843::hosted_engine::517::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) > Current state EngineDown (score: 3400) > MainThread::INFO::2021-02-09 > 18:08:23,864::states::467::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) > Engine down and local host has best score (3400), attempting to start > engine VM > MainThread::INFO::2021-02-09 > 18:08:23,894::brokerlink::73::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) > Success, was notification of state_transition (EngineDown-EngineStart) > sent? ignored > MainThread::INFO::2021-02-09 > 18:08:23,983::hosted_engine::517::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) > Current state EngineStart (score: 3400) > MainThread::INFO::2021-02-09 > 18:08:24,000::hosted_engine::895::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_clean_vdsm_state) > Ensuring VDSM state is clear for engine VM > MainThread::INFO::2021-02-09 > 18:08:24,005::hosted_engine::907::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_clean_vdsm_state) > Vdsm state for VM clean > MainThread::INFO::2021-02-09 > 18:08:24,005::hosted_engine::853::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) > Starting vm using `/usr/sbin/hosted-engine --vm-start` > MainThread::INFO::2021-02-09 > 18:08:24,519::hosted_engine::862::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) > stdout: VM in WaitForLaunch > > MainThread::INFO::2021-02-09 > 18:08:24,519::hosted_engine::863::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) > stderr: Command VM.getStats with args {'vmID': > '74b3c839-c89c-4857-ada0-95715672348a'} failed: > (code=1, message=Virtual machine does not exist: {'vmId': > '74b3c839-c89c-4857-ada0-95715672348a'}) > > MainThread::INFO::2021-02-09 > 18:08:24,519::hosted_engine::875::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_start_engine_vm) > Engine VM started on localhost > MainThread::INFO::2021-02-09 > 18:08:24,552::brokerlink::73::ovirt_hosted_engine_ha.lib.brokerlink.BrokerLink::(notify) > Success, was notification of state_transition (EngineStart-EngineStarting) > sent? ignored > MainThread::INFO::2021-02-09 > 18:08:24,565::hosted_engine::517::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_monitoring_loop) > Current state EngineStarting (score: 3400) > MainThread::INFO::2021-02-09 > 18:08:34,585::states::736::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(consume) > VM is powering up.. > MainThread::INFO::2021-02-09 > 18:08:34,590::state_decorators::99::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(check) > Timeout set to Tue Feb 9 18:18:34 2021 while transitioning <class > 'ovirt_hosted_engine_ha.agent.states.EngineStarting'> -> <class > 'ovirt_hosted_engine_ha.agent.states.EngineStarting'> > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/privacy-policy.html > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/users@ovirt.org/message/UDKODQL5A4NNI… >

2 1

Single Node Hyperconverged - Failing Gluster Setup
by jhamiltonactually＠gmail.com 09 Feb '21

09 Feb '21

Ovirt newbie here - using v 4.4.4 Have been trying for days to get this installed on my HP DL380p G6. I have 2 disk 170GB Raid 0 for OS and 6 x 330GB disk Raid 5 for Gluster. DNS all set up (that took some working out), but I just can't fathom out whats (not) happening here. Block size is returned as 512. I've had some help on Reddit where I've been told that Ovirt is seeing my single local disk ass an multipath device, which it is not/??! I think I removed the flag, but it still fails here. So, Gluster install fails quite early through, though it carries on creating all the volumes (with default settings) but then gives me the 'Deployment Failed' message :( Here is where it fails.... Any help gratefully received! TASK [fail] ******************************************************************** task path: /usr/share/cockpit/ovirt-dashboard/ansible/hc_wizard.yml:62 skipping: [ovirt-gluster.whichelo.com] => (item=[{'cmd': 'blockdev --getss /dev/sdb | grep -Po -q "512" && echo true || echo false\n', 'stdout': 'true', 'stderr': '', 'rc': 0, 'start': '2021-02-07 13:21:10.237701', 'end': '2021-02-07 13:21:10.243111', 'delta': '0:00:00.005410', 'changed': True, 'invocation': {'module_args': {'_raw_params': 'blockdev --getss /dev/sdb | grep -Po -q "512" && echo true || echo false\n', '_uses_shell': True, 'warn': True, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['true'], 'stderr_lines': [], 'failed': False, 'item': {'vgname': 'gluster_vg_sdb', 'pvname': '/dev/sdb'}, 'ansible_loop_var': 'item'}, {'cmd': 'blockdev --getss /dev/sdb | grep -Po -q "4096" && echo true || echo false\n', 'stdout': 'false', 'stderr': '', 'rc': 0, 'start': '2021-02-07 13:21:14.760897', 'end': '2021-02-07 13:21:14.766395', 'delta': '0:00:00.005498', 'chang ed': True, 'invocation': {'module_args': {'_raw_params': 'blockdev --getss /dev/sdb | grep -Po -q "4096" && echo true || echo false\n', '_uses_shell': True, 'warn': True, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}, 'stdout_lines': ['false'], 'stderr_lines': [], 'failed': False, 'item': {'vgname': 'gluster_vg_sdb', 'pvname': '/dev/sdb'}, 'ansible_loop_var': 'item'}]) => {"ansible_loop_var": "item", "changed": false, "item": [{"ansible_loop_var": "item", "changed": true, "cmd": "blockdev --getss /dev/sdb | grep -Po -q \"512\" && echo true || echo false\n", "delta": "0:00:00.005410", "end": "2021-02-07 13:21:10.243111", "failed": false, "invocation": {"module_args": {"_raw_params": "blockdev --getss /dev/sdb | grep -Po -q \"512\" && echo true || echo false\n", "_uses_shell": true, "argv": null, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, " stdin_add_newline": true, "strip_empty_ends": true, "warn": true}}, "item": {"pvname": "/dev/sdb", "vgname": "gluster_vg_sdb"}, "rc": 0, "start": "2021-02-07 13:21:10.237701", "stderr": "", "stderr_lines": [], "stdout": "true", "stdout_lines": ["true"]}, {"ansible_loop_var": "item", "changed": true, "cm d": "blockdev --getss /dev/sdb | grep -Po -q \"4096\" && echo true || echo false\n", "delta": "0:00:00.005498", "end": "2021-02-07 13:21:14.766395", "failed": false, "invocation": {"module_args": {"_raw_params": "blockdev --getss /dev/sdb | grep -Po -q \"4096\" && echo true || echo false\n", "_uses_shell": true, "argv": null, "chdir": null, "creates": null, "executable": null, "removes": null, "stdin": null, "stdin_add_newline": true, "strip_empty_ends": true, "warn": true}}, "item": {"pvname": "/dev/sdb", "vgname": "gluster_vg_sdb"}, "rc": 0, "start": "2021-02-07 13:21:14.760897", "stderr": "", "stderr_lines": [], "stdout": "false", "stdout_lines": ["false"]}], "skip_reason": "Conditional result was False"} hc_wizard.yml excerpt: - name: Check if block device is 4KN shell: > blockdev --getss {{ item.pvname }} | grep -Po -q "4096" && echo true || echo false register: is4KN with_items: "{{ gluster_infra_volume_groups }}" - fail: ################ THIS IS LINE 62 ##################################### msg: "Mix of 4K and 512 Block devices are not allowed" with_nested: - "{{ is512.results }}" - "{{ is4KN.results }}" when: item[0].stdout|bool and item[1].stdout|bool # logical block size of 512 bytes. To disable the check set # gluster_features_512B_check to false. DELETE the below task once # OVirt limitation is fixed - name: Check if disks have logical block size of 512B command: blockdev --getss {{ item.pvname }} register: logical_blk_size when: gluster_infra_volume_groups is defined and item.pvname is not search("/dev/mapper") and gluster_features_512B_check|default(false) Can anyone help?

3 5

HostedEngine VM Paused after power failure
by Ian Easter 09 Feb '21

09 Feb '21

Hello Users, We have an oVirt (4.4) environment that had 2 hosts in the cluster. We suffered from a power failure that caused the servers to be offline for some time. Once restored, one of the hosts from the cluster lost its OS raid and is not accessible. The other server has the HostedEngine vm on it but in a paused state. I have tried to manually start the vm with the hosted-engine CLI tool but it indicates that HostedEngine is running on another host. Is there any manual intervention I can accomplish here to start the HostedEngine on the second, active host server? *Thank you,* *Ian Easter* *DevOps Engineer* *TelVue Support* https://www.telvue.com/support/

4 3

Re: Newer kernel for oVirt Node NG
by Shantur Rathore 09 Feb '21

09 Feb '21

Thanks Matthew, Are you sure about this? I thought oVirt Node NG images are immutable On Tue, Feb 9, 2021 at 3:42 PM Matthew.Stier(a)fujitsu.com < Matthew.Stier(a)fujitsu.com> wrote: > Elrepo.org > > > > *From:* Shantur Rathore <rathore4u(a)gmail.com> > *Sent:* Tuesday, February 9, 2021 9:28 AM > *To:* users <users(a)ovirt.org> > *Subject:* [ovirt-users] Newer kernel for oVirt Node NG > > > > Hi oVirt Users, > > > > I am trying to test some vfio related stuff on oVirt Node NG 4.4.4 based > host. > > What would be the easiest way to have a 5.x kernel on this node? > > I don't mind compiling if it needs to. > > > > Cheers, > > Shantur >

3 3

Random Crash
by francesco＠shellrent.com 09 Feb '21

09 Feb '21

Hi all, I'm experiencing random reboot on several oVirt nodes (CentOS 7/8, oVirt 4.3/4.4 as well). Sometimes it happens three times in a day, and the more hosts I'm adding to my pool, the more I noticing. The logs are not helpful: it's like a brute poweroff cause there are no entries at all in the messages, vdsm, secure (I looked all over the logs) from the last "normal" entry (user logged in/off, normal vdsm log ecc.) until the first entry of the boot. kdump is enabled and /var/crash is empty. I used to run Xen on the servers of the same provider and I didn't have all of these frequent reboots, that's why I'm not sure it is a hardware related issue. Any advice on what enables for getting more info about this crash? Thank you for your time, Francesco

3 3

Newer kernel for oVirt Node NG
by Shantur Rathore 09 Feb '21

09 Feb '21

Hi oVirt Users, I am trying to test some vfio related stuff on oVirt Node NG 4.4.4 based host. What would be the easiest way to have a 5.x kernel on this node? I don't mind compiling if it needs to. Cheers, Shantur

1 0

after CA cert replacement, httpd is running but getting error, unable to login to the portal
by dhanaraj.ramesh＠yahoo.com 09 Feb '21

09 Feb '21

I followed https://www.ovirt.org/documentation/administration_guide/ and replaced hosted engine SSL with CA approved SSL, but getting below error and not able to login to the portal PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target please help me on this.. running ovirt Version 4.4.4.5-1.el8

1 0

async release for oVirt 4.4.4 is now available
by Lev Veyde 08 Feb '21

08 Feb '21

On February 8th the oVirt project released an async update to the following packages: - ovirt-release44-4.4.4.1 - ovirt-node-ng-installer ISO 4.4.4.1 (4.4.4-2021020810) - imgbased-1.2.16 Fixing the following security issues: - [BZ 1917684 <https://bugzilla.redhat.com/show_bug.cgi?id=1917684>] - CVE-2021-3156 <https://access.redhat.com/security/cve/CVE-2021-3156> sudo: Heap buffer overflow in argument parsing - [RHSA-2021:0150 <https://access.redhat.com/errata/RHSA-2021:0150>] - Multiple vulnerabilities within dnsmasq As well as, fixing the following bugs: - [BZ 1903777 <https://bugzilla.redhat.com/1903777>] - chronyd is disabled after upgrading RHV-H 4.4.2 -> 4.4.3 - [BZ 1909782 <https://bugzilla.redhat.com/1909782>] - /etc/openvswitch permissions broken after upgrade - [BZ 1921553 <https://bugzilla.redhat.com/1921553>] - RHVH upgrade to the latest 4.4.4-1 build will fail due to FileNotFoundError -- Lev Veyde Senior Software Engineer, RHCE | RHCVA | MCITP Red Hat Israel <https://www.redhat.com> lev(a)redhat.com | lveyde(a)redhat.com <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

The he_fqdn proposed for the engine VM resolves on this host - error?
by lejeczek 07 Feb '21

07 Feb '21

hi chaps, a newcomer here. I use cockpit to deploy hosted engine and I get this error/warning message: "The he_fqdn proposed for the engine VM resolves on this host" I should mention that if I remove the IP to which FQDN resolves off that iface(plain eth no vlans) then I get this: [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "The selected network interface is not valid"} All these errors seem bit too cryptic to me. Could you shed bit light on what is oVirt saying exactly and why it's not happy that way? many thanks, L.

4 7

Future of oVirt as RHV is converging with OpenShift
by andrew.prowse＠gmail.com 06 Feb '21

06 Feb '21

What is the future of oVirt with RHV 4.4 being the last RHV release? - see https://access.redhat.com/support/policy/updates/rhev With the release of RHV 4.4 Red Hat continues to invest in and support customers with a production-ready virtualization portfolio. The latest addition to this portfolio is OpenShift with OpenShift Virtualization, which provides continuity and roadmap delivery for the KVM-based virtualization stack currently managed by RHV. Moving forward the RHV management feature set will be converged with OpenShift and OpenShift Virtualization providing customers with requirements for containers and VMs a migration path and a common platform for deploying and managing both.

5 7