- Users - oVirt List Archives

Hosted Engine I/O scheduler
by Strahil 18 Mar '19

18 Mar '19

Dear All, I have just noticed that my Hosted Engine has a strange I/O scheduler: Last login: Sun Mar 17 18:14:26 2019 from 192.168.1.43 [root@engine ~]# cat /sys/block/vda/queue/scheduler [mq-deadline] kyber none [root@engine ~]# Based on my experience anything than noop/none is useless and performance degrading for a VM. Is there any reason that we have this scheduler ? It is quite pointless to process (and delay) the I/O in the VM and then process (and again delay) on Host Level . If there is no reason to keep the deadline, I will open a bug about it. Best Regards, Strahil Nikolov

3 2

oVirt 4.3.2 RC - ISCSI LUN
by a.e.pool＠iway.na 18 Mar '19

18 Mar '19

I have deployed the latest oVirt 4.3.2 host and the hosted engine without problems, until you have to select the storage. I configured a brand new FReenAS 11 with ISCSI on three 1TB HDDs, the oVirt engine sees the iscsi target but when trying to select the LUN it shows that the LUN is already in use. Anyone come across this issue?

1 0

Ovirt 4.3.1 problem with HA agent
by Strahil 18 Mar '19

18 Mar '19

Hi guys, After updating to 4.3.1 I had an issue where the ovirt-ha-broker was complaining that it couldn't ping the gateway. As I have seen that before - I stopped ovirt-ha-agent, ovirt-ha-broker, vdsmd, supervdsmd and sanlock on the nodes and reinitialized the lockspace. I gues s I didn't do it properly as now I receive: ovirt-ha-agent ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine.config.vm ERROR Failed extracting VM OVF from the OVF_STORE volume, falling back to initial vm.conf Any hints how to fix this ? Of course a redeploy is possible, but I prefer to recover from that. Best Regards, Strahil Nikolov

4 21

Bandwidth problem
by Leo David 18 Mar '19

18 Mar '19

Hello Everyone, I have 10Gb connections setup for all the hosts in the cluster, for both management/vm and gluster traffic ( separate network cards ) The problem is that i just cannot pass 1Gb/s traffic between vms ( even between vms running on the same hosts ! - which makes the things more weird... ). Traffic measured by using iperf tool. Is there a way I can check waht could be the problem ? Network card type, vm drivers, any suggestion ? I just do not know where to look for a possible cause. Thank you very much ! Leo -- Best regards, Leo David

2 1

Re: self-hosted ovirt-engine down
by Strahil 18 Mar '19

18 Mar '19

In order to be easier to read, you can rotate the log before starting the engine.Should be something like: hosted-engine --set-maintenance --mode=global && logrotate -f /etc/vdsm/logrotate/vdsm && hosted-engine -- vm-start Best Regards, Strahil NikolovOn Mar 16, 2019 10:52, Simone Tiraboschi <stirabos(a)redhat.com> wrote: > > > > On Sat, Mar 16, 2019 at 7:44 AM <siovelrm(a)gmail.com> wrote: >> >> Hi, I have a big problem with ovirt. I use version 4.2.7 with self-hosted. The problem is that when I try to raise the vm of the ovirt-engine with the command: hosted-engine --vm-start, it appears in the output >> "VM exists and is down, cleaning up and restarting" >> when running: hosted-engine --vm-status appears >> --== Host 1 status ==-- >> >> conf_on_shared_storage : True >> Status up-to-date : True >> Hostname : node1.softel.cu >> Host ID : 1 >> Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"} >> Score : 0 >> stopped : False >> Local maintenance : False >> crc32 : 02c3b5a4 >> local_conf_timestamp : 49529 >> Host timestamp : 49529 >> Extra metadata (valid at timestamp): >> metadata_parse_version=1 >> metadata_feature_version=1 >> timestamp=49529 (Sat Mar 16 02:39:10 2019) >> host-id=1 >> score=0 >> vm_conf_refresh_time=49529 (Sat Mar 16 02:39:11 2019) >> conf_on_shared_storage=True >> maintenance=False >> state=EngineUnexpectedlyDown >> stopped=False >> timeout=Thu Jan 1 08:49:39 1970 >> >> in /var/log/messages >> Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing network: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 >> Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing net user: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 >> Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing network: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 >> Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing net user: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 >> Mar 16 02:35:34 node1 vdsm[26151]: WARN File: /var/lib/libvirt/qemu/channels/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241.org.qemu.guest_agent.0 already removed >> >> Please help!! > > > I'd suggest to check in /var/log/vdsm/vdsm.log why it failed to start. > > >> >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/OTXJFCVCAHMYZ…

2 1

Error creating a storage domain (On Cisco UCS Only)
by nico.kruger＠darkmatter.ae 18 Mar '19

18 Mar '19

Hi Guys, I am trying to install a new cluster... I currently have a 9 node and two 6 node oVirt Clusters... (these were installed on 4.1 and upgraded to 4.2) So i want to build a new cluster, which is working fine on this HP notebook i use for testing (using single node gluster deployment) But when i try install this on my production servers which are Cisco UCS servers i keep getting this error: [ ERROR ] Error: Fault reason is "Operation Failed". Fault detail is "[Error creating a storage domain]". HTTP response code is 400. [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Error creating a storage domain]\". HTTP response code is 400."} This happens during storage creation after the hosted-engine is built and after gluster has been deployed (error happens for both single and 3 replica deployments) I just cant see how an install on one type of server is successful but not on the UCS servers (which i am running my other ovirt clusters on) BTW i dont think the issue is related to Gluster Storage Create as i tried using NFS and Local storage and get the same error (on UCS server only) I am using the ovirt-node-ng-installer-4.2.0-2019011406.el7.iso install ISO Below is a tail from ovirt-hosted-engine-setup-ansible-create_storage_domain Log file 2019-01-27 11:09:49,754+0400 INFO ansible ok {'status': 'OK', 'ansible_task': u'Fetch Datacenter name', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'task'} 2019-01-27 11:09:49,754+0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f9f9b7e2d50> kwargs 2019-01-27 11:09:50,478+0400 INFO ansible task start {'status': 'OK', 'ansible_task': u'Add NFS storage domain', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'task'} 2019-01-27 11:09:50,479+0400 DEBUG ansible on_any args TASK: Add NFS storage domain kwargs is_conditional:False 2019-01-27 11:09:51,151+0400 DEBUG var changed: host "localhost" var "otopi_storage_domain_details_nfs" type "<type 'dict'>" value: "{ "changed": false, "skip_reason": "Conditional result was False", "skipped": true }" 2019-01-27 11:09:51,151+0400 INFO ansible skipped {'status': 'SKIPPED', 'ansible_task': u'Add NFS storage domain', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'task'} 2019-01-27 11:09:51,151+0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f9f9b7e2610> kwargs 2019-01-27 11:09:51,820+0400 INFO ansible task start {'status': 'OK', 'ansible_task': u'Add glusterfs storage domain', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'task'} 2019-01-27 11:09:51,821+0400 DEBUG ansible on_any args TASK: Add glusterfs storage domain kwargs is_conditional:False 2019-01-27 11:10:02,045+0400 DEBUG var changed: host "localhost" var "otopi_storage_domain_details_gluster" type "<type 'dict'>" value: "{ "changed": false, "exception": "Traceback (most recent call last):\n File \"/tmp/ansible_ovirt_storage_domain_payload_Xous24/__main__.py\", line 682, in main\n ret = storage_domains_module.create()\n File \"/tmp/ansible_ovirt_storage_domain_payload_Xous24/ansible_ovirt_storage_domain_payload.zip/ansible/module_utils/ovirt.py\", line 587, in create\n **kwargs\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py\", line 24225, in add\n return self._internal_add(storage_domain, headers, query, wait)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 232, in _internal_add\n return future.wait() if wait else future\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 55, in wait\n return self._code(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 229, in callback\n self._check_fault(response)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 132, in _check_fault\n self._raise_error(response, body)\n File \"/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py\", line 118, in _raise_error\n raise error\nError: Fault reason is \"Operation Failed\". Fault detail is \"[Error creating a storage domain]\". HTTP response code is 400.\n", "failed": true, "msg": "Fault reason is \"Operation Failed\". Fault detail is \"[Error creating a storage domain]\". HTTP response code is 400." }" 2019-01-27 11:10:02,045+0400 DEBUG var changed: host "localhost" var "ansible_play_hosts" type "<type 'list'>" value: "[]" 2019-01-27 11:10:02,045+0400 DEBUG var changed: host "localhost" var "play_hosts" type "<type 'list'>" value: "[]" 2019-01-27 11:10:02,045+0400 DEBUG var changed: host "localhost" var "ansible_play_batch" type "<type 'list'>" value: "[]" 2019-01-27 11:10:02,046+0400 ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u'Add glusterfs storage domain', 'ansible_result': u'type: <type \'dict\'>\nstr: {\'_ansible_parsed\': True, u\'exception\': u\'Traceback (most recent call last):\\n File "/tmp/ansible_ovirt_storage_domain_payload_Xous24/__main__.py", line 682, in main\\n ret = storage_domains_module.create()\\n File "/tmp/ansible_ovirt_storage_domain_payload_Xous24/ansible_ovirt_storage_domain_pay', 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml'} 2019-01-27 11:10:02,046+0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7f9f9b859f50> kwargs ignore_errors:None 2019-01-27 11:10:02,048+0400 INFO ansible stats {'status': 'FAILED', 'ansible_playbook_duration': 39.840701, 'ansible_result': u"type: <type 'dict'>\nstr: {u'localhost': {'unreachable': 0, 'skipped': 2, 'ok': 13, 'changed': 0, 'failures': 1}}", 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_storage_domain.yml', 'ansible_type': 'finish'} 2019-01-27 11:10:02,048+0400 DEBUG ansible on_any args <ansible.executor.stats.AggregateStats object at 0x7f9f9da59a50> kwargs This issue is driving me crazy... any assistance on what i can do would be greatly appreciated Nico

5 8

vdsm should decouple with managed glusterfs services
by levin＠mydream.com.hk 18 Mar '19

18 Mar '19

Hi, I had experience two time of 3-node hyper-converged 4.2.8 ovirt cluster total outage due to vdsm reactivate the unresponsive node, and cause the multiple glusterfs daemon restart. As a result, all VM was paused and some of disk image was corrupted. At the very beginning, one of the ovirt node was overloaded due to high memory and CPU, the hosted-engine have trouble to collect status from vdsm and mark it as unresponsive and it start migrate the workload to healthy node. However, when it start migrate, second ovirt node being unresponsive where vdsm try reactivate the 1st unresponsive node and restart it's glusterd. So the gluster domain was acquiring the quorum and waiting for timeout. If 1st node reactivation is success and every other node can survive the timeout, it will be an idea case. Unfortunately, the second node cannot pick up the VM being migrated due to gluster I/O timeout, so second node at that moment was marked as unresponsive, and so on... vdsm is restarting the glusterd on the second node which cause disaster. All node are racing on gluster volume self-healing, and i can't mark the cluster as maintenance mode as well. What I can do is try to resume the paused VM via virsh and issue shutdown for each domain, also hard shutdown for un-resumable VM. After number of VM shutdown and wait the gluster healing completed, the cluster state back to normal, and I try to start the VM being manually stopped, most of them can be started normally, but number of VM was crashed or un-startable, instantly I found the image files of un-startable VM was owned by root(can't explain why), and can be restarted after chmod. Two of them still cannot start with "bad volume specification" error. One of them can start to boot loader, but the LVM metadata were lost. The impact was huge when vdsm restart the glusterd without human invention.

3 3

Enabled Base and Update repos?
by nico.kruger＠darkmatter.ae 18 Mar '19

18 Mar '19

Hi Guys, Should Centos Base and Update repos be enabled on ovirt hosts or only the dependancies and ovirt repos (the ones from rpm release)

1 0

Libgfapisupport messes disk image ownership
by Hesham Ahmed 18 Mar '19

18 Mar '19

I had reported this here: https://bugzilla.redhat.com/show_bug.cgi?id=1687126 Has anyone else faced this with 4.3.1?

3 3

Re: Ovirt 4.3.1 problem with HA agent
by Strahil 17 Mar '19

17 Mar '19

Hi Simone, I have noticed that my Engine's root disk is 'vda' just in standalone KVM. I have the feeling that was not the case before. Can someone check a default engine and post the output of lsblk ? Thanks in advance. Best Regards, Strahil NikolovOn Mar 15, 2019 12:46, Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: > > > On Fri, Mar 15, 2019 at 8:12 AM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: >> >> Ok, >> >> I have managed to recover again and no issues are detected this time. >> I guess this case is quite rare and nobody has experienced that. > > > >Hi, > >can you please explain how you fixed it? > > I have set again to global maintenance, defined the HostedEngine from the old xml (taken from old vdsm log) , defined the network and powered it off. > Set the OVF update period to 5 min , but it took several hours until the OVF_STORE were updated. Once this happened I restarted the ovirt-ha-agent ovirt-ha-broker on both nodes.Then I powered off the HostedEngine and undefined it from ovirt1. > > then I set the maintenance to 'none' and the VM powered on ovirt1. > In order to test a failure, I removed the global maintenance and powered off the HostedEngine from itself (via ssh). It was brought back to the other node. > > In order to test failure of ovirt2, I set ovirt1 in local maintenance and removed it (mode 'none') and again shutdown the VM via ssh and it started again to ovirt1. > > It seems to be working, as I have later shut down the Engine several times and it managed to start without issues. > > I'm not sure this is related, but I had detected that ovirt2 was out-of-sync of the vdsm-ovirtmgmt network , but it got fixed easily via the UI. > > > > Best Regards, > Strahil Nikolov

1 0

Are people still experiencing issues with GlusterFS on 4.3x?
by Jayme 17 Mar '19

17 Mar '19

I along with others had GlusterFS issues after 4.3 upgrades, the failed to dispatch handler issue with bricks going down intermittently. After some time it seemed to have corrected itself (at least in my enviornment) and I hadn't had any brick problems in a while. I upgraded my three node HCI cluster to 4.3.1 yesterday and again I'm running in to brick issues. They will all be up running fine then all of a sudden a brick will randomly drop and I have to force start the volume to get it back up. Have any of these Gluster issues been addressed in 4.3.2 or any other releases/patches that may be available to help the problem at this time? Thanks!

7 12

Re: Broken dependencies in CentOS7 hosts upgrading to 4.2.8 from 4.2.7
by Strahil 17 Mar '19

17 Mar '19

Here is my repolist: Last login: Sun Mar 17 12:42:53 2019 from 192.168.1.43 [root@ovirt1 ~]# yum repolist Loaded plugins: enabled_repos_upload, fastestmirror, package_upload, product- : id, search-disabled-repos, subscription-manager, vdsmupgrade This system is not registered with an entitlement server. You can use subscription-manager to register. Loading mirror speeds from cached hostfile * base: mirror.wwfx.net * extras: mirror.wwfx.net * ovirt-4.3: ftp.nluug.nl * ovirt-4.3-epel: mirror.t-home.mk * updates: mirror.wwfx.net repo id repo name statusbase/7/x86_64 CentOS-7 - Base 10,019centos-sclo-rh-release/x86_64 CentOS-7 - SCLo rh 8,113extras/7/x86_64 CentOS-7 - Extras 371ovirt-4.3/7 Latest oVirt 4.3 Release 265ovirt-4.3-centos-gluster5/x86_64 CentOS-7 - Gluster 5 112ovirt-4.3-centos-opstools/x86_64 CentOS-7 - OpsTools - release 853ovirt-4.3-centos-ovirt43/x86_64 CentOS-7 - oVirt 4.3 287ovirt-4.3-centos-qemu-ev/x86_64 CentOS-7 - QEMU EV 71ovirt-4.3-epel/x86_64 Extra Packages for Enterprise Linux 7 12,981ovirt-4.3-virtio-win-latest virtio-win builds roughly matching wh 40sac-gluster-ansible/x86_64 Copr repo for gluster-ansible owned b 18updates/7/x86_64 CentOS-7 - Updates 1,163repolist: 34,293 Uploading Enabled Repositories Report Cannot upload enabled repos report, is this client registered? [root@ovirt1 ~]# As you can see my CentOS7 - Updates is enabled. So far I have managed to upgrade 4.2.7 -> 4.2.8 -> 4.3.0 -> 4.3.1 There were some issues , but nothing related to the repos. Best Regards, Strahil NikolovOn Mar 15, 2019 18:57, Roberto Nunin <robnunin(a)gmail.com> wrote: > > Hi > I have some oVirt clusters, in various config. > > One cluster based on CentOS7 hosts, another based on ovirt-node-ng. > While the second was successfully updated from 4.2.7 to 4.2.8, attempts to update hosts of the first one ends with: > > Error: Package: vdsm-4.20.46-1.el7.x86_64 (ovirt-4.2) > Requires: libvirt-daemon-kvm >= 4.5.0-10.el7_6.3 > Installed: libvirt-daemon-kvm-4.5.0-10.el7.x86_64 (@base) > libvirt-daemon-kvm = 4.5.0-10.el7 > > Being a CentOS7 installation and not an ovirt-node-ng, I cannot follow the notice in : > https://lists.ovirt.org/archives/list/users@ovirt.org/message/AJUBXAIGXVD5U… > > The only way to have libvirt-daemon-kvm release 4-5-0.10_6.4 (and not 6.3) is to enable CentOS Updates repo. Looking at host-deploy log looks fine, but It's safe to enable that repo ? There is another, safest method to update these hosts to the latest version of 4.2 ? > > Thanks in advance > > -- > Roberto Nunin > > >

2 1

GPU passthrough
by Darin Schmidt 16 Mar '19

16 Mar '19

If I'm reading through the documents correctly, you can't use consumer grade gpus such as gtx 1080's and assign them to a VM? I'm trying to do something similar to how Linus used untaid to build several gaming machines from one. Any help would be appreciated. Thanks.

2 2

single server, self hosted engine on host, one vm wont start
by abzstrak＠gmail.com 16 Mar '19

16 Mar '19

I'm not an expert with ovirt by any means, I've been running it for about 1.5 years coming from Xen. I have a Vm with 16TiB of storage that will not boot, it has some snapshots that wont remove. The web gui says it finished removing the last snapshot, but I don't think it actually did (took it about 50 hours).... I have no idea what is wrong with the vm that wont boot, it just says it's starting and never actually starts. Eventually I'll get a message that it failed to start. The boot drive of that vm is 16GB. I removed any snapshots from it and was able to clone it to a new disk, make a new vm and attach that disk to the new vm... booted fine. Now, the storage disk is what I really need working, but I do not have enough space to clone it and I cannot unattach it from the flakey vm since it has snapshots... and it seemingly won't remove the snapshots. Right now, I think its actually still maybe attempting to remove the snapshot? in top, I see qemu-img pulling all of one core at about 100%. unfortunately its very very slow because it's only using one core (of 16). This seemed to be a thing, AFAIK, when removing/merging snapshots in that it will only run single threaded... if it takes a while I don't care, I just want to know what's going on. I can't find any logs to look at that tell me that a snapshot removal or merge is ongoing or ever happened. I cannot find any logs that ever stated the progress of any such merge either. Any pointers there? Also, my backup machine died about 2 weeks ago taking its raid 5 with it... of course... I've been busy at work and figured I'd fix or rebuild it in a couple of weeks... so I don't have a good recent backup of my data on this vm, which is why I'm trying to get it to boot up again.

1 0

self-hosted ovirt-engine down
by siovelrm＠gmail.com 16 Mar '19

16 Mar '19

Hi, I have a big problem with ovirt. I use version 4.2.7 with self-hosted. The problem is that when I try to raise the vm of the ovirt-engine with the command: hosted-engine --vm-start, it appears in the output "VM exists and is down, cleaning up and restarting" when running: hosted-engine --vm-status appears --== Host 1 status ==-- conf_on_shared_storage : True Status up-to-date : True Hostname : node1.softel.cu Host ID : 1 Engine status : {"reason": "bad vm status", "health": "bad", "vm": "down_unexpected", "detail": "Down"} Score : 0 stopped : False Local maintenance : False crc32 : 02c3b5a4 local_conf_timestamp : 49529 Host timestamp : 49529 Extra metadata (valid at timestamp): metadata_parse_version=1 metadata_feature_version=1 timestamp=49529 (Sat Mar 16 02:39:10 2019) host-id=1 score=0 vm_conf_refresh_time=49529 (Sat Mar 16 02:39:11 2019) conf_on_shared_storage=True maintenance=False state=EngineUnexpectedlyDown stopped=False timeout=Thu Jan 1 08:49:39 1970 in /var/log/messages Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing network: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing net user: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing network: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 Mar 16 02:35:34 node1 vdsm[26151]: WARN Attempting to remove a non existing net user: ovirtmgmt/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241 Mar 16 02:35:34 node1 vdsm[26151]: WARN File: /var/lib/libvirt/qemu/channels/0c3e1c08-3928-47f1-96a8-c6a8d0dc3241.org.qemu.guest_agent.0 already removed Please help!!

2 1

Self Hosted Engine failed during setup using oVirt Node 4.3
by Jagi Sarcilla 16 Mar '19

16 Mar '19

Hardware Specs Hypervisor: Supermicro A2SDi-16C-HLN4F Intel(R) Atom(TM) CPU C3955 @ 2.10GHz 16cores Samsung NVME 1TB Sandisk SSHD 1TB 256GB RAM 4 x 1G nics Storage: FreeNAS-11.2-RELEASE-U1 - NFS - iSCSI * Issues #1 using iscsi unable to discover the target using cockpit dashboard * Issue #2 using nfs can successfully connect but the host wont come up after shutdown by the installation * Error message: [ INFO ] TASK [oVirt.hosted-engine-setup : Check engine VM health] [ ERROR ] fatal: [localhost]: FAILED! => {"attempts": 120, "changed": true, "cmd": ["hosted-engine", "--vm-status", "--json"], "delta": "0:00:00.358141", "end": "2019-03-14 06:33:35.499429", "rc": 0, "start": "2019-03-14 06:33:35.141288", "stderr": "", "stderr_lines": [], "stdout": "{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=43479 (Thu Mar 14 06:33:34 2019)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=43480 (Thu Mar 14 06:33:34 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUnexpectedlyDown\\nstopped=False\\ntimeout=Thu Jan 1 07:04:46 1970\\n\", \"hostname\": \"dreamlevel-1.logistics.corp\", \"host-id\": 1, \"engine-status\": {\"reason\": \"bad vm status\", \"health\": \"bad\", \"vm\": \"down_unexpected\", \"detail\": \"Down\"}, \"score\": 0, \"stopped\": false, \"maintenance\": false, \"crc32\": \"de300a81\", \"local_conf_timestamp\": 43480, \"host-ts\": 43479}, \"global_maintenance\": false}", "stdout_lines": ["{\"1\": {\"conf_on_shared_storage\": true, \"live-data\": true, \"extra\": \"metadata_parse_version=1\\nmetadata_feature_version=1\\ntimestamp=43479 (Thu Mar 14 06:33:34 2019)\\nhost-id=1\\nscore=0\\nvm_conf_refresh_time=43480 (Thu Mar 14 06:33:34 2019)\\nconf_on_shared_storage=True\\nmaintenance=False\\nstate=EngineUnexpectedlyDown\\nstopped=False\\ntimeout=Thu Jan 1 07:04:46 1970\\n\", \"hostname\": \"dreamlevel-1.logistics.corp\", \"host-id\": 1, \"engine-status\": {\"reason\": \"bad vm status\", \"health\": \"bad\", \"vm\": \"down_unexpected\", \"detail\": \"Down\"}, \"score\": 0, \"stopped\": false, \"maintenance\": false, \"crc32\": \"de300a81\", \"local_conf_timestamp\": 43480, \"host-ts\": 43479}, \"global_maintenance\": false}"]} [ INFO ] TASK [oVirt.hosted-engine-setup : Check VM status at virt level] [ INFO ] TASK [oVirt.hosted-engine-setup : debug] [ INFO ] ok: [localhost] [ INFO ] TASK [oVirt.hosted-engine-setup : Fail if engine VM is not running] [ ERROR ] fatal: [localhost]: FAILED! => {"changed": false, "msg": "Engine VM is not running, please check vdsm logs"} 2019-03-14 06:33:41,168-0400 ERROR ansible failed {'status': 'FAILED', 'ansible_type': 'task', 'ansible_task': u'Check VM status at virt level', 'ansible_result': u"type: <type 'dict'>\nstr: {'_ansible_parsed': True, 'stderr_lines': [], u'changed': True, u'end': u'2019-03-14 06:33:39.429283', '_ansible_no_log': False, u'stdout': u'', u'cmd': u'virsh -r list | grep HostedEngine | grep running', u'rc': 1, u'stderr': u'', u'delta': u'0:00:00.118398', u'invocation': {u'module_args': {u'crea", 'ansible_host': u'localhost', 'ansible_playbook': u'/usr/share/ovirt-hosted-engine-setup/ansible/create_target_vm.yml'} 2019-03-14 06:33:41,168-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8948c2fd0> kwargs ignore_errors:True 2019-03-14 06:30:45,000-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8944dd810> kwargs 2019-03-14 06:30:50,678-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894782490> kwargs 2019-03-14 06:30:56,369-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894675490> kwargs 2019-03-14 06:31:02,059-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8950ff750> kwargs 2019-03-14 06:31:07,738-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8944dd810> kwargs 2019-03-14 06:31:13,429-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8947824d0> kwargs 2019-03-14 06:31:19,118-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8947c2390> kwargs 2019-03-14 06:31:24,797-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894750b10> kwargs 2019-03-14 06:31:30,481-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8944dd810> kwargs 2019-03-14 06:31:36,174-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894782490> kwargs 2019-03-14 06:31:41,852-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894675490> kwargs 2019-03-14 06:31:47,534-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8947c2390> kwargs 2019-03-14 06:31:53,226-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8944dd810> kwargs 2019-03-14 06:31:58,908-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8947824d0> kwargs 2019-03-14 06:32:04,592-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8945f9f10> kwargs 2019-03-14 06:32:10,266-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8944fb390> kwargs 2019-03-14 06:32:15,948-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8944dd810> kwargs 2019-03-14 06:32:21,630-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894782490> kwargs 2019-03-14 06:32:27,316-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894675490> kwargs 2019-03-14 06:32:33,016-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8945f9f10> kwargs 2019-03-14 06:32:38,702-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8944dd810> kwargs 2019-03-14 06:32:44,382-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8947824d0> kwargs 2019-03-14 06:32:50,063-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8950ff750> kwargs 2019-03-14 06:32:55,745-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894750b10> kwargs 2019-03-14 06:33:01,422-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8944dd810> kwargs 2019-03-14 06:33:07,107-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894782490> kwargs 2019-03-14 06:33:12,794-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe894675490> kwargs 2019-03-14 06:33:18,470-0400 DEBUG ansible on_any args <ansible.executor.task_result.TaskResult object at 0x7fe8950ff750> kwargs Engine VM status: EngineUnexpectedlyDown-EngineDown. * Manually start the Engine VM via virsh: error: Failed to start domain HostedEngine error: the CPU is incompatible with host CPU: Host CPU does not provide required features: pcid

3 10

Broken dependencies in CentOS7 hosts upgrading to 4.2.8 from 4.2.7
by Roberto Nunin 15 Mar '19

15 Mar '19

Hi I have some oVirt clusters, in various config. One cluster based on CentOS7 hosts, another based on ovirt-node-ng. While the second was successfully updated from 4.2.7 to 4.2.8, attempts to update hosts of the first one ends with: Error: Package: vdsm-4.20.46-1.el7.x86_64 (ovirt-4.2) Requires: libvirt-daemon-kvm >= 4.5.0-10.el7_6.3 Installed: libvirt-daemon-kvm-4.5.0-10.el7.x86_64 (@base) libvirt-daemon-kvm = 4.5.0-10.el7 Being a CentOS7 installation and not an ovirt-node-ng, I cannot follow the notice in : https://lists.ovirt.org/archives/list/users@ovirt.org/message/AJUBXAIGXVD5U… The only way to have libvirt-daemon-kvm release 4-5-0.10_6.4 (and not 6.3) is to enable CentOS Updates repo. Looking at host-deploy log looks fine, but It's safe to enable that repo ? There is another, safest method to update these hosts to the latest version of 4.2 ? Thanks in advance -- Roberto Nunin

1 0

Discard a snapshot
by Mitja Mihelič 15 Mar '19

15 Mar '19

Hi! We have run into a problem migrating a VM's disk. While doing a live disk migration it all went well until "Removing Snapshot Auto-generated for Live Storage Migration". The operation started and got stuck in the "Preparing to merge" stage. The task was visible as an async task for a couple of days, then dissapeared. The snapshot named "Auto-generated for Live Storage Migration" still exists. If we try to start the VM it produces the following error: VM VM_NAME_HERE is down with error. Exit message: Bad volume specification {u'index': 0, u'domainID': u'47ee9bde-4f18-43c2-9383-0d30a27d1ae7', 'reqsize': '0', u'format': u'cow', u'bootOrder': u'1', u'address': {u'function': u'0x0', u'bus': u'0x00', u'domain': u'0x0000', u'type': u'pci', u'slot': u'0x05'}, u'volumeID': u'3b0184b7-5c5d-4669-8b3a-04880883118a', 'apparentsize': '19193135104', u'imageID': u'35599677-250a-4586-9db1-4df8f3291164', u'discard': False, u'specParams': {}, u'readonly': u'false', u'iface': u'virtio', u'optional': u'false', u'deviceId': u'35599677-250a-4586-9db1-4df8f3291164', 'truesize': '19193135104', u'poolID': u'35628205-9fa3-41de-b914-84f9adb633e4', u'device': u'disk', u'shared': u'false', u'propagateErrors': u'off', u'type': u'disk'}. From what I understand the data access path is snapshot->"disk before snapshot". And since the snapshot is corrupted it cannot be accessed and the whole VM start process fails. I believe that the disk before the snapshot was made is still intact. How can we discard the whole snapshot so that are left only with the state before the snapshot was made? We can afford to lose the data in the snapshot. Best regards, Mitja

1 0

oVirt 4.3.1 - Remove VM greyed out
by Strahil Nikolov 15 Mar '19

15 Mar '19

Hi Community, I have the following problem.A VM was created based on template and after poweroff/shutdown it cannot be removed - the button is greyed-out. Anyone who got such an issue ?Any hint where to look for ? Best Regards,Strahil Nikolov

1 1

Migrating VMs from ovirt-4.1.1 to ovirt-4.2.8
by Wood Peter 15 Mar '19

15 Mar '19

Hi, I need to migrate a few dozen VMs from ovirt-4.1.1 to ovirt-4.2.8. I did a few following this procedure: VMs export -> Detach Export domain -> Attach Export to new ovirt -> Import VMs Importing from VMware is one step only process. All I have to do is select VMware for Source in the Import page and then import the VMs I need. I feel like I'm missing something. Is there a way to import VMs from another ovirt deployment similar to import from VMware i.e. without going through Export storage domain? Thank you, -- Peter

2 1

hosted-engine --deploy fails on iSCSI while trying to connect to retrieved ipv6 address, even while forcing ipv4 with --4
by Guillaume Pavese 15 Mar '19

15 Mar '19

I try to deploy hosted-engine 4.3.2-rc2 on iSCSI I put a ipv4 portal address and targets get discovered. However they are are returned by the Synology hosts with both ipv4 and ipv6 adresses. LUN discovery then fails while attempting to connect to ipv6 address I tried hosted-engine --deploy --4 to force ipv4 but that fails too. Please specify the storage you would like: iscsi Please specify the iSCSI portal IP address: 10.199.9.16 Please specify the iSCSI portal port [3260]: Please specify the iSCSI discover user: Please specify the iSCSI discover password: Please specify the iSCSI portal login user: Please specify the iSCSI portal login password: [ INFO ] Discovering iSCSI targets [ INFO ] TASK [ovirt.hosted_engine_setup : Execute just a specific set of steps] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Force facts gathering] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : include_tasks] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Obtain SSO token using username/password credentials] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Prepare iSCSI parameters] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch host facts] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : iSCSI discover with REST API] [ INFO ] ok: [localhost] The following targets have been found: [1] iqn.2000-01.com.synology:SVC-STO-FR-301.Target-1.2dfed4a32a TPGT: 1, portals: 10.199.9.16:3260 fe80::211:32ff:fe6d:6ddb:3260 [2] iqn.2000-01.com.synology:SVC-STO-FR-301.Target-2.2dfed4a32a TPGT: 1, portals: 10.199.9.16:3260 fe80::211:32ff:fe6d:6ddb:3260 in Host's logs : mars 15 09:32:25 vs-inf-int-kvm-fr-304-210.hostics.fr iscsid[5983]:conn 0 login rejected: initiator error (02/00) mars 15 09:32:25 vs-inf-int-kvm-fr-304-210.hostics.fr iscsid[5983]: Connection1:0 to [target: iqn.2000-01.com.synology:SVC-STO-FR-301.Target-2.2dfed4a32a, portal: 10.199.9.16,3260] through [iface: default] is shutdown. mars 15 09:32:27 vs-inf-int-kvm-fr-304-210.hostics.fr iscsid[5983]: cannot make a connection to fe80::211:32ff:fe6d:6ddb:3260 (-1,22) ... mars 15 09:33:21 vs-inf-int-kvm-fr-304-210.hostics.fr iscsid[5983]: cannot make a connection to fe80::211:32ff:fe6d:6ddb:3260 (-1,22) mars 15 09:33:24 vs-inf-int-kvm-fr-304-210.hostics.fr iscsid[5983]: cannot make a connection to fe80::211:32ff:fe6d:6ddb:3260 (-1,22) mars 15 09:33:27 vs-inf-int-kvm-fr-304-210.hostics.fr vdsm[26174]: WARN Worker blocked: <Worker name=jsonrpc/1 running <Task <JsonRpcTask {'params': {u'connectionParams': [{u'port': u'3260', u'connection': u'fe80::211:32ff:fe6d:6ddb', u'iqn': u'iqn.2000-01.com.synology:SVC-STO-FR-301.Target-2.2dfed4a32a', u'user': u'', u'tpgt': u'1', u'ipv6_enabled': u'true', u'password': '********', u'id': u'00000000-0000-0000-0000-000000000000'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 3}, 'jsonrpc': '2.0', 'method': u'StoragePool.connectStorageServer', 'id': u'31411035-df1d-4b61-ac37-2a138dbdbb00'} at 0x7fcb904ef390> timeout=60, duration=60.00 at 0x7fcb904ef410> task#=10 at 0x7fcb905977d0>, traceback: File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap self.__bootstrap_inner() File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner self.run() File: "/usr/lib64/python2.7/threading.py", line 765, in run self.__target(*self.__args, **self.__kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 195, in run ret = func(*args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run self._execute_task() File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task task() File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__ self._callable() File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 262, in __call__ self._handler(self._ctx, self._req) File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 305, in _serveRequest response = self._handle_request(req, ctx) File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 345, in _handle_request res = method(**params) File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 198, in _dynamicMethod result = fn(*methodArgs) File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1095, in connectStorageServer connectionParams) File: "/usr/lib/python2.7/site-packages/vdsm/storage/dispatcher.py", line 74, in wrapper result = ctask.prepare(func, *args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 108, in wrapper return m(self, *a, **kw) File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1179, in prepare result = self._run(func, *args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File: "<string>", line 2, in connectStorageServer File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File: "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 2411, in connectStorageServer conObj.connect() File: "/usr/lib/python2.7/site-packages/vdsm/storage/storageServer.py", line 488, in connect iscsi.addIscsiNode(self._iface, self._target, self._cred) File: "/usr/lib/python2.7/site-packages/vdsm/storage/iscsi.py", line 217, in addIscsiNode iscsiadm.node_login(iface.name, target.address, target.iqn) File: "/usr/lib/python2.7/site-packages/vdsm/storage/iscsiadm.py", line 327, in node_login portal, "-l"]) File: "/usr/lib/python2.7/site-packages/vdsm/storage/iscsiadm.py", line 122, in _runCmd return misc.execCmd(cmd, printable=printCmd, sudo=True, sync=sync) File: "/usr/lib/python2.7/site-packages/vdsm/common/commands.py", line 213, in execCmd (out, err) = p.communicate(data) File: "/usr/lib64/python2.7/site-packages/subprocess32.py", line 924, in communicate stdout, stderr = self._communicate(input, endtime, timeout) File: "/usr/lib64/python2.7/site-packages/subprocess32.py", line 1706, in _communicate orig_timeout) File: "/usr/lib64/python2.7/site-packages/subprocess32.py", line 1779, in _communicate_with_poll ready = poller.poll(self._remaining_time(endtime)) mars 15 09:33:27 vs-inf-int-kvm-fr-304-210.hostics.fr iscsid[5983]: cannot make a connection to fe80::211:32ff:fe6d:6ddb:3260 (-1,22) Guillaume Pavese Ingénieur Système et Réseau Interactiv-Group

1 0

Centos7.5 virtual machine installation is slow
by xilazz＠126.com 15 Mar '19

15 Mar '19

Software version 4.3.1.1-1. The virtual machine performed post-installation setup tasks for a long time during the process of creating Centos7.5 system.I don't know if any of you have.

1 0

upgrading failed from 4.2 to 4.3
by Abhishek Sahni 15 Mar '19

15 Mar '19

Hello Users, I am trying to upgrade ovirt-engine running on bare metal, following below guide, https://ovirt.org/release/4.3.0/ and it is getting failed every time with the below errors, engine-setup ============================ [ INFO ] Stage: Setup validation During execution engine service will be stopped (OK, Cancel) [OK]: [ ERROR ] It seems that you are running your engine inside of the hosted-engine VM and are not in "Global Maintenance" mode. In that case you should put the system into the "Global Maintenance" mode before running engine-setup, or the hosted-engine HA agent might kill the machine, which might corrupt your data. [ ERROR ] Failed to execute stage 'Setup validation': Hosted Engine setup detected, but Global Maintenance is not set. [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20190314171026-wicgh1.log [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20190314171042-setup.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed =================================== I was using hosted engine on nodes earlier but later remove the nodes and using ovirt egine on seperate bare-metal. Any pointers? -- ABHISHEK SAHNI Mob : +91-990-701-5143

2 2

Check in API if host is in state "reconfiguring networks"
by Eduardo Mayoral 14 Mar '19

14 Mar '19

Hi, I am writing some provisioning scripts using the python API in which I add a number of hosts to an ovirt engine. To minimize the time the script takes, I would like to call host_service.setup_network() and host_service.commit_net_config() for each host with async=True and then get into a loop checking every few seconds for the status of the hosts until all of them have finished (so I can go on with another task such as configuring storage). I was able to do something similar for the install phase checking if host.status equals HostStatus.MAINTENANCE , but I cannot find in the documentation how to check when a host is in status "reconfiguring network". The web interface shows this, so surely there must be a way, but I have not managed to find it in the API documentation. Could somebody help? Thanks in advance! -- Eduardo Mayoral Jimeno Systems engineer, platform department. Arsys Internet. emayoral(a)arsys.es - +34 941 620 105 - ext 2153

1 0

Re: Migrate HE between ISCSI storages
by Strahil 14 Mar '19

14 Mar '19

If you have the will, would you create a short but detailed instruction ? I think that it will be nice to have your experience documented in the ovirt docs. Best Regards, Strahil NikolovOn Mar 14, 2019 06:51, kiv(a)intercom.pro wrote: > > I went the other way. Created another HE one on a new host. On a separate server, make the export domain, connected it to both HE and migrated all vm through export / import between HE. Everything went quickly and without problems. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QR722D7VHFLPM…

1 0