- Users - oVirt List Archives

Mellanox OFED with oVirt
by Vinícius Ferrão 02 Sep '20

02 Sep '20

Hello, Anyone had success using Mellanox OFED with oVirt? Already learned some things: 1. I can’t use oVirt Node. 2. Mellanox OFED cannot be installed with mlnx-ofed-all since it breaks dnf. We need to rely on the upstream RDMA implementation. 3. The way to go is running: dnf install mlnx-ofed-dpdk-upstream-libs But after the installation I ended up with broken dnf: [root@c4140 ~]# dnf update Updating Subscription Management repositories. Last metadata expiration check: 0:03:54 ago on Tue 01 Sep 2020 11:52:41 PM -03. Error: Problem: both package mlnx-ofed-all-user-only-5.1-0.6.6.0.rhel8.2.noarch and mlnx-ofed-all-5.1-0.6.6.0.rhel8.2.noarch obsolete glusterfs-rdma - cannot install the best update candidate for package glusterfs-rdma-6.0-37.el8.x86_64 - package ovirt-host-4.4.1-4.el8ev.x86_64 requires glusterfs-rdma, but none of the providers can be installed - package mlnx-ofed-all-5.1-0.6.6.0.rhel8.2.noarch obsoletes glusterfs-rdma provided by glusterfs-rdma-6.0-37.el8.x86_64 - package glusterfs-rdma-3.12.2-40.2.el8.x86_64 requires glusterfs(x86-64) = 3.12.2-40.2.el8, but none of the providers can be installed - package glusterfs-rdma-6.0-15.el8.x86_64 requires glusterfs(x86-64) = 6.0-15.el8, but none of the providers can be installed - package glusterfs-rdma-6.0-20.el8.x86_64 requires glusterfs(x86-64) = 6.0-20.el8, but none of the providers can be installed - cannot install both glusterfs-3.12.2-40.2.el8.x86_64 and glusterfs-6.0-37.el8.x86_64 - cannot install both glusterfs-6.0-15.el8.x86_64 and glusterfs-6.0-37.el8.x86_64 - cannot install both glusterfs-6.0-20.el8.x86_64 and glusterfs-6.0-37.el8.x86_64 - cannot install the best update candidate for package ovirt-host-4.4.1-4.el8ev.x86_64 - cannot install the best update candidate for package glusterfs-6.0-37.el8.x86_64 (try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages) That are the packages installed: [root@c4140 ~]# rpm -qa *mlnx* mlnx-dpdk-19.11.0-1.51066.x86_64 mlnx-ofa_kernel-devel-5.1-OFED.5.1.0.6.6.1.rhel8u2.x86_64 mlnx-ethtool-5.4-1.51066.x86_64 mlnx-dpdk-devel-19.11.0-1.51066.x86_64 mlnx-ofa_kernel-5.1-OFED.5.1.0.6.6.1.rhel8u2.x86_64 mlnx-dpdk-doc-19.11.0-1.51066.noarch mlnx-dpdk-tools-19.11.0-1.51066.x86_64 mlnx-ofed-dpdk-upstream-libs-5.1-0.6.6.0.rhel8.2.noarch kmod-mlnx-ofa_kernel-5.1-OFED.5.1.0.6.6.1.rhel8u2.x86_64 mlnx-iproute2-5.6.0-1.51066.x86_64 And finally this is the repo that I’m using: [root@c4140 ~]# cat /etc/yum.repos.d/mellanox_mlnx_ofed.repo # # Mellanox Technologies Ltd. public repository configuration file. # For more information, refer to http://linux.mellanox.com # [mlnx_ofed_latest_base] name=Mellanox Technologies rhel8.2-$basearch mlnx_ofed latest baseurl=http://linux.mellanox.com/public/repo/mlnx_ofed/latest/rhel8.2/$bas… enabled=1 gpgkey=http://www.mellanox.com/downloads/ofed/RPM-GPG-KEY-Mellanox gpgcheck=1 So anyone had success with this?

3 2

How can Gluster be a HCI default, when it's hardly ever working?
by thomas＠hoberg.net 02 Sep '20

02 Sep '20

I've just tried to verify what you said here. As a base line I started with the 1nHCI Gluster setup. From four VMs, two legacy, two Q35 on the single node Gluster, one survived the import, one failed silently with an empty disk, two failed somewhere in the middle of qemu-img trying to write the image to the Gluster storage. For each of those two, this always happened at the same block number, a unique one per machine, not in random places, as if qemu-img reading and writing the very same image could not agree. That's two types of error and a 75% failure rate I created another domain, basically using an NFS automount export from one of the HCI nodes (a 4.3 node serving as 4.4 storage) and imported the very same VMs (source all 4.3) transported via a re-attached export domain to 4.4. Three of the for imports worked fine, no error with qemu-img writing to NFS. All VMs had full disk images and launched, which verified that there is nothing wrong with the exports at least. But there was still one, that failed with the same qemu-img error. I then tried to move the disks from NFS to Gluster, which internally is also done via qemu-img, and I had those fail every time. Gluster or HCI seems a bit of a Russian roulette for migrations, and I am wondering how much it is better for normal operations. I'm still going to try moving via a backup domain (on NFS) and moving between that and Gluster, to see if it makes any difference. I really haven't done a lot of stress testing yet with oVirt, but this experience doesn't build confidence.

2 2

HP blade G7, with be2net.
by Remulo 01 Sep '20

01 Sep '20

Hello I have some blades with 10GE interface that have / need the Emulex be2net driver, however it is no longer available on Redhat8 / Centos8. Is there any way to install ovirt to work on these machines? Here are some links to problems with Redhat. Emulex NIC using be2net driver https://access.redhat.com/solutions/1229853 https://access.redhat.com/solutions/514353 I did a lot of research and couldn't get a functional driver for ovirt. Thank you. -- Atenciosamente, Rêmulo Ferreira.

3 2

Problem installing Windows VM on 4.4.1
by fgarat＠gmail.com 01 Sep '20

01 Sep '20

Hi, I'm having problem after I upgraded to 4.4.1 with Windows machines. The installation sees no disk. Even IDE disk doesn't get detected and installation won't move forward no matter what driver i use for the disk. Any one else having this issue?. Regards, Facundo

4 3

How can you avoid breaking 4.3.11 legacy VMs imported in 4.4.1 during a migration?
by thomas＠hoberg.net 01 Sep '20

01 Sep '20

Testing the 4.3 to 4.4 migration... what I describe here is facts is mostly observations and conjecture, could be wrong, just makes writing easier... While 4.3 seems to maintain a default emulated machine type (pc-i440fx-rhel7.6.0 by default), it doesn't actually allow setting it in the cluster settings: Could be built-in, could be inherited from the default template... Most of my VMs were created with the default on 4.3. oVirt 4.4 presets that to pc-q35-rhel8.1.0 and that has implications: 1. Any VM imported from an export on a 4.3 farm, will get upgraded to Q35, which unfortunately breaks things, e.g. network adapters getting renamed as the first issue I stumbled on some Debian machines 2. If you try to compensate by lowering the cluster default from Q35 to pc-i44fx the hosted-engine will fail, because it was either built or came as Q35 and can no longer find critical devices: It evidently doesn't take/use the VM configuration data it had at the last shutdown, but seems to re-generate it according to some obscure logic, which fails here. I've tried creating a bit of backward compatibility by creating another template based on pc-i440fx, but at the time of the import, I cannot switch the template. If I try to downgrade the cluster, the hosted-engine will fail to start and I can't change the template of the hosted-engine to something Q35. Currently this leaves me in a position where I can't separate the move of VMs from 4.3 to 4.4 and the upgrade of the virtual hardware, which is a different mess for every OS in the mix of VMs. Recommendations, tips anyone? P.S. A hypervisor reconstructing the virtual hardware from anywhere but storage at every launch, is difficult to trust IMHO.

3 6

Upgrade Doc typo.
by carl langlois 01 Sep '20

01 Sep '20

Hi, Not sure if this is relevant but in the ovirt doc in the 4.2 to 4.3 upgrade the repo rpm is specified with 4,4 rpm . Thanks Carl [image: image.png]

2 2

Re: Promiscuous Mode
by thomas＠hoberg.net 31 Aug '20

31 Aug '20

Not sure it will actually do that, but if you create a new network profile, you can select a 'port mirroring' option: It is my understanding that is what you need. You may also want to deselect the filtering rules in the same place.

1 0

can't mount an export domain with 4K block size (VDO) as PosixFS (xfs)
by thomas＠hoberg.net 31 Aug '20

31 Aug '20

After this (https://devconfcz2020a.sched.com/event/YOtG/ovirt-4k-teaching-an-old-dog-ne…) I sure do not expect this (log below): Actually I am trying to evaluate just how portable oVirt storage is and this case I had prepared a USB3 HDD with VDO, which I could literally move between farms to transport VMs. Logical disks are typically large for simplicity within the VMs, QCOW2 and VDO assumed to compensate for this 'lack of planning' while the allocated storage easily fits the HDD. Once I got beyond the initial issues, I came across this somewhat unexpected issue: VDO storage uses 4k blocks all around, but evidently when you mount an export domain (or I guess any domain) as POSIX, 512byte blocks are assumed somewhere and 4k blocks rejected. I'd say that is a bug in VDSM, right? Or is there anything in the mount options to fix this? 020-08-31 18:44:40,424+0200 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=7a293dec-85b3-4b82-92b7-4e7d03b40343 (api:48) 2020-08-31 18:44:40,425+0200 INFO (periodic/0) [vdsm.api] FINISH repoStats return={'9992dc21-edf2-4951-9020-7c78f1220e02': {'code': 0, 'lastCheck': '1.6', 'delay': '0.00283651', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '25d95783-44df-4fda-b642-55fe09162149': {'code': 0, 'lastCheck': '2.2', 'delay': '0.00256944', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}, '148d9e9e-d7b8-4220-9ee8-057da96e608c': {'code': 0, 'lastCheck': '2.2', 'delay': '0.00050217', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=internal, task_id=7a293dec-85b3-4b82-92b7-4e7d03b40343 (api:54) 2020-08-31 18:44:41,121+0200 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:41,412+0200 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:41,414+0200 INFO (jsonrpc/2) [vdsm.api] START repoStats(domains=['9992dc21-edf2-4951-9020-7c78f1220e02']) from=::1,44782, task_id=657c820e-e2eb-4b1d-b2fb-1772b3af6f32 (api:48) 2020-08-31 18:44:41,414+0200 INFO (jsonrpc/2) [vdsm.api] FINISH repoStats return={'9992dc21-edf2-4951-9020-7c78f1220e02': {'code': 0, 'lastCheck': '2.6', 'delay': '0.00283651', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,44782, task_id=657c820e-e2eb-4b1d-b2fb-1772b3af6f32 (api:54) 2020-08-31 18:44:41,414+0200 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:41,965+0200 INFO (jsonrpc/3) [IOProcessClient] (Global) Starting client (__init__:308) 2020-08-31 18:44:41,984+0200 INFO (ioprocess/87332) [IOProcess] (Global) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,006+0200 INFO (jsonrpc/3) [storage.StorageDomainCache] Removing domain fe9fb0db-2743-457a-80f0-9a4edc509e9d from storage domain cache (sdc:211) 2020-08-31 18:44:42,006+0200 INFO (jsonrpc/3) [storage.StorageDomainCache] Invalidating storage domain cache (sdc:74) 2020-08-31 18:44:42,006+0200 INFO (jsonrpc/3) [vdsm.api] FINISH connectStorageServer return={'statuslist': [{'id': '00000000-0000-0000-0000-000000000000', 'status': 0}]} from=::ffff:192.168.0.87,40378, flow_id=05fe72ef-c8ac-4e03-8453-5171e5fc5f8b, task_id=b92d4144-a9db-4423-ba6b-35934d4f9200 (api:54) 2020-08-31 18:44:42,008+0200 INFO (jsonrpc/3) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 3.11 seconds (__init__:312) 2020-08-31 18:44:42,063+0200 INFO (jsonrpc/5) [vdsm.api] START getStorageDomainsList(spUUID='00000000-0000-0000-0000-000000000000', domainClass=3, storageType='', remotePath='/dev/mapper/vdo1', options=None) from=::ffff:192.168.0.87,40378, flow_id=eff548f9-b663-45dd-b8a5-5854f9b5dde8, task_id=98ca66a5-edee-40d4-9253-6a46409241cc (api:48) 2020-08-31 18:44:42,063+0200 INFO (jsonrpc/5) [storage.StorageDomainCache] Refreshing storage domain cache (resize=True) (sdc:80) 2020-08-31 18:44:42,063+0200 INFO (jsonrpc/5) [storage.ISCSI] Scanning iSCSI devices (iscsi:442) 2020-08-31 18:44:42,101+0200 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:42,128+0200 INFO (jsonrpc/5) [storage.ISCSI] Scanning iSCSI devices: 0.06 seconds (utils:390) 2020-08-31 18:44:42,129+0200 INFO (jsonrpc/5) [storage.HBA] Scanning FC devices (hba:60) 2020-08-31 18:44:42,182+0200 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call Host.ping2 succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:42,184+0200 INFO (jsonrpc/4) [vdsm.api] START repoStats(domains=['9992dc21-edf2-4951-9020-7c78f1220e02']) from=::1,44782, task_id=7b3f1eb8-f7e3-4467-93a1-27ef9721c90c (api:48) 2020-08-31 18:44:42,184+0200 INFO (jsonrpc/4) [vdsm.api] FINISH repoStats return={'9992dc21-edf2-4951-9020-7c78f1220e02': {'code': 0, 'lastCheck': '3.4', 'delay': '0.00283651', 'valid': True, 'version': 5, 'acquired': True, 'actual': True}} from=::1,44782, task_id=7b3f1eb8-f7e3-4467-93a1-27ef9721c90c (api:54) 2020-08-31 18:44:42,184+0200 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Host.getStorageRepoStats succeeded in 0.00 seconds (__init__:312) 2020-08-31 18:44:42,277+0200 INFO (jsonrpc/5) [storage.HBA] Scanning FC devices: 0.15 seconds (utils:390) 2020-08-31 18:44:42,309+0200 INFO (jsonrpc/5) [storage.Multipath] Resizing multipath devices (multipath:104) 2020-08-31 18:44:42,312+0200 INFO (jsonrpc/5) [storage.Multipath] Resizing multipath devices: 0.00 seconds (utils:390) 2020-08-31 18:44:42,312+0200 INFO (jsonrpc/5) [storage.StorageDomainCache] Refreshing storage domain cache: 0.25 seconds (utils:390) 2020-08-31 18:44:42,418+0200 INFO (tmap-569/0) [IOProcessClient] (_dev_mapper_vdo1) Starting client (__init__:308) 2020-08-31 18:44:42,445+0200 INFO (tmap-569/1) [IOProcessClient] (glusterSD/nucvirt.mtk.hoberg.net:_data) Starting client (__init__:308) 2020-08-31 18:44:42,449+0200 INFO (ioprocess/87496) [IOProcess] (_dev_mapper_vdo1) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,457+0200 INFO (tmap-569/2) [IOProcessClient] (glusterSD/nucvirt.mtk.hoberg.net:_vmstore) Starting client (__init__:308) 2020-08-31 18:44:42,465+0200 INFO (ioprocess/87508) [IOProcess] (glusterSD/nucvirt.mtk.hoberg.net:_data) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,478+0200 INFO (tmap-569/3) [IOProcessClient] (glusterSD/nucvirt.mtk.hoberg.net:_engine) Starting client (__init__:308) 2020-08-31 18:44:42,486+0200 INFO (ioprocess/87516) [IOProcess] (glusterSD/nucvirt.mtk.hoberg.net:_vmstore) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,514+0200 INFO (ioprocess/87533) [IOProcess] (glusterSD/nucvirt.mtk.hoberg.net:_engine) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,522+0200 INFO (jsonrpc/5) [IOProcessClient] (fe9fb0db-2743-457a-80f0-9a4edc509e9d) Starting client (__init__:308) 2020-08-31 18:44:42,553+0200 INFO (ioprocess/87552) [IOProcess] (fe9fb0db-2743-457a-80f0-9a4edc509e9d) Starting ioprocess (__init__:434) 2020-08-31 18:44:42,582+0200 ERROR (jsonrpc/5) [storage.HSM] Unexpected error (hsm:2843) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2829, in getStorageDomainsList dom = sdCache.produce(sdUUID=sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 115, in produce domain.getRealDomain() File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 139, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 156, in _findDomain return findMethod(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/nfsSD.py", line 146, in findDomain return NfsStorageDomain(NfsStorageDomain.findDomainPath(sdUUID)) File "/usr/lib/python3.6/site-packages/vdsm/storage/fileSD.py", line 390, in __init__ manifest.block_size, storage_block_size) File "/usr/lib/python3.6/site-packages/vdsm/storage/sd.py", line 939, in _validate_storage_block_size raise se.StorageDomainBlockSizeMismatch(block_size, storage_block_size) vdsm.storage.exception.StorageDomainBlockSizeMismatch: Block size does not match storage block size: 'block_size=512, storage_block_size=4096' 2020-08-31 18:44:42,582+0200 INFO (jsonrpc/5) [vdsm.api] FINISH getStorageDomainsList return={'domlist': []} from=::ffff:192.168.0.87,40378, flow_id=eff548f9-b663-45dd-b8a5-5854f9b5dde8, task_id=98ca66a5-edee-40d4-9253-6a46409241cc (api:54) 2020-08-31 18:44:42,582+0200 INFO (jsonrpc/5) [jsonrpc.JsonRpcServer] RPC call Host.getStorageDomains succeeded in 0.52 seconds (__init__:312) 2020-08-31 18:44:42,801+0200 INFO (jsonrpc/7) [vdsm.api] START disconnectStorageServer(domType=6, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '********', 'vfs_type': 'xfs', 'port': '', 'mnt_options': 'rw,relatime,seclabel,attr2,inode64,noquota,_netdev,x-systemd.device-timeout=0', 'iqn': '', 'connection': '/dev/mapper/vdo1', 'ipv6_enabled': 'false', 'id': 'b4a73d6a-344c-446b-a33d-7b665e78578d', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:192.168.0.87,40378, flow_id=6678917a-b29c-416c-aa2d-5abc1d5351a3, task_id=3d0cf36e-427d-4000-8857-a8a2af641b10 (api:48) 2020-08-31 18:44:42,801+0200 INFO (jsonrpc/7) [storage.Mount] unmounting /rhev/data-center/mnt/_dev_mapper_vdo1 (mount:215) 2020-08-31 18:44:42,806+0200 INFO (jsonrpc/0) [vdsm.api] START disconnectStorageServer(domType=6, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'password': '********', 'vfs_type': 'xfs', 'port': '', 'mnt_options': 'rw,relatime,seclabel,attr2,inode64,noquota,_netdev,x-systemd.device-timeout=0', 'iqn': '', 'connection': '/dev/mapper/vdo1', 'ipv6_enabled': 'false', 'id': 'b4a73d6a-344c-446b-a33d-7b665e78578d', 'user': '', 'tpgt': '1'}], options=None) from=::ffff:192.168.0.87,40378, flow_id=bb449405-5b78-4b65-a352-e2e94a269df9, task_id=09783e0a-933c-448d-b0ca-1b5d351eb042 (api:48) 2020-08-31 18:44:42,806+0200 INFO (jsonrpc/0) [storage.Mount] unmounting /rhev/data-center/mnt/_dev_mapper_vdo1 (mount:215) 2020-08-31 18:44:42,821+0200 ERROR (jsonrpc/0) [storage.HSM] Could not disconnect from storageServer (hsm:2528) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2524, in disconnectStorageServer conObj.disconnect() File "/usr/lib/python3.6/site-packages/vdsm/storage/storageServer.py", line 201, in disconnect self._mount.umount(True, True) File "/usr/lib/python3.6/site-packages/vdsm/storage/mount.py", line 217, in umount umount(self.fs_file, force=force, lazy=lazy, freeloop=freeloop) File "/usr/lib/python3.6/site-packages/vdsm/common/supervdsm.py", line 56, in __call__ return callMethod() File "/usr/lib/python3.6/site-packages/vdsm/common/supervdsm.py", line 54, in <lambda> **kwargs) File "<string>", line 2, in umount File "/usr/lib64/python3.6/multiprocessing/managers.py", line 772, in _callmethod raise convert_to_error(kind, result) vdsm.storage.mount.MountError: Command ['/usr/bin/umount', '-f', '-l', '/rhev/data-center/mnt/_dev_mapper_vdo1'] failed with rc=32 out=b'' err=b'umount: /rhev/data-center/mnt/_dev_mapper_vdo1: not mounted.\n' 2020-08-31 18:44:42,821+0200 INFO (jsonrpc/0) [storage.StorageDomainCache] Refreshing storage domain cache (resize=False) (sdc:80)

2 2

Promiscuous Mode
by Anton Louw 31 Aug '20

31 Aug '20

Hi All, I would like to deploy a VM that will monitor a specific network. I know in VMware you can set the port group to allow Promiscuous Mode. Is there something similar I can do in oVirt? Thanks Anton Louw Cloud Engineer: Storage and Virtualization ______________________________________ D: 087 805 1572 | M: N/A A: Rutherford Estate, 1 Scott Street, Waverley, Johannesburg anton.louw(a)voxtelecom.co.za www.vox.co.za

1 0

Error exporting into ova
by Gianluca Cecchi 31 Aug '20

31 Aug '20

Hello, I'm playing with export_vm_as_ova.py downloaded from the examples github: https://github.com/oVirt/ovirt-engine-sdk/blob/master/sdk/examples/export_v… My environment is oVirt 4.3.3.7 with iSCSI storage domain. It fails leaving an ova.tmp file In webadmin gui: Starting to export Vm enginecopy1 as a Virtual Appliance 7/19/1911:55:12 AM VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) 7/19/1912:25:36 PM Failed to export Vm enginecopy1 as a Virtual Appliance to path /save_ova/base/dump/myvm2.ova on Host ov301 7/19/1912:25:37 PM During export I have this qemu-img process creating the disk over the loop device: root 30878 30871 0 11:55 pts/2 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop1' vdsm vdsm 30882 30878 10 11:55 ? 00:00:00 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop1 The ova.tmp file is getting filled while command runs eg: [root@ov301 ]# du -sh /save_ova/base/dump/myvm2.ova.tmp 416M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 911M /save_ova/base/dump/myvm2.ova.tmp [root@ov301 ]# and the final generated / not completed file is in this state: [root@ov301 ]# qemu-img info /save_ova/base/dump/myvm2.ova.tmp image: /save_ova/base/dump/myvm2.ova.tmp file format: raw virtual size: 30G (32217446400 bytes) disk size: 30G [root@ov301 sysctl.d]# But I notice that the timestamp of the file is about 67 minutes after start of job and well after the notice of its failure.... [root@ov301 sysctl.d]# ll /save_ova/base/dump/ total 30963632 -rw-------. 1 root root 32217446400 Jul 19 13:02 myvm2.ova.tmp [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 30G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# In engine.log the first error I see is 30 minutes after start 2019-07-19 12:25:31,563+02 ERROR [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook execution failed: Timeout occurred while executing Ansible playbook. 2019-07-19 12:25:31,563+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Ansible playbook command has exited with value: 1 2019-07-19 12:25:31,564+02 ERROR [org.ovirt.engine.core.bll.CreateOvaCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] Failed to create OVA. Please check logs for more details: /var/log/ovirt-engine/ova/ovirt-export-ova-ansible-20190719115531-ov301-2001ddf4.log 2019-07-19 12:25:31,565+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.TeardownImageVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] START, TeardownImageVDSCommand(HostName = ov301, ImageActionsVDSCommandParameters:{hostId='8ef1ce6f-4e38-486c-b3a4-58235f1f1d06'}), log id: 3d2246f7 2019-07-19 12:25:36,569+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [2001ddf4] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ov301 command TeardownImageVDS failed: Cannot deactivate Logical Volume: ('General Storage Exception: ("5 [] [\' Logical volume fa33df49-b09d-4f86-9719-ede649542c21/0420ef47-0ad0-4cf9-babd-d89383f7536b in use.\']\\nfa33df49-b09d-4f86-9719-ede649542c21/[\'a7480dc5-b5ca-4cb3-986d-77bc12165be4\', \'0420ef47-0ad0-4cf9-babd-d89383f7536b\']",)',) In ansible playbook suggested log file I don't see anything useful. It ends with timestamps when the script has been launched. Last lines are: 2019-07-19 11:55:33,877 p=5699 u=ovirt | TASK [ovirt-ova-export-pre-pack : Retrieving the temporary path for the OVA file] *** 2019-07-19 11:55:34,198 p=5699 u=ovirt | changed: [ov301] => { "changed": true, "dest": "/save_ova/base/dump/myvm2.ova.tmp", "gid": 0, "group": "root", "mode": "0600", "owner": "root", "secontext": "system_u:object_r:nfs_t:s0", "size": 32217446912, "state": "file", "uid": 0 } 2019-07-19 11:55:34,204 p=5699 u=ovirt | TASK [ovirt-ova-pack : Run packing script] ************************************* It seems 30 minutes... for timeout? About what, ansible job? Or possibly implicit user session created when running the python script? The snapshot has been correctly deleted (as I see also in engine.log), I don't see it in webadmin gui. Any known problem? Just for test I executed again at 14:24 and I see same Ansible error at 14:54 The snapshot gets deleted, while the qemu-img command still continues.... [root@ov301 sysctl.d]# ps -ef | grep qemu-img root 13504 13501 0 14:24 pts/1 00:00:00 su -p -c qemu-img convert -T none -O qcow2 '/rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b' '/dev/loop0' vdsm vdsm 13505 13504 3 14:24 ? 00:01:26 qemu-img convert -T none -O qcow2 /rhev/data-center/mnt/blockSD/fa33df49-b09d-4f86-9719-ede649542c21/images/59a4a324-4c99-4ff5-abb1-e9bbac83292a/0420ef47-0ad0-4cf9-babd-d89383f7536b /dev/loop0 root 17587 24530 0 15:05 pts/0 00:00:00 grep --color=auto qemu-img [root@ov301 sysctl.d]# [root@ov301 sysctl.d]# du -sh /save_ova/base/dump/myvm2.ova.tmp 24G /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# ll /save_ova/base/dump/myvm2.ova.tmp -rw-------. 1 root root 32217446400 Jul 19 15:14 /save_ova/base/dump/myvm2.ova.tmp [root@ov301 sysctl.d]# and then continues until image copy completes, but at this time the job has already aborted and so the completion of the ova composition doesn't go ahead... and I remain with the ova.tmp file... How to extend timeout? Thanks in advance, Gianluca

4 11