- Users - oVirt List Archives

Libvirt ERROR cannot access backing file after importing VM from OpenStack
by Vrgotic, Marko 15 May '19

15 May '19

Dear oVirt team, When trying to start imported VM, it fails with following message: ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-2) [] EVENT_ID: VM_DOWN_ERROR(119), VM instance-00000673 is down with error. Exit message: Cannot access backing file '/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce' of storage file '/rhev/data-center/mnt/glusterSD/aws-gfs-01.awesome.lan:_gv0__he/2607c265-248c-40ad-b020-f3756454839e/images/816ac00f-ba98-4827-b5c8-42a8ba496089/8ecfcd5b-db67-4c23-9869-0e20d7553aba' (as uid:107, gid:107): No such file or directory. Platform details: Ovirt SHE Version 4.2.2.6-1.el7.centos GlusterFS, unmanaged by oVirt. VM is imported & converted from OpenStack, according to log files, successfully (one WARN, related to different MAC address): 2018-05-24 12:03:31,028+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand] (default task-29) [cc5931a2-1af5-4d65-b0b3-362588db9d3f] FINISH, GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c], VM [instance-00078f6a], VM [instance-00000814], VM [instance-0001f9ac], VM [instance-000001ff], VM [instance-0001f718], VM [instance-00000673], VM [instance-0001ecf2], VM [instance-00078d38]], log id: 7f178a5e 2018-05-24 12:48:33,722+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsNamesFromExternalProviderVDSCommand] (default task-8) [103d56e1-7449-4853-ae50-48ee94d43d77] FINISH, GetVmsNamesFromExternalProviderVDSCommand, return: [VM [instance-0001f94c], VM [instance-00078f6a], VM [instance-00000814], VM [instance-0001f9ac], VM [instance-000001ff], VM [instance-0001f718], VM [instance-00000673], VM [instance-0001ecf2], VM [instance-00078d38]], log id: 3aa178c5 2018-05-24 12:48:47,291+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand] (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] START, GetVmsFullInfoFromExternalProviderVDSCommand(HostName = aws-ovhv-01, GetVmsFromExternalProviderParameters:{hostId='cbabe1e8-9e7f-4c4b-be9c-49154953564d', url='qemu+tcp://root@172.19.0.12/system', username='null', originType='KVM', namesOfVms='[instance-00000673]'}), log id: 4c445109 2018-05-24 12:48:47,318+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFullInfoFromExternalProviderVDSCommand] (default task-17) [4bf555c7-9d64-4ecc-b059-8a60a4b27bdd] FINISH, GetVmsFullInfoFromExternalProviderVDSCommand, return: [VM [instance-00000673]], log id: 4c445109 2018-05-24 12:49:20,466+02 INFO [org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand] (default task-41) [14edb003-b4a0-4355-b3de-da2b68774fe3] Lock Acquired to object 'EngineLock:{exclusiveLocks='[instance-00000673=VM_NAME, 1f0b608f-7cfc-4b27-a876-b5d8073011a1=VM]', sharedLocks=''}' 2018-05-24 12:49:20,586+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-653408) [14edb003-b4a0-4355-b3de-da2b68774fe3] EVENT_ID: MAC_ADDRESS_IS_EXTERNAL(925), VM instance-00000673 has MAC address(es) fa:16:3e:74:18:50, which is/are out of its MAC pool definitions. 2018-05-24 12:49:21,021+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-653408) [14edb003-b4a0-4355-b3de-da2b68774fe3] EVENT_ID: IMPORTEXPORT_STARTING_IMPORT_VM(1,165), Starting to import Vm instance-00000673 to Data Center AVEUNL, Cluster AWSEUOPS 2018-05-24 12:49:28,816+02 INFO [org.ovirt.engine.core.bll.exportimport.ImportVmFromExternalProviderCommand] (EE-ManagedThreadFactory-engine-Thread-653407) [] Lock freed to object 'EngineLock:{exclusiveLocks='[instance-00000673=VM_NAME, 1f0b608f-7cfc-4b27-a876-b5d8073011a1=VM]', sharedLocks=''}' 2018-05-24 12:49:28,911+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConvertVmVDSCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [2047673e] START, ConvertVmVDSCommand(HostName = aws-ovhv-01, ConvertVmVDSParameters:{hostId='cbabe1e8-9e7f-4c4b-be9c-49154953564d', url='qemu+tcp://root@172.19.0.12/system', username='null', vmId='1f0b608f-7cfc-4b27-a876-b5d8073011a1', vmName='instance-00000673', storageDomainId='2607c265-248c-40ad-b020-f3756454839e', storagePoolId='5a5de92c-0120-0167-03cb-00000000038a', virtioIsoPath='null', compatVersion='null', Disk0='816ac00f-ba98-4827-b5c8-42a8ba496089'}), log id: 53408517 2018-05-24 12:49:29,010+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [2047673e] EVENT_ID: IMPORTEXPORT_STARTING_CONVERT_VM(1,193), Starting to convert Vm instance-00000673 2018-05-24 12:52:57,982+02 INFO [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-16) [df1d5f72-eb17-46e4-9946-20ca9809b54c] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[instance-00000673=VM_NAME]', sharedLocks='[1f0b608f-7cfc-4b27-a876-b5d8073011a1=VM]'}' 2018-05-24 12:59:24,575+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [2047673e] EVENT_ID: IMPORTEXPORT_IMPORT_VM(1,152), Vm instance-00000673 was imported successfully to Data Center AVEUNL, Cluster AWSEUOPS Than trying to start VM fails with following messages: 2018-05-24 13:00:32,085+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-653729) [] EVENT_ID: USER_STARTED_VM(153), VM instance-00000673 was started by admin@internal-authz (Host: aws-ovhv-06). 2018-05-24 13:00:33,417+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-9) [] VM '1f0b608f-7cfc-4b27-a876-b5d8073011a1'(instance-00000673) moved from 'WaitForLaunch' --> 'Down' 2018-05-24 13:00:33,436+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-9) [] EVENT_ID: VM_DOWN_ERROR(119), VM instance-00000673 is down with error. Exit message: Cannot access backing file '/var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce' of storage file '/rhev/data-center/mnt/glusterSD/aws-gfs-01.awesome.lan:_gv0__he/2607c265-248c-40ad-b020-f3756454839e/images/816ac00f-ba98-4827-b5c8-42a8ba496089/8ecfcd5b-db67-4c23-9869-0e20d7553aba' (as uid:107, gid:107): No such file or directory. 2018-05-24 13:00:33,437+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-9) [] add VM '1f0b608f-7cfc-4b27-a876-b5d8073011a1'(instance-00000673) to rerun treatment 2018-05-24 13:00:33,455+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-653732) [] EVENT_ID: USER_INITIATED_RUN_VM_FAILED(151), Failed to run VM instance-00000673 on Host aws-ovhv-06. 2018-05-24 13:00:33,460+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-653732) [] EVENT_ID: USER_FAILED_RUN_VM(54), Failed to run VM instance-00000673 (User: admin@internal-authz). Checking on the Gluster volume, directory and files exist, permissions are in order: [root@aws-ovhv-01 816ac00f-ba98-4827-b5c8-42a8ba496089] -rw-rw----. 1 vdsm kvm 14G May 24 12:59 8ecfcd5b-db67-4c23-9869-0e20d7553aba -rw-rw----. 1 vdsm kvm 1.0M May 24 12:49 8ecfcd5b-db67-4c23-9869-0e20d7553aba.lease -rw-r--r--. 1 vdsm kvm 310 May 24 12:49 8ecfcd5b-db67-4c23-9869-0e20d7553aba.meta Than I have checked image info, and noticed that backing file entry is pointing to non-existing location, which does and should not exist on oVirt hosts: [root@aws-ovhv-01 816ac00f-ba98-4827-b5c8-42a8ba496089]# qemu-img info 8ecfcd5b-db67-4c23-9869-0e20d7553aba image: 8ecfcd5b-db67-4c23-9869-0e20d7553aba file format: qcow2 virtual size: 160G (171798691840 bytes) disk size: 14G cluster_size: 65536 backing file: /var/lib/nova/instances/_base/2f4f8c5fc11bb83bcab03f4c829ddda4da8c0bce Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false Can somebody advise me how to fix, address this, as I am in need of importing 200+ VMs from OpenStack to oVirt? Kindly awaiting your reply. Marko Vrgotic

7 36

Change host names/IPs
by Davide Ferrari 15 May '19

15 May '19

Hello Is there a clean way and possibly without downtime to change the hostname and IP addresses of all the hosts in a running oVirt cluster? -- Davide Ferrari Senior Systems Engineer

4 7

CVE-2018-12130
by Ladislav Humenik 15 May '19

15 May '19

Hello, Red Hat has yesterday published a new flaw in CPU micro-architecture, and i already saw there is need to update vdsm. Will you backport the update also for ovirt community and at least for the latest 4.2.x version too? links to CVE: https://access.redhat.com/security/cve/cve-2018-12130 https://access.redhat.com/security/vulnerabilities/mds -- Kind regards Ladislav Humenik

1 0

ovirt 4.3.3 SELINUX issue
by Strahil Nikolov 14 May '19

14 May '19

Hello All, I want to warn you that selinux-policy & selinux-policy-targeted with version '3.13.1-229.el7_6.12' cause an issue with my HostedEngine where I got "Login incorrect" screen of death. I have also raised a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1710083 If you want to test your luck, increase the default grub timeout in "/etc/default/grub" to 15 and rebuild the grub menu via 'grub2-mkconfig -o /boot/grub2/grub.cfg' If the issue occurs to you - just append 'enforcing=0' to your kernel and the issue will be over.Of course, you can always rollback either from a rescue DVD or from the running 'enforcing=0' system. Best Regards,Strahil Nikolov

1 0

oVirt and Windows NFS
by ron＠prgstudios.com 14 May '19

14 May '19

Hi Everyone - Greetings from Detroit Michigan! Has anyone had any luck with using oVirt to connect to a Windows Server running an NFS service? I've made several attempts so far and am having problems getting the two to work together. I was hoping someone had some hints (besides the obvious "use Linux" bit). Thanks! Ron

3 2

Is teamd supported on 4.3?
by Valentin Bajrami 14 May '19

14 May '19

Hello guys, Next week, I'm planning to deploy ovirt-node 4.3 on a few hosts. I've been running bonds for the past years but I'd like to know if teaming (teamd) is also supported with this version. My current package version(s): OS Version: | RHEL - 7 - 6.1810.2.el7.centos OS Description: | oVirt Node 4.3.1 Kernel Version: | 3.10.0 - 957.5.1.el7.x86_64 KVM Version: | 2.12.0 - 18.el7_6.3.1 LIBVIRT Version: | libvirt-4.5.0-10.el7_6.4 VDSM Version: | vdsm-4.30.9-1.el7 SPICE Version: | 0.14.0 - 6.el7_6.1 GlusterFS Version: | glusterfs-5.3-2.el7 CEPH Version: | librbd1-10.2.5-4.el7 Open vSwitch Version: | openvswitch-2.10.1-3.el7 Kernel Features: | PTI: 1, IBRS: 0, RETP: 1 VNC Encryption: | Disabled Is anyone running teamd on this version ? Thanks in advance -- Kind regards / Met vriendelijke groet, Valentin Bajrami Target Holding

2 4

botched 3.6 -> 4.0/1/2 upgrade, how to recover
by Axel.Thimm＠01lgc.com 14 May '19

14 May '19

Hello, an old 3.6 self-hosted one-system installation was half upgraded to 4.0 and I took on the task to continue upgrading it. I managed to setup a new engine and upgraded step by step until 4.2 where the host needs to be upgraded. During that time the host was "non-responsive", but I though that this had to do with the upgrade process. Upon trying to upgrade the host the setup broke to pieces. I tried to downgrade everything and restart the old engine to no avail. I now try the way forward to try to start the new engine or create a complete new one, I don;t care much about the old engine's inherited information - all I need to to resurrect the VMs again. At the end of the day the setup is supposed to migrate to a three system hyperconverged setup anyway. How should I proceed to get a working state that fires up the VMs? Is it safe to install a hosted engine from scratch and reattach the domain? Probably the VM configuration will be lost and I will have to puzzle together the disks. I still have the 3.6 engine's backup file, if that is of any help. Should I perhaps recreate a 4.0 engine with that file and try to continue from there? I guess all information on which VMs are attached to which images etc are in the DB of the engine, so I either have to get this info off the 3.6 backup or wire them up again by inspection. BTW this is not a pure ovirt, but an RHV with self-evaluation support, but I believe that any pure ovirt solution will apply. Many thanks in advance.

5 10

Re: Is teamd supported on 4.3?
by Strahil 14 May '19

14 May '19

I'm using teaming and I don't see issues. Just cannot control the teaming device. Best Regards, Strahil NikolovOn May 13, 2019 22:26, Dominik Holler <dholler(a)redhat.com> wrote: > > On Mon, 13 May 2019 15:30:11 +0200 > Valentin Bajrami <valentin.bajrami(a)target-holding.nl> wrote: > > > Hello guys, > > > > Next week, I'm planning to deploy ovirt-node 4.3 on a few hosts. I've > > been running bonds for the past years but I'd like to know if teaming > > (teamd) is also supported with this version. > > > > No, unfortunately not. > May I ask why you want to use teaming instead of bonding? > > > > My current package version(s): > > > > OS Version: | RHEL - 7 - 6.1810.2.el7.centos > > OS Description: | oVirt Node 4.3.1 > > Kernel Version: | 3.10.0 - 957.5.1.el7.x86_64 > > KVM Version: | 2.12.0 - 18.el7_6.3.1 > > LIBVIRT Version: | libvirt-4.5.0-10.el7_6.4 > > VDSM Version: | vdsm-4.30.9-1.el7 > > SPICE Version: | 0.14.0 - 6.el7_6.1 > > GlusterFS Version: | glusterfs-5.3-2.el7 > > CEPH Version: | librbd1-10.2.5-4.el7 > > Open vSwitch Version: | openvswitch-2.10.1-3.el7 > > Kernel Features: | PTI: 1, IBRS: 0, RETP: 1 > > VNC Encryption: | Disabled > > > > Is anyone running teamd on this version ? > > > > Thanks in advance > > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/XVFQQWUJK3QTV…

3 2

Ovirt Node transfer to New Engine (it is possible)
by IT INFRA 14 May '19

14 May '19

Hi, It is possible to transfer my node to other ovirt engine (new 4.3) here's the scenario. Old Server: (no internet prod) Engine (ovirt 4.1) - Hardware failure No possible recovery Node1 (node 4.1) Node2 (node 4.1) New Server Engine (Ovirt 4.3) *note have full backup of old engine 4.1 version\ * can't re install using yum install http://resources.ovirt.org/pub/yum-repo/ovirt-release41.rpm (no more link available) any possible solution on this.. Thanks,

1 1

A "memory" issue when trying to start VM on oVirt 4.3
by Vrgotic, Marko 14 May '19

14 May '19

Dear oVirt, We are running 4.3.3 oVirt with SHE on three CentOS nodes for HA – with NFS storage. We also have two Local Storage DCs managed by same SHE. [cid:image001.png@01D50A59.F4276FD0] The issue we are reporting was present on 4.3.2 as well. The memory usage on both hosts (third one is in maintenance mode) does not go above 45%: [cid:image002.png@01D50A59.F4276FD0] I have also attached the UI Dashboard screenshot. However, when trying to start one VM: [cid:image003.png@01D50A59.F4276FD0] Engine reports that there is not enough memory and writes the following in the engine.log: 2019-05-14 11:23:10,809Z INFO [org.ovirt.engine.core.bll.RunVmCommand] (default task-14) [63aa9817-80fd-4b18-8c86-26c910a2954a] Lock Acquired to object 'EngineLock:{exclusiveLocks='[13b2299f-b4b4-48e4-b2f9-09450eff235d=VM]', sharedLocks=''}' 2019-05-14 11:23:10,819Z INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-14) [63aa9817-80fd-4b18-8c86-26c910a2954a] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='13b2299f-b4b4-48e4-b2f9-09450eff235d'}), log id: 5f80129f 2019-05-14 11:23:10,819Z INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-14) [63aa9817-80fd-4b18-8c86-26c910a2954a] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5f80129f 2019-05-14 11:23:10,833Z INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-14) [63aa9817-80fd-4b18-8c86-26c910a2954a] Candidate host 'ovirt-staging-hv-01' ('d81b296a-f3b2-49ea-9b4d-04c90f3508e9') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'Memory' (correlation id: null) 2019-05-14 11:23:10,833Z INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (default task-14) [63aa9817-80fd-4b18-8c86-26c910a2954a] Candidate host 'ovirt-staging-hv-02' ('f4fdeccd-4cf4-4ed7-9eb1-d98ac65c4ce6') was filtered out by 'VAR__FILTERTYPE__INTERNAL' filter 'Memory' (correlation id: null) 2019-05-14 11:23:10,838Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-14) [63aa9817-80fd-4b18-8c86-26c910a2954a] EVENT_ID: USER_FAILED_RUN_VM(54), Failed to run VM csm2-216-d due to a failed validation: [Cannot run VM. There is no host that satisfies current scheduling constraints. See below for details:, The host ovirt-staging-hv-01 did not satisfy internal filter Memory because its available memory is too low (0 MB) to run the VM., The host ovirt-staging-hv-01 did not satisfy internal filter Memory because its available memory is too low (0 MB) to run the VM.] (User: admin@internal-authz). 2019-05-14 11:23:10,838Z WARN [org.ovirt.engine.core.bll.RunVmCommand] (default task-14) [63aa9817-80fd-4b18-8c86-26c910a2954a] Validation of action 'RunVm' failed for user admin@internal-authz. Reasons: VAR__ACTION__RUN,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-01,$filterName Memory,$availableMem 0,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-02,$filterName Memory,$availableMem 0,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL 2019-05-14 11:23:10,838Z INFO [org.ovirt.engine.core.bll.RunVmCommand] (default task-14) [63aa9817-80fd-4b18-8c86-26c910a2954a] Lock freed to object 'EngineLock:{exclusiveLocks='[13b2299f-b4b4-48e4-b2f9-09450eff235d=VM]', sharedLocks=''}' Current memory optimization is set to “None” Please let me know if you need additional log files. Kindly awaiting your reply. Marko Vrgotic Sr. System Engineer

1 0

Re: oVirt and Windows NFS
by Strahil 14 May '19

14 May '19

Get a vanilla CentOS7 and try to access the shares to Windows. Is windows in AD ? Best Regards, Strahil NikolovOn May 14, 2019 01:31, ron(a)prgstudios.com wrote: > > Hi Everyone - Greetings from Detroit Michigan! > > Has anyone had any luck with using oVirt to connect to a Windows Server running an NFS service? > > I've made several attempts so far and am having problems getting the two to work together. I was hoping someone had some hints (besides the obvious "use Linux" bit). > > Thanks! > > Ron > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/BH4KT3ODZYEGT…

1 0

Re: Gluster volume heals and after 5 seconds has /dom_md/ids dirty again
by Strahil 14 May '19

14 May '19

I have just noticed this behaviour. I stopped and redesigned my brick (addded vdo layer) and after the heal (1000+ entries got sync) the dom_ids need healing. Running 'gluster volume heal volume_name full' heals the file and 1 min later it happens again. I will try to do a rolling reboot and see if the issues got fixes. Best Regards, Strahil NikolovOn May 13, 2019 18:14, Darrell Budic <budic(a)onholyground.com> wrote: > > I see this sometimes after rebooting a server, and it usually stops happening, generally within a few hours, I’ve never tracked it down further. Don’t know for sure, but I assume it’s related to healing and goes away once everything syncs up. > > Occasionally it turns out to be a communications problem between servers (usually an update to something screws up my firewall), so I always check my peer status when I see it and make sure all servers are talking to each other. > > > On May 13, 2019, at 4:13 AM, Andreas Elvers <andreas.elvers+ovirtforum(a)solutions.work> wrote: > > > > I restored my engine to a gluster volume named :/engine on a three node hyperconverged oVirt 4.3.3.1 cluster. Before restoring I was checking the status of the volumes. They were clean. No heal entries. All peers connected. gluster volume status looked good. Then I restored. This went well. The engine is up. But the engine gluster volume shows entries on node02 and node03. The engine was installed to node01. I have to deploy the engine to the other two hosts to reach full HA, but I bet maintenance is not possible until the volume is healed. > > > > I tried "gluster volume heal engine" also with added "full". The heal entries will disappear for a few seconds and then /dom_md/ids will pop up again. The __DIRECT_IO_TEST__ will join later. The split-brain info has no entries. Is this some kind of hidden split brain? Maybe there is data on node01 brick which got not synced to the other two nodes? I can only speculate. Gluster docs say: this should heal. But it doesn't. I have two other volumes. Those are fine. One of them containing 3 VMs that are running. I also tried to shut down the engine, so no-one was using the volume. Then heal. Same effect. Those two files will always show up. But none other. Heal can always be started successfully from any of the participating nodes. > > > > Reset the volume bricks one by one and cross fingers? > > > > [root@node03 ~]# gluster volume heal engine info > > Brick node01.infra.solutions.work:/gluster_bricks/engine/engine > > Status: Connected > > Number of entries: 0 > > > > Brick node02.infra.solutions.work:/gluster_bricks/engine/engine > > /9f4d5ae9-e01d-4b73-8b6d-e349279e9782/dom_md/ids > > /__DIRECT_IO_TEST__ > > Status: Connected > > Number of entries: 2 > > > > Brick node03.infra.solutions.work:/gluster_bricks/engine/engine > > /9f4d5ae9-e01d-4b73-8b6d-e349279e9782/dom_md/ids > > /__DIRECT_IO_TEST__ > > Status: Connected > > Number of entries: 2 > > _______________________________________________ > > Users mailing list -- users(a)ovirt.org > > To unsubscribe send an email to users-leave(a)ovirt.org > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/L3YCRPRAGPUMB… > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6XOCRXRCQOUKE…

1 0

Gluster volume heals and after 5 seconds has /dom_md/ids dirty again
by Andreas Elvers 14 May '19

14 May '19

I restored my engine to a gluster volume named :/engine on a three node hyperconverged oVirt 4.3.3.1 cluster. Before restoring I was checking the status of the volumes. They were clean. No heal entries. All peers connected. gluster volume status looked good. Then I restored. This went well. The engine is up. But the engine gluster volume shows entries on node02 and node03. The engine was installed to node01. I have to deploy the engine to the other two hosts to reach full HA, but I bet maintenance is not possible until the volume is healed. I tried "gluster volume heal engine" also with added "full". The heal entries will disappear for a few seconds and then /dom_md/ids will pop up again. The __DIRECT_IO_TEST__ will join later. The split-brain info has no entries. Is this some kind of hidden split brain? Maybe there is data on node01 brick which got not synced to the other two nodes? I can only speculate. Gluster docs say: this should heal. But it doesn't. I have two other volumes. Those are fine. One of them containing 3 VMs that are running. I also tried to shut down the engine, so no-one was using the volume. Then heal. Same effect. Those two files will always show up. But none other. Heal can always be started successfully from any of the participating nodes. Reset the volume bricks one by one and cross fingers? [root@node03 ~]# gluster volume heal engine info Brick node01.infra.solutions.work:/gluster_bricks/engine/engine Status: Connected Number of entries: 0 Brick node02.infra.solutions.work:/gluster_bricks/engine/engine /9f4d5ae9-e01d-4b73-8b6d-e349279e9782/dom_md/ids /__DIRECT_IO_TEST__ Status: Connected Number of entries: 2 Brick node03.infra.solutions.work:/gluster_bricks/engine/engine /9f4d5ae9-e01d-4b73-8b6d-e349279e9782/dom_md/ids /__DIRECT_IO_TEST__ Status: Connected Number of entries: 2

3 11

Re: oVirt Open Source Backup solution?
by Strahil 14 May '19

14 May '19

Another option is to create a snapshot, backup the snapahot and merge the disks (delete the snapshot actually). Sadly that option doesn't work with Databases, as you might inyerrupt a transaction and leave the DB in inconsistent state. Best Regards, Strahil Nikolov On May 10, 2019 15:37, Derek Atkins <derek(a)ihtfp.com> wrote: > > Hi, > > Michael Blanchard <michael(a)wanderingmad.com> writes: > > > If you haven't seen my other posts, I'm not a very experienced Linux admin, so > > I'm trying to make it as easy as possible to run and maintain. It's hard > > enough for me to not break ovirt in crazy ways > > This has nothing to do with ovirt. > > You could use rdiff-backup on any running machine, be it virtual or bare > metal. It's just a way to use a combination of diff and rsync to backup > machines. Indeed, I was using it with my vmware-based systems and, when > I migrated them to ovirt, the backups just continued working. > > > Get Outlook for Android > > -derek > -- > Derek Atkins 617-623-3745 > derek(a)ihtfp.com www.ihtfp.com > Computer and Internet Security Consultant > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/CBMHSCOBQ3MQH…

6 6

New to OVirt
by Slobodan Stevanovic 13 May '19

13 May '19

I am having hard time installing OVIrt. Gluster deployment for the single node sometimes work, but host engine deployment never works. Any one has suggestion on which release of OVirt to use as a beginner.

6 10

Permission Problem for admin portal
by s.gay＠ovea.com 13 May '19

13 May '19

Hello, I'm using the lastest ovirt release. I would like to add a cluster permission for a user. This user should only see this cluster and not other information or other cluster I've create a new user in internal or external When the ClusterAdmin role is added in the permission tab of a cluster, the user can access the entire infrastructure through the user portal or administrator portal. It can modify other VMs, ... I checked the other cluster and the user does not appear in the permissions When I add the UserRole role to the cluster and remove the ClusterAdmin role, it sees only the VMs in the cluster in the user portal When the user creating a VM, he can see all the cluster in the drop-down window for cluster selection It's possible to add only a cluster permission for users ? Thanks

3 2

HostedEngine migration from ovirt1:/engine to gluster1:/engine (new ip)
by Strahil Nikolov 13 May '19

13 May '19

Hello Community, I have added new interfaces and bonded them in order to split storage from oVirt traffic and I have one last issue. In Storage -> Storage Domains , I have the "hosted_storage" domain that is pointing to the old "ovirt1.localdomain:/engine" instead of "gluster1:/engine". I have managed to reconfigure the ha agent to bring the new storage , but it seems the engine mounts the old gluster path and this causes problem with the ha agent . How can I edit the "hosted_storage" in a safe manner to point to "gluster1:/engine" and mount options of "backup-volfile-servers=gluster2:ovirt3". Should I edit the DB ? P.S.: My google skills did not show any results on this topic and thus I'm raising it to the mail list.Thanks in advance. Best Regards,Strahil Nikolov

2 2

Upgrade success story (4.3.0->4.3.3)
by Juhani Rautiainen 13 May '19

13 May '19

Hi! Just wanted to let you know that I succesfully upgrade our cluster to 4.3.3 (FC SAN based system and no Gluster). I postponed upgrade a lot because of the difficulties I had when upgrading from 4.2.8->4.3.0. This time Hosted Engine upgrade went without any problems. The first node also upgraded itself from admin UI without any problems. The second node was different story. I couldn't upgrade it because yum complained about missing packages. This was mysterious as I just had upgraded the other node. I bypassed this by taking ovirt-4.3 repos from other node since diffing files gave results that they differed. After that node upgraded from UI but went missing after reboot. Contacting it via ILO I found out that it had turned to DHCP again (which it never should do). Somehow settings in VDSM persistent settings were still wrong and pointing to DHCP (it has always been in fixed address). Fixed this by hand by using vdsm persistent settings from the running node. Now the node reboots reliably. Anyway thanks for all the good work oVirt people put in. Small problems here and there but it's getting better by the release. -Juhani

1 0

Node losing management network address?
by Juhani Rautiainen 13 May '19

13 May '19

Hi! I had weird occurence in my two node ovirt cluster today (I have HE). I noticed that one node had ovirtmgmt network unsynchronized. I tried to resynchronize it. This led the node being rebooted by HP ILO. After reboot the node came up with DHCP address. Tried to change it back by fixing ifcfg-ovirtmgmt to original static address. It reverter back to DHCP if I tried the resync the network. I decided to remove HE from the node in order to remove the node in order to be able to add it back. After I started HE removal, address popped back to static address. I did upgrade which was pending on node and after reboot it came back with DHCP address again. After this I removed the node from the cluster, added it back and now it seems to work. I'm just wondering how I prevent this from happening again? How this unsync situation happens for instance and why it decides that DHCP is the way to go? -Juhani

5 11

VMs import over slow 1gig interface instead of fast 10gig interface?
by Jacob Green 13 May '19

13 May '19

Ok, so I want to try and more thoroughly explain my situation. I want to better understand what is happening, and if there is a way to force import over the faster connection. Today I have two Ovirt environments, one on 4.1 and one on 4.2, on the new environment we have migrated to new VLANs for Management and Data. The export domain is a NFS domain hosted on a Trunas device that has an ip address of X.X.12.100. Our host has multiple Ethernet connections, one Bonded 20Gigabit interface. That is where we have the ovirtmgmt profile. And a secondary 1 Gigabit interface that we call our VLAN12 profile or server VLAN. *The problem.... *When importing a VM from the export domain, the import is happening over the 1Gigabit interface on the VLAN12 profile. Instead of the 20gigabit Bonded interface on VLAN 100. *My understanding.... *So I probably just misunderstand how this works, but here is what I thought should happen, in the *Setup Host Network* screen for this host I have the ovirtmgmt on VLAN100 with the bonded 20Gigabit interface performing the following. "*Management, Display, Migration, VM and Default Route*." Then I have the secondary 1 Gigabit interface just doing "*VM*" on VLAN12. What I thought should happen is that anything Migration related "*/Includes importing from export domain?/*" should happen on the Migration interface, which in my case is the ovritmgmt interface on VLAN100. I have also confirmed that I can ping from the VLAN100 ovirtmgmt interface to the storage on VLAN12. So there is connectivity from that interface to the storage. What I am trying to figure out is there a way to force it to do imports over the 20Gigabit interface that resides on VLAN100 instead of the slower VLAN12 interface, that is just there for general VM connectivity. Thank you for any advice or guidance on this matter. -- Jacob Green Systems Admin American Alloy Steel

7 9

Re: OvfUpdateIntervalInMinutes restored to original value too early?
by Andreas Elvers 12 May '19

12 May '19

I suppose the /var/log/ovirt-hosted-engine-setup/engine-logs-2019-05-10T16:16:14Z/ovirt-engine/engine.log is the bootstrap engine log. It is attached. The bootstrap engine looks quite ok when glancing through the dashboard, datacenter, storage domains,… Although there are two old hosted storage domains.

1 1

Upgrade engine to 4.3.3 or setup a new engine
by Magnus Isaksson 12 May '19

12 May '19

Hello all I'm having trouble upgrading my engine to 4.3.3 from 4.2.8 The upgrade goes well, no errors but webgui wont load, it just says "waiting for engine" I have tried to set up a new machine with engine and import the DB from the old(after upgrade) still the same problem. So after quite a few tries i gave up and set up a fresh new engine 4.3.3 But can i move/migrate the hosts with running VM:s in to the new engine? In vmware vcenter it is just to add the host and all VM:s and stuff comes over, but im guessing from what ive read that it is not that simple with ovirt. Or how should i solve this? //Magnus

2 1

HostedEngine cleaned up
by Sakhi Hadebe 12 May '19

12 May '19

Hi, We have a situation where the HostedEngine was cleaned up and the VMs are no longer running. Looking at the logs we can see the drive files as: 2019-03-26T07:42:46.915838Z qemu-kvm: -drive file=/rhev/data-center/mnt/glusterSD/glustermount.goku:_vmstore/9f8ef3f6-53f2-4b02-8a6b-e171b000b420/images/b2b872cd-b468-4f14-ae20-555ed823e84b/76ed4113-51b6-44fd-a3cd-3bd64bf93685,format=qcow2,if=none,id=drive-ua-b2b872cd-b468-4f14-ae20-555ed823e84b,serial=b2b872cd-b468-4f14-ae20-555ed823e84b,werror=stop,rerror=stop,cache=none,aio=native: 'serial' is deprecated, please use the corresponding option of '-device' instead I assume this is the disk was writing to before it went down. Trying to list the file gives an error and the file is not there; ls -l /rhev/data-center/mnt/glusterSD/glustermount.goku:_vmstore/9f8ef3f6-53f2-4b02-8a6b-e171b000b420/images/b2b872cd-b468-4f14-ae20-555ed823e84b/76ed4113-51b6-44fd-a3cd-3bd64bf93685 Is there a way we can recover the VM's disk images ? NOTE: No HostedEngine backups -- Regards, Sakhi Hadebe

7 13

engine-setup failed in upgrade step to 4.3.2 with DNF errors on Centos 7.6
by Oliver Riesener 12 May '19

12 May '19

# engine-setup ``` [ INFO ] Stage: Initializing [ INFO ] Stage: Environment setup Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-aio.conf', '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'] Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20190329154037-huchep.log Version: otopi-1.8.1 (otopi-1.8.1-1.el7) [ INFO ] DNF Downloading 1 files, 0.00KB [ INFO ] DNF Downloaded Extra Packages for Enterprise Linux 7 - x86_64 [ INFO ] Stage: Environment packages setup [ INFO ] Stage: Programs detection [ INFO ] Stage: Environment setup (late) [ INFO ] Stage: Environment customization --== PRODUCT OPTIONS ==-- Set up Cinderlib integration (Currently in tech preview) (Yes, No) [No]: [ INFO ] ovirt-provider-ovn already installed, skipping. --== PACKAGES ==-- [ INFO ] Checking for product updates... [ ERROR ] DNF 'Base' object has no attribute '_group_persistor' [ INFO ] DNF Performing DNF transaction rollback [ ERROR ] Failed to execute stage 'Environment customization': 'Base' object has no attribute '_group_persistor' [ INFO ] Stage: Clean up Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20190329154037-huchep.log [ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20190329154107-setup.conf' [ INFO ] Stage: Pre-termination [ INFO ] Stage: Termination [ ERROR ] Execution of setup failed What can i do ? ```

2 2

[Users] Lifecycle / upgradepath
by Sven Kieske 11 May '19

11 May '19

Hi Community, Currently, there is no single document describing supported (which means: working ) upgrade scenarios. I think the project has matured enough, to have such an supported upgradepath, which should be considered in the development of new releases. As far as I know, currently it is supported to upgrade from x.y.z to x.y.z+1 and from x.y.z to x.y+1.z but not from x.y-1.z to x.y+1.z directly. maybe this should be put together in a wiki page at least. also it would be cool to know how long a single "release" would be supported. In this context I would define a release as a version bump from x.y.z to x.y+1.z or to x+1.y.z a bump in z would be a bugfix release. The question is, how long will we get bugfix releases for a given version? What are your thoughts? -- Mit freundlichen Grüßen / Regards Sven Kieske Systemadministrator Mittwald CM Service GmbH & Co. KG Königsberger Straße 6 32339 Espelkamp T: +49-5772-293-100 F: +49-5772-293-333 https://www.mittwald.de Geschäftsführer: Robert Meyer St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen

2 1