- Users - oVirt List Archives

Inconsisten datacenter [urgent]
by jplorier＠gmail.com 08 Oct '19

08 Oct '19

I have a gluster engine domain on 4.2 replica 3 1 arbiter and the arbiter got down. I've been trying to replace the arbiter with no success. In the meantime, the two remaining hosts are non- responsive, both with one vm (one actually with the engine). The one without the engine can't be set to maintenance because shows one vm and in non-responsive state, but in the vm list, it shows none. I manually tried to shut the vm down with virsh but no domain is listed. I can't power off the host as it's part of the gluster. Is there a way I can manually remove the ghost vm so I can set the host to maintenance and try to fix it? Regards,

3 2

update node 4.3.5 to 4.3.6
by Vincent Royer 08 Oct '19

08 Oct '19

Ovirt node on 4.3.5. Was installed from 4.3.5 iso and has never been updated GUI said update available for hosts. Here's what I did: - Global HA maintenance - yum update and engine-upgrade-check engine-setup of hosted engine - engine now at Version 4.3.6.7-1.el7 - put node 1/3 into maintenance (vms migrate off) - update host in gui until it says its done - reboot node The node reboots but is still at 4.3.5 So I try manually: yum update > Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, > package_upload, product-id, search-disabled-repos, subscription-manager, > vdsmupgrade, versionlock > This system is not registered with an entitlement server. You can use > subscription-manager to register. > Loading mirror speeds from cached hostfile > * ovirt-4.3: mirrors.rit.edu > * ovirt-4.3-epel: d2lzkl7pfhq30w.cloudfront.net > Excluding 70 updates due to versionlock (use "yum versionlock status" to > show them) > No packages marked for update > Uploading Enabled Repositories Report > Cannot upload enabled repos report, is this client registered? > [root@letty ~]# in the etc/yum.repos.d folder I have: CentOS-Base.repo CentOS-CR.repo CentOS-Debuginfo.repo CentOS-fasttrack.repo CentOS-Media.repo CentOS-Sources.repo CentOS-Vault.repo ovirt-4.3-dependencies.repo ovirt-4.3.repo So how do I update to 4.3.6? isn't it part of ovirt-4.3.repo?

1 0

Delete snapshots task hung
by Leo David 08 Oct '19

08 Oct '19

Hi Everyone, I'm waiting since 3 days for 5 x delete snapshot tasks to finish, and for some reason it seems to be stucked.For other vms snapshot removal took at most 20 mins, with havin the disks pretty much same size, and snapshots numbers. Any thoughts on how should I get this fixed ? Below, some lines from the engine.log, and it seems to show some complains regarding locks ( Failed to acquire lock and wait lock) , although I am not sure if thats the root cause: Thank you very much ! Leo 2019-10-08 09:52:48,692+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '341d9c1b-2915-48d6-a8a9-9146ab19d5f8') waiting on child command id: '329da0fd-801b-4e0d-b7c0-fbb5c2a98bb5' type:'DestroyImage' to complete 2019-10-08 09:52:48,702+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '580fa033-35fd-44f0-9979-e60e9bbf8a29') waiting on child command id: 'c00bdeb6-2e8b-4ef8-a3dc-1aaa088ae052' type:'DestroyImage' to complete 2019-10-08 09:52:49,713+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [539ba19e-0cb5-42cf-9a23-7916ee2de4a9] Command 'RemoveSnapshotSingleDiskLive' (id: 'de747f91-ec59-4e70-9345-77e16234bfe0') waiting on child command id: '10812160-cf4c-4239-bb92-1d5a847687ee' type:'DestroyImage' to complete 2019-10-08 09:52:50,725+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [baed2fa3-bcad-43b2-8164-480598bc72f3] Command 'RemoveSnapshotSingleDiskLive' (id: '4919b287-e980-4d34-a219-c08a169cd8f7') waiting on child command id: '5eceb6a8-f08e-42aa-8258-c907f5927e6c' type:'DestroyImage' to complete 2019-10-08 09:52:51,563+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:51,583+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:51,604+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:51,606+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:51,735+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshot' (id: 'c9ab1344-ae27-4934-9358-d6a7b10a4f0a') waiting on child command id: '341d9c1b-2915-48d6-a8a9-9146ab19d5f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:52,706+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@21830b5f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@676adc3e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@385a3510, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@af24d00, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@331266f2], log id: 1a3515fe 2019-10-08 09:52:52,708+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-2.domain.int, VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}), log id: 2a8c782 2019-10-08 09:52:52,745+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [539ba19e-0cb5-42cf-9a23-7916ee2de4a9] Command 'RemoveSnapshot' (id: '28ac73f3-8b07-43d4-8b4b-1df90831ed50') waiting on child command id: 'de747f91-ec59-4e70-9345-77e16234bfe0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:53,115+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 2a8c782 2019-10-08 09:52:53,116+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = host-dell-1.domain.int, GlusterVolumeAdvancedDetailsVDSParameters:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b', volumeName='samsung_sata'}), log id: 6bb38072 2019-10-08 09:52:54,659+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:54,662+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:54,663+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:54,679+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:54,755+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [4f2bedf7-b9b3-481e-936f-e180803ac1b8] Command 'RemoveSnapshot' (id: 'ceeb99c3-68be-4e0f-8dd5-3ef6509d4013') waiting on child command id: '66094952-88e3-41d9-9191-7ef8873de511' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:54,762+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [825c211d-b40d-45d7-9dc7-e5c6af28a269] Command 'RemoveSnapshotSingleDiskLive' (id: '5e9f9b03-4df9-4e31-b2e1-5efaa4fcf66c') waiting on child command id: 'b3b0a892-e652-45dd-a71f-b2f77a942491' type:'DestroyImage' to complete 2019-10-08 09:52:55,276+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@db5a3b94, log id: 6bb38072 2019-10-08 09:52:55,288+03 INFO [org.ovirt.engine.core.bll.gluster.GlusterThinDeviceService] (DefaultQuartzScheduler10) [8921c9c] Volume samsung_sata have non-thin bricks, skipping confirmed free size calculation 2019-10-08 09:52:55,295+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:55,308+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = host-dell-3.domain.int, VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}), log id: cc40e52 2019-10-08 09:52:55,376+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@723faf2a, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@faa8525, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@17581efa, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3bc224dd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@340b8a70, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6fd3a279, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@382bceb7, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1265e1a3, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7a55a195, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@86714cc, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6e5f3d01, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@538aa190, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@484eb7d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@34754e55, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2355daef], log id: cc40e52 2019-10-08 09:52:55,378+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = host-dell-3.domain.int, VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}), log id: 25557934 2019-10-08 09:52:55,436+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@629b1e0, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@18305096, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@153f73dd], log id: 25557934 2019-10-08 09:52:55,438+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-3.domain.int, VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}), log id: 1bad2497 2019-10-08 09:52:55,740+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 1bad2497 2019-10-08 09:52:55,742+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = host-dell-1.domain.int, VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}), log id: 504020b3 2019-10-08 09:52:55,890+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4a6e6ecd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@24a15dc6, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@56da4c59, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@74eab89, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3e6fb5ab, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@100e90cd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@14b2ae11, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@66d25ded, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@25537e71, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@41bf0aee, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@40fce6ee, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3435f629, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@a2ba628, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@337a25f9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@14967eee, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@f8946eb, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5844abd4], log id: 504020b3 2019-10-08 09:52:55,891+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = host-dell-1.domain.int, VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}), log id: 3af318c8 2019-10-08 09:52:56,620+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler2) [5f2fb2c6] START, GlusterTasksListVDSCommand(HostName = host-dell-1.domain.int, VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}), log id: 4dd7bc07 2019-10-08 09:52:56,681+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@1d356076, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@584ec325, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@2529c9f3, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@73eb0c6, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@31be4687], log id: 3af318c8 2019-10-08 09:52:56,682+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-1.domain.int, VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}), log id: 1f47fc7e 2019-10-08 09:52:56,774+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [825c211d-b40d-45d7-9dc7-e5c6af28a269] Command 'RemoveSnapshotSingleDiskLive' (id: '1593ccd5-4e2a-4f8c-9c10-7944b95836b3') waiting on child command id: '8b017634-d965-4273-a4a4-6281a337ed94' type:'DestroyImage' to complete 2019-10-08 09:52:57,095+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 1f47fc7e 2019-10-08 09:52:57,096+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = host-dell-4.domain.int, VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}), log id: 74cad834 2019-10-08 09:52:57,144+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@600eeeb, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@45491ad2, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@654daa09, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4764d5ec, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@46121c5d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4b0001c7, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4315e74c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@22bf8177, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@74999189, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3f5ca8e4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@531f17c4], log id: 74cad834 2019-10-08 09:52:57,145+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = host-dell-4.domain.int, VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}), log id: df580e9 2019-10-08 09:52:57,191+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@35585d07], log id: df580e9 2019-10-08 09:52:57,192+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-4.domain.int, VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}), log id: 6f19c8c0 2019-10-08 09:52:57,198+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler2) [5f2fb2c6] FINISH, GlusterTasksListVDSCommand, return: [], log id: 4dd7bc07 2019-10-08 09:52:57,432+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 6f19c8c0 2019-10-08 09:52:57,433+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = host-dell-2.domain.int, VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}), log id: 449353d0 2019-10-08 09:52:57,576+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:57,594+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:57,616+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:57,619+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:57,785+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [4f2bedf7-b9b3-481e-936f-e180803ac1b8] Command 'RemoveSnapshotSingleDiskLive' (id: '66094952-88e3-41d9-9191-7ef8873de511') waiting on child command id: '122be070-d45f-4e6b-bb61-4943a096b88b' type:'DestroyImage' to complete 2019-10-08 09:52:57,792+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [baed2fa3-bcad-43b2-8164-480598bc72f3] Command 'RemoveSnapshot' (id: '7aafe2fe-e05b-44bc-a716-c48734a8c2de') waiting on child command id: '4919b287-e980-4d34-a219-c08a169cd8f7' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:57,799+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [825c211d-b40d-45d7-9dc7-e5c6af28a269] Command 'RemoveSnapshot' (id: 'cb6e7a1d-1902-4e61-9c3d-e541cf4c6348') waiting on child command id: '5e9f9b03-4df9-4e31-b2e1-5efaa4fcf66c' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:58,345+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@65406163, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1207b8a9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@33f93de1, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6d2ae59d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@b07ca22, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@246ae91b, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5e86ce42, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@643ade02, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@51d95e7d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2288576c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4a9469a1, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@78ec643f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@63dde7e9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@30effc9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6a3e4768], log id: 449353d0 2019-10-08 09:52:58,346+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = host-dell-2.domain.int, VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}), log id: 71680b8c 2019-10-08 09:52:58,807+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '341d9c1b-2915-48d6-a8a9-9146ab19d5f8') waiting on child command id: '329da0fd-801b-4e0d-b7c0-fbb5c2a98bb5' type:'DestroyImage' to complete 2019-10-08 09:52:58,817+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '580fa033-35fd-44f0-9979-e60e9bbf8a29') waiting on child command id: 'c00bdeb6-2e8b-4ef8-a3dc-1aaa088ae052' type:'DestroyImage' to complete -- Best regards, Leo David

2 3

Problem with ovirt 4.3 and systemd-journal-gateway
by Fabrice Bacchella 08 Oct '19

08 Oct '19

I'm forwarding logs using systemd-journal-gateway (https://www.freedesktop.org/software/systemd/man/systemd-journal-gatewayd.s…) But it fails with ovirt 4.3 on CentOS 7.7: lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.7.1908 (Core) Release: 7.7.1908 Codename: Core yum update ... Error: Package: systemd-journal-gateway-219-67.el7_7.1.x86_64 (@updates) Requires: libmicrohttpd.so.10()(64bit) Removing: libmicrohttpd-0.9.33-2.el7.x86_64 (@base) libmicrohttpd.so.10()(64bit) Updated By: 1:libmicrohttpd-0.9.59-2.el7.x86_64 (ovirt-4.3-centos-opstools) ~libmicrohttpd.so.12()(64bit) Is that a known bug ?

1 0

Re: Problem with logical network
by Strahil 08 Oct '19

08 Oct '19

Can you check your hosts and especially their networks. If any network is not OK (out of sync), you can fix it from the UI. Best Regards, Strahil NikolovOn Oct 4, 2019 23:47, Alex Irmel Oviedo Solis <alexove(a)alexove.me> wrote: > > Hello, I'm trying to run a vm with two ethernet interfaces, one of them attached to ovirmnt and the other attached to a logical network named internal_servers and got this error: > > "VM is down with error. Exit message: can not get MTU interface in 'br-int': No such device2 > > Then I created the bridge manually but I get the following error: > > "VM is down with error. Exit message: Hook Error: ('',)." > > Please help me. > > Best Regards >

4 3

[ANN] oVirt Engine 4.3.6 async update is now available
by Sandro Bonazzola 08 Oct '19

08 Oct '19

The oVirt Team has just released a new version of ovirt-engine package that fixes a few high severity bugs: - [BZ 1755869 <https://bugzilla.redhat.com/1755869>] ovirt-engine throws an exception when trying to access VMs which have snapshots from unsupported compatibility levels - [BZ 1757782 <https://bugzilla.redhat.com/1757782>] Deleting a Single snapshot disk live merge 'REDUCE_IMAGE' step fails with NPE and snapshot disk remains in illegal state - [BZ 1758052 <https://bugzilla.redhat.com/1758052>] Create VM rollback fails with NPE leaving VM locked - Exception in invoking callback of command AddVmFromTemplate- Exception: javax.ejb.EJBTransactionRolledbackException We recommend to update ovirt-engine in order to avoid these issues. Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Cannot start a VM after making it headless
by Thorsten Glaser 07 Oct '19

07 Oct '19

Hi everyone, I’ve installed a Windows Server 2016 VM, enabled RDP in it, and now want to make it headless. Upon starting I get a failure; the detail message is: VM 〈name〉 is down with error. Exit message: Nicht unterstützte Konfiguration: graphics device is needed for attribute value 'display=on' in <hostdev>. “Nicht unterstützte Konfiguration” means “unsupported configuration”. Software Version:4.3.5.5-1.el7 This works on another cluster which runs: 4.1.7.6-1.el7.centos Thanks in advance, //mirabilos -- tarent solutions GmbH Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/ Tel: +49 228 54881-393 • Fax: +49 228 54881-235 HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941 Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

1 0

Thin pool best proctices
by jeremy_tourville＠hotmail.com 07 Oct '19

07 Oct '19

After a week of downtime from a thin pool that had 100% full metadata I decided to rebuild my single node. My post was formerly titled: Ovirt 4.2.7 won't start and drops to emergency console. I have backups of nearly all my VMs and I am working on importing them. I went from version 4.2.7.1 to 4.3.6. I am comfortable with LVM but I am new to administering thin pool LVM. I am asking myself what can I do to avoid this headache in the future? What practical steps should I take? -Regular backups -Monitoring -Proper configuration It's the last one that is the most difficult for me. You don't know what you don't know.... Can anyone share tips and advice? I learned that /etc/LVM has backups. It should be part of the backup strategy. I now know that thin pools have a special metadata volume that should be monitored for disk space. At the time I built the system I used the gedploy gluster config file. I had assumed the values in there were "safe". As I found out the hard way, reality proved otherwise. I was reading elsewhere about allowing the thin volume to grow 20% as needed but no real mention of how to do that or a good example. As I understand it, the config to make that happen exists in lvm.conf. Is that right? Have there been any changes in Ansible GlusterFS setup since 4.2.7 to choose different "safe" values with newer versions? I hope my comments won't be taken as critical by any Dev Team members regarding Ansible. My sole intent for this post is so I learn what to do better/right next time. I do have most of my VMs and the few I lost are not critical. It was lucky that I learned a hard lesson the easy way. Thanks for your input!

2 1

oVirt Software after upgrade
by Rick A 07 Oct '19

07 Oct '19

I'm wondering why after upgrading our hosts, when viewing the host details, it's still showing the OS Description as 4.3.3.1. We are using the oVirt Node. Shouldn't the OS Description show the version we just upgraded to? I do the upgrade using the web UI. Thanks OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: oVirt Node 4.3.3.1 <-------------------------------------- Kernel Version: 3.10.0 - 957.10.1.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.3.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.6 VDSM Version: vdsm-4.30.13-1.el7 SPICE Version: 0.14.0 - 6.el7_6.1 GlusterFS Version: glusterfs-5.5-1.el7 CEPH Version: librbd1-10.2.5-4.el7 Open vSwitch Version: openvswitch-2.10.1-3.el7 Kernel Features: PTI: 1, IBRS: 1, RETP: 0, SSBD: 3 VNC Encryption: Disabled

2 1

Re: Managed Block Storage/Ceph: Experiences from Catastrophic Hardware failure
by Strahil 07 Oct '19

07 Oct '19

Hi Dan, As CEPH support is quite new, we need DEV clarification. Hi Sandro, Who can help to clarify if Ovirt supports direct RBD LUNs presented on the VMs? Are there any limitations in the current solution ? Best Regards, Strahil NikolovOn Oct 7, 2019 13:54, Dan Poltawski <dan.poltawski(a)tnp.net.uk> wrote: > > On Mon, 2019-10-07 at 01:56 +0300, Strahil Nikolov wrote: > > I'm not very sure that you are supposed to use the CEPH by giving > > each VM direct access. > > > > Have you considered using an iSCSI gateway as an entry point for your > > storage domain ? This way oVirt will have no issues dealing with the > > rbd locks. > > > > Of course, oVirt might be able to deal with RBD locks , but that can > > be confirmed/denied by the devs. > > Thanks for your response - regarding the locks point, I realised later > that this was my own incorrect permissions given to the client. The > ceph client was detecting the broken locks when mounting the rbd device > and unable to blacklist it. I addressed this by swithcign to the > permissions 'profile rbd'. > > Regarding iSCSI, we are using this for the hosted engine. However, I am > attracted to the idea of managing block devices with individual rbd > devices to facilate individual block device level snapshotting and I > assume performance will be better. > > thanks, > > Dan > > ________________________________ > > The Networking People (TNP) Limited. Registered office: Network House, Caton Rd, Lancaster, LA1 3PE. Registered in England & Wales with company number: 07667393 > > This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

2 1