1:05 p.m.
Now I'm in a worst position, after unlocking "all" entities. The tasks
are not present anymore, the snapshots are not locked anymore, but these
5 vms are not able to start:
2019-10-08 12:54:25,088+03 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJoinPool-1-worker-7) [] EVENT_ID: VM_DOWN_ERROR(119), VM
openshift-04-os-infra-1 is down with error. Exit message: Bad volume
specification {'protocol': 'gluster', 'address':
{'function': '0x0', 'bus':
'0x00', 'domain': '0x0000', 'type': 'pci',
'slot': '0x06'}, 'serial':
'eb6331b3-ec4f-4b8e-a1cd-cb763ada9f6a', 'index': 0, 'iface':
'virtio',
'apparentsize': '29933240320', 'specParams':
{'pinToIoThread': '1'},
'cache': 'none', 'imageID':
'eb6331b3-ec4f-4b8e-a1cd-cb763ada9f6a',
'truesize': '30025396224', 'type': 'disk',
'domainID':
'97ced32d-bdb9-4913-a272-8a4a83ca3d1b', 'reqsize': '0',
'format': 'cow',
'poolID': 'c604b50e-2214-11e9-b449-00163e168814', 'device':
'disk', 'path':
'ssd-samsung-evo860/97ced32d-bdb9-4913-a272-8a4a83ca3d1b/images/eb6331b3-ec4f-4b8e-a1cd-cb763ada9f6a/
*f8b86437-e54c-4728-8100-ed05ef312212'*, 'propagateErrors': 'off',
'name':
'vda', 'bootOrder': '1', 'volumeID':
'f8b86437-e54c-4728-8100-ed05ef312212', 'diskType': 'network',
'alias':
'ua-eb6331b3-ec4f-4b8e-a1cd-cb763ada9f6a', 'hosts': [{'name':
'192.168.80.191', 'port': '0'}], 'discard': False}.
So i have checked the img file, it seems to be present an healthy:
# cd /rhev/data-center/mnt/glusterSD/192.168.80.191:
_ssd-samsung-evo860/97ced32d-bdb9-4913-a272-8a4a83ca3d1b/images/eb6331b3-ec4f-4b8e-a1cd-cb763ada9f6a
# qemu-img info *f8b86437-e54c-4728-8100-ed05ef312212*
image: f8b86437-e54c-4728-8100-ed05ef312212
file format: qcow2
virtual size: 50G (53687091200 bytes)
disk size: 28G
cluster_size: 65536
backing file: 4de5eae5-ab25-4a9e-a41a-2f7c2d8b272f
backing file format: qcow2
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
# qemu-img check *f8b86437-e54c-4728-8100-ed05ef312212*
No errors were found on the image.
456643/819200 = 55.74% allocated, 5.36% fragmented, 0.00% compressed
clusters
Image end offset: 29933240320
# ls -l
total 62430134
-rw-rw----. 1 vdsm kvm 33863434240 Oct 7 09:40
4de5eae5-ab25-4a9e-a41a-2f7c2d8b272f
-rw-rw----. 1 vdsm kvm 1048576 Jun 27 13:10
4de5eae5-ab25-4a9e-a41a-2f7c2d8b272f.lease
-rw-r--r--. 1 vdsm kvm 338 Oct 6 19:01
4de5eae5-ab25-4a9e-a41a-2f7c2d8b272f.meta
-rw-rw----. 1 vdsm kvm 29933240320 Oct 6 19:00
*f8b86437-e54c-4728-8100-ed05ef312212*
-rw-rw----. 1 vdsm kvm 1048576 Jul 1 22:56
f8b86437-e54c-4728-8100-ed05ef312212.lease
-rw-r--r--. 1 vdsm kvm 271 Oct 6 19:01
f8b86437-e54c-4728-8100-ed05ef312212.meta
Any ideea how should I proceed to have the vms to start ? I am kind of
stuck in this issue...
Thank you very much in advance,
Leo
On Tue, Oct 8, 2019 at 9:59 AM Leo David <leoalex(a)gmail.com> wrote:
Hi Everyone,
I'm waiting since 3 days for 5 x delete snapshot tasks to finish, and for
some reason it seems to be stucked.For other vms snapshot removal took at
most 20 mins, with havin the disks pretty much same size, and snapshots
numbers.
Any thoughts on how should I get this fixed ?
Below, some lines from the engine.log, and it seems to show some
complains regarding locks ( Failed to acquire lock and wait lock) ,
although I am not sure if thats the root cause:
Thank you very much !
Leo
2019-10-08 09:52:48,692+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-47)
[73016a4a-bb2f-487f-91c5-cd027b278930] Command
'RemoveSnapshotSingleDiskLive' (id:
'341d9c1b-2915-48d6-a8a9-9146ab19d5f8')
waiting on child command id: '329da0fd-801b-4e0d-b7c0-fbb5c2a98bb5'
type:'DestroyImage' to complete
2019-10-08 09:52:48,702+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-47)
[73016a4a-bb2f-487f-91c5-cd027b278930] Command
'RemoveSnapshotSingleDiskLive' (id:
'580fa033-35fd-44f0-9979-e60e9bbf8a29')
waiting on child command id: 'c00bdeb6-2e8b-4ef8-a3dc-1aaa088ae052'
type:'DestroyImage' to complete
2019-10-08 09:52:49,713+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-50)
[539ba19e-0cb5-42cf-9a23-7916ee2de4a9] Command
'RemoveSnapshotSingleDiskLive' (id:
'de747f91-ec59-4e70-9345-77e16234bfe0')
waiting on child command id: '10812160-cf4c-4239-bb92-1d5a847687ee'
type:'DestroyImage' to complete
2019-10-08 09:52:50,725+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-100)
[baed2fa3-bcad-43b2-8164-480598bc72f3] Command
'RemoveSnapshotSingleDiskLive' (id:
'4919b287-e980-4d34-a219-c08a169cd8f7')
waiting on child command id: '5eceb6a8-f08e-42aa-8258-c907f5927e6c'
type:'DestroyImage' to complete
2019-10-08 09:52:51,563+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:51,583+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:51,604+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:51,606+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:51,735+03 INFO
[org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-94)
[73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshot' (id:
'c9ab1344-ae27-4934-9358-d6a7b10a4f0a') waiting on child command id:
'341d9c1b-2915-48d6-a8a9-9146ab19d5f8'
type:'RemoveSnapshotSingleDiskLive'
to complete
2019-10-08 09:52:52,706+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterLocalPhysicalVolumeListVDSCommand, return:
[org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@21830b5f,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@676adc3e,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@385a3510,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@af24d00,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@331266f2],
log id: 1a3515fe
2019-10-08 09:52:52,708+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-2.domain.int,
VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}),
log id: 2a8c782
2019-10-08 09:52:52,745+03 INFO
[org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-22)
[539ba19e-0cb5-42cf-9a23-7916ee2de4a9] Command 'RemoveSnapshot' (id:
'28ac73f3-8b07-43d4-8b4b-1df90831ed50') waiting on child command id:
'de747f91-ec59-4e70-9345-77e16234bfe0'
type:'RemoveSnapshotSingleDiskLive'
to complete
2019-10-08 09:52:53,115+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterVDOVolumeListVDSCommand, return: [], log id: 2a8c782
2019-10-08 09:52:53,116+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterVolumeAdvancedDetailsVDSCommand(HostName =
host-dell-1.domain.int,
GlusterVolumeAdvancedDetailsVDSParameters:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b',
volumeName='samsung_sata'}), log id: 6bb38072
2019-10-08 09:52:54,659+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:54,662+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:54,663+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:54,679+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:54,755+03 INFO
[org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-21)
[4f2bedf7-b9b3-481e-936f-e180803ac1b8] Command 'RemoveSnapshot' (id:
'ceeb99c3-68be-4e0f-8dd5-3ef6509d4013') waiting on child command id:
'66094952-88e3-41d9-9191-7ef8873de511'
type:'RemoveSnapshotSingleDiskLive'
to complete
2019-10-08 09:52:54,762+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-21)
[825c211d-b40d-45d7-9dc7-e5c6af28a269] Command
'RemoveSnapshotSingleDiskLive' (id:
'5e9f9b03-4df9-4e31-b2e1-5efaa4fcf66c')
waiting on child command id: 'b3b0a892-e652-45dd-a71f-b2f77a942491'
type:'DestroyImage' to complete
2019-10-08 09:52:55,276+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterVolumeAdvancedDetailsVDSCommand, return:
org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@db5a3b94,
log id: 6bb38072
2019-10-08 09:52:55,288+03 INFO
[org.ovirt.engine.core.bll.gluster.GlusterThinDeviceService]
(DefaultQuartzScheduler10) [8921c9c] Volume samsung_sata have non-thin
bricks, skipping confirmed free size calculation
2019-10-08 09:52:55,295+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:55,308+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterLocalLogicalVolumeListVDSCommand(HostName =
host-dell-3.domain.int,
VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}),
log id: cc40e52
2019-10-08 09:52:55,376+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterLocalLogicalVolumeListVDSCommand, return:
[org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@723faf2a,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@faa8525,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@17581efa,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3bc224dd,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@340b8a70,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6fd3a279,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@382bceb7,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1265e1a3,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7a55a195,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@86714cc,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6e5f3d01,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@538aa190,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@484eb7d,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@34754e55,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2355daef],
log id: cc40e52
2019-10-08 09:52:55,378+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterLocalPhysicalVolumeListVDSCommand(HostName =
host-dell-3.domain.int,
VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}),
log id: 25557934
2019-10-08 09:52:55,436+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterLocalPhysicalVolumeListVDSCommand, return:
[org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@629b1e0,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@18305096,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@153f73dd],
log id: 25557934
2019-10-08 09:52:55,438+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-3.domain.int,
VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}),
log id: 1bad2497
2019-10-08 09:52:55,740+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterVDOVolumeListVDSCommand, return: [], log id: 1bad2497
2019-10-08 09:52:55,742+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterLocalLogicalVolumeListVDSCommand(HostName =
host-dell-1.domain.int,
VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}),
log id: 504020b3
2019-10-08 09:52:55,890+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterLocalLogicalVolumeListVDSCommand, return:
[org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4a6e6ecd,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@24a15dc6,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@56da4c59,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@74eab89,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3e6fb5ab,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@100e90cd,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@14b2ae11,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@66d25ded,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@25537e71,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@41bf0aee,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@40fce6ee,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3435f629,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@a2ba628,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@337a25f9,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@14967eee,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@f8946eb,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5844abd4],
log id: 504020b3
2019-10-08 09:52:55,891+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterLocalPhysicalVolumeListVDSCommand(HostName =
host-dell-1.domain.int,
VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}),
log id: 3af318c8
2019-10-08 09:52:56,620+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand]
(DefaultQuartzScheduler2) [5f2fb2c6] START,
GlusterTasksListVDSCommand(HostName = host-dell-1.domain.int,
VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}),
log id: 4dd7bc07
2019-10-08 09:52:56,681+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterLocalPhysicalVolumeListVDSCommand, return:
[org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@1d356076,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@584ec325,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@2529c9f3,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@73eb0c6,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@31be4687],
log id: 3af318c8
2019-10-08 09:52:56,682+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-1.domain.int,
VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}),
log id: 1f47fc7e
2019-10-08 09:52:56,774+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-32)
[825c211d-b40d-45d7-9dc7-e5c6af28a269] Command
'RemoveSnapshotSingleDiskLive' (id:
'1593ccd5-4e2a-4f8c-9c10-7944b95836b3')
waiting on child command id: '8b017634-d965-4273-a4a4-6281a337ed94'
type:'DestroyImage' to complete
2019-10-08 09:52:57,095+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterVDOVolumeListVDSCommand, return: [], log id: 1f47fc7e
2019-10-08 09:52:57,096+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterLocalLogicalVolumeListVDSCommand(HostName =
host-dell-4.domain.int,
VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}),
log id: 74cad834
2019-10-08 09:52:57,144+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterLocalLogicalVolumeListVDSCommand, return:
[org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@600eeeb,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@45491ad2,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@654daa09,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4764d5ec,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@46121c5d,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4b0001c7,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4315e74c,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@22bf8177,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@74999189,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3f5ca8e4,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@531f17c4],
log id: 74cad834
2019-10-08 09:52:57,145+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterLocalPhysicalVolumeListVDSCommand(HostName =
host-dell-4.domain.int,
VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}),
log id: df580e9
2019-10-08 09:52:57,191+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterLocalPhysicalVolumeListVDSCommand, return:
[org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@35585d07],
log id: df580e9
2019-10-08 09:52:57,192+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-4.domain.int,
VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}),
log id: 6f19c8c0
2019-10-08 09:52:57,198+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand]
(DefaultQuartzScheduler2) [5f2fb2c6] FINISH, GlusterTasksListVDSCommand,
return: [], log id: 4dd7bc07
2019-10-08 09:52:57,432+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterVDOVolumeListVDSCommand, return: [], log id: 6f19c8c0
2019-10-08 09:52:57,433+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterLocalLogicalVolumeListVDSCommand(HostName =
host-dell-2.domain.int,
VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}),
log id: 449353d0
2019-10-08 09:52:57,576+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:57,594+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:57,616+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:57,619+03 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock
'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]',
sharedLocks=''}'
2019-10-08 09:52:57,785+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-90)
[4f2bedf7-b9b3-481e-936f-e180803ac1b8] Command
'RemoveSnapshotSingleDiskLive' (id:
'66094952-88e3-41d9-9191-7ef8873de511')
waiting on child command id: '122be070-d45f-4e6b-bb61-4943a096b88b'
type:'DestroyImage' to complete
2019-10-08 09:52:57,792+03 INFO
[org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-90)
[baed2fa3-bcad-43b2-8164-480598bc72f3] Command 'RemoveSnapshot' (id:
'7aafe2fe-e05b-44bc-a716-c48734a8c2de') waiting on child command id:
'4919b287-e980-4d34-a219-c08a169cd8f7'
type:'RemoveSnapshotSingleDiskLive'
to complete
2019-10-08 09:52:57,799+03 INFO
[org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-90)
[825c211d-b40d-45d7-9dc7-e5c6af28a269] Command 'RemoveSnapshot' (id:
'cb6e7a1d-1902-4e61-9c3d-e541cf4c6348') waiting on child command id:
'5e9f9b03-4df9-4e31-b2e1-5efaa4fcf66c'
type:'RemoveSnapshotSingleDiskLive'
to complete
2019-10-08 09:52:58,345+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] FINISH,
GetGlusterLocalLogicalVolumeListVDSCommand, return:
[org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@65406163,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1207b8a9,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@33f93de1,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6d2ae59d,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@b07ca22,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@246ae91b,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5e86ce42,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@643ade02,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@51d95e7d,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2288576c,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4a9469a1,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@78ec643f,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@63dde7e9,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@30effc9,
org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6a3e4768],
log id: 449353d0
2019-10-08 09:52:58,346+03 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand]
(DefaultQuartzScheduler10) [8921c9c] START,
GetGlusterLocalPhysicalVolumeListVDSCommand(HostName =
host-dell-2.domain.int,
VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}),
log id: 71680b8c
2019-10-08 09:52:58,807+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-81)
[73016a4a-bb2f-487f-91c5-cd027b278930] Command
'RemoveSnapshotSingleDiskLive' (id:
'341d9c1b-2915-48d6-a8a9-9146ab19d5f8')
waiting on child command id: '329da0fd-801b-4e0d-b7c0-fbb5c2a98bb5'
type:'DestroyImage' to complete
2019-10-08 09:52:58,817+03 INFO
[org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback]
(EE-ManagedThreadFactory-engineScheduled-Thread-81)
[73016a4a-bb2f-487f-91c5-cd027b278930] Command
'RemoveSnapshotSingleDiskLive' (id:
'580fa033-35fd-44f0-9979-e60e9bbf8a29')
waiting on child command id: 'c00bdeb6-2e8b-4ef8-a3dc-1aaa088ae052'
type:'DestroyImage' to complete
--
Best regards, Leo David
--
Best regards, Leo David