2019-04-13 21:24:44,813+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:24:44,813+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:24:45,342+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:24:47,344+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:24:47,351+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:24:47,357+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:24:49,364+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:24:49,365+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:24:50,372+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:24:51,381+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:24:52,391+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:24:52,392+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:24:52,400+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:24:52,405+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:24:53,407+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:24:54,415+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:24:54,416+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:24:55,424+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:24:57,425+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:24:57,430+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:24:57,434+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:24:59,442+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:24:59,443+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:24:59,854+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:24:59,854+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:24:59,854+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:24:59,854+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:25:00,449+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:25:01,455+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:02,461+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:25:02,462+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:25:02,466+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:02,471+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:25:03,472+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:25:04,478+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:04,478+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:25:05,486+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:25:07,489+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:25:07,498+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:25:07,504+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:09,519+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:09,520+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:25:10,528+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:25:11,537+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:12,544+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:25:12,545+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:25:12,552+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:12,557+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:25:13,558+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:25:14,566+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:14,567+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:25:14,893+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:25:14,893+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:25:14,893+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:25:14,894+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:25:15,575+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:25:17,577+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:25:17,582+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:25:17,587+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:19,593+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:19,594+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:25:20,601+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:25:21,609+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:22,616+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:25:22,617+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:25:22,622+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:22,627+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:25:23,628+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:25:24,635+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:24,636+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:25:25,642+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:25:27,644+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:25:27,648+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:25:27,652+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:29,660+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:29,661+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:25:29,931+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:25:29,931+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:25:29,931+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:25:29,931+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:25:30,131+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:25:30,669+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:25:31,676+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:32,687+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:25:32,688+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:25:32,696+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:32,702+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:25:33,703+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:25:34,709+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:34,710+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:25:35,715+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:25:37,717+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:25:37,725+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:25:37,732+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:39,738+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:39,739+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:25:40,745+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:25:41,750+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:42,757+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:25:42,757+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:25:42,762+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:42,766+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:25:43,768+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:25:44,774+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:44,775+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:25:44,968+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:25:44,968+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:25:44,968+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:25:44,968+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:25:45,781+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:25:47,783+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:25:47,788+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:25:47,792+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:49,800+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:49,801+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:25:50,808+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:25:51,815+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:52,823+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:25:52,823+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:25:52,828+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:52,833+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:25:53,835+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:25:54,841+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:54,842+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:25:55,850+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:25:57,851+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:25:57,857+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:25:57,861+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:59,868+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:25:59,869+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:26:00,016+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:26:00,017+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:26:00,017+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:26:00,017+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:26:00,875+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:26:01,881+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:02,888+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:26:02,889+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:26:02,892+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:02,896+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:26:03,897+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:26:04,903+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:04,903+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:26:05,910+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:26:07,912+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:26:07,918+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:26:07,924+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:09,931+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:09,931+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:26:10,938+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:26:11,944+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:12,951+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:26:12,951+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:26:12,957+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:12,961+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:26:13,962+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:26:14,969+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:14,969+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:26:15,053+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:26:15,054+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:26:15,054+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:26:15,054+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:26:15,977+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:26:17,979+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:26:17,984+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:26:17,988+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:19,994+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:19,995+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:26:21,002+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:26:22,009+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:23,015+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:26:23,015+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:26:23,020+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:23,023+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:26:24,025+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:26:25,031+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:25,032+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:26:26,039+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:26:28,041+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:26:28,048+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:26:28,053+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:30,061+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:30,062+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:26:30,104+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:26:30,104+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:26:30,105+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:26:30,105+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:26:30,147+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:26:31,068+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:26:32,075+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:33,083+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:26:33,084+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:26:33,091+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:33,097+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:26:34,098+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:26:35,104+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:35,105+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:26:36,113+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:26:38,115+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:26:38,123+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:26:38,132+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:40,140+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:40,141+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:26:41,149+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:26:42,155+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:43,163+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:26:43,164+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:26:43,171+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:43,177+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:26:44,178+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:26:45,147+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:26:45,147+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:26:45,148+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:26:45,148+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:26:45,187+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:45,187+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:26:46,194+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:26:48,195+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:26:48,201+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:26:48,205+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:50,211+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:50,212+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:26:51,219+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:26:52,225+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:53,232+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:26:53,233+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:26:53,238+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:53,244+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:26:54,245+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:26:55,251+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:26:55,252+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:26:56,261+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:26:58,262+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:26:58,268+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:26:58,272+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:00,184+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:27:00,184+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:27:00,184+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:27:00,184+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:27:00,283+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:00,284+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:27:01,291+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:27:02,298+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:03,305+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:27:03,306+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:27:03,310+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:03,314+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:27:04,315+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:27:05,320+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:05,321+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:27:06,328+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:27:08,330+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:27:08,336+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:27:08,340+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:10,346+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:10,348+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:27:11,354+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:27:12,360+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:13,367+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:27:13,367+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:27:13,371+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:13,376+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:27:14,377+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:27:15,227+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:27:15,227+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:27:15,227+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:27:15,227+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:27:15,383+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:15,384+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:27:16,391+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:27:18,393+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:27:18,399+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:27:18,404+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:20,411+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:20,412+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:27:21,418+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:27:22,424+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:23,432+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:27:23,432+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:27:23,439+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:23,446+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:27:24,447+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:27:25,454+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:25,454+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:27:26,462+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:27:28,463+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:27:28,470+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:27:28,474+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:30,147+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:27:30,265+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:27:30,265+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:27:30,265+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:27:30,265+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:27:30,482+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:30,483+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:27:31,490+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:27:32,497+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:33,504+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:27:33,505+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:27:33,510+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:33,514+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:27:34,515+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:27:35,522+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:35,523+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:27:36,530+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:27:38,531+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:27:38,537+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:27:38,541+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:40,550+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:40,551+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:27:41,557+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:27:42,565+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:43,572+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:27:43,572+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:27:43,576+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:43,580+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:27:44,582+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:27:45,308+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:27:45,308+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:27:45,308+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:27:45,308+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:27:45,589+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:45,590+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:27:46,596+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:27:48,598+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:27:48,604+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:27:48,609+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:50,616+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:50,617+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:27:51,624+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:27:52,630+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:53,637+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:27:53,638+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:27:53,644+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:53,652+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:27:54,653+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:27:55,659+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:27:55,660+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:27:56,666+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:27:58,668+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:27:58,675+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:27:58,681+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:00,350+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:28:00,350+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:28:00,350+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:28:00,351+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:28:00,687+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:00,688+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:28:01,694+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:28:02,700+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:03,706+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:28:03,706+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:28:03,711+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:03,715+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:28:04,716+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:28:05,723+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:05,724+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:28:06,730+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:28:08,732+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:28:08,736+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:28:08,740+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:10,747+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:10,747+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:28:11,754+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:28:12,759+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:13,766+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:28:13,766+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:28:13,770+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:13,774+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:28:14,775+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:28:15,389+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:28:15,390+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:28:15,390+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:28:15,390+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:28:15,781+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:15,782+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:28:16,789+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:28:18,791+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:28:18,797+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:28:18,801+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:20,808+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:20,808+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:28:21,816+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:28:22,823+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:23,829+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:28:23,829+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:28:23,834+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:23,838+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:28:24,839+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:28:25,846+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:25,847+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:28:26,852+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:28:28,854+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:28:28,859+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:28:28,863+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:30,140+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:28:30,428+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:28:30,428+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:28:30,429+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:28:30,429+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:28:30,870+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:30,871+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:28:31,878+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:28:32,884+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:33,890+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:28:33,891+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:28:33,895+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:33,899+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:28:34,900+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:28:35,906+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:35,906+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:28:36,913+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:28:38,914+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:28:38,919+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:28:38,925+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:40,931+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:40,932+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:28:41,938+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:28:42,947+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:43,954+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:28:43,954+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:28:43,959+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:43,964+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:28:44,965+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:28:45,472+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:28:45,472+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:28:45,473+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:28:45,473+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:28:45,972+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:45,973+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:28:46,981+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:28:48,982+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:28:48,988+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:28:48,992+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:50,998+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:50,999+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:28:52,005+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:28:53,011+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:54,017+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:28:54,017+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:28:54,023+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:54,029+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:28:55,031+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:28:56,039+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:28:56,040+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:28:57,047+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:28:59,049+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:28:59,054+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:28:59,058+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:00,513+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:29:00,513+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:29:00,513+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:29:00,513+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:29:01,066+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:01,066+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:29:02,073+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:29:03,079+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:04,086+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:29:04,087+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:29:04,092+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:04,096+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:29:05,098+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:29:06,104+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:06,105+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:29:07,111+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:29:09,113+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:29:09,118+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:29:09,123+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:11,129+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:11,130+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:29:12,139+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:29:13,145+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:14,152+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:29:14,152+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:29:14,157+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:14,161+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:29:15,162+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:29:15,551+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:29:15,551+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:29:15,551+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:29:15,551+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:29:16,169+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:16,170+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:29:17,180+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:29:19,181+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:29:19,186+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:29:19,190+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:21,199+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:21,200+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:29:22,206+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:29:23,213+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:24,219+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:29:24,220+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:29:24,226+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:24,231+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:29:25,232+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:29:26,241+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:26,242+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:29:27,248+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:29:29,250+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:29:29,258+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:29:29,265+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:30,153+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:29:30,589+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:29:30,590+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:29:30,590+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:29:30,590+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:29:31,272+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:31,273+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:29:32,280+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:29:33,286+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:34,292+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:29:34,292+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:29:34,297+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:34,301+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:29:35,302+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:29:36,309+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:36,309+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:29:37,316+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:29:39,317+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:29:39,323+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:29:39,327+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:41,333+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:41,334+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:29:42,340+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:29:43,346+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:44,353+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:29:44,353+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:29:44,358+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:44,362+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:29:45,364+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:29:45,640+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:29:45,640+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:29:45,640+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:29:45,640+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:29:46,370+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:46,371+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:29:47,379+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:29:49,381+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:29:49,386+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:29:49,390+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:51,397+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:51,397+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:29:52,403+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:29:53,410+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:54,417+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:29:54,418+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:29:54,422+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:54,426+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:29:55,427+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:29:56,433+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:29:56,434+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:29:57,441+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:29:59,443+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:29:59,450+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:29:59,457+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:00,678+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:30:00,678+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:30:00,678+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:30:00,679+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:30:01,463+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:01,464+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:30:02,472+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:30:03,478+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:04,484+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:30:04,484+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:30:04,489+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:04,493+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:30:05,494+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:30:06,500+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:06,500+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:30:07,506+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:30:09,508+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:30:09,513+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:30:09,517+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:11,524+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:11,525+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:30:12,531+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:30:13,537+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:14,544+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:30:14,544+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:30:14,549+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:14,554+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:30:15,555+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:30:15,726+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:30:15,726+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:30:15,727+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:30:15,727+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:30:16,561+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:16,562+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:30:17,571+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:30:19,573+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:30:19,581+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:30:19,591+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:21,597+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:21,598+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:30:22,605+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:30:23,611+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:24,618+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:30:24,618+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:30:24,622+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:24,627+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:30:25,628+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:30:26,635+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:26,637+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:30:27,646+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:30:29,648+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:30:29,657+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:30:29,663+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:30,141+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:30:30,771+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:30:30,771+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:30:30,772+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:30:30,772+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:30:31,670+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:31,671+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:30:32,677+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:30:33,683+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:34,690+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:30:34,690+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:30:34,695+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:34,699+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:30:35,700+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:30:36,707+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:36,707+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:30:37,713+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:30:39,715+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:30:39,721+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:30:39,726+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:41,738+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:41,739+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:30:41,984+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'commandCoordinator' is using 0 threads out of 10, 10 threads waiting for tasks. 2019-04-13 21:30:41,984+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'default' is using 0 threads out of 1, 5 threads waiting for tasks. 2019-04-13 21:30:41,984+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engine' is using 0 threads out of 500, 8 threads waiting for tasks and 0 tasks in queue. 2019-04-13 21:30:41,984+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engineScheduled' is using 0 threads out of 100, 100 threads waiting for tasks. 2019-04-13 21:30:41,984+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engineThreadMonitoring' is using 1 threads out of 1, 0 threads waiting for tasks. 2019-04-13 21:30:41,984+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'hostUpdatesChecker' is using 0 threads out of 5, 5 threads waiting for tasks. 2019-04-13 21:30:42,744+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:30:43,752+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:44,758+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:30:44,758+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:30:44,763+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:44,766+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:30:45,768+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:30:45,809+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:30:45,809+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:30:45,809+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:30:45,809+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:30:46,774+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:46,774+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:30:47,781+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:30:49,782+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:30:49,789+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:30:49,796+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:51,803+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:51,804+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:30:52,810+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:30:53,818+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:54,826+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:30:54,827+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:30:54,832+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:54,840+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:30:55,841+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:30:56,848+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:30:56,848+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:30:57,855+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:30:59,856+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:30:59,863+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:30:59,871+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:00,846+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:31:00,846+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:31:00,846+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:31:00,846+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:31:01,878+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:01,878+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:31:02,887+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:31:03,893+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:04,900+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:31:04,902+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:31:04,909+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:04,915+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:31:05,916+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:31:06,923+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:06,923+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:31:07,930+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:31:09,932+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:31:09,938+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:31:09,944+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:11,951+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:11,952+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:31:12,959+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:31:13,965+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:14,971+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:31:14,972+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:31:14,976+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:14,982+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:31:15,891+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:31:15,891+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:31:15,891+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:31:15,891+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:31:15,984+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:31:16,990+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:16,990+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:31:17,998+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:31:19,999+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:31:20,005+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:31:20,009+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:22,016+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:22,017+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:31:23,023+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:31:24,029+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:25,035+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:31:25,036+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:31:25,040+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:25,045+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:31:26,046+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:31:27,052+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:27,053+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:31:28,059+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:31:30,061+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:31:30,066+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:31:30,071+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:30,141+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:31:30,924+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:31:30,924+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:31:30,924+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:31:30,925+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:31:32,079+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:32,080+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:31:33,088+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:31:34,095+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:35,102+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:31:35,103+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:31:35,109+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:35,119+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:31:36,120+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:31:37,126+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:37,127+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:31:38,133+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:31:40,134+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:31:40,142+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:31:40,148+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:42,156+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:42,157+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:31:43,165+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:31:44,173+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:45,181+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:31:45,181+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:31:45,186+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:45,191+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:31:45,974+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:31:45,974+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:31:45,974+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:31:45,974+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:31:46,193+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:31:47,199+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:47,200+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:31:48,206+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:31:50,208+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:31:50,216+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:31:50,223+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:52,231+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:52,232+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:31:53,239+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:31:54,248+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:55,257+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:31:55,258+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:31:55,264+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:55,273+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:31:56,274+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:31:57,280+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:31:57,281+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:31:58,288+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:32:00,290+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:32:00,301+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:32:00,308+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:01,029+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:32:01,030+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:32:01,030+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:32:01,030+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:32:02,315+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:02,316+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:32:03,325+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:32:04,331+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:05,337+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:32:05,338+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:32:05,343+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:05,347+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:32:06,349+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:32:07,355+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:07,356+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:32:08,363+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:32:10,365+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:32:10,371+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:32:10,377+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:12,385+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:12,386+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:32:13,394+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:32:14,402+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:15,410+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:32:15,411+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:32:15,417+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:15,423+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:32:16,087+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:32:16,088+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:32:16,088+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:32:16,088+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:32:16,424+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:32:17,431+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:17,431+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:32:18,438+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:32:20,440+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:32:20,446+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:32:20,450+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:22,457+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:22,458+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:32:23,464+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:32:24,471+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:25,479+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:32:25,480+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:32:25,486+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:25,492+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:32:26,493+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:32:27,500+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:27,501+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:32:28,508+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:32:30,143+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:32:30,509+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:32:30,515+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:32:30,519+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:31,126+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:32:31,126+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:32:31,127+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:32:31,127+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:32:32,527+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:32,527+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:32:33,533+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:32:34,541+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:35,548+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:32:35,548+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:32:35,553+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:35,558+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:32:36,559+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:32:37,566+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:37,566+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:32:38,580+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:32:40,582+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:32:40,588+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:32:40,594+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:42,600+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:42,601+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:32:43,608+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:32:44,617+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:45,624+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:32:45,624+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:32:45,629+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:45,634+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:32:46,175+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:32:46,176+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:32:46,176+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:32:46,176+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:32:46,635+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:32:47,643+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:47,644+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:32:48,652+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:32:50,654+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:32:50,661+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:32:50,668+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:52,677+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:52,678+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:32:53,686+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:32:54,692+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:55,698+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:32:55,699+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:32:55,703+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:55,707+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:32:56,708+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:32:57,714+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:32:57,715+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:32:58,722+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:33:00,724+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:33:00,730+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:33:00,736+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:01,215+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:33:01,215+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:33:01,216+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:33:01,216+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:33:02,745+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:02,746+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:33:03,752+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:33:04,758+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:05,763+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:33:05,764+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:33:05,769+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:05,773+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:33:06,774+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:33:07,780+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:07,781+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:33:08,788+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:33:10,789+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:33:10,796+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:33:10,801+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:12,807+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:12,808+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:33:13,814+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:33:14,820+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:15,826+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:33:15,826+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:33:15,830+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:15,834+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:33:16,262+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:33:16,262+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:33:16,262+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:33:16,262+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:33:16,835+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:33:17,842+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:17,843+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:33:18,850+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:33:20,852+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:33:20,857+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:33:20,861+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:22,868+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:22,869+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:33:23,875+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:33:24,882+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:25,889+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:33:25,889+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:33:25,894+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:25,899+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:33:26,900+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:33:27,906+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:27,906+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:33:28,912+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:33:30,139+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:33:30,913+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:33:30,918+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:33:30,923+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:31,306+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:33:31,307+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:33:31,307+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:33:31,307+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:33:32,929+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:32,930+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:33:33,938+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:33:34,944+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:35,952+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:33:35,952+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:33:35,958+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:35,964+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:33:36,965+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:33:37,973+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:37,974+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:33:38,983+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:33:40,985+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:33:40,990+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:33:40,995+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:43,002+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:43,002+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:33:44,009+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:33:45,018+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:46,024+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:33:46,024+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:33:46,030+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:46,035+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:33:46,344+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:33:46,345+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:33:46,345+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:33:46,345+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:33:47,036+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:33:48,042+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:48,043+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:33:49,050+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:33:51,052+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:33:51,057+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:33:51,062+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:53,069+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:53,069+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:33:54,075+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:33:55,082+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:56,091+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:33:56,092+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:33:56,099+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:56,106+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:33:57,107+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:33:58,113+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:33:58,114+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:33:59,121+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:34:01,122+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:34:01,128+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:34:01,133+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:01,398+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:34:01,398+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:34:01,398+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:34:01,398+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:34:03,140+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:03,140+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:34:04,146+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:34:05,154+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:06,160+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:34:06,160+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:34:06,165+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:06,168+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:34:07,170+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:34:08,175+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:08,176+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:34:09,182+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:34:11,184+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:34:11,189+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:34:11,193+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:13,200+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:13,200+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:34:14,206+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:34:15,213+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:16,220+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:34:16,221+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:34:16,226+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:16,231+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:34:16,434+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:34:16,434+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:34:16,434+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:34:16,435+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:34:17,233+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:34:18,239+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:18,239+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:34:19,246+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:34:21,247+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:34:21,253+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:34:21,257+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:23,263+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:23,264+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:34:24,268+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:34:25,272+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:26,278+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:34:26,278+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:34:26,281+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:26,284+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:34:27,285+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:34:28,291+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:28,292+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:34:29,301+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:34:30,141+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:34:31,303+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:34:31,308+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:34:31,313+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:31,469+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:34:31,469+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:34:31,470+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:34:31,470+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:34:33,324+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:33,325+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:34:34,332+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:34:35,339+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:36,346+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:34:36,347+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:34:36,353+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:36,359+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:34:37,360+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:34:38,367+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:38,367+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:34:39,376+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:34:41,378+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:34:41,383+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:34:41,387+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:43,394+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:43,394+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:34:44,401+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:34:45,408+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:46,414+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:34:46,415+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:34:46,421+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:46,426+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:34:46,506+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:34:46,506+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:34:46,507+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:34:46,507+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:34:47,428+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:34:48,434+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:48,435+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:34:49,441+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:34:51,443+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:34:51,448+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:34:51,452+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:53,458+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:53,459+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:34:54,465+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:34:55,471+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:56,477+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:34:56,477+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:34:56,482+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:56,485+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:34:57,487+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:34:58,494+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:34:58,494+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:34:59,500+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:35:01,501+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:35:01,507+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:35:01,511+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:01,545+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:35:01,545+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:35:01,545+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:35:01,546+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:35:03,518+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:03,520+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:35:04,526+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:35:05,532+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:06,540+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:35:06,541+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:35:06,546+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:06,552+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:35:07,553+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:35:08,559+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:08,560+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:35:09,566+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:35:11,568+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:35:11,574+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:35:11,580+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:13,587+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:13,587+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:35:14,594+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:35:15,600+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:16,582+02 WARN [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [] Unexpected return value: Status [code=-32603, message=Internal JSON-RPC error: {'reason': "'appsList'"}] 2019-04-13 21:35:16,582+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [] Failed in 'GetAllVmStatsVDS' method 2019-04-13 21:35:16,582+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSErrorException: Failed to GetAllVmStatsVDS, error = Internal JSON-RPC error: {'reason': "'appsList'"}, code = -32603 2019-04-13 21:35:16,583+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:35:16,607+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:35:16,608+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:35:16,613+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:16,618+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:35:17,619+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:35:18,625+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:18,626+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:35:19,632+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:35:21,634+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:35:21,641+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:35:21,646+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:23,653+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:23,653+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:35:24,660+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:35:25,666+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:26,673+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:35:26,673+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:35:26,680+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:26,687+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:35:27,689+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:35:28,696+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:28,696+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:35:29,703+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:35:30,136+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:35:31,705+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:35:31,712+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:35:31,719+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:33,726+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:33,726+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:35:34,733+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:35:35,743+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:36,591+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] No interaction with host 'onode-1.discovery.intern' for 20000 ms. 2019-04-13 21:35:36,751+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:35:36,751+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:35:36,758+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:36,766+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:35:37,767+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:35:38,602+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] No interaction with host 'onode-1.discovery.intern' for 20000 ms. 2019-04-13 21:35:38,778+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:38,779+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:35:39,604+02 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connection timeout for host 'onode-1.discovery.intern', last response arrived 23022 ms ago. 2019-04-13 21:35:39,788+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:35:39,793+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command SpmStatusVDS failed: Connection timeout for host 'onode-1.discovery.intern', last response arrived 23022 ms ago. 2019-04-13 21:35:39,840+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetAllVmStatsVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Command 'GetAllVmStatsVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: VDSGenericException: VDSNetworkException: Connection timeout for host 'onode-1.discovery.intern', last response arrived 23022 ms ago. 2019-04-13 21:35:39,840+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.PollVmStatsRefresher] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Failed to fetch vms info for host 'onode-1' - skipping VMs monitoring. 2019-04-13 21:35:39,844+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSNetworkException: Connection timeout for host 'onode-1.discovery.intern', last response arrived 23022 ms ago. 2019-04-13 21:35:39,876+02 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-229848) [] Host 'onode-1' is not responding. It will stay in Connecting state for a grace period of 88 seconds and after that an attempt to fence the host will be issued. 2019-04-13 21:35:39,876+02 WARN [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-229847) [] Host 'onode-1' is not responding. It will stay in Connecting state for a grace period of 88 seconds and after that an attempt to fence the host will be issued. 2019-04-13 21:35:39,929+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229848) [] EVENT_ID: VDS_HOST_NOT_RESPONDING_CONNECTING(9,008), Host onode-1 is not responding. It will stay in Connecting state for a grace period of 88 seconds and after that an attempt to fence the host will be issued. 2019-04-13 21:35:39,932+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229847) [] EVENT_ID: VDS_HOST_NOT_RESPONDING_CONNECTING(9,008), Host onode-1 is not responding. It will stay in Connecting state for a grace period of 88 seconds and after that an attempt to fence the host will be issued. 2019-04-13 21:35:39,933+02 INFO [org.ovirt.engine.core.bll.storage.pool.SetStoragePoolStatusCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:35:39,959+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '93f02ede-ec5c-4641-8e97-2949f98e856d' status from 'Active' to 'Unknown', reason: null 2019-04-13 21:35:39,967+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '42bb6772-12f9-4bf2-9245-43caa41c5a2a' status from 'Active' to 'Unknown', reason: null 2019-04-13 21:35:39,968+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '6a4cde72-6a6a-499e-9b6e-d490d91ca19e' status from 'Active' to 'Unknown', reason: null 2019-04-13 21:35:39,970+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '9c3f06cf-7475-448e-819b-f4f52fa7d782' status from 'Active' to 'Unknown', reason: null 2019-04-13 21:35:39,987+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [] Unable to GetStats: VDSNetworkException: VDSGenericException: VDSNetworkException: Connection timeout for host 'onode-1.discovery.intern', last response arrived 23022 ms ago. 2019-04-13 21:35:40,131+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] EVENT_ID: SYSTEM_CHANGE_STORAGE_POOL_STATUS_PROBLEMATIC_WITH_ERROR(987), Invalid status on Data Center Default. Setting Data Center status to Non Responsive (On host onode-1, Error: Network error during communication with the Host.). 2019-04-13 21:35:40,262+02 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connection timeout for host 'onode-1.discovery.intern', last response arrived 22549 ms ago. 2019-04-13 21:35:40,277+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command GetStoragePoolInfoVDS failed: Connection timeout for host 'onode-1.discovery.intern', last response arrived 22549 ms ago. 2019-04-13 21:35:40,288+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] ERROR, org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand, exception: VDSGenericException: VDSNetworkException: Connection timeout for host 'onode-1.discovery.intern', last response arrived 22549 ms ago., log id: 7fa7feb1 2019-04-13 21:35:40,303+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [234c8345] Exception 2019-04-13 21:35:40,335+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:35:41,789+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:35:41,817+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:35:41,837+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:43,843+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:43,844+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:35:44,848+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:35:45,853+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:46,857+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:35:46,857+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:35:46,859+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:46,861+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:35:47,862+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:35:48,865+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:48,865+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:35:49,869+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:35:51,871+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:35:51,875+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:35:51,878+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:53,882+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:53,883+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:35:54,888+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:35:55,891+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:56,895+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:35:56,895+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:35:56,898+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:56,900+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:35:57,901+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:35:58,906+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:35:58,906+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:35:59,912+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:36:00,339+02 WARN [org.ovirt.vdsm.jsonrpc.client.utils.retry.Retryable] (SSL Stomp Reactor) [] Retry failed 2019-04-13 21:36:00,340+02 ERROR [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Exception during connection 2019-04-13 21:36:00,348+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSNetworkException: Connection issue Connection timeout 2019-04-13 21:36:00,355+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Connection issue Connection timeout 2019-04-13 21:36:00,359+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Unable to RefreshCapabilities: ConnectException: Connection timeout 2019-04-13 21:36:00,359+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:00,364+02 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Trying to release exclusive lock which does not exist, lock key: '65c45da8-2a46-42e0-9c3a-58748546103bVDS_INIT' 2019-04-13 21:36:00,365+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand' return value 'org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturn@475f5d31' 2019-04-13 21:36:00,365+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] HostName = onode-1 2019-04-13 21:36:00,366+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.rmi.ConnectException: Connection timeout 2019-04-13 21:36:01,913+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:36:01,917+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:36:01,921+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:03,373+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:03,924+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:03,925+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-11) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:36:04,929+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:36:05,934+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:06,937+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:36:06,938+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:36:06,940+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:06,942+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:36:07,943+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:36:08,947+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:08,947+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:36:09,950+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:36:11,952+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:36:11,955+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:36:11,957+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:13,503+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:36:13,518+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:13,961+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:13,961+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:36:14,967+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:36:15,971+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:16,526+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:16,976+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:36:16,976+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:36:16,979+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:16,982+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:36:17,983+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:36:18,403+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [] IrsBroker::Failed::GetStoragePoolInfoVDS 2019-04-13 21:36:18,405+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:36:18,406+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:18,408+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [] Command 'GetStoragePoolInfoVDSCommand( GetStoragePoolInfoVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='true'})' execution failed: IRSProtocolException: 2019-04-13 21:36:18,990+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:18,991+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:36:19,997+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:36:21,415+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:21,998+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:36:22,003+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:36:22,005+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:24,009+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:24,009+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:36:24,421+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:36:24,423+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:25,016+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:36:26,022+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:27,029+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:36:27,029+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:36:27,034+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:27,038+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:36:27,431+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:28,039+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:36:29,044+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:29,044+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:36:30,050+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:36:30,139+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:36:30,437+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:36:30,438+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:32,052+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:36:32,061+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:36:32,072+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:33,446+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:34,077+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:34,078+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:36:35,082+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:36:36,090+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:36,453+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:36:36,454+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:37,094+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:36:37,094+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:36:37,098+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:37,103+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:36:38,104+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:36:39,111+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:39,111+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:36:39,462+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:40,116+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:36:42,117+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:36:42,122+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:36:42,126+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-89) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:42,469+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:36:42,473+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:44,132+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:44,133+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:36:45,141+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:36:45,484+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:46,145+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:47,149+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:36:47,149+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:36:47,151+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:47,153+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:36:48,154+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:36:48,490+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:36:48,491+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:49,160+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:49,161+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:36:50,168+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:36:51,501+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:52,169+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:36:52,174+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:36:52,176+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:54,181+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:54,182+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:36:54,509+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:36:54,511+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:54,565+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM srs was set to the Unknown status. 2019-04-13 21:36:54,570+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM timetrack was set to the Unknown status. 2019-04-13 21:36:54,575+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM win2016-01 was set to the Unknown status. 2019-04-13 21:36:54,580+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM transfer was set to the Unknown status. 2019-04-13 21:36:54,585+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM mosaico was set to the Unknown status. 2019-04-13 21:36:54,589+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM notebook was set to the Unknown status. 2019-04-13 21:36:54,595+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM remote was set to the Unknown status. 2019-04-13 21:36:54,600+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM gogs was set to the Unknown status. 2019-04-13 21:36:54,605+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM taiga was set to the Unknown status. 2019-04-13 21:36:54,612+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM ffplayout-test was set to the Unknown status. 2019-04-13 21:36:54,620+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM pxe was set to the Unknown status. 2019-04-13 21:36:54,628+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM tv-planer was set to the Unknown status. 2019-04-13 21:36:54,634+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM pass was set to the Unknown status. 2019-04-13 21:36:54,641+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM win2016-02 was set to the Unknown status. 2019-04-13 21:36:54,648+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM webdev was set to the Unknown status. 2019-04-13 21:36:54,654+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VM_SET_TO_UNKNOWN_STATUS(142), VM ffplayout was set to the Unknown status. 2019-04-13 21:36:54,654+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (EE-ManagedThreadFactory-engine-Thread-229858) [] Server failed to respond, vds_id='65c45da8-2a46-42e0-9c3a-58748546103b', vds_name='onode-1', vm_count=16, spm_status='SPM', non-responsive_timeout (seconds)=88, error: java.net.NoRouteToHostException: No route to host 2019-04-13 21:36:54,687+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229858) [] EVENT_ID: VDS_FAILURE(12), Host onode-1 is non responsive. 2019-04-13 21:36:54,695+02 INFO [org.ovirt.engine.core.bll.VdsEventListener] (EE-ManagedThreadFactory-engine-Thread-229859) [] ResourceManager::vdsNotResponding entered for Host '65c45da8-2a46-42e0-9c3a-58748546103b', 'onode-1.discovery.intern' 2019-04-13 21:36:54,941+02 INFO [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Lock Acquired to object 'EngineLock:{exclusiveLocks='[65c45da8-2a46-42e0-9c3a-58748546103b=VDS_FENCE]', sharedLocks=''}' 2019-04-13 21:36:54,975+02 INFO [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Running command: VdsNotRespondingTreatmentCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:36:55,035+02 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Running command: SshSoftFencingCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:36:55,188+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:36:55,512+02 ERROR [org.ovirt.engine.core.bll.pm.FenceProxyLocator] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Can not run fence action on host 'onode-1', no suitable proxy host was found. 2019-04-13 21:36:55,529+02 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Opening SSH Soft Fencing session on host 'onode-1.discovery.intern' 2019-04-13 21:36:56,194+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:57,201+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:36:57,201+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:36:57,207+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:57,211+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:36:57,524+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:36:57,563+02 ERROR [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] SSH Soft Fencing command failed on host 'onode-1.discovery.intern': Failed to get the session. Stdout: Stderr: 2019-04-13 21:36:57,600+02 INFO [org.ovirt.engine.core.bll.pm.SshSoftFencingCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Lock freed to object 'EngineLock:{exclusiveLocks='[65c45da8-2a46-42e0-9c3a-58748546103b=VDS_FENCE]', sharedLocks=''}' 2019-04-13 21:36:57,677+02 INFO [org.ovirt.engine.core.bll.pm.VdsKdumpDetectionCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Running command: VdsKdumpDetectionCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:36:58,212+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:36:59,218+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:36:59,219+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:37:00,225+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:37:00,521+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:00,522+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:02,227+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:37:02,234+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:37:02,241+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:03,534+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:04,246+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:04,247+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:37:05,251+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:37:06,257+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:06,541+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:06,543+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:07,266+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:37:07,267+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:37:07,273+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:07,279+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:37:08,280+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:37:09,292+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:09,293+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:37:09,553+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:10,297+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:37:12,298+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:37:12,301+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:37:12,303+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:12,562+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:12,563+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:14,311+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:14,312+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:37:15,320+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:37:15,573+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:16,325+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:17,329+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:37:17,329+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:37:17,331+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:17,333+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:37:18,334+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:37:18,579+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:18,587+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:19,341+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:19,341+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:37:20,348+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:37:21,602+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:22,350+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:37:22,355+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:37:22,360+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:24,364+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:24,365+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:37:24,609+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:24,610+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:25,371+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:37:26,378+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:27,384+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:37:27,385+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:37:27,389+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:27,393+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:37:27,641+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:27,845+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] EVENT_ID: KDUMP_FLOW_NOT_DETECTED_ON_VDS(615), Kdump flow is not in progress on host onode-1. 2019-04-13 21:37:27,846+02 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Trying to release exclusive lock which does not exist, lock key: '65c45da8-2a46-42e0-9c3a-58748546103bVDS_FENCE' 2019-04-13 21:37:27,866+02 INFO [org.ovirt.engine.core.bll.pm.VdsKdumpDetectionCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [28df0d16] Lock freed to object 'EngineLock:{exclusiveLocks='[65c45da8-2a46-42e0-9c3a-58748546103b=VDS_FENCE]', sharedLocks=''}' 2019-04-13 21:37:28,108+02 ERROR [org.ovirt.engine.core.bll.pm.FenceProxyLocator] (EE-ManagedThreadFactory-engine-Thread-229859) [] Can not run fence action on host 'onode-1', no suitable proxy host was found. 2019-04-13 21:37:28,109+02 ERROR [org.ovirt.engine.core.bll.pm.RestartVdsCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [] Failed to run RestartVdsCommand on vds 'onode-1' 2019-04-13 21:37:28,109+02 WARN [org.ovirt.engine.core.bll.pm.RestartVdsCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [] Validation of action 'RestartVds' failed for user SYSTEM. Reasons: VAR__ACTION__RESTART,VDS_NO_VDS_PROXY_FOUND,VDS_FENCE_OPERATION_FAILED,VAR__TYPE__HOST,VAR__ACTION__RESTART 2019-04-13 21:37:28,110+02 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-229859) [] Trying to release exclusive lock which does not exist, lock key: '65c45da8-2a46-42e0-9c3a-58748546103bVDS_FENCE' 2019-04-13 21:37:28,110+02 INFO [org.ovirt.engine.core.bll.pm.RestartVdsCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [] Lock freed to object 'EngineLock:{exclusiveLocks='[65c45da8-2a46-42e0-9c3a-58748546103b=VDS_FENCE]', sharedLocks=''}' 2019-04-13 21:37:28,133+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229859) [] EVENT_ID: VDS_RECOVER(14), Host onode-1 is rebooting. 2019-04-13 21:37:28,165+02 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engine-Thread-229859) [] Trying to release exclusive lock which does not exist, lock key: '65c45da8-2a46-42e0-9c3a-58748546103bVDS_FENCE' 2019-04-13 21:37:28,165+02 INFO [org.ovirt.engine.core.bll.pm.VdsNotRespondingTreatmentCommand] (EE-ManagedThreadFactory-engine-Thread-229859) [] Lock freed to object 'EngineLock:{exclusiveLocks='[65c45da8-2a46-42e0-9c3a-58748546103b=VDS_FENCE]', sharedLocks=''}' 2019-04-13 21:37:28,394+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:37:29,398+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:29,399+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:37:30,119+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:37:30,403+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:37:30,650+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:30,651+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:32,405+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:37:32,410+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:37:32,415+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:33,661+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:34,423+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:34,424+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:37:35,429+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:37:36,436+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:36,667+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:36,668+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:37,443+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:37:37,444+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:37:37,449+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:37,455+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:37:38,456+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:37:39,468+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:39,469+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:37:39,680+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:40,473+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:37:42,474+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:37:42,479+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:37:42,484+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:42,687+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:42,689+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:44,490+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:44,491+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:37:45,497+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:37:45,698+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:46,503+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:47,506+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:37:47,506+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:37:47,508+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:47,510+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:37:48,511+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:37:48,705+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:48,706+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:49,517+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:49,518+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:37:50,524+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:37:51,716+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:52,525+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:37:52,530+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:37:52,533+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:54,539+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:54,539+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:37:54,721+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:37:54,722+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:37:55,571+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:37:56,578+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:57,584+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:37:57,584+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:37:57,589+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:57,593+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:37:57,732+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:37:58,594+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:37:59,598+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:37:59,599+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:38:00,604+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:38:00,737+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:00,738+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:02,605+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:38:02,610+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:38:02,614+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:03,746+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:04,618+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:04,618+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:38:05,622+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-18) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:38:06,625+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:06,753+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:06,754+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:07,635+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:38:07,635+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:38:07,644+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:07,652+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:38:08,654+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:38:09,661+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:09,661+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:38:09,763+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:10,666+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:38:12,667+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:38:12,671+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:38:12,674+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:12,769+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:12,770+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:14,683+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:14,684+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:38:15,690+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:38:15,778+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:16,694+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:17,699+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:38:17,699+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:38:17,703+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:17,707+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:38:18,708+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:38:18,785+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:18,786+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-92) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:19,714+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:19,715+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:38:20,724+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:38:21,798+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:22,727+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:38:22,730+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:38:22,734+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:24,738+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:24,738+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:38:24,805+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:24,806+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-97) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:25,746+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-66) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:38:26,752+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:27,759+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:38:27,759+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:38:27,765+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:27,770+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:38:27,815+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:28,770+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:38:29,776+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:29,777+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:38:30,142+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:38:30,781+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:38:30,821+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:30,822+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:32,783+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:38:32,791+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:38:32,797+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:32,985+02 INFO [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-60) [] User admin@internal successfully logged in with scopes: ovirt-app-admin ovirt-app-api ovirt-app-portal ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access 2019-04-13 21:38:33,249+02 INFO [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default task-60) [19f1595b] Running command: CreateUserSessionCommand internal: false. 2019-04-13 21:38:33,291+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-60) [19f1595b] EVENT_ID: USER_VDC_LOGIN(30), User admin@internal-authz connecting from '192.168.0.1' using session 'r5QfGuCH+wtAo66j9WJmULaADkUScW5yV3nsCNcuNz5IPt8XVCXCtqzBT62QH4rut9a5nYhlfVmRv5Kk7msARg==' logged in. 2019-04-13 21:38:33,831+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:34,822+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:34,823+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:38:35,827+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:38:36,833+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:36,837+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:36,839+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:37,840+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:38:37,841+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:38:37,845+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:37,852+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:38:38,853+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:38:39,849+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:39,859+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:39,859+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:38:40,863+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:38:40,869+02 INFO [org.ovirt.engine.core.utils.servlet.ServletUtils] (default task-57) [] Can't read file '/usr/share/ovirt-engine/files/spice/SpiceVersion.txt' for request '/ovirt-engine/services/files/spice/SpiceVersion.txt' -- 404 2019-04-13 21:38:42,855+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:42,856+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:42,864+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:38:42,869+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:38:42,873+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:44,880+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:44,880+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:38:45,865+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:45,885+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:38:46,890+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-65) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:47,897+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:38:47,897+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:38:47,901+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:47,905+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:38:48,871+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:48,872+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:48,907+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:38:49,913+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:49,914+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:38:50,920+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:38:51,882+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:52,921+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:38:52,924+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:38:52,926+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:54,889+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:38:54,890+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:38:54,932+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:54,933+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:38:55,939+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:38:56,946+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:57,900+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:38:57,952+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:38:57,953+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:38:57,957+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:57,960+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:38:58,961+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:38:59,967+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:38:59,967+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:39:00,907+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:00,908+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:00,973+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-96) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:39:02,975+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:39:02,981+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:39:02,988+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:03,919+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:04,992+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:04,992+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:39:05,996+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:39:06,925+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:06,926+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:07,001+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-52) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:08,008+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:39:08,009+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:39:08,014+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:08,020+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-31) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:39:09,021+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:39:09,937+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:10,028+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:10,029+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:39:11,034+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:39:12,943+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:12,944+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:13,035+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:39:13,039+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:39:13,042+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:15,048+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:15,049+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:39:15,953+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:16,055+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:39:17,059+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:18,063+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:39:18,063+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:39:18,066+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:18,069+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:39:18,583+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command Get Host Capabilities failed: Vds timeout occured 2019-04-13 21:39:18,583+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Vds timeout occured 2019-04-13 21:39:18,596+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSNetworkException: Vds timeout occured 2019-04-13 21:39:18,713+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:18,959+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] IrsBroker::Failed::GetStoragePoolInfoVDS 2019-04-13 21:39:18,959+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:18,959+02 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] Command 'GetStoragePoolInfoVDSCommand( GetStoragePoolInfoVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='true'})' execution failed: IRSProtocolException: 2019-04-13 21:39:18,959+02 WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [] Trying to release exclusive lock which does not exist, lock key: '65c45da8-2a46-42e0-9c3a-58748546103bVDS_INIT' 2019-04-13 21:39:18,960+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand' return value 'org.ovirt.engine.core.vdsbroker.vdsbroker.VDSInfoReturn@70d411cf' 2019-04-13 21:39:18,960+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [] HostName = onode-1 2019-04-13 21:39:18,960+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:19,070+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:39:20,077+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:20,078+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:39:21,087+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:39:21,970+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:23,088+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:39:23,115+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:39:23,131+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:24,977+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:24,978+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:25,135+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:25,136+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:39:26,144+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:39:27,151+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:27,987+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:28,156+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:39:28,156+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:39:28,159+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:28,162+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:39:29,163+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-16) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:39:30,144+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:39:30,169+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:30,170+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:39:30,994+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:30,995+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:31,175+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:39:33,176+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:39:33,182+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:39:33,187+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:34,005+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:35,191+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:35,191+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:39:36,196+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:39:37,011+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:37,012+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:37,203+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:38,209+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:39:38,210+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:39:38,215+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:38,220+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:39:39,221+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:39:40,022+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:40,225+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:40,225+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:39:41,230+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:39:43,030+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:43,031+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:43,231+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:39:43,237+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:39:43,242+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:45,250+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:45,251+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:39:46,039+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:46,255+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:39:47,261+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:48,265+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:39:48,265+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:39:48,267+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:48,269+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:39:49,045+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:49,046+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:49,270+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:39:50,278+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:50,279+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-78) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:39:51,286+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-26) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:39:52,056+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:53,287+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:39:53,290+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:39:53,294+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:55,063+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:39:55,064+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-48) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:39:55,301+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:55,302+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:39:56,309+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:39:57,315+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:58,073+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:39:58,320+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:39:58,320+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:39:58,324+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:39:58,327+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:39:59,328+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:40:00,363+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:00,365+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-62) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:40:01,079+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Unable to RefreshCapabilities: NoRouteToHostException: No route to host 2019-04-13 21:40:01,085+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-45) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.NoRouteToHostException: No route to host 2019-04-13 21:40:01,387+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-72) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:40:03,389+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:40:03,397+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:40:03,404+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-2) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:04,093+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:40:04,096+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Unable to RefreshCapabilities: ConnectException: Connection refused 2019-04-13 21:40:04,097+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.ConnectException: Connection refused 2019-04-13 21:40:05,411+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:05,411+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:40:06,418+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-60) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:40:07,107+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:40:07,110+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [] Unable to RefreshCapabilities: ConnectException: Connection refused 2019-04-13 21:40:07,113+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.ConnectException: Connection refused 2019-04-13 21:40:07,424+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:08,429+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:40:08,430+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:40:08,435+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:08,439+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-82) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:40:09,440+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-46) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:40:10,122+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:40:10,124+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] Unable to RefreshCapabilities: ConnectException: Connection refused 2019-04-13 21:40:10,126+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.ConnectException: Connection refused 2019-04-13 21:40:10,446+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:10,447+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:40:11,453+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:40:13,140+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:40:13,142+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Unable to RefreshCapabilities: ConnectException: Connection refused 2019-04-13 21:40:13,145+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.ConnectException: Connection refused 2019-04-13 21:40:13,455+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:40:13,462+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:40:13,469+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-7) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:15,477+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:15,478+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:40:16,154+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:40:16,156+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Unable to RefreshCapabilities: ConnectException: Connection refused 2019-04-13 21:40:16,167+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.ConnectException: Connection refused 2019-04-13 21:40:16,484+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:40:17,491+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:18,499+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:40:18,500+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:40:18,507+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:18,513+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-38) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:40:19,175+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:40:19,177+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [] Unable to RefreshCapabilities: ConnectException: Connection refused 2019-04-13 21:40:19,179+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetCapabilitiesAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-9) [] Command 'GetCapabilitiesAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'})' execution failed: java.net.ConnectException: Connection refused 2019-04-13 21:40:19,514+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:40:20,521+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:20,522+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:40:21,529+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:40:22,190+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:40:22,320+02 WARN [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able to update response for 2019-04-13 21:40:22,320+02 WARN [org.ovirt.vdsm.jsonrpc.client.JsonRpcClient] (ResponseWorker) [] Not able to update response for 2019-04-13 21:40:22,366+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command SpmStatusVDS failed: Recovering from crash or Initializing 2019-04-13 21:40:22,366+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: Recovering from crash or Initializing 2019-04-13 21:40:22,371+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=99, message=Recovering from crash or Initializing]'} ' 2019-04-13 21:40:22,371+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] HostName = onode-1 2019-04-13 21:40:22,372+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: Recovering from crash or Initializing 2019-04-13 21:40:22,372+02 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Unable to RefreshCapabilities: VDSRecoveringException: Recovering from crash or Initializing 2019-04-13 21:40:23,531+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:40:23,537+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:40:23,541+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-15) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:25,548+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:25,548+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:40:26,556+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:40:27,564+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:28,571+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:40:28,572+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:40:28,576+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:28,581+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:40:29,582+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:40:30,156+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:40:30,588+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:30,589+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-49) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:40:31,596+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-59) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:40:33,329+02 INFO [org.ovirt.engine.core.bll.EngineBackupAwarenessManager] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] Backup check started. 2019-04-13 21:40:33,332+02 INFO [org.ovirt.engine.core.bll.EngineBackupAwarenessManager] (EE-ManagedThreadFactory-engineScheduled-Thread-6) [] Backup check completed. 2019-04-13 21:40:33,598+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:40:33,604+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:40:33,608+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:35,614+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:35,615+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:40:36,623+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:40:37,630+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:38,638+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:40:38,639+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:40:38,644+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:38,651+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:40:39,652+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:40:40,659+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:40,659+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:40:41,665+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:40:41,998+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'commandCoordinator' is using 0 threads out of 10, 10 threads waiting for tasks. 2019-04-13 21:40:41,998+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'default' is using 0 threads out of 1, 5 threads waiting for tasks. 2019-04-13 21:40:41,998+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engine' is using 0 threads out of 500, 10 threads waiting for tasks and 0 tasks in queue. 2019-04-13 21:40:41,998+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engineScheduled' is using 0 threads out of 100, 100 threads waiting for tasks. 2019-04-13 21:40:41,998+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engineThreadMonitoring' is using 1 threads out of 1, 0 threads waiting for tasks. 2019-04-13 21:40:41,999+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'hostUpdatesChecker' is using 0 threads out of 5, 5 threads waiting for tasks. 2019-04-13 21:40:43,667+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:40:43,673+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:40:43,677+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:45,684+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:45,685+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:40:46,692+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:40:47,698+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:48,704+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:40:48,705+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:40:48,710+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:48,714+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:40:49,715+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-19) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:40:50,728+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:50,729+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:40:50,954+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [] START, GetHardwareInfoAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'}), log id: 70825ca7 2019-04-13 21:40:50,955+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 70825ca7 2019-04-13 21:40:51,180+02 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [5fa1e25] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:40:51,323+02 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [7197d30c] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:40:51,370+02 INFO [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:40:51,603+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] START, ConnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='2286311a-c1c9-4dda-bea5-9c78651c6e36', connection='onode-1.discovery.intern:/mnt/store', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 4d37d2b3 2019-04-13 21:40:51,756+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:40:51,997+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] FINISH, ConnectStorageServerVDSCommand, return: {2286311a-c1c9-4dda-bea5-9c78651c6e36=0}, log id: 4d37d2b3 2019-04-13 21:40:52,005+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] START, ConnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='b4f4bcf2-e104-468e-b330-ed731ec8ef6c', connection='nas.discovery.intern:/mnt/d2600/vms', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 4ad8d92c 2019-04-13 21:40:52,246+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] FINISH, ConnectStorageServerVDSCommand, return: {b4f4bcf2-e104-468e-b330-ed731ec8ef6c=0}, log id: 4ad8d92c 2019-04-13 21:40:52,249+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] START, ConnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='07dd9ff2-afa1-4a33-b401-935ea5b8c9db', connection='nas.discovery.intern:/mnt/d2600/isos', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='false'}), log id: 2657dc50 2019-04-13 21:40:52,377+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] FINISH, ConnectStorageServerVDSCommand, return: {07dd9ff2-afa1-4a33-b401-935ea5b8c9db=0}, log id: 2657dc50 2019-04-13 21:40:52,381+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] START, ConnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='858731bf-2fb1-45a7-a693-235f675a667b', connection='nas.discovery.intern:/mnt/d2600/sicherung/vmBackups', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='false'}), log id: 38bcf88b 2019-04-13 21:40:52,506+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] FINISH, ConnectStorageServerVDSCommand, return: {858731bf-2fb1-45a7-a693-235f675a667b=0}, log id: 38bcf88b 2019-04-13 21:40:52,517+02 INFO [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] Host 'onode-1' storage connection was succeeded 2019-04-13 21:40:52,581+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229899) [344934b8] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 671d1585 2019-04-13 21:40:52,582+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229899) [344934b8] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:40:52,704+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 7da2ec79 2019-04-13 21:40:52,708+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:40:53,156+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229899) [344934b8] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 671d1585 2019-04-13 21:40:53,514+02 ERROR [org.ovirt.engine.core.bll.pm.FenceProxyLocator] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] Can not run fence action on host 'onode-1', no suitable proxy host was found. 2019-04-13 21:40:53,523+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] START, SetMOMPolicyParametersVDSCommand(HostName = onode-1, MomPolicyVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'}), log id: 1408acc9 2019-04-13 21:40:53,593+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [344934b8] FINISH, SetMOMPolicyParametersVDSCommand, return: , log id: 1408acc9 2019-04-13 21:40:53,618+02 INFO [org.ovirt.engine.core.bll.hostdev.RefreshHostDevicesCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [7e5fcf82] Running command: RefreshHostDevicesCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 21:40:53,626+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 7da2ec79 2019-04-13 21:40:53,669+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:40:53,676+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command SpmStatusVDS failed: Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory') 2019-04-13 21:40:53,677+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:40:53,677+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:40:53,757+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Waiting on merge command to complete (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) 2019-04-13 21:40:53,761+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' (id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49') waiting on child command id: 'c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4' type:'Merge' to complete 2019-04-13 21:40:53,765+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:53,853+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [7e5fcf82] EVENT_ID: VDS_DETECTED(13), Status of host onode-1 was set to Up. 2019-04-13 21:40:53,867+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [7e5fcf82] EVENT_ID: VDS_ALERT_FENCE_TEST_FAILED(9,001), Power Management test failed for Host onode-1.There is no other host in the data center that can be used to test the power management settings. 2019-04-13 21:40:53,874+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [7e5fcf82] EVENT_ID: VDS_FENCE_STATUS_FAILED(497), Failed to verify Host onode-1 power management. 2019-04-13 21:40:53,944+02 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-68) [6d5aea51] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:40:54,894+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] Fetched 16 VMs from VDS '65c45da8-2a46-42e0-9c3a-58748546103b' 2019-04-13 21:40:54,921+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '611486ee-b72f-4acc-8925-4d7b6df9af1d' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:54,940+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='611486ee-b72f-4acc-8925-4d7b6df9af1d', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 1d9f9b1b 2019-04-13 21:40:55,397+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 1d9f9b1b 2019-04-13 21:40:55,398+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '611486ee-b72f-4acc-8925-4d7b6df9af1d'(taiga) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:55,496+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM taiga is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:55,503+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '28d7d5ff-2ca1-42c6-ae8d-065c721d3066' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:55,504+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='28d7d5ff-2ca1-42c6-ae8d-065c721d3066', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 1988031a 2019-04-13 21:40:55,794+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:55,794+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-55) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Waiting on merge command to complete (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) 2019-04-13 21:40:55,943+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 1988031a 2019-04-13 21:40:55,943+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '28d7d5ff-2ca1-42c6-ae8d-065c721d3066'(mosaico) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:55,966+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM mosaico is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:55,968+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '40eaf7e2-659c-4790-8362-49fe12c5f61a' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:55,969+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='40eaf7e2-659c-4790-8362-49fe12c5f61a', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 390fa347 2019-04-13 21:40:56,415+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 390fa347 2019-04-13 21:40:56,415+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '40eaf7e2-659c-4790-8362-49fe12c5f61a'(transfer) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:56,437+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM transfer is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:56,439+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:56,441+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 393ea7dd 2019-04-13 21:40:56,800+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' (id: '1c8ebed4-5850-4083-9565-95addb69b123') waiting on child command id: '8ba1b508-f4c1-42ee-9e22-9347bd78f24f' type:'Merge' to complete 2019-04-13 21:40:56,878+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 393ea7dd 2019-04-13 21:40:56,878+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6'(gogs) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:56,907+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM gogs is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:56,909+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '6772f848-4874-4785-9bb0-88d40bfb71c4' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:56,911+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='6772f848-4874-4785-9bb0-88d40bfb71c4', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 6af506fd 2019-04-13 21:40:57,349+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 6af506fd 2019-04-13 21:40:57,349+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '6772f848-4874-4785-9bb0-88d40bfb71c4'(notebook) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:57,373+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM notebook is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:57,376+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'afa2c17f-39e4-4168-aad7-2548dd19cc26' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:57,377+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='afa2c17f-39e4-4168-aad7-2548dd19cc26', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 81c2aab 2019-04-13 21:40:57,809+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-24) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:57,816+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 81c2aab 2019-04-13 21:40:57,817+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'afa2c17f-39e4-4168-aad7-2548dd19cc26'(pass) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:57,845+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM pass is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:57,850+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'be42701b-3f74-4c81-b6ef-5f7e7806554a' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:57,851+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='be42701b-3f74-4c81-b6ef-5f7e7806554a', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 27b011a3 2019-04-13 21:40:58,290+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 27b011a3 2019-04-13 21:40:58,290+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'be42701b-3f74-4c81-b6ef-5f7e7806554a'(win2016-02) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:58,317+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM win2016-02 is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:58,319+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '5a326302-86c1-41b5-ba58-793232299713' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:58,322+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='5a326302-86c1-41b5-ba58-793232299713', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 27c337e3 2019-04-13 21:40:58,778+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 27c337e3 2019-04-13 21:40:58,778+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '5a326302-86c1-41b5-ba58-793232299713'(tv-planer) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:58,802+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM tv-planer is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:58,803+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'a40d738c-e8ec-43a8-8ae3-1fe3ebf2dfa3' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:58,804+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='a40d738c-e8ec-43a8-8ae3-1fe3ebf2dfa3', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 1fec5924 2019-04-13 21:40:58,815+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:40:58,816+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Waiting on merge command to complete (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) 2019-04-13 21:40:58,820+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:40:58,825+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-77) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:40:59,244+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 1fec5924 2019-04-13 21:40:59,244+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'a40d738c-e8ec-43a8-8ae3-1fe3ebf2dfa3'(webdev) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:59,274+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM webdev is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:59,276+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'b5713713-ac04-4a91-95b8-3c3f17eba32f' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:59,279+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='b5713713-ac04-4a91-95b8-3c3f17eba32f', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 743a90cd 2019-04-13 21:40:59,717+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 743a90cd 2019-04-13 21:40:59,717+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'b5713713-ac04-4a91-95b8-3c3f17eba32f'(timetrack) moved from 'Unknown' --> 'Down' 2019-04-13 21:40:59,742+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM timetrack is down with error. Exit message: VM terminated with error. 2019-04-13 21:40:59,743+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '139534f6-1741-4ded-8ea6-ea4e3ecdad1f' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:40:59,745+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='139534f6-1741-4ded-8ea6-ea4e3ecdad1f', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 300ec53b 2019-04-13 21:40:59,826+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Waiting on merge command to complete (jobId = a6032849-8a43-404b-aeef-80952e045501) 2019-04-13 21:41:00,183+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 300ec53b 2019-04-13 21:41:00,183+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '139534f6-1741-4ded-8ea6-ea4e3ecdad1f'(srs) moved from 'Unknown' --> 'Down' 2019-04-13 21:41:00,208+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM srs is down with error. Exit message: VM terminated with error. 2019-04-13 21:41:00,209+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '5dd5db29-143f-4d70-9469-4aaa163b7f9d' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:41:00,211+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='5dd5db29-143f-4d70-9469-4aaa163b7f9d', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 6ab0de8f 2019-04-13 21:41:00,652+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 6ab0de8f 2019-04-13 21:41:00,652+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '5dd5db29-143f-4d70-9469-4aaa163b7f9d'(win2016-01) moved from 'Unknown' --> 'Down' 2019-04-13 21:41:00,681+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM win2016-01 is down with error. Exit message: VM terminated with error. 2019-04-13 21:41:00,683+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '0f8aff0a-7915-4214-a37c-c9fee6518869' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:41:00,685+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='0f8aff0a-7915-4214-a37c-c9fee6518869', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 63eae24a 2019-04-13 21:41:00,834+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:00,835+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [49067302-e424-4846-9a36-d7cce26e4800] Waiting on merge command to complete (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) 2019-04-13 21:41:01,125+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 63eae24a 2019-04-13 21:41:01,126+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '0f8aff0a-7915-4214-a37c-c9fee6518869'(pxe) moved from 'Unknown' --> 'Down' 2019-04-13 21:41:01,147+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM pxe is down with error. Exit message: VM terminated with error. 2019-04-13 21:41:01,157+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'dd24e11f-40aa-47d2-9f0b-11cd7e83649d' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:41:01,159+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='dd24e11f-40aa-47d2-9f0b-11cd7e83649d', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 380918d7 2019-04-13 21:41:01,597+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 380918d7 2019-04-13 21:41:01,598+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'dd24e11f-40aa-47d2-9f0b-11cd7e83649d'(ffplayout-test) moved from 'Unknown' --> 'Down' 2019-04-13 21:41:01,621+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM ffplayout-test is down with error. Exit message: VM terminated with error. 2019-04-13 21:41:01,622+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'c8df4609-fe26-4b73-9137-57724aff3ca4' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:41:01,623+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='c8df4609-fe26-4b73-9137-57724aff3ca4', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 6effb487 2019-04-13 21:41:01,841+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-69) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' (id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0') waiting on child command id: '82a6eac3-5839-4b63-86c9-5b1212decb17' type:'Merge' to complete 2019-04-13 21:41:02,063+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 6effb487 2019-04-13 21:41:02,063+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM 'c8df4609-fe26-4b73-9137-57724aff3ca4'(remote) moved from 'Unknown' --> 'Down' 2019-04-13 21:41:02,087+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM remote is down with error. Exit message: VM terminated with error. 2019-04-13 21:41:02,088+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '7b5e0fdd-8757-49ce-acf9-3206a372e1c9' was reported as Down on VDS '65c45da8-2a46-42e0-9c3a-58748546103b'(onode-1) 2019-04-13 21:41:02,089+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, DestroyVDSCommand(HostName = onode-1, DestroyVmVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='7b5e0fdd-8757-49ce-acf9-3206a372e1c9', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 13b5f91 2019-04-13 21:41:02,527+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DestroyVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, DestroyVDSCommand, return: , log id: 13b5f91 2019-04-13 21:41:02,527+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmAnalyzer] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM '7b5e0fdd-8757-49ce-acf9-3206a372e1c9'(ffplayout) moved from 'Unknown' --> 'Down' 2019-04-13 21:41:02,549+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] EVENT_ID: VM_DOWN_ERROR(119), VM ffplayout is down with error. Exit message: VM terminated with error. 2019-04-13 21:41:02,635+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [a63731eb-c09b-4768-bf6f-945e52bf30e1] was removed, VM is down 2019-04-13 21:41:02,635+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [b76fee8b-a55c-48fd-8131-8495d28f8b2d] was removed, VM is down 2019-04-13 21:41:02,635+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [29e1ba11-e3bf-4d30-a48e-3fc03894dd46] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [99ffb9be-08c1-4b82-9b00-824da1c7ffad] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [74361f16-e6a9-4261-a31e-d80859374959] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [2beb2da5-965f-4ca5-a9c9-d39782647bff] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [0ae39f4c-27b6-4508-a1d5-e57205dff0fb] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [3b89837b-e878-4963-a405-846ca0b07c43] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [e4785d16-02a1-4ec5-a014-8e9fbb1d1c89] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [5cc49657-6b94-4c77-a4bd-e179dfa978e8] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [e0895ea4-b7bd-4d92-bac4-e89ba43c859c] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [a6032849-8a43-404b-aeef-80952e045501] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [ad91d1ab-d043-410a-84d3-507458329538] was removed, VM is down 2019-04-13 21:41:02,636+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmJobsMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] VM Job [46600279-7fa9-4ade-9c9e-5a6d22c572e6] was removed, VM is down 2019-04-13 21:41:02,673+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [4a070c7] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:02,755+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [1adc1c93] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:02,809+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [28196424] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:02,856+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [60db45f1] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:02,897+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [262ed1d6] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:02,940+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [5b7d5358] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:02,980+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [796325a2] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,013+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [7b8f2e60] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,043+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [42eda645] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,076+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [35df35b] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,102+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [246f614d] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,127+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [4b88b8a8] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,152+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [2f4ce40e] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,209+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [5154ce5a] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,247+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [331b7316] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,281+02 INFO [org.ovirt.engine.core.bll.ProcessDownVmCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [67b0b973] Running command: ProcessDownVmCommand internal: true. 2019-04-13 21:41:03,345+02 INFO [org.ovirt.engine.core.bll.network.host.RefreshHostCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [49f271c] Running command: RefreshHostCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 21:41:03,357+02 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [a2dbe5a] Before acquiring and wait lock 'EngineLock:{exclusiveLocks='[HOST_NETWORK65c45da8-2a46-42e0-9c3a-58748546103b=HOST_NETWORK, 65c45da8-2a46-42e0-9c3a-58748546103b=VDS]', sharedLocks=''}' 2019-04-13 21:41:03,358+02 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [a2dbe5a] Lock-wait acquired to object 'EngineLock:{exclusiveLocks='[HOST_NETWORK65c45da8-2a46-42e0-9c3a-58748546103b=HOST_NETWORK, 65c45da8-2a46-42e0-9c3a-58748546103b=VDS]', sharedLocks=''}' 2019-04-13 21:41:03,366+02 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [a2dbe5a] Running command: RefreshHostCapabilitiesCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 21:41:03,367+02 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [a2dbe5a] Before acquiring lock in order to prevent monitoring for host 'onode-1' from data-center 'Default' 2019-04-13 21:41:03,367+02 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [a2dbe5a] Lock acquired, from now a monitoring of host will be skipped for host 'onode-1' from data-center 'Default' 2019-04-13 21:41:03,709+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:03,710+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:03,717+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:41:03,717+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] HostName = onode-1 2019-04-13 21:41:03,717+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:41:03,762+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 122932f 2019-04-13 21:41:03,762+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:41:03,866+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Merge command (jobId = a63731eb-c09b-4768-bf6f-945e52bf30e1) has completed for images 'a1d583c1-e135-4e3f-b6fa-0f29a5261366'..'47c0f42e-8bda-4e3f-8337-870899238788' 2019-04-13 21:41:03,887+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Executing Live Merge command step 'MERGE_STATUS' 2019-04-13 21:41:03,908+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:03,920+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-9) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Running command: MergeStatusCommand internal: true. Entities affected : ID: 9c3f06cf-7475-448e-819b-f4f52fa7d782 Type: Storage 2019-04-13 21:41:03,922+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-9) [6149b722-e07b-427d-8cbd-6b49d409bf8c] VM down, waiting on SPM election to resolve Live Merge 2019-04-13 21:41:04,239+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 122932f 2019-04-13 21:41:04,251+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:04,251+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:04,252+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:41:04,919+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Executing Live Merge command step 'DESTROY_IMAGE' 2019-04-13 21:41:04,920+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' with children [c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4, a79d7e54-bc26-4cef-8689-642fadbde9e2, e0a2c4f9-1708-4c8e-b178-5c204a6a8688] failed when attempting to perform the next operation, marking as 'ACTIVE' 2019-04-13 21:41:04,920+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [6149b722-e07b-427d-8cbd-6b49d409bf8c] null: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.performNextOperation(RemoveSnapshotSingleDiskLiveCommand.java:133) [bll.jar:] at org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32) [bll.jar:] at org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:77) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:04,929+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49 failed child command status for step 'DESTROY_IMAGE' 2019-04-13 21:41:04,929+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshotSingleDiskLive' id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' child commands '[c73b3c40-8fa0-4eec-80cc-93cdd6b7a1d4, a79d7e54-bc26-4cef-8689-642fadbde9e2, e0a2c4f9-1708-4c8e-b178-5c204a6a8688]' executions were completed, status 'FAILED' 2019-04-13 21:41:05,942+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:05,959+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Merge command (jobId = 5cc49657-6b94-4c77-a4bd-e179dfa978e8) has completed for images '67a4dd71-69ff-49a8-b8a4-c7101c75e105'..'b7fb83ef-77ba-408b-951c-a655c708638b' 2019-04-13 21:41:05,966+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-87) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Failed invoking callback end method 'onFailed' for command 'ea8a5636-af5a-4a6e-979c-361a3070cf49' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:06,859+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [a2dbe5a] START, GetHardwareInfoAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'}), log id: 2153cad3 2019-04-13 21:41:06,863+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [a2dbe5a] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 2153cad3 2019-04-13 21:41:06,996+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Executing Live Merge command step 'MERGE_STATUS' 2019-04-13 21:41:07,000+02 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [6d77a6ec] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:41:07,015+02 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [d37fc59] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:41:07,026+02 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [d37fc59] Refresh host capabilities finished. Lock released. Monitoring can run now for host 'onode-1' from data-center 'Default' 2019-04-13 21:41:07,039+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Running command: MergeStatusCommand internal: true. Entities affected : ID: 9c3f06cf-7475-448e-819b-f4f52fa7d782 Type: Storage 2019-04-13 21:41:07,040+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229904) [d37fc59] EVENT_ID: HOST_REFRESHED_CAPABILITIES(606), Successfully refreshed the capabilities of host onode-1. 2019-04-13 21:41:07,041+02 INFO [org.ovirt.engine.core.bll.RefreshHostCapabilitiesCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [d37fc59] Lock freed to object 'EngineLock:{exclusiveLocks='[HOST_NETWORK65c45da8-2a46-42e0-9c3a-58748546103b=HOST_NETWORK, 65c45da8-2a46-42e0-9c3a-58748546103b=VDS]', sharedLocks=''}' 2019-04-13 21:41:07,041+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-10) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] VM down, waiting on SPM election to resolve Live Merge 2019-04-13 21:41:07,055+02 INFO [org.ovirt.engine.core.bll.hostdev.RefreshHostDevicesCommand] (EE-ManagedThreadFactory-engine-Thread-229904) [a2dbe5a] Running command: RefreshHostDevicesCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 21:41:08,029+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Executing Live Merge command step 'DESTROY_IMAGE' 2019-04-13 21:41:08,029+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' id: '1c8ebed4-5850-4083-9565-95addb69b123' with children [8ba1b508-f4c1-42ee-9e22-9347bd78f24f, b6335765-876e-4131-8be8-56abb5b789c0, 336b8b94-9a94-4b31-ab7b-70b2f0e54843] failed when attempting to perform the next operation, marking as 'ACTIVE' 2019-04-13 21:41:08,029+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] null: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.performNextOperation(RemoveSnapshotSingleDiskLiveCommand.java:133) [bll.jar:] at org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32) [bll.jar:] at org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:77) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:08,030+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command id: '1c8ebed4-5850-4083-9565-95addb69b123 failed child command status for step 'DESTROY_IMAGE' 2019-04-13 21:41:08,031+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshotSingleDiskLive' id: '1c8ebed4-5850-4083-9565-95addb69b123' child commands '[8ba1b508-f4c1-42ee-9e22-9347bd78f24f, b6335765-876e-4131-8be8-56abb5b789c0, 336b8b94-9a94-4b31-ab7b-70b2f0e54843]' executions were completed, status 'FAILED' 2019-04-13 21:41:08,040+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Failed invoking callback end method 'onFailed' for command 'ea8a5636-af5a-4a6e-979c-361a3070cf49' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:08,046+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-95) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:09,053+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' (id: 'a869efce-6672-4363-9082-27f3c2b86635') waiting on child command id: '521230fb-1de0-43f0-a16c-515aaff36b50' type:'Merge' to complete 2019-04-13 21:41:09,058+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Failed invoking callback end method 'onFailed' for command '1c8ebed4-5850-4083-9565-95addb69b123' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:09,072+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Merge command (jobId = 29e1ba11-e3bf-4d30-a48e-3fc03894dd46) has completed for images '44b3ceca-f0db-43da-af29-aa681df146ac'..'8fcf0d6e-a32e-46f1-8601-2960508a6986' 2019-04-13 21:41:09,081+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:09,088+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' (id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8') waiting on child command id: '6912e759-b989-4cda-a53f-749fac169a4a' type:'Merge' to complete 2019-04-13 21:41:10,105+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Merge command (jobId = a6032849-8a43-404b-aeef-80952e045501) has completed for images '2b136ed7-8b9a-45c1-9edc-64ee96c87c43'..'940f5e42-77b4-4b95-ae11-62ecf1f3a5de' 2019-04-13 21:41:11,114+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:11,126+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Executing Live Merge command step 'MERGE_STATUS' 2019-04-13 21:41:11,153+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Failed invoking callback end method 'onFailed' for command '1c8ebed4-5850-4083-9565-95addb69b123' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:11,163+02 INFO [org.ovirt.engine.core.bll.MergeCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [49067302-e424-4846-9a36-d7cce26e4800] Merge command (jobId = 46600279-7fa9-4ade-9c9e-5a6d22c572e6) has completed for images 'ff358cbb-ba3d-4fe5-8ad9-7ca4406489a1'..'149564f0-7406-4bed-a091-8d965f603283' 2019-04-13 21:41:11,168+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Running command: MergeStatusCommand internal: true. Entities affected : ID: 9c3f06cf-7475-448e-819b-f4f52fa7d782 Type: Storage 2019-04-13 21:41:11,170+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Executing Live Merge command step 'MERGE_STATUS' 2019-04-13 21:41:11,170+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-1) [6e59b274-880f-4da4-8382-2a0cf1518c6b] VM down, waiting on SPM election to resolve Live Merge 2019-04-13 21:41:11,200+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Running command: MergeStatusCommand internal: true. Entities affected : ID: 9c3f06cf-7475-448e-819b-f4f52fa7d782 Type: Storage 2019-04-13 21:41:11,202+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-2) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] VM down, waiting on SPM election to resolve Live Merge 2019-04-13 21:41:12,192+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [49067302-e424-4846-9a36-d7cce26e4800] Executing Live Merge command step 'MERGE_STATUS' 2019-04-13 21:41:12,214+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Failed invoking callback end method 'onFailed' for command 'ea8a5636-af5a-4a6e-979c-361a3070cf49' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:12,229+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-3) [49067302-e424-4846-9a36-d7cce26e4800] Running command: MergeStatusCommand internal: true. Entities affected : ID: 9c3f06cf-7475-448e-819b-f4f52fa7d782 Type: Storage 2019-04-13 21:41:12,232+02 INFO [org.ovirt.engine.core.bll.MergeStatusCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-3) [49067302-e424-4846-9a36-d7cce26e4800] VM down, waiting on SPM election to resolve Live Merge 2019-04-13 21:41:13,222+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [49067302-e424-4846-9a36-d7cce26e4800] Executing Live Merge command step 'DESTROY_IMAGE' 2019-04-13 21:41:13,223+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' with children [82a6eac3-5839-4b63-86c9-5b1212decb17, d272164f-ec1d-442e-9367-4c80ac86e976, f94dbb02-e492-426f-a288-3853975c65be] failed when attempting to perform the next operation, marking as 'ACTIVE' 2019-04-13 21:41:13,223+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [49067302-e424-4846-9a36-d7cce26e4800] null: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.performNextOperation(RemoveSnapshotSingleDiskLiveCommand.java:133) [bll.jar:] at org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32) [bll.jar:] at org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:77) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:13,224+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [49067302-e424-4846-9a36-d7cce26e4800] Command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0 failed child command status for step 'DESTROY_IMAGE' 2019-04-13 21:41:13,225+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshotSingleDiskLive' id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' child commands '[82a6eac3-5839-4b63-86c9-5b1212decb17, d272164f-ec1d-442e-9367-4c80ac86e976, f94dbb02-e492-426f-a288-3853975c65be]' executions were completed, status 'FAILED' 2019-04-13 21:41:13,237+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Executing Live Merge command step 'DESTROY_IMAGE' 2019-04-13 21:41:13,237+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' id: 'a869efce-6672-4363-9082-27f3c2b86635' with children [521230fb-1de0-43f0-a16c-515aaff36b50, 236693c4-711e-4adb-b72e-688979d45ec6, ef68a676-0d23-4142-880f-e49d5ee45f95] failed when attempting to perform the next operation, marking as 'ACTIVE' 2019-04-13 21:41:13,237+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] null: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.performNextOperation(RemoveSnapshotSingleDiskLiveCommand.java:133) [bll.jar:] at org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32) [bll.jar:] at org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:77) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:13,238+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command id: 'a869efce-6672-4363-9082-27f3c2b86635 failed child command status for step 'DESTROY_IMAGE' 2019-04-13 21:41:13,238+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshotSingleDiskLive' id: 'a869efce-6672-4363-9082-27f3c2b86635' child commands '[521230fb-1de0-43f0-a16c-515aaff36b50, 236693c4-711e-4adb-b72e-688979d45ec6, ef68a676-0d23-4142-880f-e49d5ee45f95]' executions were completed, status 'FAILED' 2019-04-13 21:41:13,249+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Executing Live Merge command step 'DESTROY_IMAGE' 2019-04-13 21:41:13,249+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' with children [6912e759-b989-4cda-a53f-749fac169a4a, 2f820a2e-7a70-4c2c-b109-1f83aca510ad, 6e16cd50-4775-4e96-9d0c-6ee63276fc34] failed when attempting to perform the next operation, marking as 'ACTIVE' 2019-04-13 21:41:13,249+02 ERROR [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] null: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.performNextOperation(RemoveSnapshotSingleDiskLiveCommand.java:133) [bll.jar:] at org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback.childCommandsExecutionEnded(SerialChildCommandsExecutionCallback.java:32) [bll.jar:] at org.ovirt.engine.core.bll.ChildCommandsCallbackBase.doPolling(ChildCommandsCallbackBase.java:77) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:175) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:13,250+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8 failed child command status for step 'DESTROY_IMAGE' 2019-04-13 21:41:13,251+02 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshotSingleDiskLive' id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' child commands '[6912e759-b989-4cda-a53f-749fac169a4a, 2f820a2e-7a70-4c2c-b109-1f83aca510ad, 6e16cd50-4775-4e96-9d0c-6ee63276fc34]' executions were completed, status 'FAILED' 2019-04-13 21:41:14,261+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [49067302-e424-4846-9a36-d7cce26e4800] Failed invoking callback end method 'onFailed' for command 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:14,265+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Failed invoking callback end method 'onFailed' for command 'a869efce-6672-4363-9082-27f3c2b86635' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:14,269+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Failed invoking callback end method 'onFailed' for command 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:14,275+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-84) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:14,286+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:14,287+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:14,291+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:41:14,293+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] HostName = onode-1 2019-04-13 21:41:14,293+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:41:14,339+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 3ba314fa 2019-04-13 21:41:14,340+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:41:14,836+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 3ba314fa 2019-04-13 21:41:14,846+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:14,846+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:14,847+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-28) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:41:15,281+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-99) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Failed invoking callback end method 'onFailed' for command '1c8ebed4-5850-4083-9565-95addb69b123' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:16,287+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [49067302-e424-4846-9a36-d7cce26e4800] Failed invoking callback end method 'onFailed' for command 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:16,291+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Failed invoking callback end method 'onFailed' for command 'a869efce-6672-4363-9082-27f3c2b86635' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:16,295+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:16,299+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Failed invoking callback end method 'onFailed' for command 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:17,649+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-54) [] Fetched 0 VMs from VDS '65c45da8-2a46-42e0-9c3a-58748546103b' 2019-04-13 21:41:18,306+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-75) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:19,313+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-20) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:20,319+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [49067302-e424-4846-9a36-d7cce26e4800] Failed invoking callback end method 'onFailed' for command 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:20,326+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Failed invoking callback end method 'onFailed' for command 'a869efce-6672-4363-9082-27f3c2b86635' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:20,330+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Failed invoking callback end method 'onFailed' for command 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:20,335+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-36) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Failed invoking callback end method 'onFailed' for command 'ea8a5636-af5a-4a6e-979c-361a3070cf49' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:21,342+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-98) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:23,349+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Failed invoking callback end method 'onFailed' for command '1c8ebed4-5850-4083-9565-95addb69b123' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:24,355+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:24,885+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:24,894+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command SpmStatusVDS failed: Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory') 2019-04-13 21:41:24,895+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:24,900+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:41:24,900+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] HostName = onode-1 2019-04-13 21:41:24,900+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:41:24,949+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 54b29d48 2019-04-13 21:41:24,950+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:41:25,397+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 54b29d48 2019-04-13 21:41:25,408+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:25,408+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:25,409+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:41:26,363+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:28,369+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [49067302-e424-4846-9a36-d7cce26e4800] Failed invoking callback end method 'onFailed' for command 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:28,373+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Failed invoking callback end method 'onFailed' for command 'a869efce-6672-4363-9082-27f3c2b86635' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:28,379+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Failed invoking callback end method 'onFailed' for command 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:28,383+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-23) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:29,393+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:30,139+02 INFO [org.ovirt.engine.core.bll.aaa.SessionDataContainer] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Not removing session '/8zzvV6N3/Te4t2poy6AtsOoOXhDKhuvj8Ce6LxGz546kDOXO/HOTpE0hjGvUgYS80GCKN7jmeENPiGf5QDRYw==', session has running commands for user 'admin@internal-authz'. 2019-04-13 21:41:30,401+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Failed invoking callback end method 'onFailed' for command 'ea8a5636-af5a-4a6e-979c-361a3070cf49' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:31,409+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' (id: '660bf836-cf16-4006-b86d-386bfce017eb') waiting on child command id: 'ea8a5636-af5a-4a6e-979c-361a3070cf49' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:33,419+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Failed invoking callback end method 'onFailed' for command '1c8ebed4-5850-4083-9565-95addb69b123' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:34,424+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' (id: 'e919f810-5c81-4f10-b47d-ca44060acef4') waiting on child command id: '1c8ebed4-5850-4083-9565-95addb69b123' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:35,442+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:35,442+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:35,448+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:41:35,448+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] HostName = onode-1 2019-04-13 21:41:35,448+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:41:35,495+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 72993c1b 2019-04-13 21:41:35,496+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:41:35,946+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 72993c1b 2019-04-13 21:41:35,956+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:35,957+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:35,957+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:41:36,429+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:38,436+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [49067302-e424-4846-9a36-d7cce26e4800] Failed invoking callback end method 'onFailed' for command 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:38,441+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Failed invoking callback end method 'onFailed' for command 'a869efce-6672-4363-9082-27f3c2b86635' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:38,445+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Failed invoking callback end method 'onFailed' for command 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' with exception 'null', the callback is marked for end method retries 2019-04-13 21:41:38,449+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-73) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' (id: '779e7018-33ea-4307-8253-daddc3e4d61c') waiting on child command id: 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:39,457+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-67) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' (id: '55c8150c-51c7-4bc3-a330-6c3caf88e676') waiting on child command id: 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:40,469+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Failed invoking callback end method 'onFailed' for command 'ea8a5636-af5a-4a6e-979c-361a3070cf49' with exception 'null', the callback is marked for end method retries but max number of retries have been attempted. The command will be marked as Failed. 2019-04-13 21:41:40,477+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Exception in invoking callback of command RemoveSnapshotSingleDiskLive (ea8a5636-af5a-4a6e-979c-361a3070cf49): NullPointerException: 2019-04-13 21:41:40,478+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Error invoking callback method 'onFailed' for 'FAILED' command 'ea8a5636-af5a-4a6e-979c-361a3070cf49' 2019-04-13 21:41:40,478+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Exception: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.onFailed(RemoveSnapshotSingleDiskLiveCommand.java:266) [bll.jar:] at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback.onFailed(RemoveSnapshotSingleDiskLiveCommandCallback.java:21) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.endCallback(CommandCallbacksPoller.java:67) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:166) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:41,489+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-61) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Command 'RemoveSnapshot' id: '660bf836-cf16-4006-b86d-386bfce017eb' child commands '[ea8a5636-af5a-4a6e-979c-361a3070cf49]' executions were completed, status 'FAILED' 2019-04-13 21:41:42,524+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [6149b722-e07b-427d-8cbd-6b49d409bf8c] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with failure. 2019-04-13 21:41:42,582+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-8) [6149b722-e07b-427d-8cbd-6b49d409bf8c] EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete snapshot 'snapshot for backup' for VM 'timetrack'. 2019-04-13 21:41:43,606+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Failed invoking callback end method 'onFailed' for command '1c8ebed4-5850-4083-9565-95addb69b123' with exception 'null', the callback is marked for end method retries but max number of retries have been attempted. The command will be marked as Failed. 2019-04-13 21:41:43,607+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Exception in invoking callback of command RemoveSnapshotSingleDiskLive (1c8ebed4-5850-4083-9565-95addb69b123): NullPointerException: 2019-04-13 21:41:43,607+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Error invoking callback method 'onFailed' for 'FAILED' command '1c8ebed4-5850-4083-9565-95addb69b123' 2019-04-13 21:41:43,607+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Exception: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.onFailed(RemoveSnapshotSingleDiskLiveCommand.java:266) [bll.jar:] at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback.onFailed(RemoveSnapshotSingleDiskLiveCommandCallback.java:21) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.endCallback(CommandCallbacksPoller.java:67) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:166) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:43,620+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Command 'RemoveSnapshot' id: 'e919f810-5c81-4f10-b47d-ca44060acef4' child commands '[1c8ebed4-5850-4083-9565-95addb69b123]' executions were completed, status 'FAILED' 2019-04-13 21:41:44,737+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with failure. 2019-04-13 21:41:44,789+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [dfe9574d-6a9c-47f1-b44b-3e6cb33e0559] EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete snapshot 'snapshot for backup' for VM 'win2016-01'. 2019-04-13 21:41:45,986+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:45,987+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:45,991+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:41:45,991+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] HostName = onode-1 2019-04-13 21:41:45,991+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:41:46,043+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 54bf9101 2019-04-13 21:41:46,043+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:41:46,494+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 54bf9101 2019-04-13 21:41:46,507+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:46,508+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:46,508+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-4) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:41:46,818+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' (id: 'a89e3833-77c6-48a9-832d-3d7f60efe190') waiting on child command id: 'a869efce-6672-4363-9082-27f3c2b86635' type:'RemoveSnapshotSingleDiskLive' to complete 2019-04-13 21:41:48,830+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Failed invoking callback end method 'onFailed' for command 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' with exception 'null', the callback is marked for end method retries but max number of retries have been attempted. The command will be marked as Failed. 2019-04-13 21:41:48,831+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Exception in invoking callback of command RemoveSnapshotSingleDiskLive (c90c52ac-c5b3-454f-a4c7-435d0fe703e0): NullPointerException: 2019-04-13 21:41:48,831+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Error invoking callback method 'onFailed' for 'FAILED' command 'c90c52ac-c5b3-454f-a4c7-435d0fe703e0' 2019-04-13 21:41:48,831+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Exception: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.onFailed(RemoveSnapshotSingleDiskLiveCommand.java:266) [bll.jar:] at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback.onFailed(RemoveSnapshotSingleDiskLiveCommandCallback.java:21) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.endCallback(CommandCallbacksPoller.java:67) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:166) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:48,850+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Failed invoking callback end method 'onFailed' for command 'a869efce-6672-4363-9082-27f3c2b86635' with exception 'null', the callback is marked for end method retries but max number of retries have been attempted. The command will be marked as Failed. 2019-04-13 21:41:48,851+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Exception in invoking callback of command RemoveSnapshotSingleDiskLive (a869efce-6672-4363-9082-27f3c2b86635): NullPointerException: 2019-04-13 21:41:48,851+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Error invoking callback method 'onFailed' for 'FAILED' command 'a869efce-6672-4363-9082-27f3c2b86635' 2019-04-13 21:41:48,851+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Exception: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.onFailed(RemoveSnapshotSingleDiskLiveCommand.java:266) [bll.jar:] at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback.onFailed(RemoveSnapshotSingleDiskLiveCommandCallback.java:21) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.endCallback(CommandCallbacksPoller.java:67) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:166) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:48,863+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Command 'RemoveSnapshot' id: 'a89e3833-77c6-48a9-832d-3d7f60efe190' child commands '[a869efce-6672-4363-9082-27f3c2b86635]' executions were completed, status 'FAILED' 2019-04-13 21:41:48,884+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [49067302-e424-4846-9a36-d7cce26e4800] Command 'RemoveSnapshot' id: '55c8150c-51c7-4bc3-a330-6c3caf88e676' child commands '[c90c52ac-c5b3-454f-a4c7-435d0fe703e0]' executions were completed, status 'FAILED' 2019-04-13 21:41:48,911+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Failed invoking callback end method 'onFailed' for command 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' with exception 'null', the callback is marked for end method retries but max number of retries have been attempted. The command will be marked as Failed. 2019-04-13 21:41:48,911+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Exception in invoking callback of command RemoveSnapshotSingleDiskLive (d6e96bc5-5312-4bbf-8459-02dcea8a25f8): NullPointerException: 2019-04-13 21:41:48,911+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Error invoking callback method 'onFailed' for 'FAILED' command 'd6e96bc5-5312-4bbf-8459-02dcea8a25f8' 2019-04-13 21:41:48,912+02 ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Exception: java.lang.NullPointerException at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommand.onFailed(RemoveSnapshotSingleDiskLiveCommand.java:266) [bll.jar:] at org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback.onFailed(RemoveSnapshotSingleDiskLiveCommandCallback.java:21) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.endCallback(CommandCallbacksPoller.java:67) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:166) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_201] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_201] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] at org.jboss.as.ee.concurrent.service.ElytronManagedThreadFactory$ElytronManagedThread.run(ElytronManagedThreadFactory.java:78) 2019-04-13 21:41:48,922+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Command 'RemoveSnapshot' id: '779e7018-33ea-4307-8253-daddc3e4d61c' child commands '[d6e96bc5-5312-4bbf-8459-02dcea8a25f8]' executions were completed, status 'FAILED' 2019-04-13 21:41:49,958+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [6e59b274-880f-4da4-8382-2a0cf1518c6b] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with failure. 2019-04-13 21:41:49,990+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [6e59b274-880f-4da4-8382-2a0cf1518c6b] EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete snapshot 'snapshot for backup' for VM 'webdev'. 2019-04-13 21:41:50,017+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [49067302-e424-4846-9a36-d7cce26e4800] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with failure. 2019-04-13 21:41:50,049+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [49067302-e424-4846-9a36-d7cce26e4800] EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete snapshot 'snapshot for backup' for VM 'win2016-02'. 2019-04-13 21:41:50,079+02 ERROR [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] Ending command 'org.ovirt.engine.core.bll.snapshots.RemoveSnapshotCommand' with failure. 2019-04-13 21:41:50,115+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [a75fcef7-755d-4998-8c0d-2b50624ac1b3] EVENT_ID: USER_REMOVE_SNAPSHOT_FINISHED_FAILURE(357), Failed to delete snapshot 'snapshot for backup' for VM 'pass'. 2019-04-13 21:41:56,475+02 INFO [org.ovirt.engine.core.bll.storage.ovfstore.OvfDataUpdater] (EE-ManagedThreadFactory-engineScheduled-Thread-56) [] Attempting to update VMs/Templates Ovf. 2019-04-13 21:41:56,546+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:56,557+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command SpmStatusVDS failed: Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory') 2019-04-13 21:41:56,558+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:56,565+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:41:56,565+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] HostName = onode-1 2019-04-13 21:41:56,565+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:41:56,667+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 59215234 2019-04-13 21:41:56,668+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:41:57,109+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 59215234 2019-04-13 21:41:57,122+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:41:57,123+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:41:57,123+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:42:07,158+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:07,159+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:07,165+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:42:07,165+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] HostName = onode-1 2019-04-13 21:42:07,165+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:42:07,263+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 5896153e 2019-04-13 21:42:07,264+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:42:07,689+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 5896153e 2019-04-13 21:42:07,700+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:07,702+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:07,702+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-30) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:42:17,733+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:17,733+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:17,739+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:42:17,739+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] HostName = onode-1 2019-04-13 21:42:17,740+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:42:17,780+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 66d07849 2019-04-13 21:42:17,781+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:42:18,228+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 66d07849 2019-04-13 21:42:18,240+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:18,241+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:18,241+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-88) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:42:28,274+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:28,282+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command SpmStatusVDS failed: Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory') 2019-04-13 21:42:28,283+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:28,288+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:42:28,288+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] HostName = onode-1 2019-04-13 21:42:28,289+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:42:28,341+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 699f7809 2019-04-13 21:42:28,343+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:42:28,792+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 699f7809 2019-04-13 21:42:28,804+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:28,804+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:28,804+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:42:38,844+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:38,845+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:38,849+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:42:38,849+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] HostName = onode-1 2019-04-13 21:42:38,849+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:42:38,885+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 227ad994 2019-04-13 21:42:38,885+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:42:39,333+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 227ad994 2019-04-13 21:42:39,343+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:39,344+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:39,344+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:42:49,376+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:49,376+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:49,381+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:42:49,381+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] HostName = onode-1 2019-04-13 21:42:49,381+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:42:49,421+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 3a6418c3 2019-04-13 21:42:49,422+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:42:49,872+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 3a6418c3 2019-04-13 21:42:49,883+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:49,884+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:49,884+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:42:59,920+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:42:59,944+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command SpmStatusVDS failed: Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory') 2019-04-13 21:42:59,944+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:42:59,950+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:42:59,951+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] HostName = onode-1 2019-04-13 21:42:59,951+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:42:59,980+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 29618405 2019-04-13 21:42:59,980+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:43:00,423+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 29618405 2019-04-13 21:43:00,433+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:43:00,434+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:43:00,434+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:43:10,469+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:43:10,470+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:43:10,476+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:43:10,476+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] HostName = onode-1 2019-04-13 21:43:10,476+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:43:10,500+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 1e46842 2019-04-13 21:43:10,500+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:43:10,977+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 1e46842 2019-04-13 21:43:10,988+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:43:10,989+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:43:10,989+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:43:21,025+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:43:21,025+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:43:21,030+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:43:21,030+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] HostName = onode-1 2019-04-13 21:43:21,030+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:43:21,054+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 436fa69c 2019-04-13 21:43:21,054+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:43:21,516+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 436fa69c 2019-04-13 21:43:21,527+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:43:21,528+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:43:21,528+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-51) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:43:31,559+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:43:31,575+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM onode-1 command SpmStatusVDS failed: Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory') 2019-04-13 21:43:31,575+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:43:31,580+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:43:31,580+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] HostName = onode-1 2019-04-13 21:43:31,580+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:43:31,602+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 54169248 2019-04-13 21:43:31,602+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:43:32,055+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 54169248 2019-04-13 21:43:32,066+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Failed in 'SpmStatusVDS' method 2019-04-13 21:43:32,066+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStatusVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] Command 'SpmStatusVDSCommand(HostName = onode-1, SpmStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'})' execution failed: VDSGenericException: VDSErrorException: Failed to SpmStatusVDS, error = Cannot inquire Lease(name='SDM', path=u'/rhev/data-center/mnt/onode-1.discovery.intern:_mnt_store/6a4cde72-6a6a-499e-9b6e-d490d91ca19e/dom_md/leases', offset=1048576): (2, 'Sanlock get hosts failure', 'No such file or directory'), code = 100 2019-04-13 21:43:32,066+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-43) [] hostFromVds::selectedVds - 'onode-1', spmStatus returned null! 2019-04-13 21:43:36,519+02 INFO [org.ovirt.engine.core.bll.ClearAllAuditLogAlertsCommand] (default task-57) [c017a9ca-5604-4961-87ea-71c57b949ac3] Running command: ClearAllAuditLogAlertsCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group AUDIT_LOG_MANAGEMENT with role type ADMIN 2019-04-13 21:43:36,568+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-57) [c017a9ca-5604-4961-87ea-71c57b949ac3] EVENT_ID: USER_CLEAR_ALL_AUDIT_LOG_ALERTS(10,606), All alerts were removed. (User: admin@internal-authz) 2019-04-13 21:43:42,108+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] Command 'org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand' return value ' TaskStatusListReturn:{status='Status [code=654, message=Not SPM]'} ' 2019-04-13 21:43:42,108+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] HostName = onode-1 2019-04-13 21:43:42,108+02 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] Command 'HSMGetAllTasksStatusesVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'})' execution failed: IRSGenericException: IRSErrorException: IRSNonOperationalException: Not SPM 2019-04-13 21:43:42,329+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 12dae4a3 2019-04-13 21:43:42,329+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:43:42,780+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 12dae4a3 2019-04-13 21:43:42,789+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] hostFromVds::selectedVds - 'onode-1', spmStatus 'Free', storage pool 'Default', storage pool version '4.3' 2019-04-13 21:43:42,806+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] starting spm on vds 'onode-1', storage pool 'Default', prevId '-1', LVER '26' 2019-04-13 21:43:42,807+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] START, SpmStartVDSCommand(HostName = onode-1, SpmStartVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', prevId='-1', prevLVER='26', storagePoolFormatType='V4', recoveryMode='Manual', SCSIFencing='false'}), log id: 6db89a38 2019-04-13 21:43:42,812+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] spmStart polling started: taskId '61a07f8b-47c0-479f-9574-be8a3c47c173' 2019-04-13 21:43:43,820+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] spmStart polling ended: taskId '61a07f8b-47c0-479f-9574-be8a3c47c173' task status 'finished' 2019-04-13 21:43:43,830+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] spmStart polling ended, spm status: SPM 2019-04-13 21:43:43,831+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] START, HSMClearTaskVDSCommand(HostName = onode-1, HSMTaskGuidBaseVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', taskId='61a07f8b-47c0-479f-9574-be8a3c47c173'}), log id: 105bd7af 2019-04-13 21:43:43,835+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] FINISH, HSMClearTaskVDSCommand, return: , log id: 105bd7af 2019-04-13 21:43:43,835+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] FINISH, SpmStartVDSCommand, return: SpmStatusResult:{SPM Id='1', SPM LVER='27', SPM Status='SPM'}, log id: 6db89a38 2019-04-13 21:43:43,838+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] Initialize Irs proxy from vds: onode-1.discovery.intern 2019-04-13 21:43:43,852+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-41) [] EVENT_ID: IRS_HOSTED_ON_VDS(204), Storage Pool Manager runs on Host onode-1.discovery.intern (Address: onode-1.discovery.intern), Data Center Default. 2019-04-13 21:43:43,858+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 21:43:43,880+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229925) [] START, SPMGetAllTasksInfoVDSCommand( IrsBaseVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='false'}), log id: 58f7ff8b 2019-04-13 21:43:44,143+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229925) [] -- executeIrsBrokerCommand: Attempting on storage pool '59ef3a18-002f-02d1-0220-000000000124' 2019-04-13 21:43:44,145+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229925) [] START, HSMGetAllTasksInfoVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'}), log id: 27cc21a5 2019-04-13 21:43:44,150+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229925) [] FINISH, HSMGetAllTasksInfoVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@69e52799, org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@5f34a70e], log id: 27cc21a5 2019-04-13 21:43:44,150+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229925) [] FINISH, SPMGetAllTasksInfoVDSCommand, return: [org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@69e52799, org.ovirt.engine.core.common.asynctasks.AsyncTaskCreationInfo@5f34a70e], log id: 58f7ff8b 2019-04-13 21:43:44,152+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engine-Thread-229925) [] Adding task '4c004745-e676-4abb-ba5f-8458a81dece5' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2019-04-13 21:43:44,153+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engine-Thread-229925) [] Adding task 'a254cbc1-ffb6-447b-99df-98985f28d837' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2019-04-13 21:43:44,173+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-229925) [] BaseAsyncTask::startPollingTask: Starting to poll task '4c004745-e676-4abb-ba5f-8458a81dece5'. 2019-04-13 21:43:44,174+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-229925) [] BaseAsyncTask::startPollingTask: Starting to poll task 'a254cbc1-ffb6-447b-99df-98985f28d837'. 2019-04-13 21:43:44,179+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engine-Thread-229925) [] Discovered 2 tasks on Storage Pool 'Default', 2 added to manager. 2019-04-13 21:43:52,076+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:43:52,077+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-25) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:02,077+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:02,078+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:12,078+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:12,079+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-79) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:22,080+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:22,080+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-13) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:32,081+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:32,081+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-86) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:42,081+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:42,082+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-64) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' is in pre-polling period and should not be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:44,582+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-57) [1de1df89-7515-4717-bf37-80440f737625] Lock Acquired to object 'EngineLock:{exclusiveLocks='', sharedLocks='[59ef3a18-002f-02d1-0220-000000000124=POOL]'}' 2019-04-13 21:44:44,669+02 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-57) [1de1df89-7515-4717-bf37-80440f737625] There are running tasks on the SPM: 'AsyncTask:{commandId='82b1e392-f315-492f-95b2-1964bc710481', rootCommandId='38b56e5c-6c7e-4a98-8795-1e404b970756', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', taskId='9c8f8d5a-2954-48b8-bbb1-777444397b19', vdsmTaskId='4c004745-e676-4abb-ba5f-8458a81dece5', stepId='null', taskType='deleteImage', status='running'} AsyncTask:{commandId='f25d92ef-9591-433b-adda-e07a034f0d89', rootCommandId='f09fc84b-a962-456c-8aa7-a3076e6b1b0f', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', taskId='f30999e9-913e-4a2c-9712-802911afa860', vdsmTaskId='a254cbc1-ffb6-447b-99df-98985f28d837', stepId='null', taskType='copyImage', status='running'}' 2019-04-13 21:44:44,669+02 WARN [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-57) [1de1df89-7515-4717-bf37-80440f737625] Validation of action 'MaintenanceNumberOfVdss' failed for user admin@internal-authz. Reasons: VAR__TYPE__HOST,VAR__ACTION__MAINTENANCE,VDS_CANNOT_MAINTENANCE_SPM_WITH_RUNNING_TASKS 2019-04-13 21:44:44,671+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-57) [1de1df89-7515-4717-bf37-80440f737625] Lock freed to object 'EngineLock:{exclusiveLocks='', sharedLocks='[59ef3a18-002f-02d1-0220-000000000124=POOL]'}' 2019-04-13 21:44:52,082+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:52,082+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:52,082+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:52,083+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] Polling and updating Async Tasks: 2 tasks, 2 tasks to poll now 2019-04-13 21:44:52,083+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:52,083+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:52,091+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] Task id 'a254cbc1-ffb6-447b-99df-98985f28d837' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:52,091+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] SPMAsyncTask::PollTask: Polling task 'a254cbc1-ffb6-447b-99df-98985f28d837' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2019-04-13 21:44:52,091+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] BaseAsyncTask::onTaskEndSuccess: Task 'a254cbc1-ffb6-447b-99df-98985f28d837' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-04-13 21:44:52,091+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'a254cbc1-ffb6-447b-99df-98985f28d837' 2019-04-13 21:44:52,092+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='false', taskId='a254cbc1-ffb6-447b-99df-98985f28d837'}), log id: 245dfdf8 2019-04-13 21:44:52,093+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] START, HSMClearTaskVDSCommand(HostName = onode-1, HSMTaskGuidBaseVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', taskId='a254cbc1-ffb6-447b-99df-98985f28d837'}), log id: 6135b833 2019-04-13 21:44:52,155+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] FINISH, HSMClearTaskVDSCommand, return: , log id: 6135b833 2019-04-13 21:44:52,155+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] FINISH, SPMClearTaskVDSCommand, return: , log id: 245dfdf8 2019-04-13 21:44:52,163+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] BaseAsyncTask::removeTaskFromDB: Removed task 'a254cbc1-ffb6-447b-99df-98985f28d837' from DataBase 2019-04-13 21:44:52,163+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] Task id '4c004745-e676-4abb-ba5f-8458a81dece5' has passed pre-polling period time and should be polled. Pre-polling period is 60000 millis. 2019-04-13 21:44:52,163+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] SPMAsyncTask::PollTask: Polling task '4c004745-e676-4abb-ba5f-8458a81dece5' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2019-04-13 21:44:52,163+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] BaseAsyncTask::onTaskEndSuccess: Task '4c004745-e676-4abb-ba5f-8458a81dece5' (Parent Command 'Unknown', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-04-13 21:44:52,163+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '4c004745-e676-4abb-ba5f-8458a81dece5' 2019-04-13 21:44:52,164+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='false', taskId='4c004745-e676-4abb-ba5f-8458a81dece5'}), log id: 5c074b8a 2019-04-13 21:44:52,165+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] START, HSMClearTaskVDSCommand(HostName = onode-1, HSMTaskGuidBaseVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', taskId='4c004745-e676-4abb-ba5f-8458a81dece5'}), log id: 503ede4e 2019-04-13 21:44:52,205+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] FINISH, HSMClearTaskVDSCommand, return: , log id: 503ede4e 2019-04-13 21:44:52,205+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] FINISH, SPMClearTaskVDSCommand, return: , log id: 5c074b8a 2019-04-13 21:44:52,211+02 INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [] BaseAsyncTask::removeTaskFromDB: Removed task '4c004745-e676-4abb-ba5f-8458a81dece5' from DataBase 2019-04-13 21:44:56,011+02 INFO [org.ovirt.engine.core.bll.pm.StartVdsCommand] (default task-57) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Lock Acquired to object 'EngineLock:{exclusiveLocks='[7e1fc8ae-a99f-4828-84fa-a111f8d9ba89=VDS_FENCE]', sharedLocks=''}' 2019-04-13 21:44:56,207+02 INFO [org.ovirt.engine.core.bll.pm.StartVdsCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Running command: StartVdsCommand internal: false. Entities affected : ID: 7e1fc8ae-a99f-4828-84fa-a111f8d9ba89 Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 21:44:56,208+02 INFO [org.ovirt.engine.core.bll.pm.StartVdsCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Power-Management: START of host 'onode-2' initiated. 2019-04-13 21:44:56,225+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] EVENT_ID: FENCE_OPERATION_STARTED(9,017), Power management start of Host onode-2 initiated. 2019-04-13 21:44:56,240+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] START, SetVdsStatusVDSCommand(HostName = onode-2, SetVdsStatusVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', status='Reboot', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 145000de 2019-04-13 21:44:56,251+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] FINISH, SetVdsStatusVDSCommand, return: , log id: 145000de 2019-04-13 21:44:56,302+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] EVENT_ID: FENCE_OPERATION_USING_AGENT_AND_PROXY_STARTED(9,020), Executing power management status on Host onode-2 using Proxy Host onode-1 and Fence Agent ipmilan:192.168.0.31. 2019-04-13 21:44:56,304+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] START, FenceVdsVDSCommand(HostName = onode-1, FenceVdsVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', targetVdsId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', action='STATUS', agent='FenceAgent:{id='04cde8a8-b4df-4ac4-9a02-4b45960a832d', hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', order='1', type='ipmilan', ip='192.168.0.31', port='null', user='ovirt', password='***', encryptOptions='false', options=''}', policy='null'}), log id: 137e2e2e 2019-04-13 21:44:56,782+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] FINISH, FenceVdsVDSCommand, return: FenceOperationResult:{status='SUCCESS', powerStatus='OFF', message=''}, log id: 137e2e2e 2019-04-13 21:44:56,881+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] EVENT_ID: FENCE_OPERATION_USING_AGENT_AND_PROXY_STARTED(9,020), Executing power management start on Host onode-2 using Proxy Host onode-1 and Fence Agent ipmilan:192.168.0.31. 2019-04-13 21:44:56,882+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] START, FenceVdsVDSCommand(HostName = onode-1, FenceVdsVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', targetVdsId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', action='START', agent='FenceAgent:{id='04cde8a8-b4df-4ac4-9a02-4b45960a832d', hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', order='1', type='ipmilan', ip='192.168.0.31', port='null', user='ovirt', password='***', encryptOptions='false', options=''}', policy='null'}), log id: 3544c914 2019-04-13 21:45:01,278+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] FINISH, FenceVdsVDSCommand, return: FenceOperationResult:{status='SUCCESS', powerStatus='UNKNOWN', message=''}, log id: 3544c914 2019-04-13 21:45:01,279+02 INFO [org.ovirt.engine.core.bll.pm.SingleAgentFenceActionExecutor] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Waiting for host 'onode-2.discovery.intern' to reach status 'ON' 2019-04-13 21:45:06,279+02 INFO [org.ovirt.engine.core.bll.pm.SingleAgentFenceActionExecutor] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Attempt 1 to get host 'onode-2.discovery.intern' status 2019-04-13 21:45:06,402+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] EVENT_ID: FENCE_OPERATION_USING_AGENT_AND_PROXY_STARTED(9,020), Executing power management status on Host onode-2 using Proxy Host onode-1 and Fence Agent ipmilan:192.168.0.31. 2019-04-13 21:45:06,403+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] START, FenceVdsVDSCommand(HostName = onode-1, FenceVdsVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', targetVdsId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', action='STATUS', agent='FenceAgent:{id='04cde8a8-b4df-4ac4-9a02-4b45960a832d', hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', order='1', type='ipmilan', ip='192.168.0.31', port='null', user='ovirt', password='***', encryptOptions='false', options=''}', policy='null'}), log id: 39904925 2019-04-13 21:45:06,894+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] FINISH, FenceVdsVDSCommand, return: FenceOperationResult:{status='SUCCESS', powerStatus='ON', message=''}, log id: 39904925 2019-04-13 21:45:06,894+02 INFO [org.ovirt.engine.core.bll.pm.SingleAgentFenceActionExecutor] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Host 'onode-2.discovery.intern' status is 'ON' 2019-04-13 21:45:06,895+02 INFO [org.ovirt.engine.core.bll.pm.StartVdsCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Power-Management: START host 'onode-2' succeeded. 2019-04-13 21:45:06,902+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] EVENT_ID: FENCE_OPERATION_SUCCEEDED(9,018), Power management start of Host onode-2 succeeded. 2019-04-13 21:45:06,904+02 INFO [org.ovirt.engine.core.bll.pm.StartVdsCommand] (EE-ManagedThreadFactory-engine-Thread-229939) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Waiting 300 seconds, for server to finish reboot process. 2019-04-13 21:45:06,910+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] EVENT_ID: USER_VDS_START(20), Host onode-2 was started by admin@internal-authz. 2019-04-13 21:45:06,919+02 INFO [org.ovirt.engine.core.bll.pm.StartVdsCommand] (EE-ManagedThreadFactory-engine-Thread-229936) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Lock freed to object 'EngineLock:{exclusiveLocks='[7e1fc8ae-a99f-4828-84fa-a111f8d9ba89=VDS_FENCE]', sharedLocks=''}' 2019-04-13 21:46:19,501+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Setting new tasks map. The map contains now 0 tasks 2019-04-13 21:46:19,501+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-93) [] Cleared all tasks of pool '59ef3a18-002f-02d1-0220-000000000124'. 2019-04-13 21:46:53,753+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] Lock Acquired to object 'EngineLock:{exclusiveLocks='', sharedLocks='[59ef3a18-002f-02d1-0220-000000000124=POOL]'}' 2019-04-13 21:46:53,938+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 21:46:53,970+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] START, SetVdsStatusVDSCommand(HostName = onode-1, SetVdsStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', status='PreparingForMaintenance', nonOperationalReason='NONE', stopSpmFailureLogged='true', maintenanceReason='null'}), log id: 49b4d9e8 2019-04-13 21:46:53,970+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] VDS 'onode-1' is spm and moved from up calling resetIrs. 2019-04-13 21:46:53,971+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='false', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', ignoreStopFailed='false'}), log id: c579b2b 2019-04-13 21:46:53,979+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] START, SpmStopVDSCommand(HostName = onode-1, SpmStopVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'}), log id: 463cabcd 2019-04-13 21:46:53,985+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] SpmStopVDSCommand::Stopping SPM on vds 'onode-1', pool id '59ef3a18-002f-02d1-0220-000000000124' 2019-04-13 21:46:54,022+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] FINISH, SpmStopVDSCommand, return: , log id: 463cabcd 2019-04-13 21:46:54,033+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] FINISH, ResetIrsVDSCommand, return: , log id: c579b2b 2019-04-13 21:46:54,043+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] FINISH, SetVdsStatusVDSCommand, return: , log id: 49b4d9e8 2019-04-13 21:46:54,046+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] Lock freed to object 'EngineLock:{exclusiveLocks='', sharedLocks='[59ef3a18-002f-02d1-0220-000000000124=POOL]'}' 2019-04-13 21:46:54,178+02 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:46:54,254+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-57) [e5f65a30-68fb-444e-bd06-dee750fdef60] EVENT_ID: USER_VDS_MAINTENANCE_WITHOUT_REASON(620), Host onode-1 was switched to Maintenance mode by admin@internal-authz. 2019-04-13 21:46:56,175+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '93f02ede-ec5c-4641-8e97-2949f98e856d' status from 'Active' to 'Unknown', reason: no reporting hosts 2019-04-13 21:46:56,177+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '42bb6772-12f9-4bf2-9245-43caa41c5a2a' status from 'Active' to 'Unknown', reason: no reporting hosts 2019-04-13 21:46:56,179+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '6a4cde72-6a6a-499e-9b6e-d490d91ca19e' status from 'Active' to 'Unknown', reason: no reporting hosts 2019-04-13 21:46:56,180+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '9c3f06cf-7475-448e-819b-f4f52fa7d782' status from 'Active' to 'Unknown', reason: no reporting hosts 2019-04-13 21:46:56,190+02 INFO [org.ovirt.engine.core.bll.storage.pool.SetStoragePoolStatusCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [4ba013e] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:46:56,203+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-80) [4ba013e] EVENT_ID: SYSTEM_CHANGE_STORAGE_POOL_STATUS_PROBLEMATIC(980), Invalid status on Data Center Default. Setting status to Non Responsive. 2019-04-13 21:46:56,259+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-42) [e5f65a30-68fb-444e-bd06-dee750fdef60] Command 'MaintenanceNumberOfVdss' id: '7a4c047e-f3dc-4ed8-b6a2-601a53a3318e' child commands '[]' executions were completed, status 'SUCCEEDED' 2019-04-13 21:46:56,815+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] Updated host status from 'Preparing for Maintenance' to 'Maintenance' in database, host 'onode-1'(65c45da8-2a46-42e0-9c3a-58748546103b) 2019-04-13 21:46:56,863+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-229955) [] Clearing cache of pool: '59ef3a18-002f-02d1-0220-000000000124' for problematic entities of VDS: 'onode-1'. 2019-04-13 21:46:56,863+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-229955) [] Removing vds '[65c45da8-2a46-42e0-9c3a-58748546103b]' from the domain in maintenance cache 2019-04-13 21:46:56,863+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-229955) [] Removing host(s) '[65c45da8-2a46-42e0-9c3a-58748546103b]' from hosts unseen domain report cache 2019-04-13 21:46:56,868+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] START, DisconnectStoragePoolVDSCommand(HostName = onode-1, DisconnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', vds_spm_id='1'}), log id: 385455df 2019-04-13 21:46:57,288+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [e5f65a30-68fb-444e-bd06-dee750fdef60] Ending command 'org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand' successfully. 2019-04-13 21:47:01,145+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] FINISH, DisconnectStoragePoolVDSCommand, return: , log id: 385455df 2019-04-13 21:47:01,152+02 INFO [org.ovirt.engine.core.bll.storage.pool.DisconnectHostFromStoragePoolServersCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6bdff425] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:47:01,245+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6bdff425] START, DisconnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='2286311a-c1c9-4dda-bea5-9c78651c6e36', connection='onode-1.discovery.intern:/mnt/store', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}, StorageServerConnections:{id='858731bf-2fb1-45a7-a693-235f675a667b', connection='nas.discovery.intern:/mnt/d2600/sicherung/vmBackups', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}, StorageServerConnections:{id='07dd9ff2-afa1-4a33-b401-935ea5b8c9db', connection='nas.discovery.intern:/mnt/d2600/isos', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}, StorageServerConnections:{id='b4f4bcf2-e104-468e-b330-ed731ec8ef6c', connection='nas.discovery.intern:/mnt/d2600/vms', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 46ec7f61 2019-04-13 21:47:01,753+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [6bdff425] FINISH, DisconnectStorageServerVDSCommand, return: {b4f4bcf2-e104-468e-b330-ed731ec8ef6c=0, 2286311a-c1c9-4dda-bea5-9c78651c6e36=0, 07dd9ff2-afa1-4a33-b401-935ea5b8c9db=0, 858731bf-2fb1-45a7-a693-235f675a667b=0}, log id: 46ec7f61 2019-04-13 21:47:18,158+02 INFO [org.ovirt.engine.core.bll.hostdeploy.HostUpgradeCheckCommand] (default task-57) [9ca15368-b64d-4a2e-a091-6788f4c402d7] Running command: HostUpgradeCheckCommand internal: false. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN 2019-04-13 21:47:18,196+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-57) [] EVENT_ID: HOST_AVAILABLE_UPDATES_STARTED(884), Started to check for available updates on host onode-1. 2019-04-13 21:47:18,202+02 INFO [org.ovirt.engine.core.bll.hostdeploy.HostUpgradeCheckInternalCommand] (EE-ManagedThreadFactory-commandCoordinator-Thread-4) [9ca15368-b64d-4a2e-a091-6788f4c402d7] Running command: HostUpgradeCheckInternalCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 21:47:18,222+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-commandCoordinator-Thread-4) [9ca15368-b64d-4a2e-a091-6788f4c402d7] Executing Ansible command: ANSIBLE_STDOUT_CALLBACK=hostupgradeplugin /usr/bin/ansible-playbook --ssh-common-args=-F /var/lib/ovirt-engine/.ssh/config --check --private-key=/etc/pki/ovirt-engine/keys/engine_id_rsa --inventory=/tmp/ansible-inventory5432350653141985698 /usr/share/ovirt-engine/playbooks/ovirt-host-upgrade.yml [Logfile: null] 2019-04-13 21:47:44,445+02 INFO [org.ovirt.engine.core.common.utils.ansible.AnsibleExecutor] (EE-ManagedThreadFactory-commandCoordinator-Thread-4) [9ca15368-b64d-4a2e-a091-6788f4c402d7] Ansible playbook command has exited with value: 0 2019-04-13 21:47:44,454+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-commandCoordinator-Thread-4) [9ca15368-b64d-4a2e-a091-6788f4c402d7] EVENT_ID: HOST_AVAILABLE_UPDATES_FINISHED(885), Check for available updates on host onode-1 was completed successfully with message 'no updates found.'. 2019-04-13 21:48:13,047+02 WARN [org.ovirt.engine.core.utils.ThreadUtils] (EE-ManagedThreadFactory-engine-Thread-229939) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] Interrupted: sleep interrupted 2019-04-13 21:48:13,094+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229939) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] START, SetVdsStatusVDSCommand(HostName = onode-2, SetVdsStatusVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', status='NonResponsive', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 3e1843e2 2019-04-13 21:48:13,167+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-229939) [3393164b-1ced-4d86-95b0-a9dd3f120b5b] FINISH, SetVdsStatusVDSCommand, return: , log id: 3e1843e2 2019-04-13 21:50:37,856+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf'. 2019-04-13 21:50:37,862+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] The file '/etc/ovirt-engine/engine.conf' doesn't exist or isn't readable. Will return an empty set of properties. 2019-04-13 21:50:37,866+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf'. 2019-04-13 21:50:37,867+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] The file '/etc/ovirt-engine/engine.conf' doesn't exist or isn't readable. Will return an empty set of properties. 2019-04-13 21:50:37,867+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-scl-postgres.conf'. 2019-04-13 21:50:37,867+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-cinderlib.conf'. 2019-04-13 21:50:37,867+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-database.conf'. 2019-04-13 21:50:37,868+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-dwh-database.conf'. 2019-04-13 21:50:37,868+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-java.conf'. 2019-04-13 21:50:37,868+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf'. 2019-04-13 21:50:37,868+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf'. 2019-04-13 21:50:37,869+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf'. 2019-04-13 21:50:37,869+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/11-setup-sso.conf'. 2019-04-13 21:50:37,869+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/20-setup-jboss-overlay.conf'. 2019-04-13 21:50:37,869+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-engine-api-explorer.conf'. 2019-04-13 21:50:37,870+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-engine-extension-aaa-jdbc.conf'. 2019-04-13 21:50:37,870+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-web-ui.conf'. 2019-04-13 21:50:37,871+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'AAA_JAAS_KEYTAB_FILE' is ''. 2019-04-13 21:50:37,871+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'AAA_JAAS_PRINCIPAL_NAME' is ''. 2019-04-13 21:50:37,871+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'AAA_JAAS_TICKET_CACHE_FILE' is ''. 2019-04-13 21:50:37,871+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'AAA_JAAS_USE_KEYTAB' is 'false'. 2019-04-13 21:50:37,871+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'AAA_JAAS_USE_TICKET_CACHE' is 'false'. 2019-04-13 21:50:37,871+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'AAA_KRB5_CONF_FILE' is ''. 2019-04-13 21:50:37,871+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ANSIBLE_PLAYBOOK_EXEC_DEFAULT_TIMEOUT' is '30'. 2019-04-13 21:50:37,872+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ANSIBLE_PLAYBOOK_VERBOSITY_LEVEL' is '1'. 2019-04-13 21:50:37,872+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_DATABASE' is 'ovirt_cinderlib'. 2019-04-13 21:50:37,872+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 21:50:37,872+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_ENABLE' is 'False'. 2019-04-13 21:50:37,872+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_HOST' is 'localhost'. 2019-04-13 21:50:37,872+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_PASSWORD' is '***'. 2019-04-13 21:50:37,872+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_PORT' is '5432'. 2019-04-13 21:50:37,872+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_SECURED' is 'False'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_URL' is 'jdbc:postgresql://localhost:5432/ovirt_cinderlib?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'CINDERLIB_DB_USER' is 'ovirt_cinderlib'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'COMMAND_COORDINATOR_THREAD_POOL_SIZE' is '10'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DASHBOARD_CACHE_UPDATE' is 'true'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DASHBOARD_INVENTORY_CACHE_UPDATE_INTERVAL' is '60'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DASHBOARD_UTILIZATION_CACHE_UPDATE_INTERVAL' is '300'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_DATABASE' is 'ovirt_engine_history'. 2019-04-13 21:50:37,873+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_HOST' is 'localhost'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_PASSWORD' is '***'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_PORT' is '5432'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_SECURED' is 'False'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-scl-postgres.conf'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_URL' is 'jdbc:postgresql://localhost:5432/ovirt_engine_history?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'DWH_DB_USER' is 'ovirt_engine_history'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-cinderlib.conf'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENFORCE_ENGINE_HEAP_PARAMS' is 'false'. 2019-04-13 21:50:37,874+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_AJP_ENABLED' is 'true'. 2019-04-13 21:50:37,875+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_AJP_PORT' is '8702'. 2019-04-13 21:50:37,875+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-database.conf'. 2019-04-13 21:50:37,875+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_API_DEFAULT_VERSION' is '4'. 2019-04-13 21:50:37,875+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_API_DEPRECATED_VERSIONS' is '3:4.0:4.3'. 2019-04-13 21:50:37,875+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-dwh-database.conf'. 2019-04-13 21:50:37,875+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_API_EXPLORER_DIRECTORY' is '/usr/share/ovirt-engine-api-explorer'. 2019-04-13 21:50:37,877+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_API_FILTER_BY_DEFAULT' is 'true'. 2019-04-13 21:50:37,877+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_API_SUPPORTED_VERSIONS' is '3, 4'. 2019-04-13 21:50:37,877+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_APPS' is 'engine.ear restapi.war apidoc.war ovirt-web-ui.war'. 2019-04-13 21:50:37,877+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_CACHE' is '/var/cache/ovirt-engine'. 2019-04-13 21:50:37,878+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_CHECK_INTERVAL' is '1000'. 2019-04-13 21:50:37,878+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_CONNECTION_TIMEOUT' is '300000'. 2019-04-13 21:50:37,878+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_DATABASE' is 'engine'. 2019-04-13 21:50:37,878+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 21:50:37,879+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_HOST' is 'localhost'. 2019-04-13 21:50:37,879+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_MAX_CONNECTIONS' is '100'. 2019-04-13 21:50:37,879+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_MIN_CONNECTIONS' is '1'. 2019-04-13 21:50:37,879+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_PASSWORD' is '***'. 2019-04-13 21:50:37,879+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_PORT' is '5432'. 2019-04-13 21:50:37,880+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_SECURED' is 'False'. 2019-04-13 21:50:37,880+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 21:50:37,880+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_URL' is 'jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 21:50:37,880+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DB_USER' is 'engine'. 2019-04-13 21:50:37,880+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DEBUG_ADDRESS' is ''. 2019-04-13 21:50:37,881+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DEPLOYMENT_SCANNER' is 'false'. 2019-04-13 21:50:37,881+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_DOC' is '/usr/share/doc/ovirt-engine'. 2019-04-13 21:50:37,881+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_ETC' is '/etc/ovirt-engine'. 2019-04-13 21:50:37,881+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_EXTENSIONS_IGNORED' is '.*?(Authz)'. 2019-04-13 21:50:37,881+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_EXTENSION_PATH' is '/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d'. 2019-04-13 21:50:37,882+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE' is '/var/lib/ovirt-engine/external_truststore'. 2019-04-13 21:50:37,882+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 21:50:37,882+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 21:50:37,882+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_FQDN' is 'ovirt.discovery.intern'. 2019-04-13 21:50:37,882+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_GROUP' is 'ovirt'. 2019-04-13 21:50:37,883+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HEAP_MAX' is '1024M'. 2019-04-13 21:50:37,883+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HEAP_MIN' is '1024M'. 2019-04-13 21:50:37,883+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HTTPS_ENABLED' is 'false'. 2019-04-13 21:50:37,883+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 21:50:37,883+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 21:50:37,884+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 21:50:37,884+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HTTPS_PORT' is 'None'. 2019-04-13 21:50:37,884+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HTTPS_PROTOCOLS' is 'TLSv1.1,TLSv1.2'. 2019-04-13 21:50:37,884+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HTTP_ENABLED' is 'false'. 2019-04-13 21:50:37,884+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_HTTP_PORT' is 'None'. 2019-04-13 21:50:37,885+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_JAVA_MODULEPATH' is '/usr/share/ovirt-engine-wildfly-overlay/modules:/usr/share/ovirt-engine/modules/common:/usr/share/ovirt-engine-extension-aaa-jdbc/modules'. 2019-04-13 21:50:37,885+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_JBOSS_BLOCKING_TIMEOUT' is '300'. 2019-04-13 21:50:37,885+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_JMX_INTERFACE' is 'loopback'. 2019-04-13 21:50:37,885+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_JVM_ARGS' is ' -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"'. 2019-04-13 21:50:37,885+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_LOG' is '/var/log/ovirt-engine'. 2019-04-13 21:50:37,886+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_LOG_TO_CONSOLE' is 'false'. 2019-04-13 21:50:37,886+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_MALLOC_ARENA_MAX' is '1'. 2019-04-13 21:50:37,886+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_MANUAL' is '/usr/share/ovirt-engine/manual'. 2019-04-13 21:50:37,886+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_MAX_USER_SESSIONS' is '-1'. 2019-04-13 21:50:37,887+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_NON_JTA_DB_MAX_CONNECTIONS' is '10'. 2019-04-13 21:50:37,887+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_NON_JTA_DB_MIN_CONNECTIONS' is '1'. 2019-04-13 21:50:37,887+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI' is '/etc/pki/ovirt-engine'. 2019-04-13 21:50:37,876+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-java.conf'. 2019-04-13 21:50:37,903+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf'. 2019-04-13 21:50:37,903+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf'. 2019-04-13 21:50:37,904+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf'. 2019-04-13 21:50:37,904+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/11-setup-sso.conf'. 2019-04-13 21:50:37,904+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/20-setup-jboss-overlay.conf'. 2019-04-13 21:50:37,905+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-engine-api-explorer.conf'. 2019-04-13 21:50:37,905+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-engine-extension-aaa-jdbc.conf'. 2019-04-13 21:50:37,906+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-web-ui.conf'. 2019-04-13 21:50:37,906+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'AAA_JAAS_KEYTAB_FILE' is ''. 2019-04-13 21:50:37,906+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'AAA_JAAS_PRINCIPAL_NAME' is ''. 2019-04-13 21:50:37,907+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'AAA_JAAS_TICKET_CACHE_FILE' is ''. 2019-04-13 21:50:37,907+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'AAA_JAAS_USE_KEYTAB' is 'false'. 2019-04-13 21:50:37,907+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'AAA_JAAS_USE_TICKET_CACHE' is 'false'. 2019-04-13 21:50:37,907+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'AAA_KRB5_CONF_FILE' is ''. 2019-04-13 21:50:37,907+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ANSIBLE_PLAYBOOK_EXEC_DEFAULT_TIMEOUT' is '30'. 2019-04-13 21:50:37,907+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ANSIBLE_PLAYBOOK_VERBOSITY_LEVEL' is '1'. 2019-04-13 21:50:37,908+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_DATABASE' is 'ovirt_cinderlib'. 2019-04-13 21:50:37,908+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 21:50:37,908+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_ENABLE' is 'False'. 2019-04-13 21:50:37,908+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_HOST' is 'localhost'. 2019-04-13 21:50:37,908+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_PASSWORD' is '***'. 2019-04-13 21:50:37,909+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_PORT' is '5432'. 2019-04-13 21:50:37,909+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_SECURED' is 'False'. 2019-04-13 21:50:37,909+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 21:50:37,909+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_URL' is 'jdbc:postgresql://localhost:5432/ovirt_cinderlib?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 21:50:37,909+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'CINDERLIB_DB_USER' is 'ovirt_cinderlib'. 2019-04-13 21:50:37,910+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'COMMAND_COORDINATOR_THREAD_POOL_SIZE' is '10'. 2019-04-13 21:50:37,910+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DASHBOARD_CACHE_UPDATE' is 'true'. 2019-04-13 21:50:37,910+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DASHBOARD_INVENTORY_CACHE_UPDATE_INTERVAL' is '60'. 2019-04-13 21:50:37,910+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DASHBOARD_UTILIZATION_CACHE_UPDATE_INTERVAL' is '300'. 2019-04-13 21:50:37,910+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_DATABASE' is 'ovirt_engine_history'. 2019-04-13 21:50:37,910+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 21:50:37,911+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_HOST' is 'localhost'. 2019-04-13 21:50:37,911+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_PASSWORD' is '***'. 2019-04-13 21:50:37,911+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_PORT' is '5432'. 2019-04-13 21:50:37,912+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_SECURED' is 'False'. 2019-04-13 21:50:37,912+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 21:50:37,912+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_URL' is 'jdbc:postgresql://localhost:5432/ovirt_engine_history?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 21:50:37,912+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'DWH_DB_USER' is 'ovirt_engine_history'. 2019-04-13 21:50:37,912+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENFORCE_ENGINE_HEAP_PARAMS' is 'false'. 2019-04-13 21:50:37,892+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_CA' is '/etc/pki/ovirt-engine/ca.pem'. 2019-04-13 21:50:37,913+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_ENGINE_CERT' is '/etc/pki/ovirt-engine/certs/engine.cer'. 2019-04-13 21:50:37,913+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_ENGINE_STORE' is '/etc/pki/ovirt-engine/keys/engine.p12'. 2019-04-13 21:50:37,913+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_ENGINE_STORE_ALIAS' is '1'. 2019-04-13 21:50:37,913+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_ENGINE_STORE_PASSWORD' is '***'. 2019-04-13 21:50:37,914+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_ENGINE_STORE_TYPE' is 'PKCS12'. 2019-04-13 21:50:37,914+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 21:50:37,914+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 21:50:37,914+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 21:50:37,914+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PROPERTIES' is ' java.awt.headless=true sun.rmi.dgc.client.gcInterval=3600000 sun.rmi.dgc.server.gcInterval=3600000 jsse.enableSNIExtension=false'. 2019-04-13 21:50:37,915+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PROXY_ENABLED' is 'true'. 2019-04-13 21:50:37,915+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PROXY_HTTPS_PORT' is '443'. 2019-04-13 21:50:37,915+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_PROXY_HTTP_PORT' is '80'. 2019-04-13 21:50:37,915+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_RESTAPI_NEGO' is 'true'. 2019-04-13 21:50:37,916+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SCHEDULED_THREAD_POOL_SIZE' is '100'. 2019-04-13 21:50:37,916+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SCHEDULER_POOL_CLASS' is 'org.ovirt.engine.core.utils.timer.SchedulerThreadPool'. 2019-04-13 21:50:37,916+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SCHEDULER_POOL_QUEUE_SIZE' is '350'. 2019-04-13 21:50:37,916+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SCHEDULER_POOL_REJECT_POLICY' is 'false'. 2019-04-13 21:50:37,917+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SCHEDULER_POOL_THREAD_SIZE' is '100'. 2019-04-13 21:50:37,917+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_webadmin' is '~'. 2019-04-13 21:50:37,917+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_webui' is '~'. 2019-04-13 21:50:37,917+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_welcome' is '~'. 2019-04-13 21:50:37,917+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_AUTH_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/sso'. 2019-04-13 21:50:37,918+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_CLIENT_ID' is 'ovirt-engine-core'. 2019-04-13 21:50:37,918+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_CLIENT_SECRET' is '***'. 2019-04-13 21:50:37,918+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_ENABLE_EXTERNAL_SSO' is 'false'. 2019-04-13 21:50:37,918+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_EXTERNAL_SSO_LOGOUT_URI' is '/ovirt-engine/callback'. 2019-04-13 21:50:37,918+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_INSTALLED_ON_ENGINE_HOST' is 'true'. 2019-04-13 21:50:37,918+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_SERVICE_CLIENT_POOL_SIZE' is '10'. 2019-04-13 21:50:37,919+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_SERVICE_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 21:50:37,919+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_SERVICE_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 21:50:37,919+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_SERVICE_SSL_PROTOCOL' is 'TLS'. 2019-04-13 21:50:37,919+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_SERVICE_SSL_VERIFY_CHAIN' is 'true'. 2019-04-13 21:50:37,919+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_SERVICE_SSL_VERIFY_HOST' is 'false'. 2019-04-13 21:50:37,919+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_SERVICE_URI' is '/ovirt-engine/sso'. 2019-04-13 21:50:37,920+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_SSO_SERVICE_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/sso'. 2019-04-13 21:50:37,920+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_STOP_INTERVAL' is '1'. 2019-04-13 21:50:37,920+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_STOP_TIME' is '10'. 2019-04-13 21:50:37,920+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_THREAD_POOL_MAX_SIZE' is '500'. 2019-04-13 21:50:37,920+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_THREAD_POOL_MIN_SIZE' is '50'. 2019-04-13 21:50:37,912+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_AJP_ENABLED' is 'true'. 2019-04-13 21:50:37,925+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_AJP_PORT' is '8702'. 2019-04-13 21:50:37,925+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_API_DEFAULT_VERSION' is '4'. 2019-04-13 21:50:37,925+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_API_DEPRECATED_VERSIONS' is '3:4.0:4.3'. 2019-04-13 21:50:37,925+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_API_EXPLORER_DIRECTORY' is '/usr/share/ovirt-engine-api-explorer'. 2019-04-13 21:50:37,925+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_API_FILTER_BY_DEFAULT' is 'true'. 2019-04-13 21:50:37,926+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_API_SUPPORTED_VERSIONS' is '3, 4'. 2019-04-13 21:50:37,926+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_APPS' is 'engine.ear restapi.war apidoc.war ovirt-web-ui.war'. 2019-04-13 21:50:37,926+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_CACHE' is '/var/cache/ovirt-engine'. 2019-04-13 21:50:37,926+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_CHECK_INTERVAL' is '1000'. 2019-04-13 21:50:37,927+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_CONNECTION_TIMEOUT' is '300000'. 2019-04-13 21:50:37,927+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_DATABASE' is 'engine'. 2019-04-13 21:50:37,927+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 21:50:37,927+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_HOST' is 'localhost'. 2019-04-13 21:50:37,927+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_MAX_CONNECTIONS' is '100'. 2019-04-13 21:50:37,928+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_MIN_CONNECTIONS' is '1'. 2019-04-13 21:50:37,928+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_PASSWORD' is '***'. 2019-04-13 21:50:37,928+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_PORT' is '5432'. 2019-04-13 21:50:37,928+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_SECURED' is 'False'. 2019-04-13 21:50:37,929+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 21:50:37,929+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_URL' is 'jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 21:50:37,929+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DB_USER' is 'engine'. 2019-04-13 21:50:37,929+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DEBUG_ADDRESS' is ''. 2019-04-13 21:50:37,930+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DEPLOYMENT_SCANNER' is 'false'. 2019-04-13 21:50:37,930+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_DOC' is '/usr/share/doc/ovirt-engine'. 2019-04-13 21:50:37,930+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_ETC' is '/etc/ovirt-engine'. 2019-04-13 21:50:37,930+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_EXTENSIONS_IGNORED' is '.*?(Authz)'. 2019-04-13 21:50:37,930+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_EXTENSION_PATH' is '/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d'. 2019-04-13 21:50:37,930+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE' is '/var/lib/ovirt-engine/external_truststore'. 2019-04-13 21:50:37,931+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 21:50:37,931+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 21:50:37,931+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_FQDN' is 'ovirt.discovery.intern'. 2019-04-13 21:50:37,931+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_GROUP' is 'ovirt'. 2019-04-13 21:50:37,931+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HEAP_MAX' is '1024M'. 2019-04-13 21:50:37,931+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HEAP_MIN' is '1024M'. 2019-04-13 21:50:37,932+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HTTPS_ENABLED' is 'false'. 2019-04-13 21:50:37,932+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 21:50:37,932+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 21:50:37,932+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 21:50:37,932+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HTTPS_PORT' is 'None'. 2019-04-13 21:50:37,932+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HTTPS_PROTOCOLS' is 'TLSv1.1,TLSv1.2'. 2019-04-13 21:50:37,933+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HTTP_ENABLED' is 'false'. 2019-04-13 21:50:37,933+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_HTTP_PORT' is 'None'. 2019-04-13 21:50:37,933+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_JAVA_MODULEPATH' is '/usr/share/ovirt-engine-wildfly-overlay/modules:/usr/share/ovirt-engine/modules/common:/usr/share/ovirt-engine-extension-aaa-jdbc/modules'. 2019-04-13 21:50:37,933+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_JBOSS_BLOCKING_TIMEOUT' is '300'. 2019-04-13 21:50:37,934+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_JMX_INTERFACE' is 'loopback'. 2019-04-13 21:50:37,934+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_JVM_ARGS' is ' -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"'. 2019-04-13 21:50:37,934+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_LOG' is '/var/log/ovirt-engine'. 2019-04-13 21:50:37,934+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_LOG_TO_CONSOLE' is 'false'. 2019-04-13 21:50:37,934+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_MALLOC_ARENA_MAX' is '1'. 2019-04-13 21:50:37,934+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_MANUAL' is '/usr/share/ovirt-engine/manual'. 2019-04-13 21:50:37,935+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_MAX_USER_SESSIONS' is '-1'. 2019-04-13 21:50:37,935+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_NON_JTA_DB_MAX_CONNECTIONS' is '10'. 2019-04-13 21:50:37,935+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_NON_JTA_DB_MIN_CONNECTIONS' is '1'. 2019-04-13 21:50:37,935+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI' is '/etc/pki/ovirt-engine'. 2019-04-13 21:50:37,936+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_CA' is '/etc/pki/ovirt-engine/ca.pem'. 2019-04-13 21:50:37,936+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_ENGINE_CERT' is '/etc/pki/ovirt-engine/certs/engine.cer'. 2019-04-13 21:50:37,937+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_ENGINE_STORE' is '/etc/pki/ovirt-engine/keys/engine.p12'. 2019-04-13 21:50:37,937+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_ENGINE_STORE_ALIAS' is '1'. 2019-04-13 21:50:37,937+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_ENGINE_STORE_PASSWORD' is '***'. 2019-04-13 21:50:37,937+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_ENGINE_STORE_TYPE' is 'PKCS12'. 2019-04-13 21:50:37,937+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 21:50:37,937+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 21:50:37,938+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 21:50:37,938+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PROPERTIES' is ' java.awt.headless=true sun.rmi.dgc.client.gcInterval=3600000 sun.rmi.dgc.server.gcInterval=3600000 jsse.enableSNIExtension=false'. 2019-04-13 21:50:37,938+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PROXY_ENABLED' is 'true'. 2019-04-13 21:50:37,938+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PROXY_HTTPS_PORT' is '443'. 2019-04-13 21:50:37,939+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_PROXY_HTTP_PORT' is '80'. 2019-04-13 21:50:37,939+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_RESTAPI_NEGO' is 'true'. 2019-04-13 21:50:37,939+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SCHEDULED_THREAD_POOL_SIZE' is '100'. 2019-04-13 21:50:37,939+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SCHEDULER_POOL_CLASS' is 'org.ovirt.engine.core.utils.timer.SchedulerThreadPool'. 2019-04-13 21:50:37,939+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SCHEDULER_POOL_QUEUE_SIZE' is '350'. 2019-04-13 21:50:37,939+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SCHEDULER_POOL_REJECT_POLICY' is 'false'. 2019-04-13 21:50:37,940+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SCHEDULER_POOL_THREAD_SIZE' is '100'. 2019-04-13 21:50:37,940+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_webadmin' is '~'. 2019-04-13 21:50:37,940+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_webui' is '~'. 2019-04-13 21:50:37,940+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_welcome' is '~'. 2019-04-13 21:50:37,940+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_AUTH_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/sso'. 2019-04-13 21:50:37,940+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_CLIENT_ID' is 'ovirt-engine-core'. 2019-04-13 21:50:37,941+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_CLIENT_SECRET' is '***'. 2019-04-13 21:50:37,941+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_ENABLE_EXTERNAL_SSO' is 'false'. 2019-04-13 21:50:37,941+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_EXTERNAL_SSO_LOGOUT_URI' is '/ovirt-engine/callback'. 2019-04-13 21:50:37,941+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_INSTALLED_ON_ENGINE_HOST' is 'true'. 2019-04-13 21:50:37,953+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_THREAD_POOL_QUEUE_SIZE' is '100'. 2019-04-13 21:50:37,954+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_THREAD_STACK' is '1M'. 2019-04-13 21:50:37,954+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_TMP' is '/var/tmp/ovirt-engine'. 2019-04-13 21:50:37,954+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_UP_MARK' is '/var/lib/ovirt-engine/engine.up'. 2019-04-13 21:50:37,954+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_URI' is '/ovirt-engine'. 2019-04-13 21:50:37,954+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_USER' is 'ovirt'. 2019-04-13 21:50:37,955+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_USR' is '/usr/share/ovirt-engine'. 2019-04-13 21:50:37,955+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_VAR' is '/var/lib/ovirt-engine'. 2019-04-13 21:50:37,955+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'ENGINE_VERBOSE_GC' is 'false'. 2019-04-13 21:50:37,955+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_CLIENT_ID' is ''. 2019-04-13 21:50:37,956+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_CLIENT_POOL_SIZE' is '10'. 2019-04-13 21:50:37,956+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_CLIENT_SECRET' is ''. 2019-04-13 21:50:37,956+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 21:50:37,956+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 21:50:37,956+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_CONNECT_TIMEOUT' is '0'. 2019-04-13 21:50:37,957+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 21:50:37,957+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE_PASSWORD' is ''. 2019-04-13 21:50:37,957+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 21:50:37,957+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_LOGOUT_END_POINT' is ''. 2019-04-13 21:50:37,957+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_READ_TIMEOUT' is '0'. 2019-04-13 21:50:37,958+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_SSL_PROTOCOL' is 'TLS'. 2019-04-13 21:50:37,941+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_SERVICE_CLIENT_POOL_SIZE' is '10'. 2019-04-13 21:50:37,958+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_SSL_VERIFY_CHAIN' is 'true'. 2019-04-13 21:50:37,958+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_SERVICE_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 21:50:37,958+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_SERVICE_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 21:50:37,958+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_SERVICE_SSL_PROTOCOL' is 'TLS'. 2019-04-13 21:50:37,958+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_SERVICE_SSL_VERIFY_CHAIN' is 'true'. 2019-04-13 21:50:37,959+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_SERVICE_SSL_VERIFY_HOST' is 'false'. 2019-04-13 21:50:37,959+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_SERVICE_URI' is '/ovirt-engine/sso'. 2019-04-13 21:50:37,959+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_SSO_SERVICE_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/sso'. 2019-04-13 21:50:37,959+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_STOP_INTERVAL' is '1'. 2019-04-13 21:50:37,959+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_STOP_TIME' is '10'. 2019-04-13 21:50:37,959+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_THREAD_POOL_MAX_SIZE' is '500'. 2019-04-13 21:50:37,959+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_THREAD_POOL_MIN_SIZE' is '50'. 2019-04-13 21:50:37,960+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_THREAD_POOL_QUEUE_SIZE' is '100'. 2019-04-13 21:50:37,960+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_THREAD_STACK' is '1M'. 2019-04-13 21:50:37,960+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_TMP' is '/var/tmp/ovirt-engine'. 2019-04-13 21:50:37,960+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_UP_MARK' is '/var/lib/ovirt-engine/engine.up'. 2019-04-13 21:50:37,960+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_URI' is '/ovirt-engine'. 2019-04-13 21:50:37,960+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_USER' is 'ovirt'. 2019-04-13 21:50:37,960+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_USR' is '/usr/share/ovirt-engine'. 2019-04-13 21:50:37,961+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_VAR' is '/var/lib/ovirt-engine'. 2019-04-13 21:50:37,961+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'ENGINE_VERBOSE_GC' is 'false'. 2019-04-13 21:50:37,961+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_CLIENT_ID' is ''. 2019-04-13 21:50:37,961+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_CLIENT_POOL_SIZE' is '10'. 2019-04-13 21:50:37,961+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_CLIENT_SECRET' is ''. 2019-04-13 21:50:37,961+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 21:50:37,962+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 21:50:37,962+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_CONNECT_TIMEOUT' is '0'. 2019-04-13 21:50:37,962+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 21:50:37,962+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE_PASSWORD' is ''. 2019-04-13 21:50:37,962+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 21:50:37,962+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_LOGOUT_END_POINT' is ''. 2019-04-13 21:50:37,963+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_READ_TIMEOUT' is '0'. 2019-04-13 21:50:37,963+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_SSL_PROTOCOL' is 'TLS'. 2019-04-13 21:50:37,963+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_SSL_VERIFY_CHAIN' is 'true'. 2019-04-13 21:50:37,963+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_SSL_VERIFY_HOST' is 'true'. 2019-04-13 21:50:37,963+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_TOKEN_END_POINT' is ''. 2019-04-13 21:50:37,963+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'EXTERNAL_OIDC_USER_INFO_END_POINT' is ''. 2019-04-13 21:50:37,963+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'HOST_CHECK_FOR_UPDATES_THREAD_POOL_SIZE' is '5'. 2019-04-13 21:50:37,964+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'JBOSS_HOME' is '/usr/share/ovirt-engine-wildfly'. 2019-04-13 21:50:37,964+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'JBOSS_RUNTIME' is '/var/lib/ovirt-engine/jboss_runtime'. 2019-04-13 21:50:37,964+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'PACKAGE_DISPLAY_VERSION' is '4.3.2.1-1.el7'. 2019-04-13 21:50:37,964+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'PACKAGE_NAME' is 'ovirt-engine'. 2019-04-13 21:50:37,964+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'PACKAGE_VERSION' is '4.3.2.1'. 2019-04-13 21:50:37,964+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SENSITIVE_KEYS' is ',ENGINE_DB_PASSWORD,DWH_DB_PASSWORD,CINDERLIB_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD,ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD,ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD,ENGINE_SSO_CLIENT_SECRET'. 2019-04-13 21:50:37,965+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_ALTERNATE_ENGINE_FQDNS' is ''. 2019-04-13 21:50:37,965+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_AUTH_LOGIN_SEQUENCE' is 'NI'. 2019-04-13 21:50:37,965+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_CALLBACK_CLIENT_POOL_SIZE' is '10'. 2019-04-13 21:50:37,965+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_CALLBACK_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 21:50:37,965+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_CALLBACK_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 21:50:37,965+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_CALLBACK_CONNECT_TIMEOUT' is '0'. 2019-04-13 21:50:37,966+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_CALLBACK_PREFIX_CHECK' is 'true'. 2019-04-13 21:50:37,966+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_CALLBACK_READ_TIMEOUT' is '0'. 2019-04-13 21:50:37,966+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_ENGINE_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/'. 2019-04-13 21:50:37,966+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_HOUSE_KEEPING_INTERVAL' is '60'. 2019-04-13 21:50:37,966+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_TOKEN_HTTP_LOGIN_SEQUENCE' is 'Nb'. 2019-04-13 21:50:37,966+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'SSO_TOKEN_TIMEOUT' is '360000'. 2019-04-13 21:50:37,966+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'THREAD_POOL_MONITORING_INTERVAL_IN_SECONDS' is '600'. 2019-04-13 21:50:37,967+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 55) [] Value of property 'sclenv' is 'rh-postgresql10'. 2019-04-13 21:50:37,958+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_SSL_VERIFY_HOST' is 'true'. 2019-04-13 21:50:37,980+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_TOKEN_END_POINT' is ''. 2019-04-13 21:50:37,980+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'EXTERNAL_OIDC_USER_INFO_END_POINT' is ''. 2019-04-13 21:50:37,980+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'HOST_CHECK_FOR_UPDATES_THREAD_POOL_SIZE' is '5'. 2019-04-13 21:50:37,980+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'JBOSS_HOME' is '/usr/share/ovirt-engine-wildfly'. 2019-04-13 21:50:37,981+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'JBOSS_RUNTIME' is '/var/lib/ovirt-engine/jboss_runtime'. 2019-04-13 21:50:37,981+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'PACKAGE_DISPLAY_VERSION' is '4.3.2.1-1.el7'. 2019-04-13 21:50:37,981+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'PACKAGE_NAME' is 'ovirt-engine'. 2019-04-13 21:50:37,981+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'PACKAGE_VERSION' is '4.3.2.1'. 2019-04-13 21:50:37,981+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SENSITIVE_KEYS' is ',ENGINE_DB_PASSWORD,DWH_DB_PASSWORD,CINDERLIB_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD,ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD,ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD,ENGINE_SSO_CLIENT_SECRET'. 2019-04-13 21:50:37,981+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_ALTERNATE_ENGINE_FQDNS' is ''. 2019-04-13 21:50:37,981+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_AUTH_LOGIN_SEQUENCE' is 'NI'. 2019-04-13 21:50:37,982+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_CALLBACK_CLIENT_POOL_SIZE' is '10'. 2019-04-13 21:50:37,982+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_CALLBACK_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 21:50:37,982+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_CALLBACK_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 21:50:37,982+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_CALLBACK_CONNECT_TIMEOUT' is '0'. 2019-04-13 21:50:37,982+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_CALLBACK_PREFIX_CHECK' is 'true'. 2019-04-13 21:50:37,982+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_CALLBACK_READ_TIMEOUT' is '0'. 2019-04-13 21:50:37,983+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_ENGINE_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/'. 2019-04-13 21:50:37,983+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_HOUSE_KEEPING_INTERVAL' is '60'. 2019-04-13 21:50:37,983+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_TOKEN_HTTP_LOGIN_SEQUENCE' is 'Nb'. 2019-04-13 21:50:37,983+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'SSO_TOKEN_TIMEOUT' is '360000'. 2019-04-13 21:50:37,983+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'THREAD_POOL_MONITORING_INTERVAL_IN_SECONDS' is '600'. 2019-04-13 21:50:37,983+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 57) [] Value of property 'sclenv' is 'rh-postgresql10'. 2019-04-13 21:50:38,031+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Loading extension 'internal-authn' 2019-04-13 21:50:38,039+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Extension 'internal-authn' loaded 2019-04-13 21:50:38,098+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Loading extension 'internal-authz' 2019-04-13 21:50:38,100+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Extension 'internal-authz' loaded 2019-04-13 21:50:38,101+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Initializing extension 'internal-authn' 2019-04-13 21:50:38,259+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Extension 'internal-authn' initialized 2019-04-13 21:50:38,260+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Initializing extension 'internal-authz' 2019-04-13 21:50:38,298+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Extension 'internal-authz' initialized 2019-04-13 21:50:38,299+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Start of enabled extensions list 2019-04-13 21:50:38,299+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Instance name: 'internal-authn', Extension name: '"ovirt-engine-extension-aaa-jdbc".authn', Version: '"1.1.10"', Notes: 'Display name: "ovirt-engine-extension-aaa-jdbc"', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: '/etc/ovirt-engine/extensions.d/internal-authn.properties', Initialized: 'true' 2019-04-13 21:50:38,299+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] Instance name: 'internal-authz', Extension name: '"ovirt-engine-extension-aaa-jdbc".authz', Version: '"1.1.10"', Notes: 'Display name: "ovirt-engine-extension-aaa-jdbc"', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: '/etc/ovirt-engine/extensions.d/internal-authz.properties', Initialized: 'true' 2019-04-13 21:50:38,299+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 57) [] End of enabled extensions list 2019-04-13 21:50:38,566+02 INFO [org.ovirt.engine.core.sso.utils.LocalizationUtils] (ServerService Thread Pool -- 57) [] Start initializing LocalizationUtils 2019-04-13 21:50:38,583+02 INFO [org.ovirt.engine.core.sso.utils.LocalizationUtils] (ServerService Thread Pool -- 57) [] Finished initializing LocalizationUtils 2019-04-13 21:50:38,669+02 INFO [org.ovirt.engine.ui.frontend.server.dashboard.DashboardDataServlet] (ServerService Thread Pool -- 43) [] Dashboard utilization cache updater initialized (update interval 300s) 2019-04-13 21:50:38,681+02 INFO [org.ovirt.engine.ui.frontend.server.dashboard.DashboardDataServlet] (ServerService Thread Pool -- 43) [] Dashboard inventory cache updater initialized (update interval 60s) 2019-04-13 21:50:39,141+02 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 53) [] Start initializing TagsDirector 2019-04-13 21:50:39,174+02 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 53) [] Tag 'root' added to tree 2019-04-13 21:50:39,231+02 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 53) [] Finished initializing TagsDirector 2019-04-13 21:50:39,281+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'HotPlugMemoryMultiplicationSizeMb' 2019-04-13 21:50:39,284+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DnsResolverConfigurationSupported' 2019-04-13 21:50:39,306+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DbJustRestored' 2019-04-13 21:50:39,312+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'ConfigDir' 2019-04-13 21:50:39,312+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'AdminDomain' 2019-04-13 21:50:39,312+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'AllowDuplicateMacAddresses' 2019-04-13 21:50:39,312+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DefaultWorkgroup' 2019-04-13 21:50:39,312+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'KeystoneAuthUrl' 2019-04-13 21:50:39,312+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'LicenseCertificateFingerPrint' 2019-04-13 21:50:39,313+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'MacPoolRanges' 2019-04-13 21:50:39,313+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'MaxMacsCountInPool' 2019-04-13 21:50:39,313+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'VdsFenceOptions' 2019-04-13 21:50:39,321+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'HotPlugMemoryMultiplicationSizeMb' 2019-04-13 21:50:39,322+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DnsResolverConfigurationSupported' 2019-04-13 21:50:39,323+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DbJustRestored' 2019-04-13 21:50:39,324+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'ConfigDir' 2019-04-13 21:50:39,324+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'AdminDomain' 2019-04-13 21:50:39,324+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'AllowDuplicateMacAddresses' 2019-04-13 21:50:39,324+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DefaultWorkgroup' 2019-04-13 21:50:39,324+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'KeystoneAuthUrl' 2019-04-13 21:50:39,324+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'LicenseCertificateFingerPrint' 2019-04-13 21:50:39,324+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'MacPoolRanges' 2019-04-13 21:50:39,325+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'MaxMacsCountInPool' 2019-04-13 21:50:39,325+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'VdsFenceOptions' 2019-04-13 21:50:39,330+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'HotPlugMemoryMultiplicationSizeMb' 2019-04-13 21:50:39,377+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DnsResolverConfigurationSupported' 2019-04-13 21:50:39,383+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DbJustRestored' 2019-04-13 21:50:39,384+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'ConfigDir' 2019-04-13 21:50:39,385+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'AdminDomain' 2019-04-13 21:50:39,385+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'AllowDuplicateMacAddresses' 2019-04-13 21:50:39,385+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'DefaultWorkgroup' 2019-04-13 21:50:39,385+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'KeystoneAuthUrl' 2019-04-13 21:50:39,385+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'LicenseCertificateFingerPrint' 2019-04-13 21:50:39,385+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'MacPoolRanges' 2019-04-13 21:50:39,385+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'MaxMacsCountInPool' 2019-04-13 21:50:39,386+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 53) [] Could not find enum value for option: 'VdsFenceOptions' 2019-04-13 21:50:39,435+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 53) [] Start initializing Backend 2019-04-13 21:50:39,436+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.common.utils.EngineThreadPools@ace8d4b 2019-04-13 21:50:39,437+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (ServerService Thread Pool -- 53) [] Initializing Thread Monitoring Service 2019-04-13 21:50:39,441+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (ServerService Thread Pool -- 53) [] Thread Monitoring Service initialized 2019-04-13 21:50:39,441+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService@18ebca74 2019-04-13 21:50:39,463+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'default' is using 0 threads out of 1, 4 threads waiting for tasks. 2019-04-13 21:50:39,470+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engineScheduled' is using 0 threads out of 100, 1 threads waiting for tasks. 2019-04-13 21:50:39,470+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engineThreadMonitoring' is using 1 threads out of 1, 0 threads waiting for tasks. 2019-04-13 21:50:39,530+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.hostedengine.PreviousHostedEngineHost@35df24da 2019-04-13 21:50:39,621+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 53) [] Started task scheduler org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl@21fa5242 2019-04-13 21:50:39,719+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 53) [] Started task scheduler org.ovirt.engine.core.utils.timer.DBSchedulerUtilQuartzImpl@4a9c46c3 2019-04-13 21:50:39,751+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.dal.utils.CacheManager@2971c2b0 2019-04-13 21:50:39,754+02 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 53) [] Loading file '/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties' 2019-04-13 21:50:39,826+02 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 53) [] Loading file '/etc/ovirt-engine/osinfo.conf.d/10-ad.properties' 2019-04-13 21:50:40,014+02 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 53) [] Start initializing ResourceManager$Proxy$_$$_WeldSubclass 2019-04-13 21:50:40,172+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 53) [] Entered VdsManager constructor 2019-04-13 21:50:40,230+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 53) [] Initialize vdsBroker 'onode-1.discovery.intern:54321' 2019-04-13 21:50:40,431+02 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 53) [] VDS '65c45da8-2a46-42e0-9c3a-58748546103b' was added to the Resource Manager 2019-04-13 21:50:40,431+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 53) [] Entered VdsManager constructor 2019-04-13 21:50:40,453+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 53) [] Initialize vdsBroker 'onode-2.discovery.intern:54321' 2019-04-13 21:50:40,529+02 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 53) [] VDS '7e1fc8ae-a99f-4828-84fa-a111f8d9ba89' was added to the Resource Manager 2019-04-13 21:50:40,530+02 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 53) [] Finished initializing ResourceManager$Proxy$_$$_WeldSubclass 2019-04-13 21:50:40,548+02 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 53) [] Start initializing dictionaries 2019-04-13 21:50:40,554+02 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 53) [] Finished initializing dictionaries 2019-04-13 21:50:40,826+02 INFO [org.ovirt.engine.core.bll.network.macpool.MacPoolUsingRanges] (ServerService Thread Pool -- 53) [] Initializing MacPoolUsingRanges:{id='59ef3a2b-01f3-02e9-0075-00000000035f'} 2019-04-13 21:50:40,830+02 INFO [org.ovirt.engine.core.bll.network.macpool.MacPoolUsingRanges] (ServerService Thread Pool -- 53) [] Finished initializing MacPoolUsingRanges:{id='59ef3a2b-01f3-02e9-0075-00000000035f'}. Available MACs in pool: 130 2019-04-13 21:50:40,838+02 INFO [org.ovirt.engine.core.bll.network.macpool.MacPoolPerCluster] (ServerService Thread Pool -- 53) [] Successfully initialized 2019-04-13 21:50:40,866+02 INFO [org.ovirt.engine.core.bll.network.macpool.MacPoolPerCluster] (ServerService Thread Pool -- 53) [] Mac pool 59ef3a2b-01f3-02e9-0075-00000000035f has 130 available free macs 2019-04-13 21:50:40,877+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 53) [] Running ovirt-engine 4.3.2.1-1.el7 2019-04-13 21:50:40,894+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.CpuFlagsManagerHandler@cfc3e25 2019-04-13 21:50:40,895+02 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 53) [] Start initializing AuditLogCleanupManager 2019-04-13 21:50:40,896+02 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 53) [] Setting audit cleanup manager to run at '35 35 3 * * ?' 2019-04-13 21:50:40,907+02 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 53) [] Finished initializing AuditLogCleanupManager 2019-04-13 21:50:40,907+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.AuditLogCleanupManager@281c5e5a 2019-04-13 21:50:40,907+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.TagsDirector@64b34449 2019-04-13 21:50:40,914+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.storage.domain.IsoDomainListSynchronizer@e842c62 2019-04-13 21:50:40,931+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.VmHandler@7bd3c073 2019-04-13 21:50:40,933+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.VdsHandler@120e7c99 2019-04-13 21:50:40,933+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.VmTemplateHandler@753d6faa 2019-04-13 21:50:40,934+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 53) [] Completed initializing handlers 2019-04-13 21:50:40,934+02 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 53) [] Start initializing ErrorTranslatorImpl 2019-04-13 21:50:40,989+02 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 53) [] Finished initializing ErrorTranslatorImpl 2019-04-13 21:50:40,989+02 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 53) [] Start initializing ErrorTranslatorImpl 2019-04-13 21:50:40,990+02 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 53) [] Finished initializing ErrorTranslatorImpl 2019-04-13 21:50:40,990+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 53) [] Mark incomplete jobs as UNKNOWN 2019-04-13 21:50:41,023+02 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 53) [] Start initializing JobRepositoryCleanupManager 2019-04-13 21:50:41,024+02 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 53) [] Finished initializing JobRepositoryCleanupManager 2019-04-13 21:50:41,025+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager@492c4103 2019-04-13 21:50:41,026+02 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 53) [] Start initializing AutoRecoveryManager 2019-04-13 21:50:41,029+02 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 53) [] Finished initializing AutoRecoveryManager 2019-04-13 21:50:41,029+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.AutoRecoveryManager@e184f64 2019-04-13 21:50:41,057+02 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 53) [] Start initializing ExecutionMessageDirector 2019-04-13 21:50:41,060+02 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 53) [] Finished initializing ExecutionMessageDirector 2019-04-13 21:50:41,067+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.VmPoolHandler@4641c9c0 2019-04-13 21:50:41,069+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.VmPoolMonitor@33f87143 2019-04-13 21:50:41,101+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.HaAutoStartVmsRunner@3e3eb29a 2019-04-13 21:50:41,107+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.quota.QuotaManager@4c2ec6b 2019-04-13 21:50:41,116+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.vdsbroker.monitoring.VmMigrationProgressMonitoring@6508189 2019-04-13 21:50:41,514+02 WARN [org.ovirt.engine.core.bll.IconLoader] (ServerService Thread Pool -- 53) [] java.lang.RuntimeException: Icon for rhel_6_9_plus_ppc64 was not found in /usr/share/ovirt-engine/icons/small 2019-04-13 21:50:41,514+02 WARN [org.ovirt.engine.core.bll.IconLoader] (ServerService Thread Pool -- 53) [] java.lang.RuntimeException: Icon for rhel_6_9_plus_ppc64 was not found in /usr/share/ovirt-engine/icons/large 2019-04-13 21:50:41,641+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.IconLoader@423a0ec5 2019-04-13 21:50:41,654+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Loading extension 'internal-authn' 2019-04-13 21:50:41,654+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Extension 'internal-authn' loaded 2019-04-13 21:50:41,656+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Loading extension 'internal-authz' 2019-04-13 21:50:41,656+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Extension 'internal-authz' loaded 2019-04-13 21:50:41,659+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Initializing extension 'internal-authn' 2019-04-13 21:50:41,681+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Extension 'internal-authn' initialized 2019-04-13 21:50:41,681+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Start of enabled extensions list 2019-04-13 21:50:41,682+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Instance name: 'internal-authn', Extension name: '"ovirt-engine-extension-aaa-jdbc".authn', Version: '"1.1.10"', Notes: 'Display name: "ovirt-engine-extension-aaa-jdbc"', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: '/etc/ovirt-engine/extensions.d/internal-authn.properties', Initialized: 'true' 2019-04-13 21:50:41,682+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] Instance name: 'internal-authz', Extension name: '"ovirt-engine-extension-aaa-jdbc".authz', Version: '"1.1.10"', Notes: 'Display name: "ovirt-engine-extension-aaa-jdbc"', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: '/etc/ovirt-engine/extensions.d/internal-authz.properties', Initialized: 'false' 2019-04-13 21:50:41,682+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 53) [] End of enabled extensions list 2019-04-13 21:50:41,787+02 INFO [org.ovirt.engine.core.bll.pm.PmHealthCheckManager] (ServerService Thread Pool -- 53) [] Finished initializing PmHealthCheckManager 2019-04-13 21:50:41,788+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.pm.PmHealthCheckManager@3d7c0965 2019-04-13 21:50:41,790+02 INFO [org.ovirt.engine.core.bll.EngineBackupAwarenessManager] (ServerService Thread Pool -- 53) [] Start initializing EngineBackupAwarenessManager 2019-04-13 21:50:41,793+02 INFO [org.ovirt.engine.core.bll.EngineBackupAwarenessManager] (ServerService Thread Pool -- 53) [] Finished initializing EngineBackupAwarenessManager 2019-04-13 21:50:41,793+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.EngineBackupAwarenessManager@5fc44c27 2019-04-13 21:50:41,794+02 INFO [org.ovirt.engine.core.bll.DataCenterCompatibilityChecker] (ServerService Thread Pool -- 53) [] Start initializing DataCenterCompatibilityChecker 2019-04-13 21:50:41,798+02 INFO [org.ovirt.engine.core.bll.DataCenterCompatibilityChecker] (ServerService Thread Pool -- 53) [] Finished initializing DataCenterCompatibilityChecker 2019-04-13 21:50:41,798+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.DataCenterCompatibilityChecker@2ff99a7 2019-04-13 21:50:41,798+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass@7fa748ab 2019-04-13 21:50:41,839+02 INFO [org.ovirt.engine.core.bll.PrevalidatingMultipleActionsRunner] (ServerService Thread Pool -- 53) [] PrevalidatingMultipleActionsRunner of type 'RefreshHostDevices' invoked with no actions 2019-04-13 21:50:41,847+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.hostdev.HostDeviceManager@427e069 2019-04-13 21:50:41,848+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyManager] (ServerService Thread Pool -- 53) [] Start initializing IrsProxyManager 2019-04-13 21:50:41,893+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyManager] (ServerService Thread Pool -- 53) [] Start initializing IrsProxyManager 2019-04-13 21:50:41,893+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyManager@388dbef2 2019-04-13 21:50:41,900+02 INFO [org.ovirt.engine.core.bll.storage.ovfstore.OvfDataUpdater] (ServerService Thread Pool -- 53) [] Initialization of OvfDataUpdater completed successfully. 2019-04-13 21:50:41,900+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.storage.ovfstore.OvfDataUpdater@adcef58 2019-04-13 21:50:41,907+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.storage.pool.StoragePoolStatusHandler@3f1c7ea7 2019-04-13 21:50:42,089+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.gluster.GlusterJobsManager@326ea34f 2019-04-13 21:50:42,110+02 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 53) [] Init VM custom properties utilities 2019-04-13 21:50:42,111+02 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 53) [] Init device custom properties utilities 2019-04-13 21:50:42,145+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 53) [] Initializing Scheduling manager 2019-04-13 21:50:42,298+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 53) [] External scheduler disabled, discovery skipped 2019-04-13 21:50:42,298+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 53) [] Start scheduling to enable vds load balancer 2019-04-13 21:50:42,300+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 53) [] Finished scheduling to enable vds load balancer 2019-04-13 21:50:42,300+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 53) [] Start HA Reservation check 2019-04-13 21:50:42,302+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 53) [] Finished HA Reservation check 2019-04-13 21:50:42,302+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 53) [] Initialized Scheduling manager 2019-04-13 21:50:42,302+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.scheduling.SchedulingManager@7169523f 2019-04-13 21:50:42,310+02 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 53) [] Initializing DWH Heart Beat 2019-04-13 21:50:42,313+02 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 53) [] DWH Heart Beat initialized 2019-04-13 21:50:42,313+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.dwh.DwhHeartBeat@145cf4d1 2019-04-13 21:50:42,370+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 53) [] Initialization of AsyncTaskManager completed successfully. 2019-04-13 21:50:42,371+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.tasks.AsyncTaskManager@2873c085 2019-04-13 21:50:42,371+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.tasks.CommandCoordinatorUtil@3d6e84ee 2019-04-13 21:50:42,372+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (ServerService Thread Pool -- 53) [] Start initializing CommandCallbacksPoller 2019-04-13 21:50:42,373+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (ServerService Thread Pool -- 53) [] Finished initializing CommandCallbacksPoller 2019-04-13 21:50:42,373+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller@1391b03c 2019-04-13 21:50:42,373+02 INFO [org.ovirt.engine.core.bll.CommandEntityCleanupManager] (ServerService Thread Pool -- 53) [] Start initializing CommandEntityCleanupManager 2019-04-13 21:50:42,374+02 INFO [org.ovirt.engine.core.bll.CommandEntityCleanupManager] (ServerService Thread Pool -- 53) [] Setting command entity cleanup manager to run at: 35 35 3 * * ? 2019-04-13 21:50:42,375+02 INFO [org.ovirt.engine.core.bll.CommandEntityCleanupManager] (ServerService Thread Pool -- 53) [] Finished initializing CommandEntityCleanupManager 2019-04-13 21:50:42,375+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.CommandEntityCleanupManager@762b5d27 2019-04-13 21:50:42,414+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ServerService Thread Pool -- 53) [] EVENT_ID: AFFINITY_RULES_ENFORCEMENT_MANAGER_START(10,780), Affinity Rules Enforcement Manager started. 2019-04-13 21:50:42,417+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.scheduling.AffinityRulesEnforcementManager@64ef4713 2019-04-13 21:50:42,419+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.CertificationValidityChecker@42d6192a 2019-04-13 21:50:42,422+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.hostdeploy.HostUpdatesCheckerService@734dedc7 2019-04-13 21:50:42,423+02 INFO [org.ovirt.engine.core.bll.IPTablesDeprecationNotifier] (ServerService Thread Pool -- 53) [] Start initializing IPTablesDeprecationNotifier 2019-04-13 21:50:42,441+02 INFO [org.ovirt.engine.core.bll.IPTablesDeprecationNotifier] (ServerService Thread Pool -- 53) [] Finished initializing IPTablesDeprecationNotifier 2019-04-13 21:50:42,442+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.IPTablesDeprecationNotifier@3b71e4af 2019-04-13 21:50:42,453+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 53) [] Start org.ovirt.engine.core.bll.network.cluster.ExternalNetworkSyncService@212c5ed 2019-04-13 21:50:45,685+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-2.discovery.intern/192.168.0.36 2019-04-13 21:51:01,984+02 INFO [org.ovirt.engine.extension.aaa.jdbc.core.Tasks] (default task-1) [] (house keeping) deleting failed logins prior to 2019-04-06 19:51:01Z. 2019-04-13 21:51:02,193+02 INFO [org.ovirt.engine.extension.aaa.jdbc.core.Tasks] (default task-1) [] (house keeping) deleting failed logins prior to 2019-04-06 19:51:02Z. 2019-04-13 21:51:02,267+02 INFO [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-1) [] User admin@internal successfully logged in with scopes: ovirt-app-admin ovirt-app-api ovirt-app-portal ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access 2019-04-13 21:51:02,822+02 INFO [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default task-1) [2cd80fe2] Running command: CreateUserSessionCommand internal: false. 2019-04-13 21:51:02,889+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-1) [2cd80fe2] EVENT_ID: USER_VDC_LOGIN(30), User admin@internal-authz connecting from '192.168.0.1' using session 'T2dqaL3jVCwZTiaUqiqQ/yOjZHKfSPcrpjmBOSofFeTl81it3MNiri/s6dI6bNi2sow33b4YlJrjbhD3IWKrQw==' logged in. 2019-04-13 21:51:03,069+02 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (default task-1) [] Reading UI plugin descriptor '/usr/share/ovirt-engine/ui-plugins/ui-extensions.json' 2019-04-13 21:51:03,077+02 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (default task-1) [] Reading UI plugin configuration '/etc/ovirt-engine/ui-plugins/ui-extensions-config.json' 2019-04-13 21:51:05,199+02 INFO [org.ovirt.engine.core.utils.servlet.ServletUtils] (default task-3) [] Can't read file '/usr/share/ovirt-engine/files/spice/SpiceVersion.txt' for request '/ovirt-engine/services/files/spice/SpiceVersion.txt' -- 404 2019-04-13 21:51:06,197+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] START, GetHardwareInfoAsyncVDSCommand(HostName = onode-2, VdsIdAndVdsVDSCommandParametersBase:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', vds='Host[onode-2,7e1fc8ae-a99f-4828-84fa-a111f8d9ba89]'}), log id: 369737ee 2019-04-13 21:51:06,212+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 369737ee 2019-04-13 21:51:06,544+02 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-57) [4731c858] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 7e1fc8ae-a99f-4828-84fa-a111f8d9ba89 Type: VDS 2019-04-13 21:51:09,951+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [] START, GetHardwareInfoAsyncVDSCommand(HostName = onode-2, VdsIdAndVdsVDSCommandParametersBase:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', vds='Host[onode-2,7e1fc8ae-a99f-4828-84fa-a111f8d9ba89]'}), log id: 8c4161 2019-04-13 21:51:09,953+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 8c4161 2019-04-13 21:51:10,191+02 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [59f66331] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:51:10,250+02 INFO [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:51:10,339+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] START, ConnectStorageServerVDSCommand(HostName = onode-2, StorageServerConnectionManagementVDSParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='2286311a-c1c9-4dda-bea5-9c78651c6e36', connection='onode-1.discovery.intern:/mnt/store', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 2ddd71fb 2019-04-13 21:51:10,686+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] FINISH, ConnectStorageServerVDSCommand, return: {2286311a-c1c9-4dda-bea5-9c78651c6e36=0}, log id: 2ddd71fb 2019-04-13 21:51:10,688+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] START, ConnectStorageServerVDSCommand(HostName = onode-2, StorageServerConnectionManagementVDSParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='b4f4bcf2-e104-468e-b330-ed731ec8ef6c', connection='nas.discovery.intern:/mnt/d2600/vms', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 5a972aca 2019-04-13 21:51:10,796+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] FINISH, ConnectStorageServerVDSCommand, return: {b4f4bcf2-e104-468e-b330-ed731ec8ef6c=0}, log id: 5a972aca 2019-04-13 21:51:10,800+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] START, ConnectStorageServerVDSCommand(HostName = onode-2, StorageServerConnectionManagementVDSParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='07dd9ff2-afa1-4a33-b401-935ea5b8c9db', connection='nas.discovery.intern:/mnt/d2600/isos', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='false'}), log id: 534b3dc3 2019-04-13 21:51:10,901+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] FINISH, ConnectStorageServerVDSCommand, return: {07dd9ff2-afa1-4a33-b401-935ea5b8c9db=0}, log id: 534b3dc3 2019-04-13 21:51:10,903+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] START, ConnectStorageServerVDSCommand(HostName = onode-2, StorageServerConnectionManagementVDSParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='858731bf-2fb1-45a7-a693-235f675a667b', connection='nas.discovery.intern:/mnt/d2600/sicherung/vmBackups', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='false'}), log id: 2fdd1e3c 2019-04-13 21:51:11,003+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] FINISH, ConnectStorageServerVDSCommand, return: {858731bf-2fb1-45a7-a693-235f675a667b=0}, log id: 2fdd1e3c 2019-04-13 21:51:11,028+02 INFO [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] Host 'onode-2' storage connection was succeeded 2019-04-13 21:51:11,194+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-3) [324a67b1] START, ConnectStoragePoolVDSCommand(HostName = onode-2, ConnectStoragePoolVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', vdsId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 50a165ed 2019-04-13 21:51:11,197+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-3) [324a67b1] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:51:11,721+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-3) [324a67b1] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 50a165ed 2019-04-13 21:51:11,797+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-3) [324a67b1] EVENT_ID: NO_FAULTY_MULTIPATHS_ON_HOST(11,501), No faulty multipath paths on host onode-2 2019-04-13 21:51:12,006+02 ERROR [org.ovirt.engine.core.bll.pm.FenceProxyLocator] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] Can not run fence action on host 'onode-2', no suitable proxy host was found. 2019-04-13 21:51:12,030+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] START, SetMOMPolicyParametersVDSCommand(HostName = onode-2, MomPolicyVDSParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89'}), log id: 48278833 2019-04-13 21:51:12,105+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [324a67b1] FINISH, SetMOMPolicyParametersVDSCommand, return: , log id: 48278833 2019-04-13 21:51:12,157+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Fetched 0 VMs from VDS '7e1fc8ae-a99f-4828-84fa-a111f8d9ba89' 2019-04-13 21:51:12,287+02 INFO [org.ovirt.engine.core.bll.hostdev.RefreshHostDevicesCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [38aa9b85] Running command: RefreshHostDevicesCommand internal: true. Entities affected : ID: 7e1fc8ae-a99f-4828-84fa-a111f8d9ba89 Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 21:51:13,023+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [38aa9b85] EVENT_ID: VDS_DETECTED(13), Status of host onode-2 was set to Up. 2019-04-13 21:51:13,094+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [38aa9b85] EVENT_ID: VDS_ALERT_FENCE_TEST_FAILED(9,001), Power Management test failed for Host onode-2.There is no other host in the data center that can be used to test the power management settings. 2019-04-13 21:51:13,123+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [38aa9b85] EVENT_ID: VDS_FENCE_STATUS_FAILED(497), Failed to verify Host onode-2 power management. 2019-04-13 21:51:13,572+02 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-33) [68858de7] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 7e1fc8ae-a99f-4828-84fa-a111f8d9ba89 Type: VDS 2019-04-13 21:51:13,566+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] START, ConnectStoragePoolVDSCommand(HostName = onode-2, ConnectStoragePoolVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', vdsId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 1ebcc5a0 2019-04-13 21:51:13,576+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 21:51:13,973+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 1ebcc5a0 2019-04-13 21:51:14,001+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] hostFromVds::selectedVds - 'onode-2', spmStatus 'Free', storage pool 'Default', storage pool version '4.3' 2019-04-13 21:51:14,033+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] starting spm on vds 'onode-2', storage pool 'Default', prevId '-1', LVER '-1' 2019-04-13 21:51:14,039+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] START, SpmStartVDSCommand(HostName = onode-2, SpmStartVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', prevId='-1', prevLVER='-1', storagePoolFormatType='V4', recoveryMode='Manual', SCSIFencing='false'}), log id: 693ba6e2 2019-04-13 21:51:14,045+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] spmStart polling started: taskId 'df608ad4-55da-43d0-962d-69868f5cf253' 2019-04-13 21:51:27,469+02 INFO [org.ovirt.engine.core.bll.ClearAllAuditLogAlertsCommand] (default task-1) [06373397-0f74-450a-a3f3-bd9df9baf2a8] Running command: ClearAllAuditLogAlertsCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group AUDIT_LOG_MANAGEMENT with role type ADMIN 2019-04-13 21:51:27,516+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-1) [06373397-0f74-450a-a3f3-bd9df9baf2a8] EVENT_ID: USER_CLEAR_ALL_AUDIT_LOG_ALERTS(10,606), All alerts were removed. (User: admin@internal-authz) 2019-04-13 21:51:33,293+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] spmStart polling ended: taskId 'df608ad4-55da-43d0-962d-69868f5cf253' task status 'finished' 2019-04-13 21:51:33,317+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] spmStart polling ended, spm status: SPM 2019-04-13 21:51:33,325+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] START, HSMClearTaskVDSCommand(HostName = onode-2, HSMTaskGuidBaseVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', taskId='df608ad4-55da-43d0-962d-69868f5cf253'}), log id: 4dc17f8a 2019-04-13 21:51:33,333+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] FINISH, HSMClearTaskVDSCommand, return: , log id: 4dc17f8a 2019-04-13 21:51:33,333+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] FINISH, SpmStartVDSCommand, return: SpmStatusResult:{SPM Id='2', SPM LVER='28', SPM Status='SPM'}, log id: 693ba6e2 2019-04-13 21:51:33,339+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] Initialize Irs proxy from vds: onode-2.discovery.intern 2019-04-13 21:51:33,365+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-12) [] EVENT_ID: IRS_HOSTED_ON_VDS(204), Storage Pool Manager runs on Host onode-2.discovery.intern (Address: onode-2.discovery.intern), Data Center Default. 2019-04-13 21:51:33,388+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-2.discovery.intern/192.168.0.36 2019-04-13 21:51:33,421+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-6) [] START, SPMGetAllTasksInfoVDSCommand( IrsBaseVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='false'}), log id: 33e5a8be 2019-04-13 21:51:33,668+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-6) [] -- executeIrsBrokerCommand: Attempting on storage pool '59ef3a18-002f-02d1-0220-000000000124' 2019-04-13 21:51:33,674+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-6) [] START, HSMGetAllTasksInfoVDSCommand(HostName = onode-2, VdsIdVDSCommandParametersBase:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89'}), log id: 1aa82dcd 2019-04-13 21:51:33,684+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-6) [] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 1aa82dcd 2019-04-13 21:51:33,684+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-6) [] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 33e5a8be 2019-04-13 21:51:33,684+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engine-Thread-6) [] Discovered no tasks on Storage Pool 'Default' 2019-04-13 21:52:56,247+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] Lock Acquired to object 'EngineLock:{exclusiveLocks='', sharedLocks='[59ef3a18-002f-02d1-0220-000000000124=POOL]'}' 2019-04-13 21:52:56,450+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] Running command: MaintenanceNumberOfVdssCommand internal: false. Entities affected : ID: 7e1fc8ae-a99f-4828-84fa-a111f8d9ba89 Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 21:52:56,465+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] START, SetVdsStatusVDSCommand(HostName = onode-2, SetVdsStatusVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', status='PreparingForMaintenance', nonOperationalReason='NONE', stopSpmFailureLogged='true', maintenanceReason='null'}), log id: 7b839835 2019-04-13 21:52:56,466+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] VDS 'onode-2' is spm and moved from up calling resetIrs. 2019-04-13 21:52:56,470+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] START, ResetIrsVDSCommand( ResetIrsVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='false', vdsId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', ignoreStopFailed='false'}), log id: 34aee24 2019-04-13 21:52:56,484+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] START, SpmStopVDSCommand(HostName = onode-2, SpmStopVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124'}), log id: 33d4ad14 2019-04-13 21:52:56,494+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] SpmStopVDSCommand::Stopping SPM on vds 'onode-2', pool id '59ef3a18-002f-02d1-0220-000000000124' 2019-04-13 21:52:56,553+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] FINISH, SpmStopVDSCommand, return: , log id: 33d4ad14 2019-04-13 21:52:56,563+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ResetIrsVDSCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] FINISH, ResetIrsVDSCommand, return: , log id: 34aee24 2019-04-13 21:52:56,580+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] FINISH, SetVdsStatusVDSCommand, return: , log id: 7b839835 2019-04-13 21:52:56,587+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] Lock freed to object 'EngineLock:{exclusiveLocks='', sharedLocks='[59ef3a18-002f-02d1-0220-000000000124=POOL]'}' 2019-04-13 21:52:56,780+02 INFO [org.ovirt.engine.core.bll.MaintenanceVdsCommand] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] Running command: MaintenanceVdsCommand internal: true. Entities affected : ID: 7e1fc8ae-a99f-4828-84fa-a111f8d9ba89 Type: VDS 2019-04-13 21:52:56,862+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-3) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] EVENT_ID: USER_VDS_MAINTENANCE_WITHOUT_REASON(620), Host onode-2 was switched to Maintenance mode by admin@internal-authz. 2019-04-13 21:52:58,542+02 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-5) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] Command 'MaintenanceNumberOfVdss' id: '71100592-01c8-466f-82d7-064bf5760e76' child commands '[]' executions were completed, status 'SUCCEEDED' 2019-04-13 21:52:59,409+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] Updated host status from 'Preparing for Maintenance' to 'Maintenance' in database, host 'onode-2'(7e1fc8ae-a99f-4828-84fa-a111f8d9ba89) 2019-04-13 21:52:59,500+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-18) [] Clearing cache of pool: '59ef3a18-002f-02d1-0220-000000000124' for problematic entities of VDS: 'onode-2'. 2019-04-13 21:52:59,501+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-18) [] Removing vds '[7e1fc8ae-a99f-4828-84fa-a111f8d9ba89]' from the domain in maintenance cache 2019-04-13 21:52:59,501+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engine-Thread-18) [] Removing host(s) '[7e1fc8ae-a99f-4828-84fa-a111f8d9ba89]' from hosts unseen domain report cache 2019-04-13 21:52:59,508+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] START, DisconnectStoragePoolVDSCommand(HostName = onode-2, DisconnectStoragePoolVDSCommandParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', vds_spm_id='2'}), log id: 5d7aa95a 2019-04-13 21:52:59,560+02 INFO [org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-53) [09e1aa11-cb9e-4ff7-94a8-399950f9f48f] Ending command 'org.ovirt.engine.core.bll.MaintenanceNumberOfVdssCommand' successfully. 2019-04-13 21:53:01,694+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [] FINISH, DisconnectStoragePoolVDSCommand, return: , log id: 5d7aa95a 2019-04-13 21:53:01,724+02 INFO [org.ovirt.engine.core.bll.storage.pool.DisconnectHostFromStoragePoolServersCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [345a83d9] Running command: DisconnectHostFromStoragePoolServersCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:53:01,755+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [345a83d9] START, DisconnectStorageServerVDSCommand(HostName = onode-2, StorageServerConnectionManagementVDSParameters:{hostId='7e1fc8ae-a99f-4828-84fa-a111f8d9ba89', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='2286311a-c1c9-4dda-bea5-9c78651c6e36', connection='onode-1.discovery.intern:/mnt/store', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}, StorageServerConnections:{id='858731bf-2fb1-45a7-a693-235f675a667b', connection='nas.discovery.intern:/mnt/d2600/sicherung/vmBackups', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}, StorageServerConnections:{id='07dd9ff2-afa1-4a33-b401-935ea5b8c9db', connection='nas.discovery.intern:/mnt/d2600/isos', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}, StorageServerConnections:{id='b4f4bcf2-e104-468e-b330-ed731ec8ef6c', connection='nas.discovery.intern:/mnt/d2600/vms', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 576d0748 2019-04-13 21:53:02,202+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.DisconnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [345a83d9] FINISH, DisconnectStorageServerVDSCommand, return: {b4f4bcf2-e104-468e-b330-ed731ec8ef6c=0, 2286311a-c1c9-4dda-bea5-9c78651c6e36=0, 07dd9ff2-afa1-4a33-b401-935ea5b8c9db=0, 858731bf-2fb1-45a7-a693-235f675a667b=0}, log id: 576d0748 2019-04-13 21:53:05,304+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '93f02ede-ec5c-4641-8e97-2949f98e856d' status from 'Active' to 'Unknown', reason: no reporting hosts 2019-04-13 21:53:05,311+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '42bb6772-12f9-4bf2-9245-43caa41c5a2a' status from 'Active' to 'Unknown', reason: no reporting hosts 2019-04-13 21:53:05,314+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '6a4cde72-6a6a-499e-9b6e-d490d91ca19e' status from 'Active' to 'Unknown', reason: no reporting hosts 2019-04-13 21:53:05,316+02 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [] Storage Pool '59ef3a18-002f-02d1-0220-000000000124' - Updating Storage Domain '9c3f06cf-7475-448e-819b-f4f52fa7d782' status from 'Active' to 'Unknown', reason: no reporting hosts 2019-04-13 21:53:05,379+02 INFO [org.ovirt.engine.core.bll.storage.pool.SetStoragePoolStatusCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [30deaf2d] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 21:53:05,414+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-29) [30deaf2d] EVENT_ID: SYSTEM_CHANGE_STORAGE_POOL_STATUS_PROBLEMATIC(980), Invalid status on Data Center Default. Setting status to Non Responsive. 2019-04-13 21:53:46,645+02 WARN [org.ovirt.engine.core.utils.ThreadUtils] (EE-ManagedThreadFactory-engine-Thread-1) [] Interrupted: sleep interrupted 2019-04-13 22:05:20,215+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf'. 2019-04-13 22:05:20,257+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] The file '/etc/ovirt-engine/engine.conf' doesn't exist or isn't readable. Will return an empty set of properties. 2019-04-13 22:05:20,257+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-scl-postgres.conf'. 2019-04-13 22:05:20,206+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/usr/share/ovirt-engine/services/ovirt-engine/ovirt-engine.conf'. 2019-04-13 22:05:20,279+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] The file '/etc/ovirt-engine/engine.conf' doesn't exist or isn't readable. Will return an empty set of properties. 2019-04-13 22:05:20,279+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-scl-postgres.conf'. 2019-04-13 22:05:20,279+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-cinderlib.conf'. 2019-04-13 22:05:20,280+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-database.conf'. 2019-04-13 22:05:20,280+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-dwh-database.conf'. 2019-04-13 22:05:20,280+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-java.conf'. 2019-04-13 22:05:20,280+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf'. 2019-04-13 22:05:20,281+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf'. 2019-04-13 22:05:20,281+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf'. 2019-04-13 22:05:20,281+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/11-setup-sso.conf'. 2019-04-13 22:05:20,281+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/20-setup-jboss-overlay.conf'. 2019-04-13 22:05:20,282+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-engine-api-explorer.conf'. 2019-04-13 22:05:20,282+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-engine-extension-aaa-jdbc.conf'. 2019-04-13 22:05:20,282+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-web-ui.conf'. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'AAA_JAAS_KEYTAB_FILE' is ''. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'AAA_JAAS_PRINCIPAL_NAME' is ''. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'AAA_JAAS_TICKET_CACHE_FILE' is ''. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'AAA_JAAS_USE_KEYTAB' is 'false'. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'AAA_JAAS_USE_TICKET_CACHE' is 'false'. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'AAA_KRB5_CONF_FILE' is ''. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ANSIBLE_PLAYBOOK_EXEC_DEFAULT_TIMEOUT' is '30'. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ANSIBLE_PLAYBOOK_VERBOSITY_LEVEL' is '1'. 2019-04-13 22:05:20,283+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_DATABASE' is 'ovirt_cinderlib'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_ENABLE' is 'False'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_HOST' is 'localhost'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_PASSWORD' is '***'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_PORT' is '5432'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_SECURED' is 'False'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_URL' is 'jdbc:postgresql://localhost:5432/ovirt_cinderlib?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 22:05:20,284+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'CINDERLIB_DB_USER' is 'ovirt_cinderlib'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'COMMAND_COORDINATOR_THREAD_POOL_SIZE' is '10'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DASHBOARD_CACHE_UPDATE' is 'true'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DASHBOARD_INVENTORY_CACHE_UPDATE_INTERVAL' is '60'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DASHBOARD_UTILIZATION_CACHE_UPDATE_INTERVAL' is '300'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_DATABASE' is 'ovirt_engine_history'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_HOST' is 'localhost'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_PASSWORD' is '***'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_PORT' is '5432'. 2019-04-13 22:05:20,285+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_SECURED' is 'False'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_URL' is 'jdbc:postgresql://localhost:5432/ovirt_engine_history?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'DWH_DB_USER' is 'ovirt_engine_history'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENFORCE_ENGINE_HEAP_PARAMS' is 'false'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_AJP_ENABLED' is 'true'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_AJP_PORT' is '8702'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_API_DEFAULT_VERSION' is '4'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_API_DEPRECATED_VERSIONS' is '3:4.0:4.3'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_API_EXPLORER_DIRECTORY' is '/usr/share/ovirt-engine-api-explorer'. 2019-04-13 22:05:20,286+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_API_FILTER_BY_DEFAULT' is 'true'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_API_SUPPORTED_VERSIONS' is '3, 4'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_APPS' is 'engine.ear restapi.war apidoc.war ovirt-web-ui.war'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_CACHE' is '/var/cache/ovirt-engine'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_CHECK_INTERVAL' is '1000'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_CONNECTION_TIMEOUT' is '300000'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_DATABASE' is 'engine'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_HOST' is 'localhost'. 2019-04-13 22:05:20,287+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_MAX_CONNECTIONS' is '100'. 2019-04-13 22:05:20,288+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_MIN_CONNECTIONS' is '1'. 2019-04-13 22:05:20,288+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_PASSWORD' is '***'. 2019-04-13 22:05:20,288+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_PORT' is '5432'. 2019-04-13 22:05:20,288+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_SECURED' is 'False'. 2019-04-13 22:05:20,288+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 22:05:20,288+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_URL' is 'jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 22:05:20,288+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DB_USER' is 'engine'. 2019-04-13 22:05:20,288+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DEBUG_ADDRESS' is ''. 2019-04-13 22:05:20,289+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DEPLOYMENT_SCANNER' is 'false'. 2019-04-13 22:05:20,289+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_DOC' is '/usr/share/doc/ovirt-engine'. 2019-04-13 22:05:20,289+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_ETC' is '/etc/ovirt-engine'. 2019-04-13 22:05:20,289+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_EXTENSIONS_IGNORED' is '.*?(Authz)'. 2019-04-13 22:05:20,289+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_EXTENSION_PATH' is '/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d'. 2019-04-13 22:05:20,289+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE' is '/var/lib/ovirt-engine/external_truststore'. 2019-04-13 22:05:20,275+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-cinderlib.conf'. 2019-04-13 22:05:20,289+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-database.conf'. 2019-04-13 22:05:20,290+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-dwh-database.conf'. 2019-04-13 22:05:20,290+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-java.conf'. 2019-04-13 22:05:20,290+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-jboss.conf'. 2019-04-13 22:05:20,290+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-pki.conf'. 2019-04-13 22:05:20,290+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/10-setup-protocols.conf'. 2019-04-13 22:05:20,291+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/11-setup-sso.conf'. 2019-04-13 22:05:20,291+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/20-setup-jboss-overlay.conf'. 2019-04-13 22:05:20,291+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-engine-api-explorer.conf'. 2019-04-13 22:05:20,291+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-engine-extension-aaa-jdbc.conf'. 2019-04-13 22:05:20,291+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Loaded file '/etc/ovirt-engine/engine.conf.d/50-ovirt-web-ui.conf'. 2019-04-13 22:05:20,292+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'AAA_JAAS_KEYTAB_FILE' is ''. 2019-04-13 22:05:20,292+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'AAA_JAAS_PRINCIPAL_NAME' is ''. 2019-04-13 22:05:20,292+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'AAA_JAAS_TICKET_CACHE_FILE' is ''. 2019-04-13 22:05:20,292+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'AAA_JAAS_USE_KEYTAB' is 'false'. 2019-04-13 22:05:20,292+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'AAA_JAAS_USE_TICKET_CACHE' is 'false'. 2019-04-13 22:05:20,292+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'AAA_KRB5_CONF_FILE' is ''. 2019-04-13 22:05:20,292+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ANSIBLE_PLAYBOOK_EXEC_DEFAULT_TIMEOUT' is '30'. 2019-04-13 22:05:20,292+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ANSIBLE_PLAYBOOK_VERBOSITY_LEVEL' is '1'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_DATABASE' is 'ovirt_cinderlib'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_ENABLE' is 'False'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_HOST' is 'localhost'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_PASSWORD' is '***'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_PORT' is '5432'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_SECURED' is 'False'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_URL' is 'jdbc:postgresql://localhost:5432/ovirt_cinderlib?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 22:05:20,293+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'CINDERLIB_DB_USER' is 'ovirt_cinderlib'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'COMMAND_COORDINATOR_THREAD_POOL_SIZE' is '10'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DASHBOARD_CACHE_UPDATE' is 'true'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DASHBOARD_INVENTORY_CACHE_UPDATE_INTERVAL' is '60'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DASHBOARD_UTILIZATION_CACHE_UPDATE_INTERVAL' is '300'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_DATABASE' is 'ovirt_engine_history'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_HOST' is 'localhost'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_PASSWORD' is '***'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_PORT' is '5432'. 2019-04-13 22:05:20,294+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_SECURED' is 'False'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_URL' is 'jdbc:postgresql://localhost:5432/ovirt_engine_history?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'DWH_DB_USER' is 'ovirt_engine_history'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENFORCE_ENGINE_HEAP_PARAMS' is 'false'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_AJP_ENABLED' is 'true'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_AJP_PORT' is '8702'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_API_DEFAULT_VERSION' is '4'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_API_DEPRECATED_VERSIONS' is '3:4.0:4.3'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_API_EXPLORER_DIRECTORY' is '/usr/share/ovirt-engine-api-explorer'. 2019-04-13 22:05:20,295+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_API_FILTER_BY_DEFAULT' is 'true'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_API_SUPPORTED_VERSIONS' is '3, 4'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_APPS' is 'engine.ear restapi.war apidoc.war ovirt-web-ui.war'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_CACHE' is '/var/cache/ovirt-engine'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_CHECK_INTERVAL' is '1000'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_CONNECTION_TIMEOUT' is '300000'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_DATABASE' is 'engine'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_DRIVER' is 'org.postgresql.Driver'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_HOST' is 'localhost'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_MAX_CONNECTIONS' is '100'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_MIN_CONNECTIONS' is '1'. 2019-04-13 22:05:20,296+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_PASSWORD' is '***'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_PORT' is '5432'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_SECURED' is 'False'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_SECURED_VALIDATION' is 'False'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_URL' is 'jdbc:postgresql://localhost:5432/engine?sslfactory=org.postgresql.ssl.NonValidatingFactory'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DB_USER' is 'engine'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DEBUG_ADDRESS' is ''. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DEPLOYMENT_SCANNER' is 'false'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_DOC' is '/usr/share/doc/ovirt-engine'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_ETC' is '/etc/ovirt-engine'. 2019-04-13 22:05:20,297+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_EXTENSIONS_IGNORED' is '.*?(Authz)'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_EXTENSION_PATH' is '/usr/share/ovirt-engine/extensions.d:/etc/ovirt-engine/extensions.d'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE' is '/var/lib/ovirt-engine/external_truststore'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_FQDN' is 'ovirt.discovery.intern'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_GROUP' is 'ovirt'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HEAP_MAX' is '1024M'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HEAP_MIN' is '1024M'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HTTPS_ENABLED' is 'false'. 2019-04-13 22:05:20,298+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HTTPS_PORT' is 'None'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HTTPS_PROTOCOLS' is 'TLSv1.1,TLSv1.2'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HTTP_ENABLED' is 'false'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_HTTP_PORT' is 'None'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_JAVA_MODULEPATH' is '/usr/share/ovirt-engine-wildfly-overlay/modules:/usr/share/ovirt-engine/modules/common:/usr/share/ovirt-engine-extension-aaa-jdbc/modules'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_JBOSS_BLOCKING_TIMEOUT' is '300'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_JMX_INTERFACE' is 'loopback'. 2019-04-13 22:05:20,299+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_JVM_ARGS' is ' -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"'. 2019-04-13 22:05:20,300+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_LOG' is '/var/log/ovirt-engine'. 2019-04-13 22:05:20,300+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_LOG_TO_CONSOLE' is 'false'. 2019-04-13 22:05:20,300+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_MALLOC_ARENA_MAX' is '1'. 2019-04-13 22:05:20,300+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_MANUAL' is '/usr/share/ovirt-engine/manual'. 2019-04-13 22:05:20,300+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_MAX_USER_SESSIONS' is '-1'. 2019-04-13 22:05:20,300+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_NON_JTA_DB_MAX_CONNECTIONS' is '10'. 2019-04-13 22:05:20,300+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_NON_JTA_DB_MIN_CONNECTIONS' is '1'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI' is '/etc/pki/ovirt-engine'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_CA' is '/etc/pki/ovirt-engine/ca.pem'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_ENGINE_CERT' is '/etc/pki/ovirt-engine/certs/engine.cer'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_ENGINE_STORE' is '/etc/pki/ovirt-engine/keys/engine.p12'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_ENGINE_STORE_ALIAS' is '1'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_ENGINE_STORE_PASSWORD' is '***'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_ENGINE_STORE_TYPE' is 'PKCS12'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 22:05:20,301+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PROPERTIES' is ' java.awt.headless=true sun.rmi.dgc.client.gcInterval=3600000 sun.rmi.dgc.server.gcInterval=3600000 jsse.enableSNIExtension=false'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PROXY_ENABLED' is 'true'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PROXY_HTTPS_PORT' is '443'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_PROXY_HTTP_PORT' is '80'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_RESTAPI_NEGO' is 'true'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SCHEDULED_THREAD_POOL_SIZE' is '100'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SCHEDULER_POOL_CLASS' is 'org.ovirt.engine.core.utils.timer.SchedulerThreadPool'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SCHEDULER_POOL_QUEUE_SIZE' is '350'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SCHEDULER_POOL_REJECT_POLICY' is 'false'. 2019-04-13 22:05:20,302+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SCHEDULER_POOL_THREAD_SIZE' is '100'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_webadmin' is '~'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_webui' is '~'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_welcome' is '~'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_AUTH_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/sso'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_CLIENT_ID' is 'ovirt-engine-core'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_CLIENT_SECRET' is '***'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_ENABLE_EXTERNAL_SSO' is 'false'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_EXTERNAL_SSO_LOGOUT_URI' is '/ovirt-engine/callback'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_INSTALLED_ON_ENGINE_HOST' is 'true'. 2019-04-13 22:05:20,303+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_SERVICE_CLIENT_POOL_SIZE' is '10'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_SERVICE_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_SERVICE_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_SERVICE_SSL_PROTOCOL' is 'TLS'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_SERVICE_SSL_VERIFY_CHAIN' is 'true'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_SERVICE_SSL_VERIFY_HOST' is 'false'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_SERVICE_URI' is '/ovirt-engine/sso'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_SSO_SERVICE_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/sso'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_STOP_INTERVAL' is '1'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_STOP_TIME' is '10'. 2019-04-13 22:05:20,304+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_THREAD_POOL_MAX_SIZE' is '500'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_THREAD_POOL_MIN_SIZE' is '50'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_THREAD_POOL_QUEUE_SIZE' is '100'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_THREAD_STACK' is '1M'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_TMP' is '/var/tmp/ovirt-engine'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_UP_MARK' is '/var/lib/ovirt-engine/engine.up'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_URI' is '/ovirt-engine'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_USER' is 'ovirt'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_USR' is '/usr/share/ovirt-engine'. 2019-04-13 22:05:20,305+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_VAR' is '/var/lib/ovirt-engine'. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'ENGINE_VERBOSE_GC' is 'false'. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_CLIENT_ID' is ''. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_CLIENT_POOL_SIZE' is '10'. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_CLIENT_SECRET' is ''. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_CONNECT_TIMEOUT' is '0'. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE_PASSWORD' is ''. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 22:05:20,306+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_LOGOUT_END_POINT' is ''. 2019-04-13 22:05:20,307+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_READ_TIMEOUT' is '0'. 2019-04-13 22:05:20,307+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_SSL_PROTOCOL' is 'TLS'. 2019-04-13 22:05:20,307+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_SSL_VERIFY_CHAIN' is 'true'. 2019-04-13 22:05:20,307+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_SSL_VERIFY_HOST' is 'true'. 2019-04-13 22:05:20,307+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_TOKEN_END_POINT' is ''. 2019-04-13 22:05:20,307+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'EXTERNAL_OIDC_USER_INFO_END_POINT' is ''. 2019-04-13 22:05:20,307+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'HOST_CHECK_FOR_UPDATES_THREAD_POOL_SIZE' is '5'. 2019-04-13 22:05:20,289+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 22:05:20,310+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 22:05:20,310+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_FQDN' is 'ovirt.discovery.intern'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_GROUP' is 'ovirt'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HEAP_MAX' is '1024M'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HEAP_MIN' is '1024M'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HTTPS_ENABLED' is 'false'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HTTPS_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HTTPS_PORT' is 'None'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HTTPS_PROTOCOLS' is 'TLSv1.1,TLSv1.2'. 2019-04-13 22:05:20,311+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HTTP_ENABLED' is 'false'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_HTTP_PORT' is 'None'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_JAVA_MODULEPATH' is '/usr/share/ovirt-engine-wildfly-overlay/modules:/usr/share/ovirt-engine/modules/common:/usr/share/ovirt-engine-extension-aaa-jdbc/modules'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_JBOSS_BLOCKING_TIMEOUT' is '300'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_JMX_INTERFACE' is 'loopback'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_JVM_ARGS' is ' -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath="/var/log/ovirt-engine/dump"'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_LOG' is '/var/log/ovirt-engine'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_LOG_TO_CONSOLE' is 'false'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_MALLOC_ARENA_MAX' is '1'. 2019-04-13 22:05:20,312+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_MANUAL' is '/usr/share/ovirt-engine/manual'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_MAX_USER_SESSIONS' is '-1'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_NON_JTA_DB_MAX_CONNECTIONS' is '10'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_NON_JTA_DB_MIN_CONNECTIONS' is '1'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI' is '/etc/pki/ovirt-engine'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_CA' is '/etc/pki/ovirt-engine/ca.pem'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_ENGINE_CERT' is '/etc/pki/ovirt-engine/certs/engine.cer'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_ENGINE_STORE' is '/etc/pki/ovirt-engine/keys/engine.p12'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_ENGINE_STORE_ALIAS' is '1'. 2019-04-13 22:05:20,313+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_ENGINE_STORE_PASSWORD' is '***'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_ENGINE_STORE_TYPE' is 'PKCS12'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_TRUST_STORE_PASSWORD' is '***'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PROPERTIES' is ' java.awt.headless=true sun.rmi.dgc.client.gcInterval=3600000 sun.rmi.dgc.server.gcInterval=3600000 jsse.enableSNIExtension=false'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PROXY_ENABLED' is 'true'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PROXY_HTTPS_PORT' is '443'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_PROXY_HTTP_PORT' is '80'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_RESTAPI_NEGO' is 'true'. 2019-04-13 22:05:20,314+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SCHEDULED_THREAD_POOL_SIZE' is '100'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SCHEDULER_POOL_CLASS' is 'org.ovirt.engine.core.utils.timer.SchedulerThreadPool'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SCHEDULER_POOL_QUEUE_SIZE' is '350'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SCHEDULER_POOL_REJECT_POLICY' is 'false'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SCHEDULER_POOL_THREAD_SIZE' is '100'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_webadmin' is '~'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_webui' is '~'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_AUTH_SEQUENCE_welcome' is '~'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_AUTH_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/sso'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_CLIENT_ID' is 'ovirt-engine-core'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_CLIENT_SECRET' is '***'. 2019-04-13 22:05:20,315+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_ENABLE_EXTERNAL_SSO' is 'false'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_EXTERNAL_SSO_LOGOUT_URI' is '/ovirt-engine/callback'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_INSTALLED_ON_ENGINE_HOST' is 'true'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_SERVICE_CLIENT_POOL_SIZE' is '10'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_SERVICE_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_SERVICE_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_SERVICE_SSL_PROTOCOL' is 'TLS'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_SERVICE_SSL_VERIFY_CHAIN' is 'true'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_SERVICE_SSL_VERIFY_HOST' is 'false'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_SERVICE_URI' is '/ovirt-engine/sso'. 2019-04-13 22:05:20,316+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_SSO_SERVICE_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/sso'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_STOP_INTERVAL' is '1'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_STOP_TIME' is '10'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_THREAD_POOL_MAX_SIZE' is '500'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_THREAD_POOL_MIN_SIZE' is '50'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_THREAD_POOL_QUEUE_SIZE' is '100'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_THREAD_STACK' is '1M'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_TMP' is '/var/tmp/ovirt-engine'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_UP_MARK' is '/var/lib/ovirt-engine/engine.up'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_URI' is '/ovirt-engine'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_USER' is 'ovirt'. 2019-04-13 22:05:20,317+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_USR' is '/usr/share/ovirt-engine'. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_VAR' is '/var/lib/ovirt-engine'. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'ENGINE_VERBOSE_GC' is 'false'. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_CLIENT_ID' is ''. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_CLIENT_POOL_SIZE' is '10'. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_CLIENT_SECRET' is ''. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_CONNECT_TIMEOUT' is '0'. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE' is '/etc/pki/ovirt-engine/.truststore'. 2019-04-13 22:05:20,318+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE_PASSWORD' is ''. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_HTTPS_PKI_TRUST_STORE_TYPE' is 'JKS'. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_LOGOUT_END_POINT' is ''. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_READ_TIMEOUT' is '0'. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_SSL_PROTOCOL' is 'TLS'. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_SSL_VERIFY_CHAIN' is 'true'. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_SSL_VERIFY_HOST' is 'true'. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_TOKEN_END_POINT' is ''. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'EXTERNAL_OIDC_USER_INFO_END_POINT' is ''. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'HOST_CHECK_FOR_UPDATES_THREAD_POOL_SIZE' is '5'. 2019-04-13 22:05:20,319+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'JBOSS_HOME' is '/usr/share/ovirt-engine-wildfly'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'JBOSS_RUNTIME' is '/var/lib/ovirt-engine/jboss_runtime'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'PACKAGE_DISPLAY_VERSION' is '4.3.2.1-1.el7'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'PACKAGE_NAME' is 'ovirt-engine'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'PACKAGE_VERSION' is '4.3.2.1'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SENSITIVE_KEYS' is ',ENGINE_DB_PASSWORD,DWH_DB_PASSWORD,CINDERLIB_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD,ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD,ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD,ENGINE_SSO_CLIENT_SECRET'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_ALTERNATE_ENGINE_FQDNS' is ''. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_AUTH_LOGIN_SEQUENCE' is 'NI'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'JBOSS_HOME' is '/usr/share/ovirt-engine-wildfly'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'JBOSS_RUNTIME' is '/var/lib/ovirt-engine/jboss_runtime'. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'PACKAGE_DISPLAY_VERSION' is '4.3.2.1-1.el7'. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'PACKAGE_NAME' is 'ovirt-engine'. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'PACKAGE_VERSION' is '4.3.2.1'. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SENSITIVE_KEYS' is ',ENGINE_DB_PASSWORD,DWH_DB_PASSWORD,CINDERLIB_DB_PASSWORD,ENGINE_PKI_TRUST_STORE_PASSWORD,ENGINE_PKI_ENGINE_STORE_PASSWORD,ENGINE_EXTERNAL_PROVIDERS_TRUST_STORE_PASSWORD,ENGINE_HTTPS_PKI_TRUST_STORE_PASSWORD,ENGINE_SSO_CLIENT_SECRET'. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_ALTERNATE_ENGINE_FQDNS' is ''. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_AUTH_LOGIN_SEQUENCE' is 'NI'. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_CALLBACK_CLIENT_POOL_SIZE' is '10'. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_CALLBACK_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 22:05:20,321+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_CALLBACK_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_CALLBACK_CONNECT_TIMEOUT' is '0'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_CALLBACK_PREFIX_CHECK' is 'true'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_CALLBACK_READ_TIMEOUT' is '0'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_ENGINE_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_HOUSE_KEEPING_INTERVAL' is '60'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_TOKEN_HTTP_LOGIN_SEQUENCE' is 'Nb'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'SSO_TOKEN_TIMEOUT' is '360000'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'THREAD_POOL_MONITORING_INTERVAL_IN_SECONDS' is '600'. 2019-04-13 22:05:20,322+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 46) [] Value of property 'sclenv' is 'rh-postgresql10'. 2019-04-13 22:05:20,320+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_CALLBACK_CLIENT_POOL_SIZE' is '10'. 2019-04-13 22:05:20,329+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_CALLBACK_CONNECTION_RETRY_COUNT' is '3'. 2019-04-13 22:05:20,329+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_CALLBACK_CONNECTION_VALIDATE_AFTER_INACTIVITY' is '100'. 2019-04-13 22:05:20,329+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_CALLBACK_CONNECT_TIMEOUT' is '0'. 2019-04-13 22:05:20,329+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_CALLBACK_PREFIX_CHECK' is 'true'. 2019-04-13 22:05:20,329+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_CALLBACK_READ_TIMEOUT' is '0'. 2019-04-13 22:05:20,329+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_ENGINE_URL' is 'https://ovirt.discovery.intern:443/ovirt-engine/'. 2019-04-13 22:05:20,329+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_HOUSE_KEEPING_INTERVAL' is '60'. 2019-04-13 22:05:20,330+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_TOKEN_HTTP_LOGIN_SEQUENCE' is 'Nb'. 2019-04-13 22:05:20,330+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'SSO_TOKEN_TIMEOUT' is '360000'. 2019-04-13 22:05:20,330+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'THREAD_POOL_MONITORING_INTERVAL_IN_SECONDS' is '600'. 2019-04-13 22:05:20,330+02 INFO [org.ovirt.engine.core.uutils.config.ShellLikeConfd] (ServerService Thread Pool -- 47) [] Value of property 'sclenv' is 'rh-postgresql10'. 2019-04-13 22:05:20,438+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Loading extension 'internal-authn' 2019-04-13 22:05:20,457+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Extension 'internal-authn' loaded 2019-04-13 22:05:20,479+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Loading extension 'internal-authz' 2019-04-13 22:05:20,480+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Extension 'internal-authz' loaded 2019-04-13 22:05:20,481+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Initializing extension 'internal-authn' 2019-04-13 22:05:20,670+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Extension 'internal-authn' initialized 2019-04-13 22:05:20,671+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Initializing extension 'internal-authz' 2019-04-13 22:05:20,731+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Extension 'internal-authz' initialized 2019-04-13 22:05:20,731+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Start of enabled extensions list 2019-04-13 22:05:20,732+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Instance name: 'internal-authn', Extension name: '"ovirt-engine-extension-aaa-jdbc".authn', Version: '"1.1.10"', Notes: 'Display name: "ovirt-engine-extension-aaa-jdbc"', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: '/etc/ovirt-engine/extensions.d/internal-authn.properties', Initialized: 'true' 2019-04-13 22:05:20,732+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] Instance name: 'internal-authz', Extension name: '"ovirt-engine-extension-aaa-jdbc".authz', Version: '"1.1.10"', Notes: 'Display name: "ovirt-engine-extension-aaa-jdbc"', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: '/etc/ovirt-engine/extensions.d/internal-authz.properties', Initialized: 'true' 2019-04-13 22:05:20,732+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 46) [] End of enabled extensions list 2019-04-13 22:05:20,815+02 INFO [org.ovirt.engine.ui.frontend.server.dashboard.DashboardDataServlet] (ServerService Thread Pool -- 55) [] Dashboard utilization cache updater initialized (update interval 300s) 2019-04-13 22:05:20,841+02 INFO [org.ovirt.engine.ui.frontend.server.dashboard.DashboardDataServlet] (ServerService Thread Pool -- 55) [] Dashboard inventory cache updater initialized (update interval 60s) 2019-04-13 22:05:20,918+02 INFO [org.ovirt.engine.core.sso.utils.LocalizationUtils] (ServerService Thread Pool -- 46) [] Start initializing LocalizationUtils 2019-04-13 22:05:20,931+02 INFO [org.ovirt.engine.core.sso.utils.LocalizationUtils] (ServerService Thread Pool -- 46) [] Finished initializing LocalizationUtils 2019-04-13 22:05:21,056+02 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 40) [] Start initializing TagsDirector 2019-04-13 22:05:21,080+02 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 40) [] Tag 'root' added to tree 2019-04-13 22:05:21,254+02 INFO [org.ovirt.engine.core.bll.TagsDirector] (ServerService Thread Pool -- 40) [] Finished initializing TagsDirector 2019-04-13 22:05:21,345+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'HotPlugMemoryMultiplicationSizeMb' 2019-04-13 22:05:21,347+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DnsResolverConfigurationSupported' 2019-04-13 22:05:21,378+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DbJustRestored' 2019-04-13 22:05:21,380+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'ConfigDir' 2019-04-13 22:05:21,380+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'AdminDomain' 2019-04-13 22:05:21,380+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'AllowDuplicateMacAddresses' 2019-04-13 22:05:21,380+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DefaultWorkgroup' 2019-04-13 22:05:21,381+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'KeystoneAuthUrl' 2019-04-13 22:05:21,381+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'LicenseCertificateFingerPrint' 2019-04-13 22:05:21,381+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'MacPoolRanges' 2019-04-13 22:05:21,381+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'MaxMacsCountInPool' 2019-04-13 22:05:21,381+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'VdsFenceOptions' 2019-04-13 22:05:21,387+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'HotPlugMemoryMultiplicationSizeMb' 2019-04-13 22:05:21,395+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DnsResolverConfigurationSupported' 2019-04-13 22:05:21,396+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DbJustRestored' 2019-04-13 22:05:21,397+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'ConfigDir' 2019-04-13 22:05:21,397+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'AdminDomain' 2019-04-13 22:05:21,397+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'AllowDuplicateMacAddresses' 2019-04-13 22:05:21,397+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DefaultWorkgroup' 2019-04-13 22:05:21,397+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'KeystoneAuthUrl' 2019-04-13 22:05:21,398+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'LicenseCertificateFingerPrint' 2019-04-13 22:05:21,398+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'MacPoolRanges' 2019-04-13 22:05:21,398+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'MaxMacsCountInPool' 2019-04-13 22:05:21,398+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'VdsFenceOptions' 2019-04-13 22:05:21,402+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'HotPlugMemoryMultiplicationSizeMb' 2019-04-13 22:05:21,444+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DnsResolverConfigurationSupported' 2019-04-13 22:05:21,450+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DbJustRestored' 2019-04-13 22:05:21,456+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'ConfigDir' 2019-04-13 22:05:21,456+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'AdminDomain' 2019-04-13 22:05:21,456+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'AllowDuplicateMacAddresses' 2019-04-13 22:05:21,456+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'DefaultWorkgroup' 2019-04-13 22:05:21,456+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'KeystoneAuthUrl' 2019-04-13 22:05:21,457+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'LicenseCertificateFingerPrint' 2019-04-13 22:05:21,457+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'MacPoolRanges' 2019-04-13 22:05:21,457+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'MaxMacsCountInPool' 2019-04-13 22:05:21,457+02 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (ServerService Thread Pool -- 40) [] Could not find enum value for option: 'VdsFenceOptions' 2019-04-13 22:05:21,521+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 40) [] Start initializing Backend 2019-04-13 22:05:21,522+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.common.utils.EngineThreadPools@275c75e0 2019-04-13 22:05:21,523+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (ServerService Thread Pool -- 40) [] Initializing Thread Monitoring Service 2019-04-13 22:05:21,526+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (ServerService Thread Pool -- 40) [] Thread Monitoring Service initialized 2019-04-13 22:05:21,526+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService@78df7ecc 2019-04-13 22:05:21,550+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'default' is using 2 threads out of 1, 0 threads waiting for tasks. 2019-04-13 22:05:21,558+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engineScheduled' is using 0 threads out of 100, 1 threads waiting for tasks. 2019-04-13 22:05:21,558+02 INFO [org.ovirt.engine.core.bll.utils.ThreadPoolMonitoringService] (EE-ManagedThreadFactory-engineThreadMonitoring-Thread-1) [] Thread pool 'engineThreadMonitoring' is using 1 threads out of 1, 0 threads waiting for tasks. 2019-04-13 22:05:21,879+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.hostedengine.PreviousHostedEngineHost@616d62b5 2019-04-13 22:05:21,996+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 40) [] Started task scheduler org.ovirt.engine.core.utils.timer.SchedulerUtilQuartzImpl@5a30398a 2019-04-13 22:05:22,137+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 40) [] Started task scheduler org.ovirt.engine.core.utils.timer.DBSchedulerUtilQuartzImpl@47f36f83 2019-04-13 22:05:22,152+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.dal.utils.CacheManager@710b46f8 2019-04-13 22:05:22,155+02 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 40) [] Loading file '/etc/ovirt-engine/osinfo.conf.d/00-defaults.properties' 2019-04-13 22:05:22,211+02 INFO [org.ovirt.engine.core.utils.osinfo.OsInfoPreferencesLoader] (ServerService Thread Pool -- 40) [] Loading file '/etc/ovirt-engine/osinfo.conf.d/10-ad.properties' 2019-04-13 22:05:22,423+02 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) [] Start initializing ResourceManager$Proxy$_$$_WeldSubclass 2019-04-13 22:05:22,686+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) [] Entered VdsManager constructor 2019-04-13 22:05:22,767+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) [] Initialize vdsBroker 'onode-1.discovery.intern:54321' 2019-04-13 22:05:22,866+02 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) [] VDS '65c45da8-2a46-42e0-9c3a-58748546103b' was added to the Resource Manager 2019-04-13 22:05:22,866+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) [] Entered VdsManager constructor 2019-04-13 22:05:22,886+02 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (ServerService Thread Pool -- 40) [] Initialize vdsBroker 'onode-2.discovery.intern:54321' 2019-04-13 22:05:22,980+02 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) [] VDS '7e1fc8ae-a99f-4828-84fa-a111f8d9ba89' was added to the Resource Manager 2019-04-13 22:05:22,980+02 INFO [org.ovirt.engine.core.vdsbroker.ResourceManager] (ServerService Thread Pool -- 40) [] Finished initializing ResourceManager$Proxy$_$$_WeldSubclass 2019-04-13 22:05:22,995+02 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 40) [] Start initializing dictionaries 2019-04-13 22:05:22,999+02 INFO [org.ovirt.engine.core.bll.CpuFlagsManagerHandler] (ServerService Thread Pool -- 40) [] Finished initializing dictionaries 2019-04-13 22:05:23,274+02 INFO [org.ovirt.engine.core.bll.network.macpool.MacPoolUsingRanges] (ServerService Thread Pool -- 40) [] Initializing MacPoolUsingRanges:{id='59ef3a2b-01f3-02e9-0075-00000000035f'} 2019-04-13 22:05:23,276+02 INFO [org.ovirt.engine.core.bll.network.macpool.MacPoolUsingRanges] (ServerService Thread Pool -- 40) [] Finished initializing MacPoolUsingRanges:{id='59ef3a2b-01f3-02e9-0075-00000000035f'}. Available MACs in pool: 130 2019-04-13 22:05:23,280+02 INFO [org.ovirt.engine.core.bll.network.macpool.MacPoolPerCluster] (ServerService Thread Pool -- 40) [] Successfully initialized 2019-04-13 22:05:23,311+02 INFO [org.ovirt.engine.core.bll.network.macpool.MacPoolPerCluster] (ServerService Thread Pool -- 40) [] Mac pool 59ef3a2b-01f3-02e9-0075-00000000035f has 130 available free macs 2019-04-13 22:05:23,333+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 40) [] Running ovirt-engine 4.3.2.1-1.el7 2019-04-13 22:05:23,333+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.CpuFlagsManagerHandler@1220edd7 2019-04-13 22:05:23,334+02 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 40) [] Start initializing AuditLogCleanupManager 2019-04-13 22:05:23,335+02 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 40) [] Setting audit cleanup manager to run at '35 35 3 * * ?' 2019-04-13 22:05:23,342+02 INFO [org.ovirt.engine.core.bll.AuditLogCleanupManager] (ServerService Thread Pool -- 40) [] Finished initializing AuditLogCleanupManager 2019-04-13 22:05:23,342+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.AuditLogCleanupManager@2aaa788b 2019-04-13 22:05:23,342+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.TagsDirector@16e6a2b8 2019-04-13 22:05:23,343+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.storage.domain.IsoDomainListSynchronizer@16ed3fa 2019-04-13 22:05:23,353+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.VmHandler@6768b7d6 2019-04-13 22:05:23,353+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.VdsHandler@13d7783 2019-04-13 22:05:23,354+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.VmTemplateHandler@68bd5c78 2019-04-13 22:05:23,354+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 40) [] Completed initializing handlers 2019-04-13 22:05:23,355+02 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 40) [] Start initializing ErrorTranslatorImpl 2019-04-13 22:05:23,377+02 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 40) [] Finished initializing ErrorTranslatorImpl 2019-04-13 22:05:23,377+02 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 40) [] Start initializing ErrorTranslatorImpl 2019-04-13 22:05:23,378+02 INFO [org.ovirt.engine.core.utils.ErrorTranslatorImpl] (ServerService Thread Pool -- 40) [] Finished initializing ErrorTranslatorImpl 2019-04-13 22:05:23,378+02 INFO [org.ovirt.engine.core.bll.Backend] (ServerService Thread Pool -- 40) [] Mark incomplete jobs as UNKNOWN 2019-04-13 22:05:23,450+02 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 40) [] Start initializing JobRepositoryCleanupManager 2019-04-13 22:05:23,451+02 INFO [org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager] (ServerService Thread Pool -- 40) [] Finished initializing JobRepositoryCleanupManager 2019-04-13 22:05:23,451+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.job.JobRepositoryCleanupManager@44928a65 2019-04-13 22:05:23,452+02 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 40) [] Start initializing AutoRecoveryManager 2019-04-13 22:05:23,595+02 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (ServerService Thread Pool -- 40) [] Finished initializing AutoRecoveryManager 2019-04-13 22:05:23,596+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.AutoRecoveryManager@4cb5bb17 2019-04-13 22:05:23,616+02 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 40) [] Start initializing ExecutionMessageDirector 2019-04-13 22:05:23,618+02 INFO [org.ovirt.engine.core.dal.job.ExecutionMessageDirector] (ServerService Thread Pool -- 40) [] Finished initializing ExecutionMessageDirector 2019-04-13 22:05:23,621+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.VmPoolHandler@44b884d8 2019-04-13 22:05:23,623+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.VmPoolMonitor@91b13e4 2019-04-13 22:05:23,646+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.HaAutoStartVmsRunner@235c5337 2019-04-13 22:05:23,648+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.quota.QuotaManager@3de24e6b 2019-04-13 22:05:23,654+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.vdsbroker.monitoring.VmMigrationProgressMonitoring@18341992 2019-04-13 22:05:23,983+02 WARN [org.ovirt.engine.core.bll.IconLoader] (ServerService Thread Pool -- 40) [] java.lang.RuntimeException: Icon for rhel_6_9_plus_ppc64 was not found in /usr/share/ovirt-engine/icons/small 2019-04-13 22:05:23,983+02 WARN [org.ovirt.engine.core.bll.IconLoader] (ServerService Thread Pool -- 40) [] java.lang.RuntimeException: Icon for rhel_6_9_plus_ppc64 was not found in /usr/share/ovirt-engine/icons/large 2019-04-13 22:05:24,107+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.IconLoader@51f4767 2019-04-13 22:05:24,118+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Loading extension 'internal-authn' 2019-04-13 22:05:24,118+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Extension 'internal-authn' loaded 2019-04-13 22:05:24,119+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Loading extension 'internal-authz' 2019-04-13 22:05:24,119+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Extension 'internal-authz' loaded 2019-04-13 22:05:24,120+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Initializing extension 'internal-authn' 2019-04-13 22:05:24,147+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Extension 'internal-authn' initialized 2019-04-13 22:05:24,148+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Start of enabled extensions list 2019-04-13 22:05:24,148+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Instance name: 'internal-authn', Extension name: '"ovirt-engine-extension-aaa-jdbc".authn', Version: '"1.1.10"', Notes: 'Display name: "ovirt-engine-extension-aaa-jdbc"', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: '/etc/ovirt-engine/extensions.d/internal-authn.properties', Initialized: 'true' 2019-04-13 22:05:24,148+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] Instance name: 'internal-authz', Extension name: '"ovirt-engine-extension-aaa-jdbc".authz', Version: '"1.1.10"', Notes: 'Display name: "ovirt-engine-extension-aaa-jdbc"', License: 'ASL 2.0', Home: 'http://www.ovirt.org', Author 'The oVirt Project', Build interface Version: '0', File: '/etc/ovirt-engine/extensions.d/internal-authz.properties', Initialized: 'false' 2019-04-13 22:05:24,148+02 INFO [org.ovirt.engine.core.extensions.mgr.ExtensionsManager] (ServerService Thread Pool -- 40) [] End of enabled extensions list 2019-04-13 22:05:24,205+02 INFO [org.ovirt.engine.core.bll.pm.PmHealthCheckManager] (ServerService Thread Pool -- 40) [] Finished initializing PmHealthCheckManager 2019-04-13 22:05:24,205+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.pm.PmHealthCheckManager@219d4f93 2019-04-13 22:05:24,206+02 INFO [org.ovirt.engine.core.bll.EngineBackupAwarenessManager] (ServerService Thread Pool -- 40) [] Start initializing EngineBackupAwarenessManager 2019-04-13 22:05:24,208+02 INFO [org.ovirt.engine.core.bll.EngineBackupAwarenessManager] (ServerService Thread Pool -- 40) [] Finished initializing EngineBackupAwarenessManager 2019-04-13 22:05:24,208+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.EngineBackupAwarenessManager@40ef3c98 2019-04-13 22:05:24,208+02 INFO [org.ovirt.engine.core.bll.DataCenterCompatibilityChecker] (ServerService Thread Pool -- 40) [] Start initializing DataCenterCompatibilityChecker 2019-04-13 22:05:24,224+02 INFO [org.ovirt.engine.core.bll.DataCenterCompatibilityChecker] (ServerService Thread Pool -- 40) [] Finished initializing DataCenterCompatibilityChecker 2019-04-13 22:05:24,224+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.DataCenterCompatibilityChecker@6be0e2f3 2019-04-13 22:05:24,224+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass@525218eb 2019-04-13 22:05:24,252+02 INFO [org.ovirt.engine.core.bll.PrevalidatingMultipleActionsRunner] (ServerService Thread Pool -- 40) [] PrevalidatingMultipleActionsRunner of type 'RefreshHostDevices' invoked with no actions 2019-04-13 22:05:24,288+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.hostdev.HostDeviceManager@3da1f53a 2019-04-13 22:05:24,289+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyManager] (ServerService Thread Pool -- 40) [] Start initializing IrsProxyManager 2019-04-13 22:05:24,308+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyManager] (ServerService Thread Pool -- 40) [] Start initializing IrsProxyManager 2019-04-13 22:05:24,308+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyManager@33999efd 2019-04-13 22:05:24,309+02 INFO [org.ovirt.engine.core.bll.storage.ovfstore.OvfDataUpdater] (ServerService Thread Pool -- 40) [] Initialization of OvfDataUpdater completed successfully. 2019-04-13 22:05:24,309+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.storage.ovfstore.OvfDataUpdater@41229061 2019-04-13 22:05:24,312+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.storage.pool.StoragePoolStatusHandler@6a350ebc 2019-04-13 22:05:24,429+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.gluster.GlusterJobsManager@6b3eff6 2019-04-13 22:05:24,444+02 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 40) [] Init VM custom properties utilities 2019-04-13 22:05:24,444+02 INFO [org.ovirt.engine.core.bll.InitBackendServicesOnStartupBean] (ServerService Thread Pool -- 40) [] Init device custom properties utilities 2019-04-13 22:05:24,479+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) [] Initializing Scheduling manager 2019-04-13 22:05:24,607+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) [] External scheduler disabled, discovery skipped 2019-04-13 22:05:24,608+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) [] Start scheduling to enable vds load balancer 2019-04-13 22:05:24,608+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) [] Finished scheduling to enable vds load balancer 2019-04-13 22:05:24,609+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) [] Start HA Reservation check 2019-04-13 22:05:24,609+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) [] Finished HA Reservation check 2019-04-13 22:05:24,609+02 INFO [org.ovirt.engine.core.bll.scheduling.SchedulingManager] (ServerService Thread Pool -- 40) [] Initialized Scheduling manager 2019-04-13 22:05:24,610+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.scheduling.SchedulingManager@36722e4d 2019-04-13 22:05:24,621+02 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 40) [] Initializing DWH Heart Beat 2019-04-13 22:05:24,622+02 INFO [org.ovirt.engine.core.bll.dwh.DwhHeartBeat] (ServerService Thread Pool -- 40) [] DWH Heart Beat initialized 2019-04-13 22:05:24,622+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.dwh.DwhHeartBeat@1c356d1d 2019-04-13 22:05:24,683+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (ServerService Thread Pool -- 40) [] Initialization of AsyncTaskManager completed successfully. 2019-04-13 22:05:24,683+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.tasks.AsyncTaskManager@182a5c29 2019-04-13 22:05:24,684+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.tasks.CommandCoordinatorUtil@566c7657 2019-04-13 22:05:24,685+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (ServerService Thread Pool -- 40) [] Start initializing CommandCallbacksPoller 2019-04-13 22:05:24,690+02 INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (ServerService Thread Pool -- 40) [] Finished initializing CommandCallbacksPoller 2019-04-13 22:05:24,690+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller@53b6987d 2019-04-13 22:05:24,691+02 INFO [org.ovirt.engine.core.bll.CommandEntityCleanupManager] (ServerService Thread Pool -- 40) [] Start initializing CommandEntityCleanupManager 2019-04-13 22:05:24,692+02 INFO [org.ovirt.engine.core.bll.CommandEntityCleanupManager] (ServerService Thread Pool -- 40) [] Setting command entity cleanup manager to run at: 35 35 3 * * ? 2019-04-13 22:05:24,700+02 INFO [org.ovirt.engine.core.bll.CommandEntityCleanupManager] (ServerService Thread Pool -- 40) [] Finished initializing CommandEntityCleanupManager 2019-04-13 22:05:24,701+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.CommandEntityCleanupManager@5e0fb132 2019-04-13 22:05:24,755+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ServerService Thread Pool -- 40) [] EVENT_ID: AFFINITY_RULES_ENFORCEMENT_MANAGER_START(10,780), Affinity Rules Enforcement Manager started. 2019-04-13 22:05:24,758+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.scheduling.AffinityRulesEnforcementManager@5342c2a8 2019-04-13 22:05:24,761+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.CertificationValidityChecker@21fc0475 2019-04-13 22:05:24,765+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.hostdeploy.HostUpdatesCheckerService@2352c18d 2019-04-13 22:05:24,766+02 INFO [org.ovirt.engine.core.bll.IPTablesDeprecationNotifier] (ServerService Thread Pool -- 40) [] Start initializing IPTablesDeprecationNotifier 2019-04-13 22:05:24,769+02 INFO [org.ovirt.engine.core.bll.IPTablesDeprecationNotifier] (ServerService Thread Pool -- 40) [] Finished initializing IPTablesDeprecationNotifier 2019-04-13 22:05:24,769+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.IPTablesDeprecationNotifier@7ff62366 2019-04-13 22:05:24,775+02 INFO [org.ovirt.engine.core.bll.ServiceLoader] (ServerService Thread Pool -- 40) [] Start org.ovirt.engine.core.bll.network.cluster.ExternalNetworkSyncService@2ea46526 2019-04-13 22:07:07,659+02 INFO [org.ovirt.engine.extension.aaa.jdbc.core.Tasks] (default task-6) [] (house keeping) deleting failed logins prior to 2019-04-06 20:07:07Z. 2019-04-13 22:07:07,872+02 INFO [org.ovirt.engine.extension.aaa.jdbc.core.Tasks] (default task-6) [] (house keeping) deleting failed logins prior to 2019-04-06 20:07:07Z. 2019-04-13 22:07:07,953+02 INFO [org.ovirt.engine.core.sso.utils.AuthenticationUtils] (default task-6) [] User admin@internal successfully logged in with scopes: ovirt-app-admin ovirt-app-api ovirt-app-portal ovirt-ext=auth:sequence-priority=~ ovirt-ext=revoke:revoke-all ovirt-ext=token-info:authz-search ovirt-ext=token-info:public-authz-search ovirt-ext=token-info:validate ovirt-ext=token:password-access 2019-04-13 22:07:08,641+02 INFO [org.ovirt.engine.core.bll.aaa.CreateUserSessionCommand] (default task-6) [78efd0b3] Running command: CreateUserSessionCommand internal: false. 2019-04-13 22:07:08,736+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-6) [78efd0b3] EVENT_ID: USER_VDC_LOGIN(30), User admin@internal-authz connecting from '192.168.0.1' using session 'UXVIZMkH6NFMJMlaQlpMMsv04LeLyeKTL1KxWR7QJLwvNLzmsvZDXRUa6n3pqagwhb4JgUnK6pqh3O9Z/iQowA==' logged in. 2019-04-13 22:07:08,904+02 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (default task-6) [] Reading UI plugin descriptor '/usr/share/ovirt-engine/ui-plugins/ui-extensions.json' 2019-04-13 22:07:08,910+02 INFO [org.ovirt.engine.ui.frontend.server.gwt.plugin.PluginDataManager] (default task-6) [] Reading UI plugin configuration '/etc/ovirt-engine/ui-plugins/ui-extensions-config.json' 2019-04-13 22:07:10,480+02 INFO [org.ovirt.engine.core.utils.servlet.ServletUtils] (default task-5) [] Can't read file '/usr/share/ovirt-engine/files/spice/SpiceVersion.txt' for request '/ovirt-engine/services/files/spice/SpiceVersion.txt' -- 404 2019-04-13 22:07:19,838+02 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (default task-6) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] Lock Acquired to object 'EngineLock:{exclusiveLocks='[65c45da8-2a46-42e0-9c3a-58748546103b=VDS]', sharedLocks=''}' 2019-04-13 22:07:20,019+02 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (EE-ManagedThreadFactory-engine-Thread-2) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] Running command: ActivateVdsCommand internal: false. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 22:07:20,021+02 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (EE-ManagedThreadFactory-engine-Thread-2) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] Before acquiring lock in order to prevent monitoring for host 'onode-1' from data-center 'Default' 2019-04-13 22:07:20,021+02 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (EE-ManagedThreadFactory-engine-Thread-2) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] Lock acquired, from now a monitoring of host will be skipped for host 'onode-1' from data-center 'Default' 2019-04-13 22:07:20,041+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] START, SetVdsStatusVDSCommand(HostName = onode-1, SetVdsStatusVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', status='Unassigned', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 3926c5e8 2019-04-13 22:07:20,068+02 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-2) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] FINISH, SetVdsStatusVDSCommand, return: , log id: 3926c5e8 2019-04-13 22:07:20,085+02 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (EE-ManagedThreadFactory-engine-Thread-2) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] Activate host finished. Lock released. Monitoring can run now for host 'onode-1' from data-center 'Default' 2019-04-13 22:07:20,115+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-2) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] EVENT_ID: VDS_ACTIVATE(16), Activation of host onode-1 initiated by admin@internal-authz. 2019-04-13 22:07:20,146+02 INFO [org.ovirt.engine.core.bll.ActivateVdsCommand] (EE-ManagedThreadFactory-engine-Thread-2) [9fe1a95d-5ba3-4d53-a556-8b3b23d2c2cd] Lock freed to object 'EngineLock:{exclusiveLocks='[65c45da8-2a46-42e0-9c3a-58748546103b=VDS]', sharedLocks=''}' 2019-04-13 22:07:22,973+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 22:07:48,738+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [] START, GetHardwareInfoAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'}), log id: 33c9ab81 2019-04-13 22:07:48,741+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 33c9ab81 2019-04-13 22:07:48,939+02 INFO [org.ovirt.engine.core.bll.HandleVdsCpuFlagsOrClusterChangedCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-58) [dbfb802] Running command: HandleVdsCpuFlagsOrClusterChangedCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 22:07:52,495+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] START, GetHardwareInfoAsyncVDSCommand(HostName = onode-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vds='Host[onode-1,65c45da8-2a46-42e0-9c3a-58748546103b]'}), log id: 7e0e088a 2019-04-13 22:07:52,497+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetHardwareInfoAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [] FINISH, GetHardwareInfoAsyncVDSCommand, return: , log id: 7e0e088a 2019-04-13 22:07:52,684+02 INFO [org.ovirt.engine.core.bll.InitVdsOnUpCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [7f4feb06] Running command: InitVdsOnUpCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 22:07:52,779+02 INFO [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 59ef3a18-002f-02d1-0220-000000000124 Type: StoragePool 2019-04-13 22:07:52,885+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] START, ConnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='2286311a-c1c9-4dda-bea5-9c78651c6e36', connection='onode-1.discovery.intern:/mnt/store', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 119fe567 2019-04-13 22:07:53,326+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] FINISH, ConnectStorageServerVDSCommand, return: {2286311a-c1c9-4dda-bea5-9c78651c6e36=0}, log id: 119fe567 2019-04-13 22:07:53,332+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] START, ConnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='b4f4bcf2-e104-468e-b330-ed731ec8ef6c', connection='nas.discovery.intern:/mnt/d2600/vms', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='true'}), log id: 3a15f7be 2019-04-13 22:07:53,534+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] FINISH, ConnectStorageServerVDSCommand, return: {b4f4bcf2-e104-468e-b330-ed731ec8ef6c=0}, log id: 3a15f7be 2019-04-13 22:07:53,538+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] START, ConnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='07dd9ff2-afa1-4a33-b401-935ea5b8c9db', connection='nas.discovery.intern:/mnt/d2600/isos', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='false'}), log id: 1076e11c 2019-04-13 22:07:53,675+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] FINISH, ConnectStorageServerVDSCommand, return: {07dd9ff2-afa1-4a33-b401-935ea5b8c9db=0}, log id: 1076e11c 2019-04-13 22:07:53,680+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] START, ConnectStorageServerVDSCommand(HostName = onode-1, StorageServerConnectionManagementVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', storageType='NFS', connectionList='[StorageServerConnections:{id='858731bf-2fb1-45a7-a693-235f675a667b', connection='nas.discovery.intern:/mnt/d2600/sicherung/vmBackups', iqn='null', vfsType='null', mountOptions='null', nfsVersion='AUTO', nfsRetrans='null', nfsTimeo='null', iface='null', netIfaceName='null'}]', sendNetworkEventOnFailure='false'}), log id: 4a69bd76 2019-04-13 22:07:53,830+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] FINISH, ConnectStorageServerVDSCommand, return: {858731bf-2fb1-45a7-a693-235f675a667b=0}, log id: 4a69bd76 2019-04-13 22:07:53,840+02 INFO [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] Host 'onode-1' storage connection was succeeded 2019-04-13 22:07:53,889+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-4) [42fd5658] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 6b1fc26d 2019-04-13 22:07:53,892+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-4) [42fd5658] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 22:07:54,469+02 INFO [org.ovirt.engine.core.vdsbroker.monitoring.VmsStatisticsFetcher] (EE-ManagedThreadFactory-engineScheduled-Thread-10) [] Fetched 0 VMs from VDS '65c45da8-2a46-42e0-9c3a-58748546103b' 2019-04-13 22:07:54,502+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engine-Thread-4) [42fd5658] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 6b1fc26d 2019-04-13 22:07:54,592+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-4) [42fd5658] EVENT_ID: NO_FAULTY_MULTIPATHS_ON_HOST(11,501), No faulty multipath paths on host onode-1 2019-04-13 22:07:54,818+02 ERROR [org.ovirt.engine.core.bll.pm.FenceProxyLocator] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] Can not run fence action on host 'onode-1', no suitable proxy host was found. 2019-04-13 22:07:54,828+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] START, SetMOMPolicyParametersVDSCommand(HostName = onode-1, MomPolicyVDSParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'}), log id: 59b27b36 2019-04-13 22:07:54,895+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [42fd5658] FINISH, SetMOMPolicyParametersVDSCommand, return: , log id: 59b27b36 2019-04-13 22:07:54,954+02 INFO [org.ovirt.engine.core.bll.hostdev.RefreshHostDevicesCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [4a97a86d] Running command: RefreshHostDevicesCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDSAction group MANIPULATE_HOST with role type ADMIN 2019-04-13 22:07:55,374+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [4a97a86d] EVENT_ID: VDS_DETECTED(13), Status of host onode-1 was set to Up. 2019-04-13 22:07:55,395+02 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [4a97a86d] EVENT_ID: VDS_ALERT_FENCE_TEST_FAILED(9,001), Power Management test failed for Host onode-1.There is no other host in the data center that can be used to test the power management settings. 2019-04-13 22:07:55,402+02 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [4a97a86d] EVENT_ID: VDS_FENCE_STATUS_FAILED(497), Failed to verify Host onode-1 power management. 2019-04-13 22:07:55,607+02 INFO [org.ovirt.engine.core.bll.HandleVdsVersionCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-34) [62b07458] Running command: HandleVdsVersionCommand internal: true. Entities affected : ID: 65c45da8-2a46-42e0-9c3a-58748546103b Type: VDS 2019-04-13 22:07:55,853+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] START, ConnectStoragePoolVDSCommand(HostName = onode-1, ConnectStoragePoolVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vdsId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', masterVersion='47'}), log id: 15f67b52 2019-04-13 22:07:55,856+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Executing with domain map: {6a4cde72-6a6a-499e-9b6e-d490d91ca19e=active, 93f02ede-ec5c-4641-8e97-2949f98e856d=active, 42bb6772-12f9-4bf2-9245-43caa41c5a2a=active, 9c3f06cf-7475-448e-819b-f4f52fa7d782=active} 2019-04-13 22:07:56,296+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] FINISH, ConnectStoragePoolVDSCommand, return: , log id: 15f67b52 2019-04-13 22:07:56,359+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] hostFromVds::selectedVds - 'onode-1', spmStatus 'Free', storage pool 'Default', storage pool version '4.3' 2019-04-13 22:07:56,373+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] starting spm on vds 'onode-1', storage pool 'Default', prevId '-1', LVER '-1' 2019-04-13 22:07:56,378+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] START, SpmStartVDSCommand(HostName = onode-1, SpmStartVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', storagePoolId='59ef3a18-002f-02d1-0220-000000000124', prevId='-1', prevLVER='-1', storagePoolFormatType='V4', recoveryMode='Manual', SCSIFencing='false'}), log id: 129e7de5 2019-04-13 22:07:56,386+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] spmStart polling started: taskId '25b68fdb-2dd8-4aed-89a5-0eda39cd92c9' 2019-04-13 22:08:13,995+02 INFO [org.ovirt.engine.core.bll.ClearAllAuditLogAlertsCommand] (default task-6) [9f6cb43a-982c-44f3-a0e9-3865ab20b9ab] Running command: ClearAllAuditLogAlertsCommand internal: false. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group AUDIT_LOG_MANAGEMENT with role type ADMIN 2019-04-13 22:08:14,033+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-6) [9f6cb43a-982c-44f3-a0e9-3865ab20b9ab] EVENT_ID: USER_CLEAR_ALL_AUDIT_LOG_ALERTS(10,606), All alerts were removed. (User: admin@internal-authz) 2019-04-13 22:08:15,551+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] spmStart polling ended: taskId '25b68fdb-2dd8-4aed-89a5-0eda39cd92c9' task status 'finished' 2019-04-13 22:08:15,563+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] spmStart polling ended, spm status: SPM 2019-04-13 22:08:15,568+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] START, HSMClearTaskVDSCommand(HostName = onode-1, HSMTaskGuidBaseVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', taskId='25b68fdb-2dd8-4aed-89a5-0eda39cd92c9'}), log id: fa247ed 2019-04-13 22:08:15,574+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] FINISH, HSMClearTaskVDSCommand, return: , log id: fa247ed 2019-04-13 22:08:15,574+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] FINISH, SpmStartVDSCommand, return: SpmStatusResult:{SPM Id='1', SPM LVER='29', SPM Status='SPM'}, log id: 129e7de5 2019-04-13 22:08:15,578+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] Initialize Irs proxy from vds: onode-1.discovery.intern 2019-04-13 22:08:15,588+02 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-14) [] EVENT_ID: IRS_HOSTED_ON_VDS(204), Storage Pool Manager runs on Host onode-1.discovery.intern (Address: onode-1.discovery.intern), Data Center Default. 2019-04-13 22:08:15,598+02 INFO [org.ovirt.vdsm.jsonrpc.client.reactors.ReactorClient] (SSL Stomp Reactor) [] Connecting to onode-1.discovery.intern/192.168.0.35 2019-04-13 22:08:15,673+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-7) [] START, SPMGetAllTasksInfoVDSCommand( IrsBaseVDSCommandParameters:{storagePoolId='59ef3a18-002f-02d1-0220-000000000124', ignoreFailoverLimit='false'}), log id: 48756a62 2019-04-13 22:08:15,819+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-7) [] -- executeIrsBrokerCommand: Attempting on storage pool '59ef3a18-002f-02d1-0220-000000000124' 2019-04-13 22:08:15,824+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-7) [] START, HSMGetAllTasksInfoVDSCommand(HostName = onode-1, VdsIdVDSCommandParametersBase:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b'}), log id: 4474161e 2019-04-13 22:08:15,834+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-7) [] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 4474161e 2019-04-13 22:08:15,834+02 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (EE-ManagedThreadFactory-engine-Thread-7) [] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 48756a62 2019-04-13 22:08:15,835+02 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engine-Thread-7) [] Discovered no tasks on Storage Pool 'Default' 2019-04-13 22:08:50,264+02 INFO [org.ovirt.engine.core.bll.RunVmCommand] (default task-5) [6180f9cb-8bd6-4267-923b-6adef080adb4] Lock Acquired to object 'EngineLock:{exclusiveLocks='[cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6=VM]', sharedLocks=''}' 2019-04-13 22:08:50,395+02 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-5) [6180f9cb-8bd6-4267-923b-6adef080adb4] START, IsVmDuringInitiatingVDSCommand( IsVmDuringInitiatingVDSCommandParameters:{vmId='cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6'}), log id: 24481a56 2019-04-13 22:08:50,395+02 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (default task-5) [6180f9cb-8bd6-4267-923b-6adef080adb4] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 24481a56 2019-04-13 22:08:50,605+02 INFO [org.ovirt.engine.core.bll.RunVmCommand] (EE-ManagedThreadFactory-engine-Thread-12) [6180f9cb-8bd6-4267-923b-6adef080adb4] Running command: RunVmCommand internal: false. Entities affected : ID: cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6 Type: VMAction group RUN_VM with role type USER 2019-04-13 22:08:50,741+02 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (EE-ManagedThreadFactory-engine-Thread-12) [6180f9cb-8bd6-4267-923b-6adef080adb4] START, UpdateVmDynamicDataVDSCommand( UpdateVmDynamicDataVDSCommandParameters:{hostId='null', vmId='cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6', vmDynamic='org.ovirt.engine.core.common.businessentities.VmDynamic@db25f4d2'}), log id: 2a9ffc19 2019-04-13 22:08:50,766+02 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (EE-ManagedThreadFactory-engine-Thread-12) [6180f9cb-8bd6-4267-923b-6adef080adb4] FINISH, UpdateVmDynamicDataVDSCommand, return: , log id: 2a9ffc19 2019-04-13 22:08:50,818+02 INFO [org.ovirt.engine.core.vdsbroker.CreateVDSCommand] (EE-ManagedThreadFactory-engine-Thread-12) [6180f9cb-8bd6-4267-923b-6adef080adb4] START, CreateVDSCommand( CreateVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6', vm='VM [gogs]'}), log id: 580a5a62 2019-04-13 22:08:50,823+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-12) [6180f9cb-8bd6-4267-923b-6adef080adb4] START, CreateBrokerVDSCommand(HostName = onode-1, CreateVDSCommandParameters:{hostId='65c45da8-2a46-42e0-9c3a-58748546103b', vmId='cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6', vm='VM [gogs]'}), log id: 4c9887bf 2019-04-13 22:08:51,061+02 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateBrokerVDSCommand] (EE-ManagedThreadFactory-engine-Thread-12) [6180f9cb-8bd6-4267-923b-6adef080adb4] VM gogs cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6 2097152 2097152 8388608 16 oVirt OS-NAME: OS-VERSION: HOST-SERIAL: cb6ca3d9-ff5b-4c5d-91f9-08a980af1de6 Westmere