Parts of engine.log follow. - Part 1 repeats every 5 minutes, all the time - Part 2 is when I try to detach storage domain from the GUI (mixed with some logs as in 1) plus logs related to a VM start initiated by one of my colleagues). You will notice that engine reports some VM as "not responding". It happens during the process of removing the defective storage domain, but in fact VMs are fine and their state reverts back to normal when the task of removing the storage domain finally fails. Thank you for your help. ===== part 1 ===== 2015-01-06 03:50:00,016 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-81) [5ca5ae35] Autorecovering 1 storage domains 2015-01-06 03:50:00,017 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-81) [5ca5ae35] Autorecovering storage domains id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 2015-01-06 03:50:00,020 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-81) [789ace0] Running command: ConnectDomainToStorageCommand internal: true. Entities affected : ID: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 Type: Storage 2015-01-06 03:50:00,022 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-81) [789ace0] ConnectDomainToStorage. Before Connect all hosts to pool. Time:1/6/15 3:50 AM 2015-01-06 03:50:00,208 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-12) [789ace0] START, ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 65ce62da 2015-01-06 03:50:00,215 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-37) [789ace0] START, ConnectStorageServerVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 50bc0a6d 2015-01-06 03:50:00,241 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-29) [789ace0] START, ConnectStorageServerVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: e40ca91 2015-01-06 03:50:00,246 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-32) [789ace0] START, ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 5ddb16e 2015-01-06 03:53:00,216 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-12) [789ace0] Command ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 03:53:00,222 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-12) [789ace0] FINISH, ConnectStorageServerVDSCommand, log id: 65ce62da 2015-01-06 03:53:00,223 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-37) [789ace0] Command ConnectStorageServerVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 03:53:00,224 ERROR [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (org.ovirt.thread.pool-8-thread-12) [789ace0] Failed to connect host n2orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 03:53:00,230 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-37) [789ace0] FINISH, ConnectStorageServerVDSCommand, log id: 50bc0a6d 2015-01-06 03:53:00,245 ERROR [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (org.ovirt.thread.pool-8-thread-37) [789ace0] Failed to connect host n4orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 03:53:00,249 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-29) [789ace0] Command ConnectStorageServerVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 03:53:00,264 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-29) [789ace0] FINISH, ConnectStorageServerVDSCommand, log id: e40ca91 2015-01-06 03:53:00,265 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-32) [789ace0] Command ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 03:53:00,266 ERROR [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (org.ovirt.thread.pool-8-thread-29) [789ace0] Failed to connect host n3orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 03:53:00,272 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-32) [789ace0] FINISH, ConnectStorageServerVDSCommand, log id: 5ddb16e 2015-01-06 03:53:00,287 ERROR [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (org.ovirt.thread.pool-8-thread-32) [789ace0] Failed to connect host n1orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 03:53:00,301 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-81) [789ace0] ConnectDomainToStorage. After Connect all hosts to pool. Time:1/6/15 3:53 AM ===== part 2 ===== 2015-01-06 09:55:00,009 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-9) [71cf84f9] Autorecovering 1 storage domains 2015-01-06 09:55:00,011 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-9) [71cf84f9] Autorecovering storage domains id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 2015-01-06 09:55:00,013 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-9) [6ce74644] Running command: ConnectDomainToStorageCommand internal: true. Entities affected : ID: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 Type: Storage 2015-01-06 09:55:00,016 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-9) [6ce74644] ConnectDomainToStorage. Before Connect all hosts to pool. Time:1/6/15 9:55 AM 2015-01-06 09:55:00,201 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-49) [6ce74644] START, ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2bb163a4 2015-01-06 09:55:00,203 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-18) [6ce74644] START, ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 49562818 2015-01-06 09:55:00,212 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-39) [6ce74644] START, ConnectStorageServerVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 4cb88aac 2015-01-06 09:55:00,214 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-7) [6ce74644] START, ConnectStorageServerVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 23eae7f9 2015-01-06 09:55:57,598 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp--127.0.0.1-8702-6) Invalid locale found in configuration: 2015-01-06 09:55:57,600 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp--127.0.0.1-8702-6) Invalid locale found in configuration: 2015-01-06 09:56:00,437 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp--127.0.0.1-8702-11) Invalid locale found in configuration: 2015-01-06 09:56:00,438 INFO [org.ovirt.engine.core.utils.servlet.UnsupportedLocaleHelper] (ajp--127.0.0.1-8702-11) Invalid locale found in configuration: 2015-01-06 09:56:10,268 INFO [org.ovirt.engine.core.bll.aaa.LoginAdminUserCommand] (ajp--127.0.0.1-8702-3) Running command: LoginAdminUserCommand internal: false. 2015-01-06 09:56:10,276 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: User admin logged in. 2015-01-06 09:56:10,388 INFO [org.ovirt.engine.core.bll.aaa.LoginUserCommand] (ajp--127.0.0.1-8702-9) Running command: LoginUserCommand internal: false. 2015-01-06 09:56:24,013 INFO [org.ovirt.engine.core.bll.RunVmCommand] (ajp--127.0.0.1-8702-13) [c2fb019] Lock Acquired to object EngineLock [exclusiveLocks= key: e2abb477-f807-470f-ae20-a7205e690638 value: VM , sharedLocks= ] 2015-01-06 09:56:24,134 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-13) [c2fb019] START, IsVmDuringInitiatingVDSCommand( vmId = e2abb477-f807-470f-ae20-a7205e690638), log id: 5c2ceac9 2015-01-06 09:56:24,136 INFO [org.ovirt.engine.core.vdsbroker.IsVmDuringInitiatingVDSCommand] (ajp--127.0.0.1-8702-13) [c2fb019] FINISH, IsVmDuringInitiatingVDSCommand, return: false, log id: 5c2ceac9 2015-01-06 09:56:24,357 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] Running command: RunVmCommand internal: false. Entities affected : ID: e2abb477-f807-470f-ae20-a7205e690638 Type: VMAction group VM_BASIC_OPERATIONS with role type USER 2015-01-06 09:56:24,500 INFO [org.ovirt.engine.core.bll.scheduling.policyunits.HaReservationWeightPolicyUnit] (org.ovirt.thread.pool-8-thread-9) [c2fb019] Started HA reservation scoring method 2015-01-06 09:56:24,561 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] START, UpdateVmDynamicDataVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, vmDynamic=org.ovirt.engine.core.common.businessentities.VmDynamic@57214812), log id: 63eaa18e 2015-01-06 09:56:24,573 INFO [org.ovirt.engine.core.vdsbroker.UpdateVmDynamicDataVDSCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] FINISH, UpdateVmDynamicDataVDSCommand, log id: 63eaa18e 2015-01-06 09:56:24,612 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] START, CreateVmVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, vmId=e2abb477-f807-470f-ae20-a7205e690638, vm=VM [ciril-recette]), log id: 5c6b51df 2015-01-06 09:56:24,647 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] START, CreateVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, vmId=e2abb477-f807-470f-ae20-a7205e690638, vm=VM [ciril-recette]), log id: 6bcd4ecc 2015-01-06 09:56:24,691 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand kvmEnable=true,keyboardLayout=en-us,nice=0,pitReinjection=false,displayNetwork=ovirtmgmt,copyPasteEnable=true,timeOffset=0,transparentHugePages=true,vmId=e2abb477-f807-470f-ae20-a7205e690638,acpiEnable=true,custom={device_5cc74d49-61c4-4887-8cf3-41135fbdee99device_ee0562ba-7750-4857-8151-f422de37a388device_41d6489a-07ab-481d-b35e-a14e007df6cd=VmDevice {vmId=e2abb477-f807-470f-ae20-a7205e690638, deviceId=41d6489a-07ab-481d-b35e-a14e007df6cd, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_5cc74d49-61c4-4887-8cf3-41135fbdee99device_ee0562ba-7750-4857-8151-f422de37a388device_41d6489a-07ab-481d-b35e-a14e007df6cddevice_e3fa7b29-5b4f-462d-bcf4-298916996aaf=VmDevice {vmId=e2abb477-f807-470f-ae20-a7205e690638, deviceId=e3fa7b29-5b4f-462d-bcf4-298916996aaf, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_5cc74d49-61c4-4887-8cf3-41135fbdee99=VmDevice {vmId=e2abb477-f807-470f-ae20-a7205e690638, deviceId=5cc74d49-61c4-4887-8cf3-41135fbdee99, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_5cc74d49-61c4-4887-8cf3-41135fbdee99device_ee0562ba-7750-4857-8151-f422de37a388=VmDevice {vmId=e2abb477-f807-470f-ae20-a7205e690638, deviceId=ee0562ba-7750-4857-8151-f422de37a388, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}},spiceSslCipherSuite=DEFAULT,memSize=6000,smp=4,emulatedMachine=rhel6.5.0,vmType=kvm,memGuaranteedSize=1500,display=qxl,smartcardEnable=false,bootMenuEnable=false,numaTune={mode=interleave, nodeset=0,1},spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard,smpCoresPerSocket=1,maxVCpus=16,devices=[{address={bus=0x00, domain=0x0000, slot=0x02, type=pci, function=0x0}, specParams={ram=65536, vram=32768, heads=1}, device=qxl, type=video, deviceId=061ed0c0-22f5-4099-8bee-44a3dfa19956}, {shared=false, iface=ide, index=2, address={unit=0, bus=1, target=0, controller=0, type=drive}, specParams={path=}, path=, device=cdrom, type=disk, readonly=true, deviceId=dd759f21-7cca-4d56-8f0f-348d61492714}, {shared=false, index=0, volumeID=82528148-a4db-4892-87e6-b41c7b2f3ab8, propagateErrors=off, format=raw, type=disk, iface=virtio, bootOrder=1, address={bus=0x00, domain=0x0000, slot=0x06, type=pci, function=0x0}, domainID=05d4e406-de6d-47dc-8414-bdc7381b6d4a, imageID=f09ba312-5078-4070-801c-4a8c7e28d0aa, specParams={}, optional=false, device=disk, poolID=00000002-0002-0002-0002-00000000000f, readonly=false, deviceId=f09ba312-5078-4070-801c-4a8c7e28d0aa}, {nicModel=pv, address={bus=0x00, domain=0x0000, slot=0x03, type=pci, function=0x0}, specParams={outbound={}, inbound={}}, macAddr=00:1a:4a:70:41:a3, device=bridge, linkActive=true, type=interface, filter=vdsm-no-mac-spoofing, network=Cluster_VM, deviceId=b2e197f0-00f4-44f4-bd5c-32824d872c95}, {address={bus=0x00, domain=0x0000, slot=0x07, type=pci, function=0x0}, specParams={model=virtio}, device=memballoon, type=balloon, deviceId=e0a803e7-eb97-4285-a49f-6e99a5df974c}, {index=0, model=virtio-scsi, address={bus=0x00, domain=0x0000, slot=0x04, type=pci, function=0x0}, specParams={}, device=scsi, type=controller, deviceId=bd1cf13f-212b-4ceb-96c7-ee4950f7a7d3}, {address={bus=0x00, domain=0x0000, slot=0x05, type=pci, function=0x0}, specParams={}, device=virtio-serial, type=controller, deviceId=a2df4f4f-2920-4243-ba9a-26206057de6e}],vmName=ciril-recette,cpuType=Westmere,fileTransferEnable=true 2015-01-06 09:56:24,839 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateVDSCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] FINISH, CreateVDSCommand, log id: 6bcd4ecc 2015-01-06 09:56:24,852 INFO [org.ovirt.engine.core.vdsbroker.CreateVmVDSCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] FINISH, CreateVmVDSCommand, return: WaitForLaunch, log id: 5c6b51df 2015-01-06 09:56:24,854 INFO [org.ovirt.engine.core.bll.RunVmCommand] (org.ovirt.thread.pool-8-thread-9) [c2fb019] Lock freed to object EngineLock [exclusiveLocks= key: e2abb477-f807-470f-ae20-a7205e690638 value: VM , sharedLocks= ] 2015-01-06 09:56:24,873 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-9) [c2fb019] Correlation ID: c2fb019, Job ID: 42d86800-fc18-42d6-98a6-0cf205a85a2c, Call Stack: null, Custom Event ID: -1, Message: VM ciril-recette was started by admin (Host: n4orna). 2015-01-06 09:56:28,296 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) [128f72d7] VM ciril-recette e2abb477-f807-470f-ae20-a7205e690638 moved from WaitForLaunch --> PoweringUp 2015-01-06 09:56:28,298 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-73) [128f72d7] START, FullListVdsCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, vds=Host[n4orna,920bf64c-62f5-4b12-a69e-eef9936576c5], vmIds=[e2abb477-f807-470f-ae20-a7205e690638]), log id: 79d27c1c 2015-01-06 09:56:28,307 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FullListVdsCommand] (DefaultQuartzScheduler_Worker-73) [128f72d7] FINISH, FullListVdsCommand, return: [{displaySecurePort=5909, kvmEnable=true, nicModel=rtl8139,pv, keyboardLayout=en-us, displayIp=10.99.23.4, pauseCode=NOERR, pitReinjection=false, nice=0, displayNetwork=ovirtmgmt, copyPasteEnable=true, timeOffset=0, transparentHugePages=true, vmId=e2abb477-f807-470f-ae20-a7205e690638, acpiEnable=true, custom={device_5cc74d49-61c4-4887-8cf3-41135fbdee99device_ee0562ba-7750-4857-8151-f422de37a388device_41d6489a-07ab-481d-b35e-a14e007df6cd=VmDevice {vmId=e2abb477-f807-470f-ae20-a7205e690638, deviceId=41d6489a-07ab-481d-b35e-a14e007df6cd, device=spicevmc, type=CHANNEL, bootOrder=0, specParams={}, address={port=3, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel2, customProperties={}, snapshotId=null}, device_5cc74d49-61c4-4887-8cf3-41135fbdee99device_ee0562ba-7750-4857-8151-f422de37a388device_41d6489a-07ab-481d-b35e-a14e007df6cddevice_e3fa7b29-5b4f-462d-bcf4-298916996aaf=VmDevice {vmId=e2abb477-f807-470f-ae20-a7205e690638, deviceId=e3fa7b29-5b4f-462d-bcf4-298916996aaf, device=ide, type=CONTROLLER, bootOrder=0, specParams={}, address={bus=0x00, domain=0x0000, type=pci, slot=0x01, function=0x1}, managed=false, plugged=true, readOnly=false, deviceAlias=ide0, customProperties={}, snapshotId=null}, device_5cc74d49-61c4-4887-8cf3-41135fbdee99=VmDevice {vmId=e2abb477-f807-470f-ae20-a7205e690638, deviceId=5cc74d49-61c4-4887-8cf3-41135fbdee99, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=1, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel0, customProperties={}, snapshotId=null}, device_5cc74d49-61c4-4887-8cf3-41135fbdee99device_ee0562ba-7750-4857-8151-f422de37a388=VmDevice {vmId=e2abb477-f807-470f-ae20-a7205e690638, deviceId=ee0562ba-7750-4857-8151-f422de37a388, device=unix, type=CHANNEL, bootOrder=0, specParams={}, address={port=2, bus=0, controller=0, type=virtio-serial}, managed=false, plugged=true, readOnly=false, deviceAlias=channel1, customProperties={}, snapshotId=null}}, spiceSslCipherSuite=DEFAULT, memSize=6000, smp=4, displayPort=-1, emulatedMachine=rhel6.5.0, vmType=kvm, status=Up, memGuaranteedSize=1500, display=qxl, pid=23762, smartcardEnable=false, bootMenuEnable=false, spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, numaTune={nodeset=0,1, mode=interleave}, smpCoresPerSocket=1, maxVCpus=16, clientIp=, devices=[Ljava.lang.Object;@69d250bf, vmName=ciril-recette, fileTransferEnable=true, cpuType=Westmere}], log id: 79d27c1c 2015-01-06 09:56:28,328 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-73) [128f72d7] Received a spice Device without an address when processing VM e2abb477-f807-470f-ae20-a7205e690638 devices, skipping device: {specParams={spiceSecureChannels=smain,sinputs,scursor,splayback,srecord,sdisplay,susbredir,ssmartcard, keyMap=en-us, displayNetwork=ovirtmgmt, copyPasteEnable=true, displayIp=10.99.23.4}, device=spice, tlsPort=5909, type=graphics} 2015-01-06 09:56:46,936 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-55) [27e4f6a6] VM ciril-recette e2abb477-f807-470f-ae20-a7205e690638 moved from PoweringUp --> Up 2015-01-06 09:56:47,065 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-55) [27e4f6a6] Correlation ID: c2fb019, Job ID: 42d86800-fc18-42d6-98a6-0cf205a85a2c, Call Stack: null, Custom Event ID: -1, Message: VM ciril-recette started on Host n4orna 2015-01-06 09:57:57,112 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (ajp--127.0.0.1-8702-6) [631b7fb5] Lock Acquired to object EngineLock [exclusiveLocks= key: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 value: STORAGE , sharedLocks= ] 2015-01-06 09:57:57,517 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] Running command: DetachStorageDomainFromPoolCommand internal: false. Entities affected : ID: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN 2015-01-06 09:57:57,520 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] Start detach storage domain 2015-01-06 09:57:57,530 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] Detach storage domain: before connect 2015-01-06 09:57:57,720 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-50) [631b7fb5] START, ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 13e1b92c 2015-01-06 09:57:57,728 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-21) [631b7fb5] START, ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 7b9a83cf 2015-01-06 09:57:57,737 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-37) [631b7fb5] START, ConnectStorageServerVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 2a3e2925 2015-01-06 09:57:57,759 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-16) [631b7fb5] START, ConnectStorageServerVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 6386df71 2015-01-06 09:58:00,209 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-49) [6ce74644] Command ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 09:58:00,214 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-18) [6ce74644] Command ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 09:58:00,220 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-39) [6ce74644] Command ConnectStorageServerVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 09:58:00,215 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-49) [6ce74644] FINISH, ConnectStorageServerVDSCommand, log id: 2bb163a4 2015-01-06 09:58:00,227 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-39) [6ce74644] FINISH, ConnectStorageServerVDSCommand, log id: 4cb88aac 2015-01-06 09:58:00,225 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-7) [6ce74644] Command ConnectStorageServerVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 09:58:00,221 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-18) [6ce74644] FINISH, ConnectStorageServerVDSCommand, log id: 49562818 2015-01-06 09:58:00,236 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-7) [6ce74644] FINISH, ConnectStorageServerVDSCommand, log id: 23eae7f9 2015-01-06 09:58:00,230 ERROR [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (org.ovirt.thread.pool-8-thread-39) [6ce74644] Failed to connect host n3orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 09:58:00,228 ERROR [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (org.ovirt.thread.pool-8-thread-49) [6ce74644] Failed to connect host n2orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 09:58:00,239 ERROR [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (org.ovirt.thread.pool-8-thread-7) [6ce74644] Failed to connect host n4orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 09:58:00,237 ERROR [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (org.ovirt.thread.pool-8-thread-18) [6ce74644] Failed to connect host n1orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 09:58:00,294 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-9) [6ce74644] ConnectDomainToStorage. After Connect all hosts to pool. Time:1/6/15 9:58 AM 2015-01-06 09:58:03,032 INFO [org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand] (org.ovirt.thread.pool-8-thread-6) [4205431d] Running command: SyncLunsInfoForBlockStorageDomainCommand internal: true. Entities affected : ID: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 Type: Storage 2015-01-06 09:58:03,111 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-6) [4205431d] START, GetVGInfoVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, VGID=Zqoys8-LKeb-QJIG-TRDR-dnkP-0RWd-ftbOom), log id: a46928a 2015-01-06 09:58:03,225 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-6) [4205431d] Failed in GetVGInfoVDS method 2015-01-06 09:58:03,227 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-6) [4205431d] Command org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand return value OneVGReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=506, mMessage=Volume Group does not exist: ('vg_uuid: Zqoys8-LKeb-QJIG-TRDR-dnkP-0RWd-ftbOom',)]] 2015-01-06 09:58:03,229 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-6) [4205431d] HostName = n2orna 2015-01-06 09:58:03,231 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-6) [4205431d] Command GetVGInfoVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, VGID=Zqoys8-LKeb-QJIG-TRDR-dnkP-0RWd-ftbOom) execution failed. Exception: VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetVGInfoVDS, error = Volume Group does not exist: ('vg_uuid: Zqoys8-LKeb-QJIG-TRDR-dnkP-0RWd-ftbOom',), code = 506 2015-01-06 09:58:03,234 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-6) [4205431d] FINISH, GetVGInfoVDSCommand, log id: a46928a 2015-01-06 09:58:03,236 ERROR [org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand] (org.ovirt.thread.pool-8-thread-6) [4205431d] Command org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to GetVGInfoVDS, error = Volume Group does not exist: ('vg_uuid: Zqoys8-LKeb-QJIG-TRDR-dnkP-0RWd-ftbOom',), code = 506 (Failed with error VolumeGroupDoesNotExist and code 506) 2015-01-06 09:58:06,526 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-46) domain 34c95a44-db7f-4d0f-ba13-5f06a7feefe7:fujitsu_backup_rsync in problem. vds: n1orna 2015-01-06 09:58:06,529 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-46) Host 60d8c75b-38f1-4cd0-b162-729285eadefd has reported new storage access problem to the following domains 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 marking it for storage connections and pool metadata refresh (report id: 6a081ad8-201f-48e4-ba56-5a581fac4475) 2015-01-06 09:58:06,559 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-49) Host 38893029-0eb8-4d19-a28f-07680d8d6868 has reported new storage access problem to the following domains 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 marking it for storage connections and pool metadata refresh (report id: 922d1075-c975-4d9f-87d1-db8d5aa4059d) 2015-01-06 09:58:06,644 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-57) [6e3156c6] Running storage connections refresh for hosts [38893029-0eb8-4d19-a28f-07680d8d6868, 60d8c75b-38f1-4cd0-b162-729285eadefd] 2015-01-06 09:58:06,648 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-8-thread-1) [265f9df9] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000000f Type: StoragePool 2015-01-06 09:58:06,648 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-8-thread-42) [4f7c5a1d] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000000f Type: StoragePool 2015-01-06 09:58:06,700 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-24) Host 920bf64c-62f5-4b12-a69e-eef9936576c5 has reported new storage access problem to the following domains 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 marking it for storage connections and pool metadata refresh (report id: 12dfe3fe-966a-486e-8eda-89575648545f) 2015-01-06 09:58:06,762 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-1) [265f9df9] START, ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 70a8393b 2015-01-06 09:58:06,769 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-42) [4f7c5a1d] START, ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 396e347 2015-01-06 09:58:15,975 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-31) Host ab744426-2294-4c0b-aaf5-08ebb162f542 has reported new storage access problem to the following domains 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 marking it for storage connections and pool metadata refresh (report id: 18fffe35-403c-413e-acb1-8d173aa447da) 2015-01-06 09:58:52,862 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-1) [7face9e1] VM ciril-recette e2abb477-f807-470f-ae20-a7205e690638 moved from Up --> RebootInProgress 2015-01-06 09:59:23,829 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-98) [4c461177] VM ciril-recette e2abb477-f807-470f-ae20-a7205e690638 moved from RebootInProgress --> Up 2015-01-06 09:59:48,453 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-35) domain 05d4e406-de6d-47dc-8414-bdc7381b6d4a:MSA1-O2-vd01 in problem. vds: n3orna 2015-01-06 10:00:34,503 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) [289a4e02] VM ldap-recette ad20808d-b62c-4087-a41a-31ed7672e364 moved from Up --> NotResponding 2015-01-06 10:00:34,550 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) [289a4e02] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM ldap-recette is not responding. 2015-01-06 10:00:34,572 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-94) [289a4e02] VM ciril 627fbbe4-4812-4218-a512-cc1ed26124fd moved from Up --> NotResponding 2015-01-06 10:00:34,618 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-94) [289a4e02] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM ciril is not responding. 2015-01-06 10:00:50,138 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) [50e93ebc] VM antivirus2 1437b20a-d3cb-4537-ace5-2dee650f561f moved from Up --> NotResponding 2015-01-06 10:00:50,183 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) [50e93ebc] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM antivirus2 is not responding. 2015-01-06 10:00:50,188 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) [50e93ebc] VM rsst-formation 064f2b1f-f245-4f34-9747-27c28ce388df moved from Up --> NotResponding 2015-01-06 10:00:50,257 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) [50e93ebc] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM rsst-formation is not responding. 2015-01-06 10:00:50,262 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-96) [50e93ebc] VM forum c94e5f1f-1f10-47e5-b4bb-87d1ef31caf9 moved from Up --> NotResponding 2015-01-06 10:00:50,303 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-96) [50e93ebc] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM forum is not responding. 2015-01-06 10:00:57,729 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-50) [631b7fb5] Command ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 10:00:57,737 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-21) [631b7fb5] Command ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 10:00:57,745 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-37) [631b7fb5] Command ConnectStorageServerVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 10:00:57,741 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-50) [631b7fb5] FINISH, ConnectStorageServerVDSCommand, log id: 13e1b92c 2015-01-06 10:00:57,760 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-37) [631b7fb5] FINISH, ConnectStorageServerVDSCommand, log id: 2a3e2925 2015-01-06 10:00:57,750 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-21) [631b7fb5] FINISH, ConnectStorageServerVDSCommand, log id: 7b9a83cf 2015-01-06 10:00:57,764 ERROR [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-37) [631b7fb5] Failed to connect host n4orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 10:00:57,762 ERROR [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-50) [631b7fb5] Failed to connect host n2orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 10:00:57,767 ERROR [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-21) [631b7fb5] Failed to connect host n1orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 10:00:57,767 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-16) [631b7fb5] Command ConnectStorageServerVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 10:00:57,868 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-16) [631b7fb5] FINISH, ConnectStorageServerVDSCommand, log id: 6386df71 2015-01-06 10:00:57,871 ERROR [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-16) [631b7fb5] Failed to connect host n3orna to storage domain (name: fujitsu_backup_rsync, id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7). Exception: org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022): org.ovirt.engine.core.common.errors.VdcBLLException: VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:117) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.RunVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:58) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.runConnectionStorageToDomain(ISCSIStorageHelper.java:31) [bll.jar:] at org.ovirt.engine.core.bll.storage.ISCSIStorageHelper.connectStorageToDomainByVdsId(ISCSIStorageHelper.java:227) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:336) [bll.jar:] at org.ovirt.engine.core.bll.storage.StorageDomainCommandBase$3.call(StorageDomainCommandBase.java:331) [bll.jar:] at org.ovirt.engine.core.utils.threadpool.ThreadPoolUtil$InternalCallable.call(ThreadPoolUtil.java:112) [utils.jar:] at java.util.concurrent.FutureTask.run(FutureTask.java:262) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [rt.jar:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [rt.jar:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [rt.jar:1.7.0_71] 2015-01-06 10:00:57,902 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] Detach storage domain: after connect 2015-01-06 10:00:57,906 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] START, DetachStorageDomainVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000000f, ignoreFailoverLimit = false, storageDomainId = 34c95a44-db7f-4d0f-ba13-5f06a7feefe7, masterDomainId = 00000000-0000-0000-0000-000000000000, masterVersion = 1, force = false), log id: 7e14d5fa 2015-01-06 10:01:05,877 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-89) [3c054f3d] VM voip-admin 34d7033b-72bd-4c61-9a24-a73c5541955c moved from Up --> NotResponding 2015-01-06 10:01:05,925 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-89) [3c054f3d] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM voip-admin is not responding. 2015-01-06 10:01:06,770 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-1) [265f9df9] Command ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 10:01:06,776 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-42) [4f7c5a1d] Command ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-06 10:01:06,785 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-1) [265f9df9] FINISH, ConnectStorageServerVDSCommand, log id: 70a8393b 2015-01-06 10:01:06,796 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-42) [4f7c5a1d] FINISH, ConnectStorageServerVDSCommand, log id: 396e347 2015-01-06 10:01:06,799 ERROR [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-8-thread-1) [265f9df9] Command org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) 2015-01-06 10:01:06,801 ERROR [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-8-thread-42) [4f7c5a1d] Command org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) 2015-01-06 10:01:06,813 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-57) [6e3156c6] Submitting to the event queue pool refresh for hosts [38893029-0eb8-4d19-a28f-07680d8d6868, 60d8c75b-38f1-4cd0-b162-729285eadefd] 2015-01-06 10:01:06,816 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-37) Running storage pool metadata refresh for hosts {1} 2015-01-06 10:01:06,844 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-47) START, ConnectStoragePoolVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, vdsId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, masterVersion = 1), log id: 377e2518 2015-01-06 10:01:06,844 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-26) START, ConnectStoragePoolVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, vdsId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, masterVersion = 1), log id: 2d074200 2015-01-06 10:01:15,895 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] Failed in DetachStorageDomainVDS method 2015-01-06 10:01:15,898 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] IrsBroker::Failed::DetachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to DetachStorageDomainVDS, error = Storage domain does not exist: ('34c95a44-db7f-4d0f-ba13-5f06a7feefe7',), code = 358 2015-01-06 10:01:21,229 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-26) FINISH, ConnectStoragePoolVDSCommand, log id: 2d074200 2015-01-06 10:01:22,578 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) [2ba9b406] VM packetfence adef4f71-d322-4307-b1d1-c0988bd4efcb moved from Up --> NotResponding 2015-01-06 10:01:22,624 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) [2ba9b406] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM packetfence is not responding. 2015-01-06 10:01:22,629 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) [2ba9b406] VM afs1 2256fed4-ff9e-4520-9e89-fe0bca30ec1b moved from Up --> NotResponding 2015-01-06 10:01:22,672 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) [2ba9b406] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM afs1 is not responding. 2015-01-06 10:01:22,676 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-86) [2ba9b406] VM piros e22277e0-723f-40b6-a46d-ca8f821807e7 moved from Up --> NotResponding 2015-01-06 10:01:22,719 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler_Worker-86) [2ba9b406] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VM piros is not responding. 2015-01-06 10:03:21,860 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] START, SpmStopVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f), log id: 15cd9c64 2015-01-06 10:03:22,004 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] SpmStopVDSCommand::Stopping SPM on vds n2orna, pool id 00000002-0002-0002-0002-00000000000f 2015-01-06 10:03:22,032 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-47) FINISH, ConnectStoragePoolVDSCommand, log id: 377e2518 2015-01-06 10:03:22,037 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-3) domain 75a3e25a-2ea6-4b95-9d9f-9fe791e38e97:MSA2-O2 in problem. vds: n2orna 2015-01-06 10:03:22,047 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-46) starting processDomainRecovery for domain 34c95a44-db7f-4d0f-ba13-5f06a7feefe7:fujitsu_backup_rsync 2015-01-06 10:03:22,179 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] FINISH, SpmStopVDSCommand, log id: 15cd9c64 2015-01-06 10:03:22,181 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) [631b7fb5] Irs placed on server 38893029-0eb8-4d19-a28f-07680d8d6868 failed. Proceed Failover 2015-01-06 10:03:22,219 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-46) Domain 34c95a44-db7f-4d0f-ba13-5f06a7feefe7:fujitsu_backup_rsync was reported by all hosts in status UP as problematic. Moving the domain to NonOperational. 2015-01-06 10:03:22,251 INFO [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-46) [74a0283] Failed to Acquire Lock to object EngineLock [exclusiveLocks= key: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 value: STORAGE , sharedLocks= key: 00000002-0002-0002-0002-00000000000f value: POOL ] 2015-01-06 10:03:22,258 WARN [org.ovirt.engine.core.bll.storage.DeactivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-46) [74a0283] CanDoAction of action DeactivateStorageDomain failed. Reasons:VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ACTION_TYPE_FAILED_OBJECT_LOCKED 2015-01-06 10:03:22,279 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-46) [74a0283] Removing vds [] from the domain in maintenance cache 2015-01-06 10:03:22,282 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-46) [74a0283] Removing host(s) [] from hosts unseen domain report cache 2015-01-06 10:03:22,300 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000000f Type: StoragePool 2015-01-06 10:03:22,306 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (org.ovirt.thread.pool-8-thread-17) [422d495e] Storage Pool 00000002-0002-0002-0002-00000000000f - Updating Storage Domain 05d4e406-de6d-47dc-8414-bdc7381b6d4a status from Active to Unknown, reason : null 2015-01-06 10:03:22,311 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (org.ovirt.thread.pool-8-thread-17) [422d495e] Storage Pool 00000002-0002-0002-0002-00000000000f - Updating Storage Domain 75a3e25a-2ea6-4b95-9d9f-9fe791e38e97 status from Active to Unknown, reason : null 2015-01-06 10:03:22,315 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (org.ovirt.thread.pool-8-thread-17) [422d495e] Storage Pool 00000002-0002-0002-0002-00000000000f - Updating Storage Domain 5a8e48d4-25f0-46f4-9682-e92785e9057a status from Active to Unknown, reason : null 2015-01-06 10:03:22,319 INFO [org.ovirt.engine.core.vdsbroker.storage.StoragePoolDomainHelper] (org.ovirt.thread.pool-8-thread-17) [422d495e] Storage Pool 00000002-0002-0002-0002-00000000000f - Updating Storage Domain 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 status from Active to Unknown, reason : null 2015-01-06 10:03:22,363 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-17) [422d495e] Correlation ID: 422d495e, Call Stack: null, Custom Event ID: -1, Message: Data Center is being initialized, please wait for initialization to complete. 2015-01-06 10:03:22,381 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) [422d495e] hostFromVds::selectedVds - n3orna, spmStatus Free, storage pool Default 2015-01-06 10:03:22,391 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) [422d495e] starting spm on vds n3orna, storage pool Default, prevId -1, LVER -1 2015-01-06 10:03:22,418 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] START, SpmStartVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, prevId=-1, prevLVER=-1, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 51aadbb5 2015-01-06 10:03:22,446 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] spmStart polling started: taskId = 58d4fff4-d230-4c6b-aae3-62b7ff427098 2015-01-06 10:03:26,697 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM ldap-recette ad20808d-b62c-4087-a41a-31ed7672e364 moved from NotResponding --> Up 2015-01-06 10:03:26,699 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM voip-admin 34d7033b-72bd-4c61-9a24-a73c5541955c moved from NotResponding --> Up 2015-01-06 10:03:26,702 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM packetfence adef4f71-d322-4307-b1d1-c0988bd4efcb moved from NotResponding --> Up 2015-01-06 10:03:26,704 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM afs1 2256fed4-ff9e-4520-9e89-fe0bca30ec1b moved from NotResponding --> Up 2015-01-06 10:03:26,707 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM piros e22277e0-723f-40b6-a46d-ca8f821807e7 moved from NotResponding --> Up 2015-01-06 10:03:26,709 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM ciril 627fbbe4-4812-4218-a512-cc1ed26124fd moved from NotResponding --> Up 2015-01-06 10:03:26,712 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM antivirus2 1437b20a-d3cb-4537-ace5-2dee650f561f moved from NotResponding --> Up 2015-01-06 10:03:26,714 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM rsst-formation 064f2b1f-f245-4f34-9747-27c28ce388df moved from NotResponding --> Up 2015-01-06 10:03:26,716 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-100) [7ad552fa] VM forum c94e5f1f-1f10-47e5-b4bb-87d1ef31caf9 moved from NotResponding --> Up 2015-01-06 10:03:28,136 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] spmStart polling ended: taskId = 58d4fff4-d230-4c6b-aae3-62b7ff427098 task status = finished 2015-01-06 10:03:28,155 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] spmStart polling ended, spm status: SPM 2015-01-06 10:03:28,191 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] START, HSMClearTaskVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, taskId=58d4fff4-d230-4c6b-aae3-62b7ff427098), log id: 3c3c7472 2015-01-06 10:03:28,199 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] FINISH, HSMClearTaskVDSCommand, log id: 3c3c7472 2015-01-06 10:03:28,201 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@146b8db3, log id: 51aadbb5 2015-01-06 10:03:28,212 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) [422d495e] Initialize Irs proxy from vds: 10.99.23.3 2015-01-06 10:03:28,248 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-17) [422d495e] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host n3orna (Address: 10.99.23.3). 2015-01-06 10:03:28,273 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-26) [422d495e] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000000f, ignoreFailoverLimit = false), log id: 5be8ceb1 2015-01-06 10:03:28,530 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] Failed in DetachStorageDomainVDS method 2015-01-06 10:03:28,533 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] IrsBroker::Failed::DetachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to DetachStorageDomainVDS, error = Storage domain does not exist: ('34c95a44-db7f-4d0f-ba13-5f06a7feefe7',), code = 358 2015-01-06 10:03:28,688 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] START, SpmStopVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f), log id: 430bdc38 2015-01-06 10:03:28,696 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] SpmStopVDSCommand::Stopping SPM on vds n3orna, pool id 00000002-0002-0002-0002-00000000000f 2015-01-06 10:03:28,943 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStopVDSCommand] (org.ovirt.thread.pool-8-thread-17) [422d495e] FINISH, SpmStopVDSCommand, log id: 430bdc38 2015-01-06 10:03:28,946 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) [422d495e] Irs placed on server ab744426-2294-4c0b-aaf5-08ebb162f542 failed. Proceed Failover 2015-01-06 10:03:29,052 INFO [org.ovirt.engine.core.bll.storage.SetStoragePoolStatusCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] Running command: SetStoragePoolStatusCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000000f Type: StoragePool 2015-01-06 10:03:29,099 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-17) [18526185] Correlation ID: 18526185, Call Stack: null, Custom Event ID: -1, Message: Data Center is being initialized, please wait for initialization to complete. 2015-01-06 10:03:29,113 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) [18526185] hostFromVds::selectedVds - n4orna, spmStatus Free, storage pool Default 2015-01-06 10:03:29,123 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) [18526185] starting spm on vds n4orna, storage pool Default, prevId -1, LVER -1 2015-01-06 10:03:29,151 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] START, SpmStartVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, prevId=-1, prevLVER=-1, storagePoolFormatType=V3, recoveryMode=Manual, SCSIFencing=false), log id: 3992a29c 2015-01-06 10:03:29,164 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] spmStart polling started: taskId = d2c1864d-c71c-4378-a4a5-97d0afafd775 2015-01-06 10:03:31,478 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] spmStart polling ended: taskId = d2c1864d-c71c-4378-a4a5-97d0afafd775 task status = finished 2015-01-06 10:03:31,490 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] spmStart polling ended, spm status: SPM 2015-01-06 10:03:31,518 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] START, HSMClearTaskVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, taskId=d2c1864d-c71c-4378-a4a5-97d0afafd775), log id: 29266dd2 2015-01-06 10:03:31,526 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] FINISH, HSMClearTaskVDSCommand, log id: 29266dd2 2015-01-06 10:03:31,528 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SpmStartVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] FINISH, SpmStartVDSCommand, return: org.ovirt.engine.core.common.businessentities.SpmStatusResult@59cd7a77, log id: 3992a29c 2015-01-06 10:03:31,539 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) [18526185] Initialize Irs proxy from vds: 10.99.23.4 2015-01-06 10:03:31,575 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-17) [18526185] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Storage Pool Manager runs on Host n4orna (Address: 10.99.23.4). 2015-01-06 10:03:31,599 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18526185] START, SPMGetAllTasksInfoVDSCommand( storagePoolId = 00000002-0002-0002-0002-00000000000f, ignoreFailoverLimit = false), log id: 5e2b03ab 2015-01-06 10:03:31,711 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] Failed in DetachStorageDomainVDS method 2015-01-06 10:03:31,714 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] IrsBroker::Failed::DetachStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to DetachStorageDomainVDS, error = Storage domain does not exist: ('34c95a44-db7f-4d0f-ba13-5f06a7feefe7',), code = 358 2015-01-06 10:03:31,718 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DetachStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] FINISH, DetachStorageDomainVDSCommand, log id: 7e14d5fa 2015-01-06 10:03:31,718 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18526185] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000000f 2015-01-06 10:03:31,721 ERROR [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] Command org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to DetachStorageDomainVDS, error = Storage domain does not exist: ('34c95a44-db7f-4d0f-ba13-5f06a7feefe7',), code = 358 (Failed with error StorageDomainDoesNotExist and code 358) 2015-01-06 10:03:31,741 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] Command [id=3b4688b2-b626-470c-9ac9-4a65b0ef0864]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatusSnapshot [id=storagePoolId = 00000002-0002-0002-0002-00000000000f, storageId = 34c95a44-db7f-4d0f-ba13-5f06a7feefe7, status=Inactive]. 2015-01-06 10:03:31,761 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-17) [18526185] Correlation ID: 631b7fb5, Job ID: 780fb013-3a4c-4cc7-9bfd-a755989a6eeb, Call Stack: null, Custom Event ID: -1, Message: Failed to detach Storage Domain fujitsu_backup_rsync to Data Center Default. (User: admin) 2015-01-06 10:03:31,771 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18526185] START, HSMGetAllTasksInfoVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5), log id: 32a82db7 2015-01-06 10:03:31,773 INFO [org.ovirt.engine.core.bll.storage.DetachStorageDomainFromPoolCommand] (org.ovirt.thread.pool-8-thread-17) [18526185] Lock freed to object EngineLock [exclusiveLocks= key: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 value: STORAGE , sharedLocks= ] 2015-01-06 10:03:31,779 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18526185] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 32a82db7 2015-01-06 10:03:31,782 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-13) [18526185] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 5e2b03ab 2015-01-06 10:03:31,782 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-26) [422d495e] -- executeIrsBrokerCommand: Attempting on storage pool 00000002-0002-0002-0002-00000000000f 2015-01-06 10:03:31,785 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-13) [18526185] Discovered no tasks on Storage Pool Default 2015-01-06 10:03:31,815 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-26) [422d495e] START, HSMGetAllTasksInfoVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5), log id: 4eb41b3a 2015-01-06 10:03:31,825 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-26) [422d495e] FINISH, HSMGetAllTasksInfoVDSCommand, return: [], log id: 4eb41b3a 2015-01-06 10:03:31,829 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMGetAllTasksInfoVDSCommand] (org.ovirt.thread.pool-8-thread-26) [422d495e] FINISH, SPMGetAllTasksInfoVDSCommand, return: [], log id: 5be8ceb1 2015-01-06 10:03:31,831 INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (org.ovirt.thread.pool-8-thread-26) [422d495e] Discovered no tasks on Storage Pool Default 2015-01-06 10:03:32,008 INFO [org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand] (org.ovirt.thread.pool-8-thread-32) [afbbe0a] Running command: SyncLunsInfoForBlockStorageDomainCommand internal: true. Entities affected : ID: 05d4e406-de6d-47dc-8414-bdc7381b6d4a Type: Storage 2015-01-06 10:03:32,023 INFO [org.ovirt.engine.core.bll.storage.SyncLunsInfoForBlockStorageDomainCommand] (org.ovirt.thread.pool-8-thread-7) [73793c2b] Running command: SyncLunsInfoForBlockStorageDomainCommand internal: true. Entities affected : ID: 75a3e25a-2ea6-4b95-9d9f-9fe791e38e97 Type: Storage 2015-01-06 10:03:32,075 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-32) [afbbe0a] START, GetVGInfoVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, VGID=CtELsP-UYZb-zgRH-bVz7-lijD-Elfy-frzEJp), log id: 19fe6181 2015-01-06 10:03:32,091 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-7) [73793c2b] START, GetVGInfoVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, VGID=dEB6XY-cNBu-hsoz-avXf-m6Vp-4jOq-eDMCBK), log id: 14fc74a1 2015-01-06 10:03:32,102 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-55) [27e4f6a6] Running storage connections refresh for hosts [ab744426-2294-4c0b-aaf5-08ebb162f542, 920bf64c-62f5-4b12-a69e-eef9936576c5] 2015-01-06 10:03:32,107 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-8-thread-27) [4bd57497] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000000f Type: StoragePool 2015-01-06 10:03:32,107 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-8-thread-15) [2ad98c66] Running command: ConnectHostToStoragePoolServersCommand internal: true. Entities affected : ID: 00000002-0002-0002-0002-00000000000f Type: StoragePool 2015-01-06 10:03:32,163 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-27) [4bd57497] START, ConnectStorageServerVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = NFS, connectionList = [{ id: 79bae336-2d76-4986-bf1e-50e350cde9f7, connection: cluster.33sdis.fr:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1acd4935 2015-01-06 10:03:32,167 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-15) [2ad98c66] START, ConnectStorageServerVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = NFS, connectionList = [{ id: 79bae336-2d76-4986-bf1e-50e350cde9f7, connection: cluster.33sdis.fr:/var/lib/exports/iso, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 10087d1 2015-01-06 10:03:32,196 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-15) [2ad98c66] FINISH, ConnectStorageServerVDSCommand, return: {79bae336-2d76-4986-bf1e-50e350cde9f7=0}, log id: 10087d1 2015-01-06 10:03:32,199 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-8-thread-15) [2ad98c66] Host n3orna storage connection was succeeded 2015-01-06 10:03:32,643 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-32) [afbbe0a] FINISH, GetVGInfoVDSCommand, return: [LUNs [id=3600c0ff000149e9d22c0105401000000, physicalVolumeId=OMa7pu-EyxG-mdNM-9EP5-eP2f-2GPV-tWiexe, volumeGroupId=CtELsP-UYZb-zgRH-bVz7-lijD-Elfy-frzEJp, serial=SHP_P2000_G3_FC_00c0ff149e9d000022c0105401000000, lunMapping=0, vendorId=HP, productId=P2000 G3 FC, _lunConnections=[], deviceSize=5583, vendorName=HP, pathsDictionary={sdb=true, sdd=true}, lunType=UNKNOWN, status=null, diskId=null, diskAlias=null, storageDomainId=05d4e406-de6d-47dc-8414-bdc7381b6d4a, storageDomainName=null]], log id: 19fe6181 2015-01-06 10:03:32,649 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVGInfoVDSCommand] (org.ovirt.thread.pool-8-thread-7) [73793c2b] FINISH, GetVGInfoVDSCommand, return: [LUNs [id=3600c0ff00015986a35d98a5401000000, physicalVolumeId=oWeQjL-dOc0-G9dH-wu1g-8pCd-jGjG-rzL0ZM, volumeGroupId=dEB6XY-cNBu-hsoz-avXf-m6Vp-4jOq-eDMCBK, serial=SHP_P2000_G3_FC_00c0ff15986a000035d98a5401000000, lunMapping=0, vendorId=HP, productId=P2000 G3 FC, _lunConnections=[], deviceSize=5583, vendorName=HP, pathsDictionary={sdc=true, sde=true}, lunType=UNKNOWN, status=null, diskId=null, diskAlias=null, storageDomainId=75a3e25a-2ea6-4b95-9d9f-9fe791e38e97, storageDomainName=null]], log id: 14fc74a1 2015-01-06 10:03:32,664 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-27) [4bd57497] FINISH, ConnectStorageServerVDSCommand, return: {79bae336-2d76-4986-bf1e-50e350cde9f7=0}, log id: 1acd4935 2015-01-06 10:03:32,668 INFO [org.ovirt.engine.core.bll.storage.ConnectHostToStoragePoolServersCommand] (org.ovirt.thread.pool-8-thread-27) [4bd57497] Host n4orna storage connection was succeeded 2015-01-06 10:03:32,671 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-55) [27e4f6a6] Submitting to the event queue pool refresh for hosts [ab744426-2294-4c0b-aaf5-08ebb162f542, 920bf64c-62f5-4b12-a69e-eef9936576c5] 2015-01-06 10:03:32,675 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-35) Running storage pool metadata refresh for hosts {1} 2015-01-06 10:03:32,704 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-10) START, ConnectStoragePoolVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, vdsId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, masterVersion = 1), log id: 66dc8feb 2015-01-06 10:03:32,705 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-41) START, ConnectStoragePoolVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, vdsId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, masterVersion = 1), log id: 1a6cc20f 2015-01-06 10:03:34,083 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-10) FINISH, ConnectStoragePoolVDSCommand, log id: 66dc8feb 2015-01-06 10:03:34,307 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-41) FINISH, ConnectStoragePoolVDSCommand, log id: 1a6cc20f 2015-01-06 10:03:37,386 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-47) Host 60d8c75b-38f1-4cd0-b162-729285eadefd no longer storage access problem to any relevant domain clearing its report (report id: 6a081ad8-201f-48e4-ba56-5a581fac4475) 2015-01-06 10:03:37,392 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-47) Domain 75a3e25a-2ea6-4b95-9d9f-9fe791e38e97:MSA2-O2 recovered from problem. vds: n1orna 2015-01-06 10:03:37,465 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-34) Host 38893029-0eb8-4d19-a28f-07680d8d6868 no longer storage access problem to any relevant domain clearing its report (report id: 922d1075-c975-4d9f-87d1-db8d5aa4059d) 2015-01-06 10:03:37,471 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-34) Domain 75a3e25a-2ea6-4b95-9d9f-9fe791e38e97:MSA2-O2 recovered from problem. vds: n2orna 2015-01-06 10:03:37,475 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-34) Domain 75a3e25a-2ea6-4b95-9d9f-9fe791e38e97:MSA2-O2 has recovered from problem. No active host in the DC is reporting it as problematic, so clearing the domain recovery timer. 2015-01-06 10:03:37,634 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-11) Host 920bf64c-62f5-4b12-a69e-eef9936576c5 no longer storage access problem to any relevant domain clearing its report (report id: 12dfe3fe-966a-486e-8eda-89575648545f) 2015-01-06 10:03:39,263 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) Host ab744426-2294-4c0b-aaf5-08ebb162f542 no longer storage access problem to any relevant domain clearing its report (report id: 18fffe35-403c-413e-acb1-8d173aa447da) 2015-01-06 10:03:39,268 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) Domain 05d4e406-de6d-47dc-8414-bdc7381b6d4a:MSA1-O2-vd01 recovered from problem. vds: n3orna 2015-01-06 10:03:39,270 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-17) Domain 05d4e406-de6d-47dc-8414-bdc7381b6d4a:MSA1-O2-vd01 has recovered from problem. No active host in the DC is reporting it as problematic, so clearing the domain recovery timer. 2015-01-06 10:03:44,310 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (DefaultQuartzScheduler_Worker-45) [5770eda1] No hosts has reported storage access problem to domains, clearing the handled hosts reports map 2015-01-06 10:05:00,006 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-29) [1cbf3978] Autorecovering 1 storage domains 2015-01-06 10:05:00,009 INFO [org.ovirt.engine.core.bll.AutoRecoveryManager] (DefaultQuartzScheduler_Worker-29) [1cbf3978] Autorecovering storage domains id: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 2015-01-06 10:05:00,014 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-29) [344e83b5] Running command: ConnectDomainToStorageCommand internal: true. Entities affected : ID: 34c95a44-db7f-4d0f-ba13-5f06a7feefe7 Type: Storage 2015-01-06 10:05:00,019 INFO [org.ovirt.engine.core.bll.storage.ConnectDomainToStorageCommand] (DefaultQuartzScheduler_Worker-29) [344e83b5] ConnectDomainToStorage. Before Connect all hosts to pool. Time:1/6/15 10:05 AM 2015-01-06 10:05:00,209 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-37) [344e83b5] START, ConnectStorageServerVDSCommand(HostName = n4orna, HostId = 920bf64c-62f5-4b12-a69e-eef9936576c5, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 1d6d563c 2015-01-06 10:05:00,210 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-28) [344e83b5] START, ConnectStorageServerVDSCommand(HostName = n1orna, HostId = 60d8c75b-38f1-4cd0-b162-729285eadefd, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 12a471c7 2015-01-06 10:05:00,258 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-34) [344e83b5] START, ConnectStorageServerVDSCommand(HostName = n3orna, HostId = ab744426-2294-4c0b-aaf5-08ebb162f542, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 70547eab 2015-01-06 10:05:00,258 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-19) [344e83b5] START, ConnectStorageServerVDSCommand(HostName = n2orna, HostId = 38893029-0eb8-4d19-a28f-07680d8d6868, storagePoolId = 00000002-0002-0002-0002-00000000000f, storageType = ISCSI, connectionList = [{ id: 1b9f3167-3236-431e-93c2-ab5ee18eba04, connection: 10.23.2.199, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };{ id: ea5971f8-e1a0-42e3-826d-b95e9031ce53, connection: 10.23.2.198, iqn: iqn.1999-06.com.fujitsu-siemens:0907d7d0e3.a, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 6438f79b 2015-01-06 10:06:11,746 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-83) [453f31bd] VM ciril-recette e2abb477-f807-470f-ae20-a7205e690638 moved from Up --> RebootInProgress 2015-01-06 10:06:39,611 INFO [org.ovirt.engine.core.vdsbroker.VdsUpdateRunTimeInfo] (DefaultQuartzScheduler_Worker-84) [537321db] VM ciril-recette e2abb477-f807-470f-ae20-a7205e690638 moved from RebootInProgress --> Up -- Ce courriel et tous les fichiers attachés qu'il contient sont confidentiels et destinés exclusivement à la personne à laquelle ils sont adressés. Si vous avez reçu ce courriel par erreur, merci de le retourner à son expéditeur et de le détruire. Il est rappelé que tout message électronique est susceptible d'alteration au cours de son acheminement sur internet. Seuls les documents officiels du SDIS sont de nature à engager sa responsabilité. Les idées ou opinions présentées dans ce courriel sont celles de son auteur et ne représentent pas nécessairement celles du SDIS de la Gironde.