Dear all, We have a small issue on our ovirt environment. When I try to add a fibre storage pool, I get the following error: 2015-01-21 08:24:48,705 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-22) Storage domain e5d59e58-6408-4f80-911e-a30d0e7ca1fe:BuranIsoDomain is not visible to one or more hosts. Since the domains type is ISO, hosts status will not be changed to non-operational 2015-01-21 08:24:54,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command CreateStorageDomainVDSCommand(HostName = saturnus1, HostId = 1180a1f6-635e-47f6-bba1-871d8c432de0, storageDomain=StorageDomainStatic[StoragePoolOracle01, fd6c6779-8353-42f6-b2ff-0c670e4b8a73], args=qEZ3pE-03I3-5w9M-1XFN-ArBH-2d2e-fSVRF3) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-21 08:24:54,769 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) 2015-01-21 08:24:54,810 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand. 2015-01-21 08:24:54,839 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) [2c666536] Correlation ID: 2c666536, Job ID: 35bc8058-ab79-40b5-be1e-877b88362261, Call Stack: null, Custom Event ID: -1, Message: Failed to add Storage Domain StoragePoolOracle01. (User: admin) 2015-01-21 08:24:55,107 WARN [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-25) In the GUI you'll see this error: Error while executing action New SAN Storage Domain: Network error during communication with the Host. All the hosts are up and all the rest of our storage is also up. Kind regards, Koen
I noticed that there was a difference in the number of attached LUN's between the hypervisors. This is because we have a VM with direct LUN's. Do these Lun's on this particular vm also be attached to the other hypervisors or to 1 hypervisor only? 2015-01-21 8:29 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
Dear all,
We have a small issue on our ovirt environment. When I try to add a fibre storage pool, I get the following error: 2015-01-21 08:24:48,705 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-22) Storage domain e5d59e58-6408-4f80-911e-a30d0e7ca1fe:BuranIsoDomain is not visible to one or more hosts. Since the domains type is ISO, hosts status will not be changed to non-operational 2015-01-21 08:24:54,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command CreateStorageDomainVDSCommand(HostName = saturnus1, HostId = 1180a1f6-635e-47f6-bba1-871d8c432de0, storageDomain=StorageDomainStatic[StoragePoolOracle01, fd6c6779-8353-42f6-b2ff-0c670e4b8a73], args=qEZ3pE-03I3-5w9M-1XFN-ArBH-2d2e-fSVRF3) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-21 08:24:54,769 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) 2015-01-21 08:24:54,810 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand. 2015-01-21 08:24:54,839 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) [2c666536] Correlation ID: 2c666536, Job ID: 35bc8058-ab79-40b5-be1e-877b88362261, Call Stack: null, Custom Event ID: -1, Message: Failed to add Storage Domain StoragePoolOracle01. (User: admin) 2015-01-21 08:24:55,107 WARN [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-25)
In the GUI you'll see this error:
Error while executing action New SAN Storage Domain: Network error during communication with the Host.
All the hosts are up and all the rest of our storage is also up.
Kind regards,
Koen
Added the direct lun, so all hypervisor can see it, no effect.. Still the same error 2015-01-21 10:39 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
I noticed that there was a difference in the number of attached LUN's between the hypervisors. This is because we have a VM with direct LUN's. Do these Lun's on this particular vm also be attached to the other hypervisors or to 1 hypervisor only?
2015-01-21 8:29 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
Dear all,
We have a small issue on our ovirt environment. When I try to add a fibre storage pool, I get the following error: 2015-01-21 08:24:48,705 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-22) Storage domain e5d59e58-6408-4f80-911e-a30d0e7ca1fe:BuranIsoDomain is not visible to one or more hosts. Since the domains type is ISO, hosts status will not be changed to non-operational 2015-01-21 08:24:54,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command CreateStorageDomainVDSCommand(HostName = saturnus1, HostId = 1180a1f6-635e-47f6-bba1-871d8c432de0, storageDomain=StorageDomainStatic[StoragePoolOracle01, fd6c6779-8353-42f6-b2ff-0c670e4b8a73], args=qEZ3pE-03I3-5w9M-1XFN-ArBH-2d2e-fSVRF3) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-21 08:24:54,769 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) 2015-01-21 08:24:54,810 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand. 2015-01-21 08:24:54,839 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) [2c666536] Correlation ID: 2c666536, Job ID: 35bc8058-ab79-40b5-be1e-877b88362261, Call Stack: null, Custom Event ID: -1, Message: Failed to add Storage Domain StoragePoolOracle01. (User: admin) 2015-01-21 08:24:55,107 WARN [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-25)
In the GUI you'll see this error:
Error while executing action New SAN Storage Domain: Network error during communication with the Host.
All the hosts are up and all the rest of our storage is also up.
Kind regards,
Koen
Ok, after rebooting all the hypervisors I'm left with the following errors: Thread-16::ERROR::2015-01-21 13:39:42,644::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 6cf8c48e-fbed-4b68-b376-57eab3039878 Thread-16::ERROR::2015-01-21 13:39:42,644::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 6cf8c48e-fbed-4b68-b376-57eab3039878 Thread-18::ERROR::2015-01-21 13:39:42,696::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain dc662b6f-00d4-4b9a-a320-0f5ecf5da45e Thread-18::ERROR::2015-01-21 13:39:42,698::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain dc662b6f-00d4-4b9a-a320-0f5ecf5da45e Thread-15::ERROR::2015-01-21 13:39:44,349::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain e1bbe4ab-3717-4a9a-ada2-9d175ab38262 Thread-15::ERROR::2015-01-21 13:39:44,350::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain e1bbe4ab-3717-4a9a-ada2-9d175ab38262 Thread-17::ERROR::2015-01-21 13:39:44,477::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain e33afd52-812e-45f7-b45b-8b049fc0f31c Thread-17::ERROR::2015-01-21 13:39:44,479::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain e33afd52-812e-45f7-b45b-8b049fc0f31c Thread-18::ERROR::2015-01-21 13:44:44,407::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain dc662b6f-00d4-4b9a-a320-0f5ecf5da45e Thread-18::ERROR::2015-01-21 13:44:44,408::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain dc662b6f-00d4-4b9a-a320-0f5ecf5da45e Thread-16::ERROR::2015-01-21 13:44:44,795::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 6cf8c48e-fbed-4b68-b376-57eab3039878 Thread-16::ERROR::2015-01-21 13:44:44,796::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 6cf8c48e-fbed-4b68-b376-57eab3039878 Thread-15::ERROR::2015-01-21 13:44:45,900::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain e1bbe4ab-3717-4a9a-ada2-9d175ab38262 Thread-15::ERROR::2015-01-21 13:44:45,901::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain e1bbe4ab-3717-4a9a-ada2-9d175ab38262 Thread-17::ERROR::2015-01-21 13:44:45,934::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain e33afd52-812e-45f7-b45b-8b049fc0f31c Thread-17::ERROR::2015-01-21 13:44:45,942::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain e33afd52-812e-45f7-b45b-8b049fc0f31 Kind regards, Koen 2015-01-21 11:20 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
Added the direct lun, so all hypervisor can see it, no effect.. Still the same error
2015-01-21 10:39 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
I noticed that there was a difference in the number of attached LUN's between the hypervisors. This is because we have a VM with direct LUN's. Do these Lun's on this particular vm also be attached to the other hypervisors or to 1 hypervisor only?
2015-01-21 8:29 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
Dear all,
We have a small issue on our ovirt environment. When I try to add a fibre storage pool, I get the following error: 2015-01-21 08:24:48,705 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-22) Storage domain e5d59e58-6408-4f80-911e-a30d0e7ca1fe:BuranIsoDomain is not visible to one or more hosts. Since the domains type is ISO, hosts status will not be changed to non-operational 2015-01-21 08:24:54,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command CreateStorageDomainVDSCommand(HostName = saturnus1, HostId = 1180a1f6-635e-47f6-bba1-871d8c432de0, storageDomain=StorageDomainStatic[StoragePoolOracle01, fd6c6779-8353-42f6-b2ff-0c670e4b8a73], args=qEZ3pE-03I3-5w9M-1XFN-ArBH-2d2e-fSVRF3) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-21 08:24:54,769 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) 2015-01-21 08:24:54,810 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand. 2015-01-21 08:24:54,839 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) [2c666536] Correlation ID: 2c666536, Job ID: 35bc8058-ab79-40b5-be1e-877b88362261, Call Stack: null, Custom Event ID: -1, Message: Failed to add Storage Domain StoragePoolOracle01. (User: admin) 2015-01-21 08:24:55,107 WARN [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-25)
In the GUI you'll see this error:
Error while executing action New SAN Storage Domain: Network error during communication with the Host.
All the hosts are up and all the rest of our storage is also up.
Kind regards,
Koen
Solved... 2015-01-21 13:46 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
Ok, after rebooting all the hypervisors I'm left with the following errors: Thread-16::ERROR::2015-01-21 13:39:42,644::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 6cf8c48e-fbed-4b68-b376-57eab3039878 Thread-16::ERROR::2015-01-21 13:39:42,644::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 6cf8c48e-fbed-4b68-b376-57eab3039878 Thread-18::ERROR::2015-01-21 13:39:42,696::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain dc662b6f-00d4-4b9a-a320-0f5ecf5da45e Thread-18::ERROR::2015-01-21 13:39:42,698::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain dc662b6f-00d4-4b9a-a320-0f5ecf5da45e Thread-15::ERROR::2015-01-21 13:39:44,349::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain e1bbe4ab-3717-4a9a-ada2-9d175ab38262 Thread-15::ERROR::2015-01-21 13:39:44,350::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain e1bbe4ab-3717-4a9a-ada2-9d175ab38262 Thread-17::ERROR::2015-01-21 13:39:44,477::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain e33afd52-812e-45f7-b45b-8b049fc0f31c Thread-17::ERROR::2015-01-21 13:39:44,479::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain e33afd52-812e-45f7-b45b-8b049fc0f31c Thread-18::ERROR::2015-01-21 13:44:44,407::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain dc662b6f-00d4-4b9a-a320-0f5ecf5da45e Thread-18::ERROR::2015-01-21 13:44:44,408::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain dc662b6f-00d4-4b9a-a320-0f5ecf5da45e Thread-16::ERROR::2015-01-21 13:44:44,795::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 6cf8c48e-fbed-4b68-b376-57eab3039878 Thread-16::ERROR::2015-01-21 13:44:44,796::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 6cf8c48e-fbed-4b68-b376-57eab3039878 Thread-15::ERROR::2015-01-21 13:44:45,900::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain e1bbe4ab-3717-4a9a-ada2-9d175ab38262 Thread-15::ERROR::2015-01-21 13:44:45,901::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain e1bbe4ab-3717-4a9a-ada2-9d175ab38262 Thread-17::ERROR::2015-01-21 13:44:45,934::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain e33afd52-812e-45f7-b45b-8b049fc0f31c Thread-17::ERROR::2015-01-21 13:44:45,942::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain e33afd52-812e-45f7-b45b-8b049fc0f31
Kind regards,
Koen
2015-01-21 11:20 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
Added the direct lun, so all hypervisor can see it, no effect.. Still the same error
2015-01-21 10:39 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
I noticed that there was a difference in the number of attached LUN's between the hypervisors. This is because we have a VM with direct LUN's. Do these Lun's on this particular vm also be attached to the other hypervisors or to 1 hypervisor only?
2015-01-21 8:29 GMT+01:00 Koen Vanoppen <vanoppen.koen@gmail.com>:
Dear all,
We have a small issue on our ovirt environment. When I try to add a fibre storage pool, I get the following error: 2015-01-21 08:24:48,705 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxyData] (org.ovirt.thread.pool-8-thread-22) Storage domain e5d59e58-6408-4f80-911e-a30d0e7ca1fe:BuranIsoDomain is not visible to one or more hosts. Since the domains type is ISO, hosts status will not be changed to non-operational 2015-01-21 08:24:54,764 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.CreateStorageDomainVDSCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command CreateStorageDomainVDSCommand(HostName = saturnus1, HostId = 1180a1f6-635e-47f6-bba1-871d8c432de0, storageDomain=StorageDomainStatic[StoragePoolOracle01, fd6c6779-8353-42f6-b2ff-0c670e4b8a73], args=qEZ3pE-03I3-5w9M-1XFN-ArBH-2d2e-fSVRF3) execution failed. Exception: VDSNetworkException: java.util.concurrent.TimeoutException 2015-01-21 08:24:54,769 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Command org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: java.util.concurrent.TimeoutException (Failed with error VDS_NETWORK_ERROR and code 5022) 2015-01-21 08:24:54,810 ERROR [org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand] (ajp--127.0.0.1-8702-3) [2c666536] Transaction rolled-back for command: org.ovirt.engine.core.bll.storage.AddSANStorageDomainCommand. 2015-01-21 08:24:54,839 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-3) [2c666536] Correlation ID: 2c666536, Job ID: 35bc8058-ab79-40b5-be1e-877b88362261, Call Stack: null, Custom Event ID: -1, Message: Failed to add Storage Domain StoragePoolOracle01. (User: admin) 2015-01-21 08:24:55,107 WARN [org.ovirt.engine.core.bll.AddVmFromScratchCommand] (DefaultQuartzScheduler_Worker-25)
In the GUI you'll see this error:
Error while executing action New SAN Storage Domain: Network error during communication with the Host.
All the hosts are up and all the rest of our storage is also up.
Kind regards,
Koen
participants (1)
-
Koen Vanoppen