Hi, The root cause of my problem was that there was a stale NFS mount on my hosts. They still had an nfs mount active. Killing the ioprocess processes that were keeping the mount busy and then unmounting the nfs mounts allowed me to activate the iso and export domain again. Regards, Rik On 04/28/2015 01:48 PM, Rik Theys wrote:
Hi,
I migrated my engine from CentOS 6 to CentOS 7 by taking an engine-backup on the CentOS 6 install and running the restore on a CentOS 7.1 install.
This worked rather well. I can log into the admin webinterface and see my still running VM's.
The only issue I'm facing is that the hosts can no longer access the export and ISO domain (which are nfs exports on my engine).
When I try to activate the storage domain on a host I get the following message in the engine log (see below).
It seems the engine thinks the storage domain does not exist. I copied the files from the old installation into the same directories on the new installation and I can nfs mount them manually from the hosts. I can also nfs mount it on the engine itself.
Any idea on how to debug this?
My engine is running 3.5.1 (actually 3.5.2 now as it just got upgraded, but the upgrade did not change anything regarding this bug).
Is there a way to remove the export/iso domain? I can not detach it from my data centers using the web interface.
Regards,
Rik
2015-04-28 12:59:19,271 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (ajp--127.0.0.1-8702-8) [450f2bb2] Lock Acquired to object EngineLock [exclusiveLocks= key: 31ba6486-d4ef-45ae-a184-8296185ef79b value: STORAGE , sharedLocks= ] 2015-04-28 12:59:19,330 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] Running command: ActivateStorageDomainCommand internal: false. Entities affected : ID: 31ba6486-d4ef-45ae-a184-8296185ef79b Type: StorageAction group MANIPULATE_STORAGE_DOMAIN with role type ADMIN 2015-04-28 12:59:19,360 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] Lock freed to object EngineLock [exclusiveLocks= key: 31ba6486-d4ef-45ae-a184-8296185ef79b value: STORAGE , sharedLocks= ] 2015-04-28 12:59:19,362 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] ActivateStorage Domain. Before Connect all hosts to pool. Time:4/28/15 12:59 PM 2015-04-28 12:59:19,383 INFO [org.ovirt.engine.core.bll.storage.ConnectStorageToVdsCommand] (org.ovirt.thread.pool-8-thread-28) [3e09aa16] Running command: ConnectStorageToVdsCommand internal: true. Entities affected : ID: aaa00000-0000-0000-0000-123456789aaa Type: SystemAction group CREATE_STORAGE_DOMAIN with role type ADMIN 2015-04-28 12:59:19,388 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-28) [3e09aa16] START, ConnectStorageServerVDSCommand(HostName = stadius-virt2, HostId = 7212971a-d38a-42e7-8e6a-24d3396dfa6a, storagePoolId = 00000000-0000-0000-0000-000000000000, storageType = NFS, connectionList = [{ id: 5f18ed21-8c71-4e71-874a-a6a8594c3138, connection: iron:/var/lib/exports/export-domain, iqn: null, vfsType: null, mountOptions: null, nfsVersion: null, nfsRetrans: null, nfsTimeo: null };]), log id: 13d9ec07 2015-04-28 12:59:19,409 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStorageServerVDSCommand] (org.ovirt.thread.pool-8-thread-28) [3e09aa16] FINISH, ConnectStorageServerVDSCommand, return: {5f18ed21-8c71-4e71-874a-a6a8594c3138=0}, log id: 13d9ec07 2015-04-28 12:59:19,417 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] START, ActivateStorageDomainVDSCommand( storagePoolId = e7bdba88-e718-41a9-8d2b-0ca79c517630, ignoreFailoverLimit = false, storageDomainId = 31ba6486-d4ef-45ae-a184-8296185ef79b), log id: 1da2c4c4 2015-04-28 12:59:19,774 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] Failed in ActivateStorageDomainVDS method 2015-04-28 12:59:19,781 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] IrsBroker::Failed::ActivateStorageDomainVDS due to: IRSErrorException: IRSGenericException: IRSErrorException: Failed to ActivateStorageDomainVDS, error = Storage domain does not exist: ('31ba6486-d4ef-45ae-a184-8296185ef79b',), code = 358 2015-04-28 12:59:19,793 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] FINISH, ActivateStorageDomainVDSCommand, log id: 1da2c4c4 2015-04-28 12:59:19,794 ERROR [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] Command org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand throw Vdc Bll exception. With error message VdcBLLException: org.ovirt.engine.core.vdsbroker.irsbroker.IRSErrorException: IRSGenericException: IRSErrorException: Failed to ActivateStorageDomainVDS, error = Storage domain does not exist: ('31ba6486-d4ef-45ae-a184-8296185ef79b',), code = 358 (Failed with error StorageDomainDoesNotExist and code 358) 2015-04-28 12:59:19,819 INFO [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] Command [id=4e3a7128-ad1f-48fd-b871-ebf89d6cfb87]: Compensating CHANGED_STATUS_ONLY of org.ovirt.engine.core.common.businessentities.StoragePoolIsoMap; snapshot: EntityStatusSnapshot [id=storagePoolId = e7bdba88-e718-41a9-8d2b-0ca79c517630, storageId = 31ba6486-d4ef-45ae-a184-8296185ef79b, status=Inactive]. 2015-04-28 12:59:19,832 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (org.ovirt.thread.pool-8-thread-25) [450f2bb2] Correlation ID: 450f2bb2, Job ID: 982d9fcc-7b9c-4159-b8c5-ef9414c76dee, Call Stack: null, Custom Event ID: -1, Message: Failed to activate Storage Domain iron-export (Data Center SENSORED) by XXXXX@XXXX
-- Rik Theys System Engineer KU Leuven - Dept. Elektrotechniek (ESAT) Kasteelpark Arenberg 10 bus 2440 - B-3001 Leuven-Heverlee +32(0)16/32.11.07 ---------------------------------------------------------------- <<Any errors in spelling, tact or fact are transmission errors>>