[Users] Unable to activate iSCSI domain after crash of host

Fri Feb 7 08:53:52 UTC 2014

the host can't see the domain
If you fixed everything and the host can now see the domain this mail be 
a cache issue - can you please reboot the host?

Dafna

On 02/07/2014 07:21 AM, Gianluca Cecchi wrote:
> Hello,
> Fedora 19 with 3.3.3.
> Only one host configured.
> After crash of host I'm not able to activate storage domain again.
>
> Any way to recover?
> Gianluca
>
> in engine.log
> 2014-02-07 08:11:12,602 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
> (DefaultQuartzScheduler_Worker-32) [
> 60d513d1] HostName = ovnode03
> 2014-02-07 08:11:12,602 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand]
> (DefaultQuartzScheduler_Worker-32) [
> 60d513d1] Command HSMGetAllTasksStatusesVDS execution failed.
> Exception: IRSNonOperationalException: IRSGenericException:
> IRSErrorException: IR
> SNonOperationalException: Not SPM: ()
> 2014-02-07 08:11:12,613 INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
> (DefaultQuartzScheduler_Worker-32) [60d513d1] hostFr
> omVds::selectedVds - ovnode03, spmStatus Unknown_Pool, storage pool ISCSI
> 2014-02-07 08:11:12,615 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (DefaultQuartzScheduler_Worker-32) [60d5
> 13d1] START, ConnectStoragePoolVDSCommand(HostName = ovnode03, HostId
> = b6f8f68f-4f9e-4c87-918b-aa1ff60f575a, storagePoolId =
> 546cd29c-7249-473
> 3-8fd5-317cff38ed71, vds_spm_id = 1, masterDomainId =
> f741671e-6480-4d7b-b357-8cf6e8d2c0f1, masterVersion = 2), log id:
> 3e99a2c6
> 2014-02-07 08:11:15,747 INFO
> [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand]
> (ajp--127.0.0.1-8702-4) [465b0976] Lock Acquired
>   to object EngineLock [exclusiveLocks= key:
> f741671e-6480-4d7b-b357-8cf6e8d2c0f1 value: STORAGE
> , sharedLocks= ]
> 2014-02-07 08:11:15,759 INFO
> [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand]
> (pool-6-thread-49) [465b0976] Running command: A
> ctivateStorageDomainCommand internal: false. Entities affected :  ID:
> f741671e-6480-4d7b-b357-8cf6e8d2c0f1 Type: Storage
> 2014-02-07 08:11:15,762 INFO
> [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand]
> (pool-6-thread-49) [465b0976] Lock freed to obje
> ct EngineLock [exclusiveLocks= key:
> f741671e-6480-4d7b-b357-8cf6e8d2c0f1 value: STORAGE
> , sharedLocks= ]
> 2014-02-07 08:11:15,763 INFO
> [org.ovirt.engine.core.bll.storage.ActivateStorageDomainCommand]
> (pool-6-thread-49) [465b0976] ActivateStorage Do
> main. Before Connect all hosts to pool. Time:2/7/14 8:11 AM
> 2014-02-07 08:11:15,765 INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.ActivateStorageDomainVDSCommand]
> (pool-6-thread-49) [465b0976] START,
> ActivateStorageDomainVDSCommand( storagePoolId =
> 546cd29c-7249-4733-8fd5-317cff38ed71, ignoreFailoverLimit = false,
> storageDomainId = f741671e-
> 6480-4d7b-b357-8cf6e8d2c0f1), log id: da4b270
> 2014-02-07 08:11:16,739 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (DefaultQuartzScheduler_Worker-32) [60d5
> 13d1] Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand
> return value
>   StatusOnlyReturnForXmlRpc [mStatus=StatusForXmlRpc [mCode=304,
> mMessage=Cannot find master domain:
> 'spUUID=546cd29c-7249-4733-8fd5-317cff38ed7
> 1, msdUUID=f741671e-6480-4d7b-b357-8cf6e8d2c0f1']]
> 2014-02-07 08:11:16,740 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (DefaultQuartzScheduler_Worker-32) [60d5
> 13d1] HostName = ovnode03
> 2014-02-07 08:11:16,740 ERROR
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (DefaultQuartzScheduler_Worker-32) [60d5
> 13d1] Command ConnectStoragePoolVDS execution failed. Exception:
> IRSNoMasterDomainException: IRSGenericException: IRSErrorException:
> IRSNoMaste
> rDomainException: Cannot find master domain:
> 'spUUID=546cd29c-7249-4733-8fd5-317cff38ed71,
> msdUUID=f741671e-6480-4d7b-b357-8cf6e8d2c0f1'
> 2014-02-07 08:11:16,741 INFO
> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
> (DefaultQuartzScheduler_Worker-32) [60d5
> 13d1] FINISH, ConnectStoragePoolVDSCommand, log id: 3e99a2c6
>
>
> In vdsm.log I get:
>
> Thread-85157::ERROR::2014-02-07
> 08:12:44,774::task::850::TaskManager.Task::(_setError)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5
> ffe675`::Unexpected error
> Traceback (most recent call last):
>    File "/usr/share/vdsm/storage/task.py", line 857, in _run
>      return fn(*args, **kargs)
>    File "/usr/share/vdsm/logUtils.py", line 45, in wrapper
>      res = f(*args, **kwargs)
>    File "/usr/share/vdsm/storage/hsm.py", line 1008, in connectStoragePool
>      masterVersion, options)
>    File "/usr/share/vdsm/storage/hsm.py", line 1062, in _connectStoragePool
>      res = pool.connect(hostID, scsiKey, msdUUID, masterVersion)
>    File "/usr/share/vdsm/storage/sp.py", line 699, in connect
>      self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
>    File "/usr/share/vdsm/storage/sp.py", line 1244, in __rebuild
>      masterVersion=masterVersion)
>    File "/usr/share/vdsm/storage/sp.py", line 1603, in getMasterDomain
>      raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID)
> StoragePoolMasterNotFound: Cannot find master domain:
> 'spUUID=546cd29c-7249-4733-8fd5-317cff38ed71, msdUUID=f741671e-6480-4
> d7b-b357-8cf6e8d2c0f1'
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,774::task::869::TaskManager.Task::(_run)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe67
> 5`::Task._run: b9f3c2b5-18fa-4135-96f7-c152b5ffe675
> ('546cd29c-7249-4733-8fd5-317cff38ed71', 1,
> '546cd29c-7249-4733-8fd5-31
> 7cff38ed71', 'f741671e-6480-4d7b-b357-8cf6e8d2c0f1', 2) {} failed -
> stopping task
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,775::task::1194::TaskManager.Task::(stop)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe6
> 75`::stopping in state preparing (force False)
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,775::task::974::TaskManager.Task::(_decref)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ff
> e675`::ref 1 aborting True
> Thread-85157::INFO::2014-02-07
> 08:12:44,775::task::1151::TaskManager.Task::(prepare)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe675`::aborting: Task is
> aborted: 'Cannot find master domain' - code 304
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,775::task::1156::TaskManager.Task::(prepare)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe675`::Prepare: aborted: Cannot
> find master domain
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,776::task::974::TaskManager.Task::(_decref)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe675`::ref 0 aborting True
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,776::task::909::TaskManager.Task::(_doAbort)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe675`::Task._doAbort: force
> False
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,776::resourceManager::976::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,776::task::579::TaskManager.Task::(_updateState)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe675`::moving from state
> preparing -> state aborting
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,777::task::534::TaskManager.Task::(__state_aborting)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe675`::_aborting: recover policy
> none
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,777::task::579::TaskManager.Task::(_updateState)
> Task=`b9f3c2b5-18fa-4135-96f7-c152b5ffe675`::moving from state
> aborting -> state failed
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,777::resourceManager::939::ResourceManager.Owner::(releaseAll)
> Owner.releaseAll requests {} resources {}
> Thread-85157::DEBUG::2014-02-07
> 08:12:44,777::resourceManager::976::ResourceManager.Owner::(cancelAll)
> Owner.cancelAll requests {}
> Thread-85157::ERROR::2014-02-07
> 08:12:44,777::dispatcher::67::Storage.Dispatcher.Protect::(run)
> {'status': {'message': "Cannot find master domain:
> 'spUUID=546cd29c-7249-4733-8fd5-317cff38ed71,
> msdUUID=f741671e-6480-4d7b-b357-8cf6e8d2c0f1'", 'code
> ': 304}}
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

-- 
Dafna Ron