[Users] Unable to activate data center after host reboot.

Sat Apr 27 11:58:54 EDT 2013

Hi Ian,

Thanks for pointing me in that direction. I found the following:

------------------------------------------------------------------
engine=# select * from storage_pool_iso_map;
               storage_id              | storage_pool_id            | 
status | owner
--------------------------------------+--------------------------------------+--------+-------
  025062dd-d69e-40ac-9031-c080b51647ae | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      5 |     0
  503e6677-2d51-4668-ab17-f914bdb57c19 | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      0 |     0
  d30604e3-3ca8-470a-8da4-18c9402f7504 | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      0 |     0
(3 rows)
------------------------------------------------------------------

But changed it to using the query in the thread you suggested:

------------------------------------------------------------------
engine=# select * from storage_pool_iso_map;
               storage_id              | storage_pool_id            | 
status | owner
--------------------------------------+--------------------------------------+--------+-------
  025062dd-d69e-40ac-9031-c080b51647ae | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      0 |     0
  503e6677-2d51-4668-ab17-f914bdb57c19 | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      0 |     0
  d30604e3-3ca8-470a-8da4-18c9402f7504 | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      0 |     0
------------------------------------------------------------------

When I try to activate the data centre, it changes to:

------------------------------------------------------------------
engine=# select * from storage_pool_iso_map;
               storage_id              | storage_pool_id            | 
status | owner
--------------------------------------+--------------------------------------+--------+-------
  025062dd-d69e-40ac-9031-c080b51647ae | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      4 |     0
  503e6677-2d51-4668-ab17-f914bdb57c19 | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      0 |     0
  d30604e3-3ca8-470a-8da4-18c9402f7504 | 
2b633f9a-8840-11e1-9ed7-17e6c491db84 |      0 |     0
------------------------------------------------------------------

However, I'm still unable to activate the data centre.Engine.log is 
giving me the following:

------------------------------------------------------------------
2013-04-28 00:55:33,297 INFO 
[org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-5) 
Initializing Host: BigServer
2013-04-28 00:55:33,331 WARN 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-5) domain 503e6677-2d51-4668-ab17-f914bdb57c19 
in problem. vds: BigServer
2013-04-28 00:55:33,332 WARN 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-5) domain d30604e3-3ca8-470a-8da4-18c9402f7504 
in problem. vds: BigServer
2013-04-28 00:55:39,391 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-17) hostFromVds::selectedVds - BigServer, 
spmStatus Unknown_Pool, storage pool Default
2013-04-28 00:55:39,594 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] 
(QuartzScheduler_Worker-17) START, ConnectStoragePoolVDSCommand(vdsId = 
aeac7aa6-8840-11e1-99aa-7fbd3e56e26e, storagePoolId = 
2b633f9a-8840-11e1-9ed7-17e6c491db84, vds_spm_id = 1, masterDomainId = 
025062dd-d69e-40ac-9031-c080b51647ae, masterVersion = 1), log id: 132803ba
2013-04-28 00:55:41,425 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] 
(QuartzScheduler_Worker-17) Command 
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand 
return value
  Class Name: 
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
mStatus                       Class Name: 
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
mCode                         304
mMessage                      Cannot find master domain: 
'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84, 
msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'

2013-04-28 00:55:41,425 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] 
(QuartzScheduler_Worker-17) Vds: BigServer
2013-04-28 00:55:41,433 ERROR 
[org.ovirt.engine.core.vdsbroker.VDSCommandBase] 
(QuartzScheduler_Worker-17) Command ConnectStoragePoolVDS execution 
failed. Exception: IRSNoMasterDomainException: IRSGenericException: 
IRSErrorException: IRSNoMasterDomainException: Cannot find master 
domain: 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84, 
msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'
2013-04-28 00:55:41,434 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] 
(QuartzScheduler_Worker-17) FINISH, ConnectStoragePoolVDSCommand, log 
id: 132803ba
2013-04-28 00:55:41,436 INFO 
[org.ovirt.engine.core.utils.transaction.TransactionSupport] 
(QuartzScheduler_Worker-17) transaction rolled back
2013-04-28 00:55:41,436 ERROR 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-17) IrsBroker::Failed::GetStoragePoolInfoVDS
2013-04-28 00:55:41,437 ERROR 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-17) Exception: IRSGenericException: 
IRSErrorException: IRSNoMasterDomainException: Cannot find master 
domain: 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84, 
msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'
2013-04-28 00:55:41,642 WARN 
[org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] 
(QuartzScheduler_Worker-17) Adding new value to configuration cache.
2013-04-28 00:55:41,643 WARN 
[org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] 
(QuartzScheduler_Worker-17) Didnt find the value of DBI18NPrefix in DB 
for version general - using default:
2013-04-28 00:55:41,645 WARN 
[org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] 
(QuartzScheduler_Worker-17) Adding new value to configuration cache.
2013-04-28 00:55:41,645 WARN 
[org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] 
(QuartzScheduler_Worker-17) Didnt find the value of DBPagingType in DB 
for version general - using default: Offset
2013-04-28 00:55:41,646 WARN 
[org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] 
(QuartzScheduler_Worker-17) Adding new value to configuration cache.
2013-04-28 00:55:41,646 WARN 
[org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] 
(QuartzScheduler_Worker-17) Didnt find the value of DBPagingSyntax in DB 
for version general - using default: OFFSET (%1$s -1) LIMIT %2$s
2013-04-28 00:55:41,648 WARN 
[org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] 
(QuartzScheduler_Worker-17) Adding new value to configuration cache.
2013-04-28 00:55:41,648 WARN 
[org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] 
(QuartzScheduler_Worker-17) Didnt find the value of DBSearchTemplate in 
DB for version general - using default: SELECT * FROM (%2$s) %1$s) as T1 
%3$s
2013-04-28 00:55:41,692 INFO 
[org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] 
(QuartzScheduler_Worker-17) Running command: 
ReconstructMasterDomainCommand internal: true. Entities affected :  ID: 
025062dd-d69e-40ac-9031-c080b51647ae Type: Storage
2013-04-28 00:55:41,765 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] 
(QuartzScheduler_Worker-17) START, 
MarkPoolInReconstructModeVDSCommand(storagePoolId = 
2b633f9a-8840-11e1-9ed7-17e6c491db84, ignoreFailoverLimit = false, 
compatabilityVersion = null, reconstructMarkAction = ClearCache), log 
id: 548c6afe
2013-04-28 00:55:41,766 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-17) clearing cache for problematic entities in 
pool 2b633f9a-8840-11e1-9ed7-17e6c491db84
2013-04-28 00:55:41,766 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] 
(QuartzScheduler_Worker-17) FINISH, MarkPoolInReconstructModeVDSCommand, 
log id: 548c6afe
2013-04-28 00:55:43,630 WARN 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-24) domain 503e6677-2d51-4668-ab17-f914bdb57c19 
in problem. vds: BigServer
2013-04-28 00:55:43,631 WARN 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-24) domain d30604e3-3ca8-470a-8da4-18c9402f7504 
in problem. vds: BigServer
2013-04-28 00:55:51,830 INFO 
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] 
(QuartzScheduler_Worker-39) hostFromVds::selectedVds - BigServer, 
spmStatus Unknown_Pool, storage pool Default
2013-04-28 00:55:51,845 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] 
(QuartzScheduler_Worker-39) START, ConnectStoragePoolVDSCommand(vdsId = 
aeac7aa6-8840-11e1-99aa-7fbd3e56e26e, storagePoolId = 
2b633f9a-8840-11e1-9ed7-17e6c491db84, vds_spm_id = 1, masterDomainId = 
025062dd-d69e-40ac-9031-c080b51647ae, masterVersion = 1), log id: 3b237a89
2013-04-28 00:55:53,628 INFO 
[org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] 
(QuartzScheduler_Worker-39) Command 
org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand 
return value
  Class Name: 
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
mStatus                       Class Name: 
org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
mCode                         304
mMessage                      Cannot find master domain: 
'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84, 
msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'
------------------------------------------------------------------

... and so on.

Any more help will be much appreciated - I'm utterly confused by what's 
going on.

Thanks again,

--Steven.