Hello Vered, Thanks for your email. I'm happy to say that as of about an hour ago, the problem is solved - it was a problem with a 3.6 kernel and VDSM 4.9 not playing nice with an NFS storage domain. As far as I was aware, the packages on the server had not been updated - so I didn't think to check the kernel version. It turns out that at some point the kernel had been updated but, of course, the update didn't 'kick in' until the machine was rebooted. It's a testing machine and a few developers have root access - I guess one of them had, at some point, done the deed and neglected to document it. Thank you to everybody for their help pointing me in the right direction. It was a trivial fix once the problem was identified. Thanks again, --Steven. On 28/04/2013 4:17 PM, Vered Volansky wrote:
Hi Steven,
Please send the full vdsm.log . I'd like to see it from the moment the problem first occurred - when you got the following message - "No valid Data Storage Domains are available in Data Center Default (please check your storage infrastructure)".
Best Regards, Vered
----- Original Message -----
From: "Steven Wiltshire" <stevenwiltshire@gmail.com> To: users@ovirt.org Sent: Saturday, April 27, 2013 6:58:54 PM Subject: Re: [Users] Unable to activate data center after host reboot.
Hi Ian,
Thanks for pointing me in that direction. I found the following:
------------------------------------------------------------------ engine=# select * from storage_pool_iso_map; storage_id | storage_pool_id | status | owner --------------------------------------+--------------------------------------+--------+------- 025062dd-d69e-40ac-9031-c080b51647ae | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 5 | 0 503e6677-2d51-4668-ab17-f914bdb57c19 | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0 d30604e3-3ca8-470a-8da4-18c9402f7504 | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0 (3 rows) ------------------------------------------------------------------
But changed it to using the query in the thread you suggested:
------------------------------------------------------------------ engine=# select * from storage_pool_iso_map; storage_id | storage_pool_id | status | owner --------------------------------------+--------------------------------------+--------+------- 025062dd-d69e-40ac-9031-c080b51647ae | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0 503e6677-2d51-4668-ab17-f914bdb57c19 | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0 d30604e3-3ca8-470a-8da4-18c9402f7504 | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0 ------------------------------------------------------------------
When I try to activate the data centre, it changes to:
------------------------------------------------------------------ engine=# select * from storage_pool_iso_map; storage_id | storage_pool_id | status | owner --------------------------------------+--------------------------------------+--------+------- 025062dd-d69e-40ac-9031-c080b51647ae | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 4 | 0 503e6677-2d51-4668-ab17-f914bdb57c19 | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0 d30604e3-3ca8-470a-8da4-18c9402f7504 | 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0 ------------------------------------------------------------------
However, I'm still unable to activate the data centre.Engine.log is giving me the following:
------------------------------------------------------------------ 2013-04-28 00:55:33,297 INFO [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-5) Initializing Host: BigServer 2013-04-28 00:55:33,331 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) domain 503e6677-2d51-4668-ab17-f914bdb57c19 in problem. vds: BigServer 2013-04-28 00:55:33,332 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-5) domain d30604e3-3ca8-470a-8da4-18c9402f7504 in problem. vds: BigServer 2013-04-28 00:55:39,391 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) hostFromVds::selectedVds - BigServer, spmStatus Unknown_Pool, storage pool Default 2013-04-28 00:55:39,594 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-17) START, ConnectStoragePoolVDSCommand(vdsId = aeac7aa6-8840-11e1-99aa-7fbd3e56e26e, storagePoolId = 2b633f9a-8840-11e1-9ed7-17e6c491db84, vds_spm_id = 1, masterDomainId = 025062dd-d69e-40ac-9031-c080b51647ae, masterVersion = 1), log id: 132803ba 2013-04-28 00:55:41,425 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-17) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 304 mMessage Cannot find master domain: 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84, msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'
2013-04-28 00:55:41,425 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-17) Vds: BigServer 2013-04-28 00:55:41,433 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (QuartzScheduler_Worker-17) Command ConnectStoragePoolVDS execution failed. Exception: IRSNoMasterDomainException: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84, msdUUID=025062dd-d69e-40ac-9031-c080b51647ae' 2013-04-28 00:55:41,434 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-17) FINISH, ConnectStoragePoolVDSCommand, log id: 132803ba 2013-04-28 00:55:41,436 INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (QuartzScheduler_Worker-17) transaction rolled back 2013-04-28 00:55:41,436 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) IrsBroker::Failed::GetStoragePoolInfoVDS 2013-04-28 00:55:41,437 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) Exception: IRSGenericException: IRSErrorException: IRSNoMasterDomainException: Cannot find master domain: 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84, msdUUID=025062dd-d69e-40ac-9031-c080b51647ae' 2013-04-28 00:55:41,642 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-17) Adding new value to configuration cache. 2013-04-28 00:55:41,643 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-17) Didnt find the value of DBI18NPrefix in DB for version general - using default: 2013-04-28 00:55:41,645 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-17) Adding new value to configuration cache. 2013-04-28 00:55:41,645 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-17) Didnt find the value of DBPagingType in DB for version general - using default: Offset 2013-04-28 00:55:41,646 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-17) Adding new value to configuration cache. 2013-04-28 00:55:41,646 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-17) Didnt find the value of DBPagingSyntax in DB for version general - using default: OFFSET (%1$s -1) LIMIT %2$s 2013-04-28 00:55:41,648 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-17) Adding new value to configuration cache. 2013-04-28 00:55:41,648 WARN [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils] (QuartzScheduler_Worker-17) Didnt find the value of DBSearchTemplate in DB for version general - using default: SELECT * FROM (%2$s) %1$s) as T1 %3$s 2013-04-28 00:55:41,692 INFO [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand] (QuartzScheduler_Worker-17) Running command: ReconstructMasterDomainCommand internal: true. Entities affected : ID: 025062dd-d69e-40ac-9031-c080b51647ae Type: Storage 2013-04-28 00:55:41,765 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-17) START, MarkPoolInReconstructModeVDSCommand(storagePoolId = 2b633f9a-8840-11e1-9ed7-17e6c491db84, ignoreFailoverLimit = false, compatabilityVersion = null, reconstructMarkAction = ClearCache), log id: 548c6afe 2013-04-28 00:55:41,766 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-17) clearing cache for problematic entities in pool 2b633f9a-8840-11e1-9ed7-17e6c491db84 2013-04-28 00:55:41,766 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand] (QuartzScheduler_Worker-17) FINISH, MarkPoolInReconstructModeVDSCommand, log id: 548c6afe 2013-04-28 00:55:43,630 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) domain 503e6677-2d51-4668-ab17-f914bdb57c19 in problem. vds: BigServer 2013-04-28 00:55:43,631 WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-24) domain d30604e3-3ca8-470a-8da4-18c9402f7504 in problem. vds: BigServer 2013-04-28 00:55:51,830 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (QuartzScheduler_Worker-39) hostFromVds::selectedVds - BigServer, spmStatus Unknown_Pool, storage pool Default 2013-04-28 00:55:51,845 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (QuartzScheduler_Worker-39) START, ConnectStoragePoolVDSCommand(vdsId = aeac7aa6-8840-11e1-99aa-7fbd3e56e26e, storagePoolId = 2b633f9a-8840-11e1-9ed7-17e6c491db84, vds_spm_id = 1, masterDomainId = 025062dd-d69e-40ac-9031-c080b51647ae, masterVersion = 1), log id: 3b237a89 2013-04-28 00:55:53,628 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase] (QuartzScheduler_Worker-39) Command org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand return value Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc mStatus Class Name: org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc mCode 304 mMessage Cannot find master domain: 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84, msdUUID=025062dd-d69e-40ac-9031-c080b51647ae' ------------------------------------------------------------------
... and so on.
Any more help will be much appreciated - I'm utterly confused by what's going on.
Thanks again,
--Steven.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users