[Users] Unable to activate data center after host reboot.
Steven Wiltshire
stevenwiltshire at gmail.com
Sun Apr 28 07:47:16 UTC 2013
Hello Vered,
Thanks for your email.
I'm happy to say that as of about an hour ago, the problem is solved -
it was a problem with a 3.6 kernel and VDSM 4.9 not playing nice with an
NFS storage domain.
As far as I was aware, the packages on the server had not been updated -
so I didn't think to check the kernel version. It turns out that at some
point the kernel had been updated but, of course, the update didn't
'kick in' until the machine was rebooted. It's a testing machine and a
few developers have root access - I guess one of them had, at some
point, done the deed and neglected to document it.
Thank you to everybody for their help pointing me in the right
direction. It was a trivial fix once the problem was identified.
Thanks again,
--Steven.
On 28/04/2013 4:17 PM, Vered Volansky wrote:
> Hi Steven,
>
> Please send the full vdsm.log .
> I'd like to see it from the moment the problem first occurred - when you got the following message -
> "No valid Data Storage Domains are available in Data Center Default
> (please check your storage infrastructure)".
>
> Best Regards,
> Vered
>
> ----- Original Message -----
>> From: "Steven Wiltshire" <stevenwiltshire at gmail.com>
>> To: users at ovirt.org
>> Sent: Saturday, April 27, 2013 6:58:54 PM
>> Subject: Re: [Users] Unable to activate data center after host reboot.
>>
>> Hi Ian,
>>
>> Thanks for pointing me in that direction. I found the following:
>>
>> ------------------------------------------------------------------
>> engine=# select * from storage_pool_iso_map;
>> storage_id | storage_pool_id |
>> status | owner
>> --------------------------------------+--------------------------------------+--------+-------
>> 025062dd-d69e-40ac-9031-c080b51647ae |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 5 | 0
>> 503e6677-2d51-4668-ab17-f914bdb57c19 |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0
>> d30604e3-3ca8-470a-8da4-18c9402f7504 |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0
>> (3 rows)
>> ------------------------------------------------------------------
>>
>> But changed it to using the query in the thread you suggested:
>>
>> ------------------------------------------------------------------
>> engine=# select * from storage_pool_iso_map;
>> storage_id | storage_pool_id |
>> status | owner
>> --------------------------------------+--------------------------------------+--------+-------
>> 025062dd-d69e-40ac-9031-c080b51647ae |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0
>> 503e6677-2d51-4668-ab17-f914bdb57c19 |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0
>> d30604e3-3ca8-470a-8da4-18c9402f7504 |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0
>> ------------------------------------------------------------------
>>
>> When I try to activate the data centre, it changes to:
>>
>> ------------------------------------------------------------------
>> engine=# select * from storage_pool_iso_map;
>> storage_id | storage_pool_id |
>> status | owner
>> --------------------------------------+--------------------------------------+--------+-------
>> 025062dd-d69e-40ac-9031-c080b51647ae |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 4 | 0
>> 503e6677-2d51-4668-ab17-f914bdb57c19 |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0
>> d30604e3-3ca8-470a-8da4-18c9402f7504 |
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84 | 0 | 0
>> ------------------------------------------------------------------
>>
>> However, I'm still unable to activate the data centre.Engine.log is
>> giving me the following:
>>
>> ------------------------------------------------------------------
>> 2013-04-28 00:55:33,297 INFO
>> [org.ovirt.engine.core.vdsbroker.VdsManager] (QuartzScheduler_Worker-5)
>> Initializing Host: BigServer
>> 2013-04-28 00:55:33,331 WARN
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-5) domain 503e6677-2d51-4668-ab17-f914bdb57c19
>> in problem. vds: BigServer
>> 2013-04-28 00:55:33,332 WARN
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-5) domain d30604e3-3ca8-470a-8da4-18c9402f7504
>> in problem. vds: BigServer
>> 2013-04-28 00:55:39,391 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-17) hostFromVds::selectedVds - BigServer,
>> spmStatus Unknown_Pool, storage pool Default
>> 2013-04-28 00:55:39,594 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
>> (QuartzScheduler_Worker-17) START, ConnectStoragePoolVDSCommand(vdsId =
>> aeac7aa6-8840-11e1-99aa-7fbd3e56e26e, storagePoolId =
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84, vds_spm_id = 1, masterDomainId =
>> 025062dd-d69e-40ac-9031-c080b51647ae, masterVersion = 1), log id: 132803ba
>> 2013-04-28 00:55:41,425 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (QuartzScheduler_Worker-17) Command
>> org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand
>> return value
>> Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>> mStatus Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>> mCode 304
>> mMessage Cannot find master domain:
>> 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84,
>> msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'
>>
>>
>> 2013-04-28 00:55:41,425 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (QuartzScheduler_Worker-17) Vds: BigServer
>> 2013-04-28 00:55:41,433 ERROR
>> [org.ovirt.engine.core.vdsbroker.VDSCommandBase]
>> (QuartzScheduler_Worker-17) Command ConnectStoragePoolVDS execution
>> failed. Exception: IRSNoMasterDomainException: IRSGenericException:
>> IRSErrorException: IRSNoMasterDomainException: Cannot find master
>> domain: 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84,
>> msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'
>> 2013-04-28 00:55:41,434 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
>> (QuartzScheduler_Worker-17) FINISH, ConnectStoragePoolVDSCommand, log
>> id: 132803ba
>> 2013-04-28 00:55:41,436 INFO
>> [org.ovirt.engine.core.utils.transaction.TransactionSupport]
>> (QuartzScheduler_Worker-17) transaction rolled back
>> 2013-04-28 00:55:41,436 ERROR
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-17) IrsBroker::Failed::GetStoragePoolInfoVDS
>> 2013-04-28 00:55:41,437 ERROR
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-17) Exception: IRSGenericException:
>> IRSErrorException: IRSNoMasterDomainException: Cannot find master
>> domain: 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84,
>> msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'
>> 2013-04-28 00:55:41,642 WARN
>> [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils]
>> (QuartzScheduler_Worker-17) Adding new value to configuration cache.
>> 2013-04-28 00:55:41,643 WARN
>> [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils]
>> (QuartzScheduler_Worker-17) Didnt find the value of DBI18NPrefix in DB
>> for version general - using default:
>> 2013-04-28 00:55:41,645 WARN
>> [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils]
>> (QuartzScheduler_Worker-17) Adding new value to configuration cache.
>> 2013-04-28 00:55:41,645 WARN
>> [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils]
>> (QuartzScheduler_Worker-17) Didnt find the value of DBPagingType in DB
>> for version general - using default: Offset
>> 2013-04-28 00:55:41,646 WARN
>> [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils]
>> (QuartzScheduler_Worker-17) Adding new value to configuration cache.
>> 2013-04-28 00:55:41,646 WARN
>> [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils]
>> (QuartzScheduler_Worker-17) Didnt find the value of DBPagingSyntax in DB
>> for version general - using default: OFFSET (%1$s -1) LIMIT %2$s
>> 2013-04-28 00:55:41,648 WARN
>> [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils]
>> (QuartzScheduler_Worker-17) Adding new value to configuration cache.
>> 2013-04-28 00:55:41,648 WARN
>> [org.ovirt.engine.core.dal.dbbroker.generic.DBConfigUtils]
>> (QuartzScheduler_Worker-17) Didnt find the value of DBSearchTemplate in
>> DB for version general - using default: SELECT * FROM (%2$s) %1$s) as T1
>> %3$s
>> 2013-04-28 00:55:41,692 INFO
>> [org.ovirt.engine.core.bll.storage.ReconstructMasterDomainCommand]
>> (QuartzScheduler_Worker-17) Running command:
>> ReconstructMasterDomainCommand internal: true. Entities affected : ID:
>> 025062dd-d69e-40ac-9031-c080b51647ae Type: Storage
>> 2013-04-28 00:55:41,765 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand]
>> (QuartzScheduler_Worker-17) START,
>> MarkPoolInReconstructModeVDSCommand(storagePoolId =
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84, ignoreFailoverLimit = false,
>> compatabilityVersion = null, reconstructMarkAction = ClearCache), log
>> id: 548c6afe
>> 2013-04-28 00:55:41,766 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-17) clearing cache for problematic entities in
>> pool 2b633f9a-8840-11e1-9ed7-17e6c491db84
>> 2013-04-28 00:55:41,766 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.MarkPoolInReconstructModeVDSCommand]
>> (QuartzScheduler_Worker-17) FINISH, MarkPoolInReconstructModeVDSCommand,
>> log id: 548c6afe
>> 2013-04-28 00:55:43,630 WARN
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-24) domain 503e6677-2d51-4668-ab17-f914bdb57c19
>> in problem. vds: BigServer
>> 2013-04-28 00:55:43,631 WARN
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-24) domain d30604e3-3ca8-470a-8da4-18c9402f7504
>> in problem. vds: BigServer
>> 2013-04-28 00:55:51,830 INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand]
>> (QuartzScheduler_Worker-39) hostFromVds::selectedVds - BigServer,
>> spmStatus Unknown_Pool, storage pool Default
>> 2013-04-28 00:55:51,845 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
>> (QuartzScheduler_Worker-39) START, ConnectStoragePoolVDSCommand(vdsId =
>> aeac7aa6-8840-11e1-99aa-7fbd3e56e26e, storagePoolId =
>> 2b633f9a-8840-11e1-9ed7-17e6c491db84, vds_spm_id = 1, masterDomainId =
>> 025062dd-d69e-40ac-9031-c080b51647ae, masterVersion = 1), log id: 3b237a89
>> 2013-04-28 00:55:53,628 INFO
>> [org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase]
>> (QuartzScheduler_Worker-39) Command
>> org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand
>> return value
>> Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusOnlyReturnForXmlRpc
>> mStatus Class Name:
>> org.ovirt.engine.core.vdsbroker.vdsbroker.StatusForXmlRpc
>> mCode 304
>> mMessage Cannot find master domain:
>> 'spUUID=2b633f9a-8840-11e1-9ed7-17e6c491db84,
>> msdUUID=025062dd-d69e-40ac-9031-c080b51647ae'
>> ------------------------------------------------------------------
>>
>> ... and so on.
>>
>> Any more help will be much appreciated - I'm utterly confused by what's
>> going on.
>>
>> Thanks again,
>>
>> --Steven.
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
More information about the Users
mailing list