[Users] latest vdsm cannot read ib device speeds causing storage attach fail

[Dead Horse]

Indeed reverting back to an older vdsm clears up the above issue. However
now I the issue is see is:
Thread-18::ERROR::2013-01-23
15:50:42,885::task::833::TaskManager.Task::(_setError)
Task=`08709e68-bcbc-40d8-843a-d69d4df40ac6`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 840, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 42, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 923, in connectStoragePool
    masterVersion, options)
  File "/usr/share/vdsm/storage/hsm.py", line 970, in _connectStoragePool
    res = pool.connect(hostID, scsiKey, msdUUID, masterVersion)
  File "/usr/share/vdsm/storage/sp.py", line 643, in connect
    self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
  File "/usr/share/vdsm/storage/sp.py", line 1167, in __rebuild
    self.masterDomain = self.getMasterDomain(msdUUID=msdUUID,
masterVersion=masterVersion)
  File "/usr/share/vdsm/storage/sp.py", line 1506, in getMasterDomain
    raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID)
StoragePoolMasterNotFound: Cannot find master domain:
'spUUID=f90a0d1c-06ca-11e2-a05b-00151712f280,
msdUUID=67534cca-1327-462a-b455-a04464084b31'
Thread-18::DEBUG::2013-01-23
15:50:42,887::task::852::TaskManager.Task::(_run)
Task=`08709e68-bcbc-40d8-843a-d69d4df40ac6`::Task._run:
08709e68-bcbc-40d8-843a-d69d4df40ac6
('f90a0d1c-06ca-11e2-a05b-00151712f280', 2,
'f90a0d1c-06ca-11e2-a05b-00151712f280',
'67534cca-1327-462a-b455-a04464084b31', 433) {} failed - stopping task

This is with vdsm built from
commit25a2d8572ad32352227c98a86631300fbd6523c1
- DHC


On Wed, Jan 23, 2013 at 10:44 AM, Dead Horse
<deadhorseconsulting at gmail.com>wrote:

> VDSM was built from:
> commit 166138e37e75767b32227746bb671b1dab9cdd5e
>
> Attached is the full vdsm log
>
> I should also note that from engine perspective it sees the master storage
> domain as locked and the others as unknown.
>
>
> On Wed, Jan 23, 2013 at 2:49 AM, Dan Kenigsberg <danken at redhat.com> wrote:
>
>> On Tue, Jan 22, 2013 at 04:02:24PM -0600, Dead Horse wrote:
>> > Any ideas on this one? (from VDSM log):
>> > Thread-25::DEBUG::2013-01-22
>> > 15:35:29,065::BindingXMLRPC::914::vds::(wrapper) client
>> [3.57.111.30]::call
>> > getCapabilities with () {}
>> > Thread-25::ERROR::2013-01-22 15:35:29,113::netinfo::159::root::(speed)
>> > cannot read ib0 speed
>> > Traceback (most recent call last):
>> >   File "/usr/lib64/python2.6/site-packages/vdsm/netinfo.py", line 155,
>> in
>> > speed
>> >     s = int(file('/sys/class/net/%s/speed' % dev).read())
>> > IOError: [Errno 22] Invalid argument
>> >
>> > Causes VDSM to fail to attach storage
>>
>> I doubt that this is the cause of the failure, as vdsm has always
>> reported "0" for ib devices, and still is.
>>
>> Does a former version works with your Engine?
>> Could you share more of your vdsm.log? I suppose the culprit lies in one
>> one of the storage-related commands, not in statistics retrieval.
>>
>> >
>> > Engine side sees:
>> > ERROR [org.ovirt.engine.core.bll.storage.NFSStorageHelper]
>> > (QuartzScheduler_Worker-96) [553ef26e] The connection with details
>> > 192.168.0.1:/ovirt/ds failed because of error code 100 and error
>> message
>> > is: general exception
>> > 2013-01-22 15:35:30,160 INFO
>> > [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand]
>> > (QuartzScheduler_Worker-96) [1ab78378] Running command:
>> > SetNonOperationalVdsCommand internal: true. Entities affected :  ID:
>> > 8970b3fe-1faf-11e2-bc1f-00151712f280 Type: VDS
>> > 2013-01-22 15:35:30,200 INFO
>> > [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
>> > (QuartzScheduler_Worker-96) [1ab78378] START,
>> > SetVdsStatusVDSCommand(HostName = kezan, HostId =
>> > 8970b3fe-1faf-11e2-bc1f-00151712f280, status=NonOperational,
>> > nonOperationalReason=STORAGE_DOMAIN_UNREACHABLE), log id: 4af5c4cd
>> > 2013-01-22 15:35:30,211 INFO
>> > [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand]
>> > (QuartzScheduler_Worker-96) [1ab78378] FINISH, SetVdsStatusVDSCommand,
>> log
>> > id: 4af5c4cd
>> > 2013-01-22 15:35:30,242 ERROR
>> > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>> > (QuartzScheduler_Worker-96) [1ab78378] Try to add duplicate audit log
>> > values with the same name. Type: VDS_SET_NONOPERATIONAL_DOMAIN. Value:
>> > storagepoolname
>> >
>> > Engine = latest master
>> > VDSM = latest master
>>
>> Since "latest master" is an unstable reference by definition, I'm sure
>> that History would thank you if you post the exact version (git hash?)
>> of the code.
>>
>> > node = el6
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20130123/16e895ab/attachment-0001.html>