[Users] Default storage pool won't activate after reboot

[Ian Levesque]

Hello,

Since a recent reboot, my default data storage pool has been inactive and attempts to activate it fail.

Installed RPMs:
  vdsm-4.9.6-0.150.git63061a9.el6.x86_64
  ovirt-engine-3.1.0_0001-1.8.el6.x86_64 

Relevant-looking output in the vdsm.log file (more @ http://pastebin.com/raw.php?i=cYaFLPEF):

Thread-68465::ERROR::2012-05-09 14:05:00,491::task::853::TaskManager.Task::(_setError) Task=`937d1fa9-4636-42f1-a6cb-c4cc56b9f2fd`::Unexpected error
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 861, in _run
    return fn(*args, **kargs)
  File "/usr/share/vdsm/logUtils.py", line 38, in wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 526, in getSpmStatus
    pool = self.getPool(spUUID)
  File "/usr/share/vdsm/storage/hsm.py", line 263, in getPool
    raise se.StoragePoolUnknown(spUUID)
StoragePoolUnknown: Unknown pool id, pool not connected: ('af5bcc86-898a-11e1-9632-003048c85226',)
Thread-68465::DEBUG::2012-05-09 14:05:00,491::task::872::TaskManager.Task::(_run) Task=`937d1fa9-4636-42f1-a6cb-c4cc56b9f2fd`::Task._run: 937d1fa9-4636-42f1-a6cb-c4cc56b9f2fd ('af5bcc86-898a-11e1-9632-003048c85226',) {} failed - stopping task


The share /is/ mounted on the node:

# mount | grep cmcd-db-vip
cmcd-db-vip.in.hwlab:/vmstore/ovirt on /rhev/data-center/mnt/cmcd-db-vip.in.hwlab:_vmstore_ovirt type nfs (rw,soft,nosharecache,timeo=600,retrans=6,nfsvers=3,addr=10.0.10.13)

# ls -l /rhev/data-center/mnt/cmcd-db-vip.in.hwlab:_vmstore_ovirt/
total 0
drwxr-xr-x 5 vdsm kvm 45 Apr 13 12:23 4ddb9dce-1ede-459a-9b9f-52b035622517
drwxr-xr-x 5 vdsm kvm 45 Apr 12 16:17 4ea29e75-ab27-4bc9-bc46-1740d07f4cba
drwxr-xr-x 5 vdsm kvm 45 Apr 19 11:40 e12a0f53-ee72-44bc-ad26-93f9b4613c6c

The closest google turns up is a thread from March, in which there was some mention of a change in the way metadata is read. Might be a red herring, but here's that thread: http://lists.ovirt.org/pipermail/users/2012-March/001423.html

Thanks for any insight!
Ian