[ovirt-users] Storage domains not found by vdsm after a reboot

[Charles Kozler]

I am facing the same issue here as well. The engine comes up and web UI is
reachable. Initial login takes about 6 minutes to finally let me in and
then once I am in under the events tab there is events for "storage domain
<uid> does not exist" yet they are all there. After this comes
'reconstructing master domain' and it tries to cycle through my 2 storage
domains not including ISO_UPLOAD and hosted_engine domains. Eventually it
will either 1.) Settle on one and actually able to bring it up master
domain or 2.) they all stay down and I have to manually activate one

Its not really an issue since on three tests now I have recovered fine but
it required some manual intervention on at least one occasion but otherwise
it just flaps about until it can settle on one and actually bring it up

Clocking it today its usually like this:

7 minutes for HE to come up on node 1 and access to web UI
+6 minutes while hanging on logging in to web UI
+9 minutes for one of the two storage domains to get activated as master

Total around 20 minutes before entire cluster is usable.

On Sat, Dec 3, 2016 at 11:14 AM, Yoann Laissus <yoann.laissus at gmail.com>
wrote:

> Hello,
>
> I'm running into some weird issues with vdsm and my storage domains
> after a reboot or a shutdown. I can't manage to figure out what's
> going on...
>
> Currently, my cluster (4.0.5 with hosted engine) is composed of one
> main node. (and another inactive one but unrelated to this issue).
> It has local storage exposed to oVirt via 3 NFS exports (one specific
> for the hosted engine vm) reachable from my local network.
>
> When I wan't to shutdown or reboot my main host (and so the whole
> cluster), I use a custom script :
> 1. Shutdown all VM
> 2. Shutdown engine VM
> 3. Stop HA agent and broker
> 4. Stop vdsmd
> 5. Release the sanlock on the hosted engine SD
> 6. Shutdown / Reboot
>
> It works just fine, but at the next boot, VDSM takes at least 10-15
> minutes to find storage domains, except the hosted engine one. The
> engine loops trying to reconstruct the SPM.
> During this time, vdsClient getConnectedStoragePoolsList returns nothing.
> getStorageDomainsList returns only the hosted engine domain.
> NFS exports are mountable from another server.
>
> But when I restart vdsm manually after the boot, it seems to detect
> immediately the storage domains.
>
> Is there some kind of staled storage data used by vdsm and a timeout
> to invalidate them ?
> Am I missing something on the vdsm side in my shutdown procedure ?
>
> Thanks !
>
> Engine and vdsm logs are attached.
>
>
> --
> Yoann Laissus
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161203/23648ca5/attachment-0001.html>