[ovirt-users] storage issue's with oVirt 3.5.1 + Nexenta NFS

Tue Apr 21 14:43:37 UTC 2015


Am 21.04.2015 um 16:09 schrieb Maikel vd Mosselaar:
> Hi Fred,
> 
> 
> This is one of the nodes from yesterday around 01:00 (20-04-15). The
> issue started around 01:00.
> https://bpaste.net/raw/67542540a106
> 
> The VDSM logs are very big so i am unable to paste a bigger part of the
> logfile, i don't know what the maximum allowed attachment size is of the
> mailing list?
> 
> dmesg on the one the nodes (despite this message the storage is still
> accessible):
> https://bpaste.net/raw/67da167aa300
> 
Flaky Network? NFS / Lockd Processes saturated @ Nexenta?

> 
> 
> Kind regards,
> 
> Maikel
> 
> On 04/21/2015 02:32 PM, Fred Rolland wrote:
>> Hi,
>>
>> Can you please attach VDSM logs ?
>>
>> Thanks,
>>
>> Fred
>>
>> ----- Original Message -----
>>> From: "Maikel vd Mosselaar" <m.vandemosselaar at smoose.nl>
>>> To: users at ovirt.org
>>> Sent: Monday, April 20, 2015 3:25:38 PM
>>> Subject: [ovirt-users] storage issue's with oVirt 3.5.1 + Nexenta NFS
>>>
>>> Hi,
>>>
>>> We are running ovirt 3.5.1 with 3 nodes and seperate engine.
>>>
>>> All on CentOS 6.6:
>>> 3 x nodes
>>> 1 x engine
>>>
>>> 1 x storage nexenta with NFS
>>>
>>> For multiple weeks we are experiencing issues of our nodes that cannot
>>> access the storage at random moments (atleast thats what the nodes
>>> think).
>>>
>>> When the nodes are complaining about a unavailable storage then the load
>>> rises up to +200 on all three nodes, this causes that all running VMs
>>> are unaccessible. During this process oVirt event viewer shows some i/o
>>> storage error messages, when this happens random VMs get paused and will
>>> not be resumed anymore (this almost happens every time but not all the
>>> VMs get paused).
>>>
>>> During the event we tested the accessibility from the nodes to the
>>> storage and it looks like it is working normal, at least we can do a
>>> normal
>>> "ls" on the storage without any delay of showing the contents.
>>>
>>> We tried multiple things that we thought it causes this issue but
>>> nothing worked so far.
>>> * rebooting storage / nodes / engine.
>>> * disabling offsite rsync backups.
>>> * moved the biggest VMs with highest load to different platform outside
>>> of oVirt.
>>> * checked the wsize and rsize on the nfs mounts, storage and nodes are
>>> correct according to the "NFS troubleshooting page" on ovirt.org.
>>>
>>> The environment is running in production so we are not free to test
>>> everything.
>>>
>>> I can provide log files if needed.
>>>
>>> Kind Regards,
>>>
>>> Maikel
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users