Hi Fred, This is one of the nodes from yesterday around 01:00 (20-04-15). The issue started around 01:00. https://bpaste.net/raw/67542540a106 The VDSM logs are very big so i am unable to paste a bigger part of the logfile, i don't know what the maximum allowed attachment size is of the mailing list? dmesg on the one the nodes (despite this message the storage is still accessible): https://bpaste.net/raw/67da167aa300 Kind regards, Maikel On 04/21/2015 02:32 PM, Fred Rolland wrote:
Hi,
Can you please attach VDSM logs ?
Thanks,
Fred
----- Original Message -----
From: "Maikel vd Mosselaar" <m.vandemosselaar@smoose.nl> To: users@ovirt.org Sent: Monday, April 20, 2015 3:25:38 PM Subject: [ovirt-users] storage issue's with oVirt 3.5.1 + Nexenta NFS
Hi,
We are running ovirt 3.5.1 with 3 nodes and seperate engine.
All on CentOS 6.6: 3 x nodes 1 x engine
1 x storage nexenta with NFS
For multiple weeks we are experiencing issues of our nodes that cannot access the storage at random moments (atleast thats what the nodes think).
When the nodes are complaining about a unavailable storage then the load rises up to +200 on all three nodes, this causes that all running VMs are unaccessible. During this process oVirt event viewer shows some i/o storage error messages, when this happens random VMs get paused and will not be resumed anymore (this almost happens every time but not all the VMs get paused).
During the event we tested the accessibility from the nodes to the storage and it looks like it is working normal, at least we can do a normal "ls" on the storage without any delay of showing the contents.
We tried multiple things that we thought it causes this issue but nothing worked so far. * rebooting storage / nodes / engine. * disabling offsite rsync backups. * moved the biggest VMs with highest load to different platform outside of oVirt. * checked the wsize and rsize on the nfs mounts, storage and nodes are correct according to the "NFS troubleshooting page" on ovirt.org.
The environment is running in production so we are not free to test everything.
I can provide log files if needed.
Kind Regards,
Maikel
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users