[ovirt-users] Ovirt 3.5 host gluster storage connection failure

[Ravishankar N]

On 12/23/2015 11:00 PM, Steve Dainard wrote:
> I've attached the client gluster log starting at the first log of the
> same day as failure.
Nothing significant in the client log after the crash and subsequent 
remount. The ENODATA warnings can be ignored. There was a patch 
(http://review.gluster.org/#/c/12015/) to change the log level, let me 
check if it has made it to a recent release version.
>
> The brick logs are all 0 file size on all of the replica 3 nodes... I
> just set 'gluster volume set vm-storage diagnostics.brick-log-level
> WARNING' but I'm not immediately seeing any logging to disk.
>
> I've also attached the compute1 vdsm.log file, which over the same
> time period is able to dd successfully so perhaps this discounts a
> storage side issue? I've also attached compute2 (failed node) for
> comparison.
>
>
> Ravi - I'm not familiar with core files, would this be in a non-devel
> version of gluster? Or is this something I can enable? I don't mind
> enabling it now if it could help diagnose a future issue.

You should get the core files even on the non-devel versions. I think 
the method to enable core files and its location  is distribution 
specific (depending on whether it uses abrt, systemd etc.)  but you can 
check for ulimit, and /proc/sys/kernel/core_pattern settings in general. 
On my Fedora 23 machine, I get them on /core.pid_of_the_process>.

The timestamp from the logs should also help in identifying the core file:

frame : type(0) op(0)
patchset: git://git.gluster.com/glusterfs.git
signal received: 6
time of crash:
2015-12-22 23:04:00



Regards,
Ravi