<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Thu, Sep 24, 2015 at 9:06 AM, Ramesh Nachimuthu <span dir="ltr"><<a href="mailto:rnachimu@redhat.com" target="_blank">rnachimu@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span class="">
<br>
<br>
<div>On 09/24/2015 11:28 AM, Nir Soffer
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Thu, Sep 24, 2015 at 7:37 AM,
Ramesh Nachimuthu <span dir="ltr"><<a href="mailto:rnachimu@redhat.com" target="_blank"></a><a href="mailto:rnachimu@redhat.com" target="_blank">rnachimu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span> <br>
<br>
<div>On 09/24/2015 02:38 AM, Darrell Budic wrote:<br>
</div>
<blockquote type="cite"> This is a known issue in
overt 3.5.x and below. It’s been solved in the
upcoming ovirt 3.6.
<div><br>
</div>
<div>Related to <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1172905" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1172905</a>,
the fix involved setting up a special cgroup for
the mount, but i can’t find the exact details atm.</div>
<div><br>
</div>
</blockquote>
<br>
</span> I have vdsm 4.17.6-0.el7.centos already
installed on the hosts. So I am not sure above bug <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1172905" target="_blank">1172905</a> fixes this correctly.<br>
</div>
</blockquote>
<div><br>
</div>
<div>I think the root cause is the same - qemu cannot
recover from glusterfs unmount, and the only way to resume
the vm is to restart it with a fresh mount.</div>
<div><br>
</div>
<div>The mentioned bug handle the case where stopping vdsm
kills the glusterfs mount helper. This issue is fixed in
3.6. </div>
<div><br>
</div>
<div>The issue here seems different. I suggest you open a
bug so gluster guys can investigate this.</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
<br></span>
Seems like I am hitting the issue reported in bz
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1171261" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1171261</a>. <br></div></blockquote><div><br></div><div>Indeed.</div><div><br></div><div>I would open an ovirt bug anyway and make it depend on the glusterfs bug.</div><div><br></div><div>We need a way to track this issues, and having no ovirt/rhev hides this issue.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
<br>
Regards,<br>
Ramesh<div><div class="h5"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>Nir</div>
<div><br>
</div>
<div><br>
</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"> <br>
Regards,<br>
Ramesh
<div>
<div><br>
<br>
<blockquote type="cite">
<div><br>
<div>
<blockquote type="cite">
<div>On Sep 23, 2015, at 7:38 AM, Ramesh
Nachimuthu <<a href="mailto:rnachimu@redhat.com" target="_blank">rnachimu@redhat.com</a>>
wrote:</div>
<br>
<div>
<div text="#000000" bgcolor="#FFFFFF"> <br>
<br>
<div>On 09/22/2015 05:57 PM, Alastair
Neil wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">You need to set the
gluster.server-quorum-ratio to 51%</div>
<div class="gmail_extra"><br>
</div>
</blockquote>
<br>
I did that. But still I am facing the
same issue. VM get paused when I do some
I/O using fio on some disks backed by
gluster. I am not able to resume the VM
after this. Now only way is to bring
down the VM and run again. It runs
successfully on the same host without
any issue.<br>
<br>
Regards,<br>
Ramesh<br>
<br>
<blockquote type="cite">
<div class="gmail_extra">
<div class="gmail_quote">On 22
September 2015 at 08:25, Ramesh
Nachimuthu <span dir="ltr"><<a href="mailto:rnachimu@redhat.com" target="_blank"></a><a href="mailto:rnachimu@redhat.com" target="_blank">rnachimu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span> <br>
<br>
<div>On 09/22/2015 05:43 PM,
Alastair Neil wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">what are
the gluster-quorum-type
and gluster.server-quorum-ratio
settings on the volume?</div>
<div class="gmail_extra"><br>
</div>
</blockquote>
<br>
</span>
<div style="outline-style:none">
<div style="overflow:hidden;text-overflow:ellipsis;white-space:nowrap"><b>cluster.server-quorum-type</b>:server<br>
<div title="" style="outline-style:none">
<div style="overflow:hidden;text-overflow:ellipsis;white-space:nowrap"><b>cluster.quorum-type</b>:auto<br>
<b>gluster.server-quorum-ratio
is not set.</b><br>
<br>
</div>
</div>
One brick process is
purposefully killed but
remaining two bricks are
up and running.<br>
<br>
Regards,<br>
Ramesh<br>
</div>
</div>
<span> <br>
<blockquote type="cite">
<div class="gmail_extra">
<div class="gmail_quote">On
22 September 2015 at
06:24, Ramesh
Nachimuthu <span dir="ltr"><<a href="mailto:rnachimu@redhat.com" target="_blank"></a><a href="mailto:rnachimu@redhat.com" target="_blank">rnachimu@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
Hi,<br>
<br>
I am not able
to resume a VM
which was paused
because of gluster
client quorum
issue. Here is
what happened in
my setup. <br>
<br>
1. Created a
gluster storage
domain which is
backed by gluster
volume with
replica 3. <br>
2. Killed one
brick process. So
only two bricks
are running in
replica 3 setup.<br>
3. Created two VMs<br>
4. Started some IO
using fio on both
of the VMs<br>
5. After some time
got the following
error in gluster
mount and VMs
moved to paused
state.<br>
" <span style="color:rgb(51,51,51);font-family:monospace;font-size:11px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:13.2px;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none;background-color:rgb(255,255,255)">server
<a href="http://10.70.45.17:49217/" target="_blank">10.70.45.17:49217</a>
has not
responded in the
last 42 seconds,
disconnecting."<br>
"</span><span style="color:rgb(51,51,51);font-family:monospace;font-size:11px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:13.2px;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none;background-color:rgb(255,255,255)"><span style="color:rgb(51,51,51);font-family:monospace;font-size:11px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:13.2px;text-align:left;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;display:inline!important;float:none;background-color:rgb(255,255,255)">vmstore-replicate-0:
e16d1e40-2b6e-4f19-977d-e099f465dfc6:
Failing WRITE
as quorum is
not met</span>"<br>
more
gluster mount
logs at <a href="http://pastebin.com/UmiUQq0F" target="_blank"></a><a href="http://pastebin.com/UmiUQq0F" target="_blank">http://pastebin.com/UmiUQq0F</a><br>
</span>6. After
some time gluster
quorum is active
and I am able to
write the the
gluster file
system.<br>
7. When I try to
resume the VM it
doesn't work and I
got following
error in vdsm log.<br>
<a href="http://pastebin.com/aXiamY15" target="_blank"></a><a href="http://pastebin.com/aXiamY15" target="_blank">http://pastebin.com/aXiamY15</a><br>
<br>
<br>
Regards,<br>
Ramesh<br>
<br>
</div>
<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank"></a><a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank"></a><a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</span></div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div></div>