1:28 a.m.
--Apple-Mail=_6EA3CA6C-F1CA-43B1-A0ED-5EC9B2DD6690
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
I am having an issue in a Hosted Engine GlusterFS setup. I have 4 =
hosts in a cluster, with the Engined being hosted on the Cluster. This =
follows the pattern shown in the docs for a glusterized setup, except =
that I have 4 hosts. I have engine, data, iso and export storage =
domains all as glusterfs on a replica 3 glusterfs on the first 3 hosts. =
These gluster volumes are running on an SSD Hardware Raid 6, which is =
identical on all the hosts. All the hosts have a second Raid 6 Array =
with Physical Hard Drives and I have created a second data storage =
domain as a glusterfs across all 4 hosts as a stripe 2 replica 2 and =
have added it to the Data Center. However if I use this second Storage =
Domain as the boot disk for a VM, or as second disk for a VM that is =
already running, the VM will become non-responsive as soon as the VM =
starts using this disk. Happens during the OS install if the VM is =
using this storage domain for its boot disk, or if I try copying =
anything large to it when it is a second disk for a VM that has its boot =
drive on the Master Data Storage Domain. =20
If I mount the gluster volume that is this second storage domain on one =
of the hosts directly or any other machine on my local network, the =
gluster volume works fine. It is only when it is used as a storage =
domain (second data domain) on VMs in the cluster.
Once the vm becomes non-responsive it cannot be stopped, removed or =
destroyed without restarting the host machine that the VM is currently =
running on. The 4 hosts are connected via 10gig ethernet, so should =
not be a network issue.
Any ideas?
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net =
<mailto:gary@eraalaska.net>
Systems Manager www.flyravn.com =
<http://www.flyravn.com/>
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your =
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D =
Matt 22:39
------------------------------------------------------------------------
--Apple-Mail=_6EA3CA6C-F1CA-43B1-A0ED-5EC9B2DD6690
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">I am having an issue in a Hosted Engine GlusterFS setup. =
I have 4 hosts in a cluster, with the Engined being hosted on the =
Cluster. This follows the pattern shown in the docs for a =
glusterized setup, except that I have 4 hosts. I have engine, =
data, iso and export storage domains all as glusterfs on a replica 3 =
glusterfs on the first 3 hosts. These gluster volumes are running =
on an SSD Hardware Raid 6, which is identical on all the hosts. =
All the hosts have a second Raid 6 Array with Physical Hard Drives =
and I have created a second data storage domain as a glusterfs across =
all 4 hosts as a stripe 2 replica 2 and have added it to the Data =
Center. However if I use this second Storage Domain as the boot =
disk for a VM, or as second disk for a VM that is already running, the =
VM will become non-responsive as soon as the VM starts using this disk. =
Happens during the OS install if the VM is using this storage =
domain for its boot disk, or if I try copying anything large to it when =
it is a second disk for a VM that has its boot drive on the Master Data =
Storage Domain. <div class=3D""><br
class=3D""></div><div =
class=3D"">If I mount the gluster volume that is this second storage =
domain on one of the hosts directly or any other machine on my local =
network, the gluster volume works fine. It is only when it is used =
as a storage domain (second data domain) on VMs in the =
cluster.</div><div class=3D""><br
class=3D""></div><div class=3D"">Once =
the vm becomes non-responsive it cannot be stopped, removed or destroyed =
without restarting the host machine that the VM is currently running on. =
The 4 hosts are connected via 10gig ethernet, so should not be a =
network issue.</div><div class=3D""><br
class=3D""></div><div =
class=3D""><br class=3D""></div><div
class=3D"">Any ideas?</div><div =
class=3D""><br class=3D""></div><div
class=3D"">Gary</div><div =
class=3D""><br class=3D""></div><div
class=3D""><div class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); font-size: 12px; letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really =
loving the record green up date! Summmer!! yourself=E2=80=9D M=
att 22:39</span></font><div style=3D"color: rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Menlo;" =
class=3D""></div><font face=3D"Menlo"
style=3D"font-size: 12px;" =
class=3D""></font><span style=3D"color: rgb(0, 0, 0);
letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><font =
face=3D"Menlo" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"color: rgb(0, 0, 0);
=
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><font
=
face=3D"Menlo" style=3D"font-size: 12px;"
class=3D""><br =
class=3D""></font></div></div><span style=3D"color:
rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 12px;" =
class=3D""><br
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span
style=3D"color: =
rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br class=3D""></div></body></html>=
--Apple-Mail=_6EA3CA6C-F1CA-43B1-A0ED-5EC9B2DD6690--
4:28 p.m.
Hi Gary,
Do you have other disks on this storage domain?
Have you tried to use other VMs with disks on this storage domain?
Is this disk is preallocated? If not can you try to create a pre-allocate
disk and re-try
Regards,
Maor
On Sat, Nov 5, 2016 at 2:28 AM, Gary Pedretty <gary(a)ravnalaska.net> wrote:
I am having an issue in a Hosted Engine GlusterFS setup. I have 4
hosts
in a cluster, with the Engined being hosted on the Cluster. This follows
the pattern shown in the docs for a glusterized setup, except that I have 4
hosts. I have engine, data, iso and export storage domains all as
glusterfs on a replica 3 glusterfs on the first 3 hosts. These gluster
volumes are running on an SSD Hardware Raid 6, which is identical on all
the hosts. All the hosts have a second Raid 6 Array with Physical Hard
Drives and I have created a second data storage domain as a glusterfs
across all 4 hosts as a stripe 2 replica 2 and have added it to the Data
Center. However if I use this second Storage Domain as the boot disk for a
VM, or as second disk for a VM that is already running, the VM will become
non-responsive as soon as the VM starts using this disk. Happens during
the OS install if the VM is using this storage domain for its boot disk, or
if I try copying anything large to it when it is a second disk for a VM
that has its boot drive on the Master Data Storage Domain.
If I mount the gluster volume that is this second storage domain on one of
the hosts directly or any other machine on my local network, the gluster
volume works fine. It is only when it is used as a storage domain (second
data domain) on VMs in the cluster.
Once the vm becomes non-responsive it cannot be stopped, removed or
destroyed without restarting the host machine that the VM is currently
running on. The 4 hosts are connected via 10gig ethernet, so should not
be a network issue.
Any ideas?
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net
<gary(a)eraalaska.net>
Systems Manager www.flyravn.com
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ “Love your neighbor as
Really loving the record green up date! Summmer!! yourself” Matt 22:39
------------------------------------------------------------------------
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
6:50 a.m.
--Apple-Mail=_06BC44F7-FB91-448D-94C1-8C5321567393
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
As a storage domain, this gluster volume will not work whether it is =
preallocated or thin provision. It will work as a straight gluster =
volume mounted directly to any VM on the ovirt Cluster, or any physical =
machine, just not as a data storage domain in the Data Center.
Are there restrictions to having more than one data storage domain that =
has it gluster volumes on the same hosts that are also part of the Data =
Center and Cluster?
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net =
<mailto:gary@eraalaska.net>
Systems Manager www.flyravn.com =
<http://www.flyravn.com/>
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your =
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D =
Matt 22:39
------------------------------------------------------------------------
On Nov 6, 2016, at 6:28 AM, Maor Lipchuk <mlipchuk(a)redhat.com>
wrote:
=20
Hi Gary,
=20
Do you have other disks on this storage domain?
Have you tried to use other VMs with disks on this storage domain?
Is this disk is preallocated? If not can you try to create a =
pre-allocate disk
and re-try
=20
Regards,
Maor
=20
=20
=20
On Sat, Nov 5, 2016 at 2:28 AM, Gary Pedretty <gary(a)ravnalaska.net =
<mailto:gary@ravnalaska.net>> wrote:
I am having an issue in a Hosted Engine GlusterFS setup. I have 4 =
hosts in a cluster, with the Engined being hosted on the Cluster. This =
follows the pattern shown in the docs for a glusterized setup, except =
that I have 4 hosts. I have engine, data, iso and export storage =
domains all as glusterfs on a replica 3 glusterfs on the first 3 hosts. =
These gluster volumes are running on an SSD Hardware Raid 6, which is =
identical on all the hosts. All the hosts have a second Raid 6 Array =
with Physical Hard Drives and I have created a second data storage =
domain as a glusterfs across all 4 hosts as a stripe 2 replica 2 and =
have added it to the Data Center. However if I use this second Storage =
Domain as the boot disk for a VM, or as second disk for a VM that is =
already running, the VM will become non-responsive as soon as the VM =
starts using this disk. Happens during the OS install if the VM is =
using this storage domain for its boot disk, or if I try copying =
anything large to it when it is a second disk for a VM that has its boot =
drive on the Master Data Storage Domain.=20
=20
If I mount the gluster volume that is this second storage domain on =
one of the
hosts directly or any other machine on my local network, the =
gluster volume works fine. It is only when it is used as a storage =
domain (second data domain) on VMs in the cluster.
=20
Once the vm becomes non-responsive it cannot be stopped, removed or =
destroyed
without restarting the host machine that the VM is currently =
running on. The 4 hosts are connected via 10gig ethernet, so should =
not be a network issue.
=20
=20
Any ideas?
=20
Gary
--Apple-Mail=_06BC44F7-FB91-448D-94C1-8C5321567393
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">As a storage domain, this gluster volume will not work =
whether it is preallocated or thin provision. It will work as a =
straight gluster volume mounted directly to any VM on the ovirt Cluster, =
or any physical machine, just not as a data storage domain in the Data =
Center.<div class=3D""><br class=3D""></div><div
class=3D"">Are there =
restrictions to having more than one data storage domain that has it =
gluster volumes on the same hosts that are also part of the Data Center =
and Cluster?<br class=3D""><div class=3D""><br
class=3D""></div><div =
class=3D""><br class=3D""></div><div
class=3D"">Gary</div><div =
class=3D""><br class=3D""></div><div
class=3D""><br class=3D""><div =
class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); font-size: 12px; letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really =
loving the record green up date! Summmer!! yourself=E2=80=9D M=
att 22:39</span></font><div style=3D"color: rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Menlo;" =
class=3D""></div><font face=3D"Menlo"
style=3D"font-size: 12px;" =
class=3D""></font><span style=3D"color: rgb(0, 0, 0);
letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><font =
face=3D"Menlo" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"color: rgb(0, 0, 0);
=
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><font
=
face=3D"Menlo" style=3D"font-size: 12px;"
class=3D""><br =
class=3D""></font></div></div><span style=3D"color:
rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 12px;" =
class=3D""><br
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span
style=3D"color: =
rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br class=3D""><div><blockquote type=3D"cite"
class=3D""><div =
class=3D"">On Nov 6, 2016, at 6:28 AM, Maor Lipchuk <<a =
href=3D"mailto:mlipchuk@redhat.com"
class=3D"">mlipchuk(a)redhat.com</a>&gt;=
wrote:</div><br class=3D"Apple-interchange-newline"><div
class=3D""><div =
dir=3D"ltr" style=3D"font-family: LucidaGrande; font-size: 13px; =
font-style: normal; font-variant-caps: normal; font-weight: normal; =
letter-spacing: normal; orphans: auto; text-align: start; text-indent: =
0px; text-transform: none; white-space: normal; widows: auto; =
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D"">Hi =
Gary,<div class=3D""><br class=3D""></div><div
class=3D"">Do you have =
other disks on this storage domain?</div><div class=3D"">Have you
tried =
to use other VMs with disks on this storage domain?</div><div =
class=3D"">Is this disk is preallocated? If not can you try to create a =
pre-allocate disk and re-try</div><div class=3D""><br =
class=3D""></div><div
class=3D"">Regards,</div><div =
class=3D"">Maor</div><div class=3D""><br
class=3D""></div><div =
class=3D""><br class=3D""></div></div><div
class=3D"gmail_extra" =
style=3D"font-family: LucidaGrande; font-size: 13px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;"><br class=3D""><div =
class=3D"gmail_quote">On Sat, Nov 5, 2016 at 2:28 AM, Gary Pedretty<span
=
class=3D"Apple-converted-space"> </span><span
dir=3D"ltr" =
class=3D""><<a href=3D"mailto:gary@ravnalaska.net"
target=3D"_blank" =
class=3D"">gary(a)ravnalaska.net</a>&gt;</span><span =
class=3D"Apple-converted-space"> </span>wrote:<br =
class=3D""><blockquote class=3D"gmail_quote"
style=3D"margin: 0px 0px =
0px 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, =
204); border-left-style: solid; padding-left: 1ex;"><div =
style=3D"word-wrap: break-word;" class=3D"">I am having an issue in
a =
Hosted Engine GlusterFS setup. I have 4 hosts in a cluster, with =
the Engined being hosted on the Cluster. This follows the pattern =
shown in the docs for a glusterized setup, except that I have 4 hosts. =
I have engine, data, iso and export storage domains all as =
glusterfs on a replica 3 glusterfs on the first 3 hosts. These =
gluster volumes are running on an SSD Hardware Raid 6, which is =
identical on all the hosts. All the hosts have a second Raid 6 =
Array with Physical Hard Drives and I have created a second data storage =
domain as a glusterfs across all 4 hosts as a stripe 2 replica 2 =
and have added it to the Data Center. However if I use this second =
Storage Domain as the boot disk for a VM, or as second disk for a VM =
that is already running, the VM will become non-responsive as soon as =
the VM starts using this disk. Happens during the OS install if =
the VM is using this storage domain for its boot disk, or if I try =
copying anything large to it when it is a second disk for a VM that has =
its boot drive on the Master Data Storage Domain. <div =
class=3D""><br class=3D""></div><div
class=3D"">If I mount the gluster =
volume that is this second storage domain on one of the hosts directly =
or any other machine on my local network, the gluster volume works =
fine. It is only when it is used as a storage domain (second data =
domain) on VMs in the cluster.</div><div class=3D""><br =
class=3D""></div><div class=3D"">Once the vm becomes
non-responsive it =
cannot be stopped, removed or destroyed without restarting the host =
machine that the VM is currently running on. The 4 hosts are =
connected via 10gig ethernet, so should not be a network =
issue.</div><div class=3D""><br
class=3D""></div><div class=3D""><br =
class=3D""></div><div class=3D"">Any
ideas?</div><div class=3D""><br =
class=3D""></div><div =
class=3D"">Gary</div></div></blockquote></div></div></div></blockquote></d=
iv><br class=3D""></div></div></body></html>=
--Apple-Mail=_06BC44F7-FB91-448D-94C1-8C5321567393--
7:50 a.m.
On Mon, Nov 7, 2016 at 11:20 AM, Gary Pedretty <gary(a)ravnalaska.net> wrote:
As a storage domain, this gluster volume will not work whether it is
preallocated or thin provision. It will work as a straight gluster volume
mounted directly to any VM on the ovirt Cluster, or any physical machine,
just not as a data storage domain in the Data Center.
Are there restrictions to having more than one data storage domain that
has it gluster volumes on the same hosts that are also part of the Data
Center and Cluster?
There are no such restrictions.
However your volume configuration seems suspect -"stripe 2 replica 2". Can
you provide gluster volume info of your second storage domain gluster
volume? The mount logs of the volume (under
/var/log/glusterfs/rhev-datacenter..<volname>.log) from the host where the
volume is being mounted will also help.
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net
<gary(a)eraalaska.net>
Systems Manager www.flyravn.com
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ “Love your neighbor as
Really loving the record green up date! Summmer!! yourself” Matt 22:39
------------------------------------------------------------------------
On Nov 6, 2016, at 6:28 AM, Maor Lipchuk <mlipchuk(a)redhat.com> wrote:
Hi Gary,
Do you have other disks on this storage domain?
Have you tried to use other VMs with disks on this storage domain?
Is this disk is preallocated? If not can you try to create a pre-allocate
disk and re-try
Regards,
Maor
On Sat, Nov 5, 2016 at 2:28 AM, Gary Pedretty <gary(a)ravnalaska.net> wrote:
> I am having an issue in a Hosted Engine GlusterFS setup. I have 4 hosts
> in a cluster, with the Engined being hosted on the Cluster. This follows
> the pattern shown in the docs for a glusterized setup, except that I have 4
> hosts. I have engine, data, iso and export storage domains all as
> glusterfs on a replica 3 glusterfs on the first 3 hosts. These gluster
> volumes are running on an SSD Hardware Raid 6, which is identical on all
> the hosts. All the hosts have a second Raid 6 Array with Physical Hard
> Drives and I have created a second data storage domain as a glusterfs
> across all 4 hosts as a stripe 2 replica 2 and have added it to the Data
> Center. However if I use this second Storage Domain as the boot disk for a
> VM, or as second disk for a VM that is already running, the VM will become
> non-responsive as soon as the VM starts using this disk. Happens during
> the OS install if the VM is using this storage domain for its boot disk, or
> if I try copying anything large to it when it is a second disk for a VM
> that has its boot drive on the Master Data Storage Domain.
>
> If I mount the gluster volume that is this second storage domain on one
> of the hosts directly or any other machine on my local network, the gluster
> volume works fine. It is only when it is used as a storage domain (second
> data domain) on VMs in the cluster.
>
> Once the vm becomes non-responsive it cannot be stopped, removed or
> destroyed without restarting the host machine that the VM is currently
> running on. The 4 hosts are connected via 10gig ethernet, so should not
> be a network issue.
>
>
> Any ideas?
>
> Gary
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
10:57 a.m.
--Apple-Mail=_74966C91-4D7B-4631-992A-7104BF8281C2
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
[root@fai-kvm-1-gfs admin]# gluster volume status data2
Status of volume: data2
Gluster process TCP Port RDMA Port Online =
Pid
=
--------------------------------------------------------------------------=
----
Brick fai-kvm-1-vmn.ravnalaska.net:/kvm2/gl
uster/data2/brick 49156 0 Y =
3484
Brick fai-kvm-2-vmn.ravnalaska.net:/kvm2/gl
uster/data2/brick 49156 0 Y =
34791
Brick fai-kvm-3-vmn.ravnalaska.net:/kvm2/gl
uster/data2/brick 49156 0 Y =
177340
Brick fai-kvm-4-vmn.ravnalaska.net:/kvm2/gl
uster/data2/brick 49152 0 Y =
146038
NFS Server on localhost 2049 0 Y =
40844
Self-heal Daemon on localhost N/A N/A Y =
40865
NFS Server on fai-kvm-2-gfs.ravnalaska.net 2049 0 Y =
99905
Self-heal Daemon on fai-kvm-2-gfs.ravnalask
a.net N/A N/A Y =
99915
NFS Server on fai-kvm-4-gfs.ravnalaska.net 2049 0 Y =
176305
Self-heal Daemon on fai-kvm-4-gfs.ravnalask
a.net N/A N/A Y =
176326
NFS Server on fai-kvm-3-gfs.ravnalaska.net 2049 0 Y =
226271
Self-heal Daemon on fai-kvm-3-gfs.ravnalask
a.net N/A N/A Y =
226287
Task Status of Volume data2
=
--------------------------------------------------------------------------=
----
There are no active volume tasks
[root@fai-kvm-1-gfs admin]# gluster volume info data2
Volume Name: data2
Type: Striped-Replicate
Volume ID: 20f85c9a-541b-4df4-9dba-44c5179bbfb0
Status: Started
Number of Bricks: 1 x 2 x 2 =3D 4
Transport-type: tcp
Bricks:
Brick1: fai-kvm-1-vmn.ravnalaska.net:/kvm2/gluster/data2/brick
Brick2: fai-kvm-2-vmn.ravnalaska.net:/kvm2/gluster/data2/brick
Brick3: fai-kvm-3-vmn.ravnalaska.net:/kvm2/gluster/data2/brick
Brick4: fai-kvm-4-vmn.ravnalaska.net:/kvm2/gluster/data2/brick
Options Reconfigured:
performance.readdir-ahead: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
cluster.eager-lock: enable
network.remote-dio: enable
cluster.quorum-type: auto
cluster.server-quorum-type: server
storage.owner-uid: 36
storage.owner-gid: 36
See attached file for the mount log.
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net =
<mailto:gary@eraalaska.net>
Systems Manager www.flyravn.com =
<http://www.flyravn.com/>
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your =
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D =
Matt 22:39
------------------------------------------------------------------------
On Nov 6, 2016, at 9:50 PM, Sahina Bose <sabose(a)redhat.com>
wrote:
=20
However your volume configuration seems suspect -"stripe 2 replica 2". =
Can you provide gluster volume info of your second storage domain =
gluster volume? The mount logs of the volume (under =
/var/log/glusterfs/rhev-datacenter..<volname>.log) from the host where =
the volume is being mounted will also help.
=20
--Apple-Mail=_74966C91-4D7B-4631-992A-7104BF8281C2
Content-Type: multipart/mixed;
boundary="Apple-Mail=_D3985462-55E4-4FE5-A078-30EC039EC062"
--Apple-Mail=_D3985462-55E4-4FE5-A078-30EC039EC062
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=us-ascii
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D"">[root@fai-kvm-1-gfs admin]#
gluster volume =
status data2</div><div class=3D"">Status of volume:
data2</div><div =
class=3D"">Gluster process
=
TCP Port =
RDMA Port Online Pid</div><div =
class=3D"">---------------------------------------------------------------=
---------------</div><div class=3D"">Brick <a =
href=3D"http://fai-kvm-1-vmn.ravnalaska.net" =
class=3D"">fai-kvm-1-vmn.ravnalaska.net</a>:/kvm2/gl</div><div
=
class=3D"">uster/data2/brick
=
49156
0 =
Y
3484</div><div =
class=3D"">Brick <a href=3D"http://fai-kvm-2-vmn.ravnalaska.net"
=
class=3D"">fai-kvm-2-vmn.ravnalaska.net</a>:/kvm2/gl</div><div
=
class=3D"">uster/data2/brick
=
49156
0 =
Y
34791</div><div =
class=3D"">Brick <a href=3D"http://fai-kvm-3-vmn.ravnalaska.net"
=
class=3D"">fai-kvm-3-vmn.ravnalaska.net</a>:/kvm2/gl</div><div
=
class=3D"">uster/data2/brick
=
49156
0 =
Y
=
177340</div><div class=3D"">Brick <a =
href=3D"http://fai-kvm-4-vmn.ravnalaska.net" =
class=3D"">fai-kvm-4-vmn.ravnalaska.net</a>:/kvm2/gl</div><div
=
class=3D"">uster/data2/brick
=
49152
0 =
Y
=
146038</div><div class=3D"">NFS Server on localhost
=
2049
=
0 Y
=
40844</div><div class=3D"">Self-heal Daemon on localhost
=
N/A
N/A =
Y
40865</div><div class=3D"">NFS =
Server on <a href=3D"http://fai-kvm-2-gfs.ravnalaska.net" =
class=3D"">fai-kvm-2-gfs.ravnalaska.net</a> 2049
=
0 Y
=
99905</div><div class=3D"">Self-heal Daemon on =
fai-kvm-2-gfs.ravnalask</div><div class=3D""><a
href=3D"http://a.net" =
class=3D"">a.net</a>
=
=
N/A N/A
Y =
99915</div><div class=3D"">NFS
Server on <a =
href=3D"http://fai-kvm-4-gfs.ravnalaska.net" =
class=3D"">fai-kvm-4-gfs.ravnalaska.net</a> 2049
=
0 Y
=
176305</div><div class=3D"">Self-heal Daemon on =
fai-kvm-4-gfs.ravnalask</div><div class=3D""><a
href=3D"http://a.net" =
class=3D"">a.net</a>
=
=
N/A N/A
Y =
176326</div><div class=3D"">NFS
Server on <a =
href=3D"http://fai-kvm-3-gfs.ravnalaska.net" =
class=3D"">fai-kvm-3-gfs.ravnalaska.net</a> 2049
=
0 Y
=
226271</div><div class=3D"">Self-heal Daemon on =
fai-kvm-3-gfs.ravnalask</div><div class=3D""><a
href=3D"http://a.net" =
class=3D"">a.net</a>
=
=
N/A N/A
Y =
226287</div><div class=3D""><br
class=3D""></div><div=
class=3D"">Task Status of Volume data2</div><div =
class=3D"">---------------------------------------------------------------=
---------------</div><div class=3D"">There are no active volume =
tasks</div><div class=3D""><br
class=3D""></div><div class=3D""><br =
class=3D""></div><div class=3D""><div
class=3D"">[root@fai-kvm-1-gfs =
admin]# gluster volume info data2</div><div class=3D""><br =
class=3D""></div><div class=3D"">Volume Name:
data2</div><div =
class=3D"">Type: Striped-Replicate</div><div
class=3D"">Volume ID: =
20f85c9a-541b-4df4-9dba-44c5179bbfb0</div><div class=3D"">Status: =
Started</div><div class=3D"">Number of Bricks: 1 x 2 x 2 =3D
4</div><div =
class=3D"">Transport-type: tcp</div><div
class=3D"">Bricks:</div><div =
class=3D"">Brick1: <a
href=3D"http://fai-kvm-1-vmn.ravnalaska.net" =
class=3D"">fai-kvm-1-vmn.ravnalaska.net</a>:/kvm2/gluster/data2/brick</div=
<div class=3D"">Brick2: <a
href=3D"http://fai-kvm-2-vmn.ravnalaska.net" =
class=3D"">fai-kvm-2-vmn.ravnalaska.net</a>:/kvm2/gluster/data2/brick</div=
<div class=3D"">Brick3: <a
href=3D"http://fai-kvm-3-vmn.ravnalaska.net" =
class=3D"">fai-kvm-3-vmn.ravnalaska.net</a>:/kvm2/gluster/data2/brick</div=
<div class=3D"">Brick4: <a
href=3D"http://fai-kvm-4-vmn.ravnalaska.net" =
class=3D"">fai-kvm-4-vmn.ravnalaska.net</a>:/kvm2/gluster/data2/brick</div=
<div class=3D"">Options
Reconfigured:</div><div =
class=3D"">performance.readdir-ahead: on</div><div =
class=3D"">performance.quick-read: off</div><div =
class=3D"">performance.read-ahead: off</div><div
class=3D""><a =
href=3D"http://performance.io"
class=3D"">performance.io</a>-cache: =
off</div><div class=3D"">performance.stat-prefetch:
off</div><div =
class=3D"">cluster.eager-lock: enable</div><div =
class=3D"">network.remote-dio: enable</div><div =
class=3D"">cluster.quorum-type: auto</div><div =
class=3D"">cluster.server-quorum-type: server</div><div =
class=3D"">storage.owner-uid: 36</div><div
class=3D"">storage.owner-gid: =
36</div></div><div class=3D""><br
class=3D""></div><div class=3D""><br =
class=3D""></div><div class=3D"">See attached file for
the mount =
log.</div><div class=3D""><br
class=3D""></div><div =
class=3D"">Gary</div><div class=3D""><br
class=3D""></div><div =
class=3D""></div></body></html>=
--Apple-Mail=_D3985462-55E4-4FE5-A078-30EC039EC062
Content-Disposition: attachment;
filename=glusterlog.txt
Content-Type: text/plain;
x-unix-mode=0644;
name="glusterlog.txt"
Content-Transfer-Encoding: quoted-printable
more rhev-data-center-mnt-glusterSD-glustermount2:data2.log-20161106
[2016-11-04 22:54:32.118587] I [MSGID: 100030] [glusterfsd.c:2338:main] =
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version =
3.7.16 (args: /usr/sbin/glusterfs --volfile-server=3Dglustermou
nt2 --volfile-server=3Dfai-kvm-1-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-2-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-3-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-4-vmn.ravnalaska.net --volfile-id
=3Ddata2 /rhev/data-center/mnt/glusterSD/glustermount2:data2)
[2016-11-04 22:54:32.128807] I [MSGID: 101190] =
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread =
with index 1
[2016-11-04 22:54:32.138959] I [MSGID: 101190] =
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread =
with index 2
[2016-11-04 22:54:32.139878] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-0: parent translators are ready, attempting connect on =
transport
[2016-11-04 22:54:32.142057] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-1: parent translators are ready, attempting connect on =
transport
[2016-11-04 22:54:32.142331] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-0: changing port to 49156 (from 0)
[2016-11-04 22:54:32.143961] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-2: parent translators are ready, attempting connect on =
transport
[2016-11-04 22:54:32.146123] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-0: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-04 22:54:32.146308] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-1: changing port to 49156 (from 0)
[2016-11-04 22:54:32.147618] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-3: parent translators are ready, attempting connect on =
transport
[2016-11-04 22:54:32.149874] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-0: =
Connected to data2-client-0, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-04 22:54:32.149891] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-0: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-04 22:54:32.149941] I [MSGID: 108005] =
[afr-common.c:4299:afr_notify] 0-data2-replicate-0: Subvolume =
'data2-client-0' came back up; going online.
[2016-11-04 22:54:32.150000] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-0: =
Server lk version =3D 1
[2016-11-04 22:54:32.150080] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-1: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-04 22:54:32.150274] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-2: changing port to 49156 (from 0)
Final graph:
=
+-------------------------------------------------------------------------=
-----+
1: volume data2-client-0
2: type protocol/client
3: option clnt-lk-version 1
4: option volfile-checksum 0
5: option volfile-key data2
6: option client-version 3.7.16
7: option process-uuid =
fai-kvm-1-gfs.ravnalaska.net-105264-2016/11/04-22:54:32:114962-data2-clien=
t-0-0-0
8: option fops-version 1298437
9: option ping-timeout 42
10: option remote-host fai-kvm-1-vmn.ravnalaska.net
11: option remote-subvolume /kvm2/gluster/data2/brick
12: option transport-type socket
13: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
14: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
15: option filter-O_DIRECT enable
16: option send-gids true
17: end-volume
18:
19: volume data2-client-1
20: type protocol/client
21: option clnt-lk-version 1
22: option volfile-checksum 0
23: option volfile-key data2
24: option client-version 3.7.16
25: option process-uuid =
fai-kvm-1-gfs.ravnalaska.net-105264-2016/11/04-22:54:32:114962-data2-clien=
t-1-0-0
26: option fops-version 1298437
27: option ping-timeout 42
28: option remote-host fai-kvm-2-vmn.ravnalaska.net
29: option remote-subvolume /kvm2/gluster/data2/brick
30: option transport-type socket
31: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
32: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
33: option filter-O_DIRECT enable
34: option send-gids true
35: end-volume
36:
37: volume data2-replicate-0
38: type cluster/replicate
39: option eager-lock enable
40: option quorum-type auto
41: subvolumes data2-client-0 data2-client-1
42: end-volume
43:
44: volume data2-client-2
45: type protocol/client
46: option ping-timeout 42
47: option remote-host fai-kvm-3-vmn.ravnalaska.net
48: option remote-subvolume /kvm2/gluster/data2/brick
49: option transport-type socket
50: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
51: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
52: option filter-O_DIRECT enable
53: option send-gids true
54: end-volume
55:
56: volume data2-client-3
57: type protocol/client
58: option ping-timeout 42
59: option remote-host fai-kvm-4-vmn.ravnalaska.net
60: option remote-subvolume /kvm2/gluster/data2/brick
61: option transport-type socket
62: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
63: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
64: option filter-O_DIRECT enable
65: option send-gids true
66: end-volume
67:
68: volume data2-replicate-1
69: type cluster/replicate
70: option eager-lock enable
71: option quorum-type auto
72: subvolumes data2-client-2 data2-client-3
73: end-volume
74:
75: volume data2-stripe-0
76: type cluster/stripe
77: subvolumes data2-replicate-0 data2-replicate-1
78: end-volume
79:
80: volume data2-dht
81: type cluster/distribute
82: subvolumes data2-stripe-0
83: end-volume
84:
85: volume data2-write-behind
86: type performance/write-behind
87: subvolumes data2-dht
88: end-volume
89:
90: volume data2-readdir-ahead
91: type performance/readdir-ahead
92: subvolumes data2-write-behind
93: end-volume
94:
95: volume data2-open-behind
96: type performance/open-behind
97: subvolumes data2-readdir-ahead
98: end-volume
99:
100: volume data2
101: type debug/io-stats
102: option log-level INFO
103: option latency-measurement off
104: option count-fop-hits off
105: subvolumes data2-open-behind
106: end-volume
107:
108: volume meta-autoload
109: type meta
110: subvolumes data2
111: end-volume
112:
=
+-------------------------------------------------------------------------=
-----+
[2016-11-04 22:54:32.151927] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-1: =
Connected to data2-client-1, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-04 22:54:32.151939] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-1: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-04 22:54:32.152083] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-1: =
Server lk version =3D 1
[2016-11-04 22:54:32.152341] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-3: changing port to 49152 (from 0)
[2016-11-04 22:54:32.154296] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-2: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-04 22:54:32.154774] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-2: =
Connected to data2-client-2, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-04 22:54:32.154789] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-2: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-04 22:54:32.154821] I [MSGID: 108005] =
[afr-common.c:4299:afr_notify] 0-data2-replicate-1: Subvolume =
'data2-client-2' came back up; going online.
[2016-11-04 22:54:32.154916] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-2: =
Server lk version =3D 1
[2016-11-04 22:54:32.156348] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-3: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-04 22:54:32.156860] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-3: =
Connected to data2-client-3, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-04 22:54:32.156874] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-3: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-04 22:54:32.160382] I [fuse-bridge.c:5171:fuse_graph_setup] =
0-fuse: switched to graph 0
[2016-11-04 22:54:32.160451] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-3: =
Server lk version =3D 1
[2016-11-04 22:54:32.160524] I [fuse-bridge.c:4083:fuse_init] =
0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 =
kernel 7.22
[2016-11-04 22:54:32.161171] I [MSGID: 108031] =
[afr-common.c:2071:afr_local_discovery_cbk] 0-data2-replicate-0: =
selecting local read_child data2-client-0
[2016-11-05 00:05:37.796730] W [glusterfsd.c:1251:cleanup_and_exit] =
(-->/lib64/libpthread.so.0(+0x7dc5) [0x7fe90527adc5] =
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7fe9068f5915] =
-->/usr/sbin/
glusterfs(cleanup_and_exit+0x6b) [0x7fe9068f578b] ) 0-: received signum =
(15), shutting down
[2016-11-05 00:05:37.796770] I [fuse-bridge.c:5719:fini] 0-fuse: =
Unmounting '/rhev/data-center/mnt/glusterSD/glustermount2:data2'.
[2016-11-05 00:13:04.440431] I [MSGID: 100030] [glusterfsd.c:2338:main] =
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version =
3.7.16 (args: /usr/sbin/glusterfs --volfile-server=3Dglustermou
nt2 --volfile-server=3Dfai-kvm-1-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-2-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-3-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-4-vmn.ravnalaska.net --volfile-id
=3Ddata2 /rhev/data-center/mnt/glusterSD/glustermount2:data2)
[2016-11-05 00:13:04.452151] I [MSGID: 101190] =
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread =
with index 1
[2016-11-05 00:13:04.463593] I [MSGID: 101190] =
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread =
with index 2
[2016-11-05 00:13:04.464394] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-0: parent translators are ready, attempting connect on =
transport
[2016-11-05 00:13:04.466698] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-1: parent translators are ready, attempting connect on =
transport
[2016-11-05 00:13:04.466885] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-0: changing port to 49156 (from 0)
[2016-11-05 00:13:04.468672] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-2: parent translators are ready, attempting connect on =
transport
[2016-11-05 00:13:04.470686] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-0: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-05 00:13:04.470849] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-1: changing port to 49156 (from 0)
[2016-11-05 00:13:04.472375] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-3: parent translators are ready, attempting connect on =
transport
[2016-11-05 00:13:04.475429] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-0: =
Connected to data2-client-0, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-05 00:13:04.475448] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-0: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-05 00:13:04.475503] I [MSGID: 108005] =
[afr-common.c:4299:afr_notify] 0-data2-replicate-0: Subvolume =
'data2-client-0' came back up; going online.
[2016-11-05 00:13:04.475615] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-0: =
Server lk version =3D 1
[2016-11-05 00:13:04.475687] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-1: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-05 00:13:04.475857] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-2: changing port to 49156 (from 0)
Final graph:
=
+-------------------------------------------------------------------------=
-----+
1: volume data2-client-0
2: type protocol/client
3: option clnt-lk-version 1
4: option volfile-checksum 0
5: option volfile-key data2
6: option client-version 3.7.16
7: option process-uuid =
fai-kvm-1-gfs.ravnalaska.net-13785-2016/11/05-00:13:04:437247-data2-client=
-0-0-0
8: option fops-version 1298437
9: option ping-timeout 42
10: option remote-host fai-kvm-1-vmn.ravnalaska.net
11: option remote-subvolume /kvm2/gluster/data2/brick
12: option transport-type socket
13: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
14: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
15: option filter-O_DIRECT enable
16: option send-gids true
17: end-volume
18:
19: volume data2-client-1
20: type protocol/client
21: option clnt-lk-version 1
22: option volfile-checksum 0
23: option volfile-key data2
24: option client-version 3.7.16
25: option process-uuid =
fai-kvm-1-gfs.ravnalaska.net-13785-2016/11/05-00:13:04:437247-data2-client=
-1-0-0
26: option fops-version 1298437
27: option ping-timeout 42
28: option remote-host fai-kvm-2-vmn.ravnalaska.net
29: option remote-subvolume /kvm2/gluster/data2/brick
30: option transport-type socket
31: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
32: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
33: option filter-O_DIRECT enable
34: option send-gids true
35: end-volume
36:
37: volume data2-replicate-0
38: type cluster/replicate
39: option eager-lock enable
40: option quorum-type auto
41: subvolumes data2-client-0 data2-client-1
42: end-volume
43:
44: volume data2-client-2
45: type protocol/client
46: option ping-timeout 42
47: option remote-host fai-kvm-3-vmn.ravnalaska.net
48: option remote-subvolume /kvm2/gluster/data2/brick
49: option transport-type socket
50: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
51: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
52: option filter-O_DIRECT enable
53: option send-gids true
54: end-volume
55:
56: volume data2-client-3
57: type protocol/client
58: option ping-timeout 42
59: option remote-host fai-kvm-4-vmn.ravnalaska.net
60: option remote-subvolume /kvm2/gluster/data2/brick
61: option transport-type socket
62: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
63: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
64: option filter-O_DIRECT enable
65: option send-gids true
66: end-volume
67:
68: volume data2-replicate-1
69: type cluster/replicate
70: option eager-lock enable
71: option quorum-type auto
72: subvolumes data2-client-2 data2-client-3
73: end-volume
74:
75: volume data2-stripe-0
76: type cluster/stripe
77: subvolumes data2-replicate-0 data2-replicate-1
78: end-volume
79:
80: volume data2-dht
81: type cluster/distribute
82: subvolumes data2-stripe-0
83: end-volume
84:
85: volume data2-write-behind
86: type performance/write-behind
87: subvolumes data2-dht
88: end-volume
89:
90: volume data2-readdir-ahead
91: type performance/readdir-ahead
92: subvolumes data2-write-behind
93: end-volume
94:
95: volume data2-open-behind
96: type performance/open-behind
97: subvolumes data2-readdir-ahead
98: end-volume
99:
100: volume data2
101: type debug/io-stats
102: option log-level INFO
103: option latency-measurement off
104: option count-fop-hits off
105: subvolumes data2-open-behind
106: end-volume
107:
108: volume meta-autoload
109: type meta
110: subvolumes data2
111: end-volume
112:
=
+-------------------------------------------------------------------------=
-----+
[2016-11-05 00:13:04.477682] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-1: =
Connected to data2-client-1, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-05 00:13:04.477702] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-1: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-05 00:13:04.477834] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-1: =
Server lk version =3D 1
[2016-11-05 00:13:04.478162] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-3: changing port to 49152 (from 0)
[2016-11-05 00:13:04.480593] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-2: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-05 00:13:04.480961] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-2: =
Connected to data2-client-2, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-05 00:13:04.480982] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-2: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-05 00:13:04.481020] I [MSGID: 108005] =
[afr-common.c:4299:afr_notify] 0-data2-replicate-1: Subvolume =
'data2-client-2' came back up; going online.
[2016-11-05 00:13:04.481120] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-2: =
Server lk version =3D 1
[2016-11-05 00:13:04.483500] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-3: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-05 00:13:04.483956] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-3: =
Connected to data2-client-3, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-05 00:13:04.483973] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-3: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-05 00:13:04.488332] I [fuse-bridge.c:5171:fuse_graph_setup] =
0-fuse: switched to graph 0
[2016-11-05 00:13:04.488889] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-3: =
Server lk version =3D 1
[2016-11-05 00:13:04.488933] I [fuse-bridge.c:4083:fuse_init] =
0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 =
kernel 7.22
[2016-11-05 00:13:04.489627] I [MSGID: 108031] =
[afr-common.c:2071:afr_local_discovery_cbk] 0-data2-replicate-0: =
selecting local read_child data2-client-0
[2016-11-05 00:52:07.549997] I [fuse-bridge.c:5012:fuse_thread_proc] =
0-fuse: unmounting /rhev/data-center/mnt/glusterSD/glustermount2:data2
[2016-11-05 00:52:07.550544] W [glusterfsd.c:1251:cleanup_and_exit] =
(-->/lib64/libpthread.so.0(+0x7dc5) [0x7f40b1c24dc5] =
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7f40b329f915] =
-->/usr/sbin/
glusterfs(cleanup_and_exit+0x6b) [0x7f40b329f78b] ) 0-: received signum =
(15), shutting down
[2016-11-05 00:52:07.550583] I [fuse-bridge.c:5719:fini] 0-fuse: =
Unmounting '/rhev/data-center/mnt/glusterSD/glustermount2:data2'.
[2016-11-05 00:52:39.557750] I [MSGID: 100030] [glusterfsd.c:2338:main] =
0-/usr/sbin/glusterfs: Started running /usr/sbin/glusterfs version =
3.7.16 (args: /usr/sbin/glusterfs --volfile-server=3Dglustermou
nt2 --volfile-server=3Dfai-kvm-1-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-2-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-3-vmn.ravnalaska.net =
--volfile-server=3Dfai-kvm-4-vmn.ravnalaska.net --volfile-id
=3Ddata2 /rhev/data-center/mnt/glusterSD/glustermount2:data2)
[2016-11-05 00:52:39.567614] I [MSGID: 101190] =
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread =
with index 1
[2016-11-05 00:52:39.578148] I [MSGID: 101190] =
[event-epoll.c:632:event_dispatch_epoll_worker] 0-epoll: Started thread =
with index 2
[2016-11-05 00:52:39.579039] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-0: parent translators are ready, attempting connect on =
transport
[2016-11-05 00:52:39.581159] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-1: parent translators are ready, attempting connect on =
transport
[2016-11-05 00:52:39.581483] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-0: changing port to 49156 (from 0)
[2016-11-05 00:52:39.583109] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-2: parent translators are ready, attempting connect on =
transport
[2016-11-05 00:52:39.585673] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-0: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-05 00:52:39.585849] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-1: changing port to 49156 (from 0)
[2016-11-05 00:52:39.587044] I [MSGID: 114020] [client.c:2113:notify] =
0-data2-client-3: parent translators are ready, attempting connect on =
transport
[2016-11-05 00:52:39.589467] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-0: =
Connected to data2-client-0, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-05 00:52:39.589499] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-0: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-05 00:52:39.589571] I [MSGID: 108005] =
[afr-common.c:4299:afr_notify] 0-data2-replicate-0: Subvolume =
'data2-client-0' came back up; going online.
[2016-11-05 00:52:39.589654] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-0: =
Server lk version =3D 1
[2016-11-05 00:52:39.589753] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-1: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-05 00:52:39.590026] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-2: changing port to 49156 (from 0)
[2016-11-05 00:52:39.590074] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-1: =
Connected to data2-client-1, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-05 00:52:39.590088] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-1: Server =
and Client lk-version numbers are not same, reopening the fds
Final graph:
=
+-------------------------------------------------------------------------=
-----+
1: volume data2-client-0
2: type protocol/client
3: option clnt-lk-version 1
4: option volfile-checksum 0
5: option volfile-key data2
6: option client-version 3.7.16
7: option process-uuid =
fai-kvm-1-gfs.ravnalaska.net-30520-2016/11/05-00:52:39:553646-data2-client=
-0-0-0
8: option fops-version 1298437
9: option ping-timeout 42
10: option remote-host fai-kvm-1-vmn.ravnalaska.net
11: option remote-subvolume /kvm2/gluster/data2/brick
12: option transport-type socket
13: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
14: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
15: option filter-O_DIRECT enable
16: option send-gids true
17: end-volume
18:
19: volume data2-client-1
20: type protocol/client
21: option clnt-lk-version 1
22: option volfile-checksum 0
23: option volfile-key data2
24: option client-version 3.7.16
25: option process-uuid =
fai-kvm-1-gfs.ravnalaska.net-30520-2016/11/05-00:52:39:553646-data2-client=
-1-0-0
26: option fops-version 1298437
27: option ping-timeout 42
28: option remote-host fai-kvm-2-vmn.ravnalaska.net
29: option remote-subvolume /kvm2/gluster/data2/brick
30: option transport-type socket
31: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
32: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
33: option filter-O_DIRECT enable
34: option send-gids true
35: end-volume
36:
37: volume data2-replicate-0
38: type cluster/replicate
39: option eager-lock enable
40: option quorum-type auto
41: subvolumes data2-client-0 data2-client-1
42: end-volume
43:
44: volume data2-client-2
45: type protocol/client
46: option ping-timeout 42
47: option remote-host fai-kvm-3-vmn.ravnalaska.net
48: option remote-subvolume /kvm2/gluster/data2/brick
49: option transport-type socket
50: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
51: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
52: option filter-O_DIRECT enable
53: option send-gids true
54: end-volume
55:
56: volume data2-client-3
57: type protocol/client
58: option ping-timeout 42
59: option remote-host fai-kvm-4-vmn.ravnalaska.net
60: option remote-subvolume /kvm2/gluster/data2/brick
61: option transport-type socket
62: option username 0b7a59b1-4aab-4041-a271-bd8a07e5cba5
63: option password 6dca1e9d-f6a8-490b-bb60-babc56eb131a
64: option filter-O_DIRECT enable
65: option send-gids true
66: end-volume
67:
68: volume data2-replicate-1
69: type cluster/replicate
70: option eager-lock enable
71: option quorum-type auto
72: subvolumes data2-client-2 data2-client-3
73: end-volume
74:
75: volume data2-stripe-0
76: type cluster/stripe
77: subvolumes data2-replicate-0 data2-replicate-1
78: end-volume
79:
80: volume data2-dht
81: type cluster/distribute
82: subvolumes data2-stripe-0
83: end-volume
84:
85: volume data2-write-behind
86: type performance/write-behind
87: subvolumes data2-dht
88: end-volume
89:
90: volume data2-readdir-ahead
91: type performance/readdir-ahead
92: subvolumes data2-write-behind
93: end-volume
94:
95: volume data2-open-behind
96: type performance/open-behind
97: subvolumes data2-readdir-ahead
98: end-volume
99:
100: volume data2
101: type debug/io-stats
102: option log-level INFO
103: option latency-measurement off
104: option count-fop-hits off
105: subvolumes data2-open-behind
106: end-volume
107:
108: volume meta-autoload
109: type meta
110: subvolumes data2
111: end-volume
112:
=
+-------------------------------------------------------------------------=
-----+
[2016-11-05 00:52:39.591502] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-1: =
Server lk version =3D 1
[2016-11-05 00:52:39.591817] I [rpc-clnt.c:1960:rpc_clnt_reconfig] =
0-data2-client-3: changing port to 49152 (from 0)
[2016-11-05 00:52:39.593714] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-2: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-05 00:52:39.594285] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-2: =
Connected to data2-client-2, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-05 00:52:39.594308] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-2: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-05 00:52:39.594365] I [MSGID: 108005] =
[afr-common.c:4299:afr_notify] 0-data2-replicate-1: Subvolume =
'data2-client-2' came back up; going online.
[2016-11-05 00:52:39.594552] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-2: =
Server lk version =3D 1
[2016-11-05 00:52:39.595616] I [MSGID: 114057] =
[client-handshake.c:1437:select_server_supported_programs] =
0-data2-client-3: Using Program GlusterFS 3.3, Num (1298437), Version =
(330)
[2016-11-05 00:52:39.596115] I [MSGID: 114046] =
[client-handshake.c:1213:client_setvolume_cbk] 0-data2-client-3: =
Connected to data2-client-3, attached to remote volume =
'/kvm2/gluster/data2/brick'.
[2016-11-05 00:52:39.596130] I [MSGID: 114047] =
[client-handshake.c:1224:client_setvolume_cbk] 0-data2-client-3: Server =
and Client lk-version numbers are not same, reopening the fds
[2016-11-05 00:52:39.600510] I [fuse-bridge.c:5171:fuse_graph_setup] =
0-fuse: switched to graph 0
[2016-11-05 00:52:39.600578] I [MSGID: 114035] =
[client-handshake.c:193:client_set_lk_version_cbk] 0-data2-client-3: =
Server lk version =3D 1
[2016-11-05 00:52:39.600654] I [fuse-bridge.c:4083:fuse_init] =
0-glusterfs-fuse: FUSE inited with protocol versions: glusterfs 7.22 =
kernel 7.22
[2016-11-05 00:52:39.601452] I [MSGID: 108031] =
[afr-common.c:2071:afr_local_discovery_cbk] 0-data2-replicate-0: =
selecting local read_child data2-client-0
[2016-11-05 00:52:41.619852] I [fuse-bridge.c:5012:fuse_thread_proc] =
0-fuse: unmounting /rhev/data-center/mnt/glusterSD/glustermount2:data2
[2016-11-05 00:52:41.620095] W [glusterfsd.c:1251:cleanup_and_exit] =
(-->/lib64/libpthread.so.0(+0x7dc5) [0x7f3444227dc5] =
-->/usr/sbin/glusterfs(glusterfs_sigwaiter+0xe5) [0x7f34458a2915] =
-->/usr/sbin/
glusterfs(cleanup_and_exit+0x6b) [0x7f34458a278b] ) 0-: received signum =
(15), shutting down
[2016-11-05 00:52:41.620110] I [fuse-bridge.c:5719:fini] 0-fuse: =
Unmounting '/rhev/data-center/mnt/glusterSD/glustermount2:data2'.
[root@fai-kvm-1-gfs glusterfs]#=
--Apple-Mail=_D3985462-55E4-4FE5-A078-30EC039EC062
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D""></div><div
class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); font-size: 12px; letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really =
loving the record green up date! Summmer!! yourself=E2=80=9D M=
att 22:39</span></font><div style=3D"color: rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Menlo;" =
class=3D""></div><font face=3D"Menlo"
style=3D"font-size: 12px;" =
class=3D""></font><span style=3D"color: rgb(0, 0, 0);
letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><font =
face=3D"Menlo" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"color: rgb(0, 0, 0);
=
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><font
=
face=3D"Menlo" style=3D"font-size: 12px;"
class=3D""><br =
class=3D""></font></div></div><span style=3D"color:
rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 12px;" =
class=3D""><br
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span
style=3D"color: =
rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br class=3D""><div><blockquote type=3D"cite"
class=3D""><div =
class=3D"">On Nov 6, 2016, at 9:50 PM, Sahina Bose <<a =
href=3D"mailto:sabose@redhat.com"
class=3D"">sabose(a)redhat.com</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><div
class=3D""><span =
style=3D"font-family: LucidaGrande; font-size: 13px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; float: none; display: inline =
!important;" class=3D"">However your volume configuration seems suspect
=
-"stripe 2 replica 2". Can you provide gluster volume info of your =
second storage domain gluster volume? The mount logs of the volume =
(under /var/log/glusterfs/rhev-datacenter..<volname>.log) from the =
host where the volume is being mounted will also help.</span><br =
style=3D"font-family: LucidaGrande; font-size: 13px; font-style: normal; =
font-variant-caps: normal; font-weight: normal; letter-spacing: normal; =
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><span
style=3D"font-family: =
LucidaGrande; font-size: 13px; font-style: normal; font-variant-caps: =
normal; font-weight: normal; letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
float: none; display: inline !important;" =
class=3D""> </span></div></blockquote></div><br
=
class=3D""></body></html>=
--Apple-Mail=_D3985462-55E4-4FE5-A078-30EC039EC062--
--Apple-Mail=_74966C91-4D7B-4631-992A-7104BF8281C2--
12:10 p.m.
On Mon, Nov 7, 2016 at 3:27 PM, Gary Pedretty <gary(a)ravnalaska.net> wrote:
[root@fai-kvm-1-gfs admin]# gluster volume status data2
Status of volume: data2
Gluster process TCP Port RDMA Port Online
Pid
------------------------------------------------------------
------------------
Brick fai-kvm-1-vmn.ravnalaska.net:/kvm2/gl
uster/data2/brick 49156 0 Y
3484
Brick fai-kvm-2-vmn.ravnalaska.net:/kvm2/gl
uster/data2/brick 49156 0 Y
34791
Brick fai-kvm-3-vmn.ravnalaska.net:/kvm2/gl
uster/data2/brick 49156 0 Y
177340
Brick fai-kvm-4-vmn.ravnalaska.net:/kvm2/gl
uster/data2/brick 49152 0 Y
146038
NFS Server on localhost 2049 0 Y
40844
Self-heal Daemon on localhost N/A N/A Y
40865
NFS Server on fai-kvm-2-gfs.ravnalaska.net 2049 0 Y
99905
Self-heal Daemon on fai-kvm-2-gfs.ravnalask
a.net N/A N/A Y
99915
NFS Server on fai-kvm-4-gfs.ravnalaska.net 2049 0 Y
176305
Self-heal Daemon on fai-kvm-4-gfs.ravnalask
a.net N/A N/A Y
176326
NFS Server on fai-kvm-3-gfs.ravnalaska.net 2049 0 Y
226271
Self-heal Daemon on fai-kvm-3-gfs.ravnalask
a.net N/A N/A Y
226287
Task Status of Volume data2
------------------------------------------------------------
------------------
There are no active volume tasks
[root@fai-kvm-1-gfs admin]# gluster volume info data2
Volume Name: data2
Type: Striped-Replicate
Volume ID: 20f85c9a-541b-4df4-9dba-44c5179bbfb0
Status: Started
Number of Bricks: 1 x 2 x 2 = 4
Transport-type: tcp
Bricks:
Brick1: fai-kvm-1-vmn.ravnalaska.net:/kvm2/gluster/data2/brick
Brick2: fai-kvm-2-vmn.ravnalaska.net:/kvm2/gluster/data2/brick
Brick3: fai-kvm-3-vmn.ravnalaska.net:/kvm2/gluster/data2/brick
Brick4: fai-kvm-4-vmn.ravnalaska.net:/kvm2/gluster/data2/brick
Options Reconfigured:
performance.readdir-ahead: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.stat-prefetch: off
cluster.eager-lock: enable
network.remote-dio: enable
cluster.quorum-type: auto
cluster.server-quorum-type: server
storage.owner-uid: 36
storage.owner-gid: 36
See attached file for the mount log.
Striped-Replicate is no longer supported in GlusterFS upstream. Instead,
you should be using a Distribute-Replicate with sharding enabled. Also when
using a gluster volume as storage domain, it is recommended to use replica
3.
From the mount logs, there is no indication as to why the volume is
unmounted frequently. Could you try again with a replica 3 volume that has
sharding enabled?
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net
<gary(a)eraalaska.net>
Systems Manager www.flyravn.com
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ “Love your neighbor as
Really loving the record green up date! Summmer!! yourself” Matt 22:39
------------------------------------------------------------------------
On Nov 6, 2016, at 9:50 PM, Sahina Bose <sabose(a)redhat.com> wrote:
However your volume configuration seems suspect -"stripe 2 replica 2". Can
you provide gluster volume info of your second storage domain gluster
volume? The mount logs of the volume (under
/var/log/glusterfs/rhev-datacenter..<volname>.log)
from the host where the volume is being mounted will also help.
5:55 a.m.
--Apple-Mail=_5640DDB5-E478-43B7-AB26-61A2B8B4688F
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
Solved:
Changing the second storage domain to a glusterfs Distributed Replicate =
with sharding turned on, works great. Thanks for the solution.
Gary
------------------------------------------------------------------------
Gary Pedretty gary(a)ravnalaska.net =
<mailto:gary@eraalaska.net>
Systems Manager www.flyravn.com =
<http://www.flyravn.com/>
Ravn Alaska /\ 907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238 fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your =
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D =
Matt 22:39
------------------------------------------------------------------------
On Nov 7, 2016, at 2:10 AM, Sahina Bose <sabose(a)redhat.com>
wrote:
=20
=20
=20
On Mon, Nov 7, 2016 at 3:27 PM, Gary Pedretty <gary(a)ravnalaska.net =
<mailto:gary@ravnalaska.net>> wrote:
[root@fai-kvm-1-gfs admin]# gluster volume status data2
Status of volume: data2
Gluster process TCP Port RDMA Port =
Online Pid
=
--------------------------------------------------------------------------=
----
Brick fai-kvm-1-vmn.ravnalaska.net =
<http://fai-kvm-1-vmn.ravnalaska.net/>:/kvm2/gl
uster/data2/brick 49156 0 Y
=
3484
Brick fai-kvm-2-vmn.ravnalaska.net =
<http://fai-kvm-2-vmn.ravnalaska.net/>:/kvm2/gl
uster/data2/brick 49156 0 Y
=
34791
Brick fai-kvm-3-vmn.ravnalaska.net =
<http://fai-kvm-3-vmn.ravnalaska.net/>:/kvm2/gl
uster/data2/brick 49156 0 Y
=
177340
Brick fai-kvm-4-vmn.ravnalaska.net =
<http://fai-kvm-4-vmn.ravnalaska.net/>:/kvm2/gl
uster/data2/brick 49152 0 Y
=
146038
NFS Server on localhost 2049 0 Y
=
40844
Self-heal Daemon on localhost N/A N/A Y
=
40865
NFS Server on fai-kvm-2-gfs.ravnalaska.net =
<http://fai-kvm-2-gfs.ravnalaska.net/> 2049 0 Y =
99905
Self-heal Daemon on fai-kvm-2-gfs.ravnalask
a.net <http://a.net/> N/A =
N/A
Y 99915
NFS Server on fai-kvm-4-gfs.ravnalaska.net =
<http://fai-kvm-4-gfs.ravnalaska.net/> 2049 0 Y =
176305
Self-heal Daemon on fai-kvm-4-gfs.ravnalask
a.net <http://a.net/> N/A =
N/A
Y 176326
NFS Server on fai-kvm-3-gfs.ravnalaska.net =
<http://fai-kvm-3-gfs.ravnalaska.net/> 2049 0 Y =
226271
Self-heal Daemon on fai-kvm-3-gfs.ravnalask
a.net <http://a.net/> N/A =
N/A
Y 226287
>=20
> Task Status of Volume data2
=
--------------------------------------------------------------------------=
----
There are no active volume tasks
=20
=20
[root@fai-kvm-1-gfs admin]# gluster volume info data2
=20
Volume Name: data2
Type: Striped-Replicate
Volume ID: 20f85c9a-541b-4df4-9dba-44c5179bbfb0
Status: Started
Number of Bricks: 1 x 2 x 2 =3D 4
Transport-type: tcp
Bricks:
Brick1: fai-kvm-1-vmn.ravnalaska.net =
<http://fai-kvm-1-vmn.ravnalaska.net/>:/kvm2/gluster/data2/brick
Brick2: fai-kvm-2-vmn.ravnalaska.net =
<http://fai-kvm-2-vmn.ravnalaska.net/>:/kvm2/gluster/data2/brick
Brick3: fai-kvm-3-vmn.ravnalaska.net =
<http://fai-kvm-3-vmn.ravnalaska.net/>:/kvm2/gluster/data2/brick
Brick4: fai-kvm-4-vmn.ravnalaska.net =
<http://fai-kvm-4-vmn.ravnalaska.net/>:/kvm2/gluster/data2/brick
Options Reconfigured:
performance.readdir-ahead: on
performance.quick-read: off
performance.read-ahead: off
performance.io <http://performance.io/>-cache: off
performance.stat-prefetch: off
cluster.eager-lock: enable
network.remote-dio: enable
cluster.quorum-type: auto
cluster.server-quorum-type: server
storage.owner-uid: 36
storage.owner-gid: 36
=20
=20
See attached file for the mount log.
=20
=20
Striped-Replicate is no longer supported in GlusterFS upstream. =
Instead, you
should be using a Distribute-Replicate with sharding =
enabled. Also when using a gluster volume as storage domain, it is =
recommended to use replica 3.
=20
=46rom the mount logs, there is no indication as to why the volume is =
unmounted
frequently. Could you try again with a replica 3 volume that =
has sharding enabled?
> =20
>=20
> Gary
>=20
>=20
=
------------------------------------------------------------------------
Gary Pedretty =
gary(a)ravnalaska.net <mailto:gary@eraalaska.net>
Systems Manager =
www.flyravn.com <http://www.flyravn.com/>
Ravn Alaska /\ =
907-450-7251
5245 Airport Industrial Road / \/\ 907-450-7238
=
fax
Fairbanks, Alaska 99709 /\ / \ \ Second greatest =
commandment
Serving All of Alaska / \/ /\ \ \/\ =E2=80=9CLove your
=
neighbor as
Really loving the record green up date! Summmer!! yourself=E2=80=9D
=
Matt 22:39
=
------------------------------------------------------------------------
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
=20
> On Nov 6, 2016, at 9:50 PM, Sahina Bose <sabose(a)redhat.com =
<mailto:sabose@redhat.com>> wrote:
>=20
> However your volume configuration seems suspect -"stripe 2 replica =
2". Can you provide gluster volume info of your second storage domain =
gluster volume? The mount logs of the volume (under =
/var/log/glusterfs/rhev-datacenter..<volname>.log) from the host where =
the volume is being mounted will also help.
> =20
=20
=20
=20
--Apple-Mail=_5640DDB5-E478-43B7-AB26-61A2B8B4688F
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D"">Solved:<div class=3D""><br
class=3D""></div><div =
class=3D"">Changing the second storage domain to a glusterfs Distributed =
Replicate with sharding turned on, works great. Thanks for the =
solution.</div><div class=3D""><br
class=3D""></div><div class=3D""><br =
class=3D""></div><div
class=3D"">Gary</div><div class=3D""><br =
class=3D""></div><div class=3D""><br
class=3D""></div><div class=3D""><div=
class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><div
style=3D"orphans: =
auto; text-align: start; text-indent: 0px; widows: auto; word-wrap: =
break-word; -webkit-nbsp-mode: space; -webkit-line-break: =
after-white-space;" class=3D""><div style=3D"orphans: auto;
text-align: =
start; text-indent: 0px; widows: auto; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"orphans: auto; text-align: start;
text-indent: =
0px; widows: auto; word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); font-size: 12px; letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px;" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div><div class=3D"">Gary Pedretty
=
=
<a =
href=3D"mailto:gary@eraalaska.net" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size: 12px;"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family: Monaco;" class=3D""><font
face=3D"Menlo" =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-size: 12px;"
class=3D"">Really =
loving the record green up date! Summmer!! yourself=E2=80=9D M=
att 22:39</span></font><div style=3D"color: rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Menlo;" =
class=3D""></div><font face=3D"Menlo"
style=3D"font-size: 12px;" =
class=3D""></font><span style=3D"color: rgb(0, 0, 0);
letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; font-size: 12px;" class=3D""><font =
face=3D"Menlo" class=3D""><div =
class=3D"">---------------------------------------------------------------=
---------</div></font></span><div style=3D"color: rgb(0, 0, 0);
=
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=3D""><font
=
face=3D"Menlo" style=3D"font-size: 12px;"
class=3D""><br =
class=3D""></font></div></div><span style=3D"color:
rgb(0, 0, 0); =
letter-spacing: normal; text-transform: none; white-space: normal; =
word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: 12px;" =
class=3D""><br
class=3D"Apple-interchange-newline"></span></div><span =
style=3D"color: rgb(0, 0, 0); letter-spacing: normal; text-transform: =
none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: =
0px; font-size: 12px;" class=3D""><br =
class=3D"Apple-interchange-newline"></span></div><span
style=3D"color: =
rgb(0, 0, 0); letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-size: =
12px;" class=3D""><br
class=3D"Apple-interchange-newline"></span></div><br=
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"></div><br =
class=3D"Apple-interchange-newline"><br =
class=3D"Apple-interchange-newline">
</div>
<br class=3D""><div><blockquote type=3D"cite"
class=3D""><div =
class=3D"">On Nov 7, 2016, at 2:10 AM, Sahina Bose <<a =
href=3D"mailto:sabose@redhat.com"
class=3D"">sabose(a)redhat.com</a>&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><div
class=3D""><div =
dir=3D"ltr" class=3D""><br class=3D""><div
class=3D"gmail_extra"><br =
class=3D""><div class=3D"gmail_quote">On Mon, Nov 7, 2016 at
3:27 PM, =
Gary Pedretty <span dir=3D"ltr" class=3D""><<a =
href=3D"mailto:gary@ravnalaska.net" target=3D"_blank" =
class=3D"">gary(a)ravnalaska.net</a>&gt;</span> wrote:<br =
class=3D""><blockquote class=3D"gmail_quote"
style=3D"margin:0 0 0 =
.8ex;border-left:1px #ccc solid;padding-left:1ex"><div =
style=3D"word-wrap:break-word" class=3D""><div =
class=3D"">[root@fai-kvm-1-gfs admin]# gluster volume status =
data2</div><div class=3D"">Status of volume:
data2</div><div =
class=3D"">Gluster process
=
TCP Port =
RDMA Port Online Pid</div><div =
class=3D"">------------------------------<wbr =
class=3D"">------------------------------<wbr =
class=3D"">------------------</div><div
class=3D"">Brick <a =
href=3D"http://fai-kvm-1-vmn.ravnalaska.net/" target=3D"_blank" =
class=3D"">fai-kvm-1-vmn.ravnalaska.net</a>:/<wbr =
class=3D"">kvm2/gl</div><div
class=3D"">uster/data2/brick =
=
49156 0
Y =
3484</div><div class=3D"">Brick <a =
href=3D"http://fai-kvm-2-vmn.ravnalaska.net/" target=3D"_blank" =
class=3D"">fai-kvm-2-vmn.ravnalaska.net</a>:/<wbr =
class=3D"">kvm2/gl</div><div
class=3D"">uster/data2/brick =
=
49156 0
Y =
34791</div><div class=3D"">Brick <a =
href=3D"http://fai-kvm-3-vmn.ravnalaska.net/" target=3D"_blank" =
class=3D"">fai-kvm-3-vmn.ravnalaska.net</a>:/<wbr =
class=3D"">kvm2/gl</div><div
class=3D"">uster/data2/brick =
=
49156 0
Y =
177340</div><div class=3D"">Brick <a =
href=3D"http://fai-kvm-4-vmn.ravnalaska.net/" target=3D"_blank" =
class=3D"">fai-kvm-4-vmn.ravnalaska.net</a>:/<wbr =
class=3D"">kvm2/gl</div><div
class=3D"">uster/data2/brick =
=
49152 0
Y =
146038</div><div class=3D"">NFS Server on
localhost =
2049 =
0 Y
=
40844</div><div class=3D"">Self-heal Daemon on localhost
=
N/A
N/A =
Y
40865</div><div =
class=3D"">NFS Server on <a
href=3D"http://fai-kvm-2-gfs.ravnalaska.net/" =
target=3D"_blank" class=3D"">fai-kvm-2-gfs.ravnalaska.net</a>
2049 =
0 Y
=
99905</div><div class=3D"">Self-heal Daemon on =
fai-kvm-2-gfs.ravnalask</div><div class=3D""><a
href=3D"http://a.net/" =
target=3D"_blank" class=3D"">a.net</a>
=
=
N/A
N/A =
Y
99915</div><div class=3D"">NFS =
Server on <a href=3D"http://fai-kvm-4-gfs.ravnalaska.net/" =
target=3D"_blank" class=3D"">fai-kvm-4-gfs.ravnalaska.net</a>
2049 =
0 Y
=
176305</div><div class=3D"">Self-heal Daemon on =
fai-kvm-4-gfs.ravnalask</div><div class=3D""><a
href=3D"http://a.net/" =
target=3D"_blank" class=3D"">a.net</a>
=
=
N/A
N/A =
Y
176326</div><div class=3D"">NFS=
Server on <a href=3D"http://fai-kvm-3-gfs.ravnalaska.net/" =
target=3D"_blank" class=3D"">fai-kvm-3-gfs.ravnalaska.net</a>
2049 =
0 Y
=
226271</div><div class=3D"">Self-heal Daemon on =
fai-kvm-3-gfs.ravnalask</div><div class=3D""><a
href=3D"http://a.net/" =
target=3D"_blank" class=3D"">a.net</a>
=
=
N/A
N/A =
Y
226287</div><div class=3D""><br=
class=3D""></div><div class=3D"">Task Status of Volume
data2</div><div =
class=3D"">------------------------------<wbr =
class=3D"">------------------------------<wbr =
class=3D"">------------------</div><div
class=3D"">There are no active =
volume tasks</div><div class=3D""><br
class=3D""></div><div class=3D""><br=
class=3D""></div><div class=3D""><div
class=3D"">[root@fai-kvm-1-gfs =
admin]# gluster volume info data2</div><div class=3D""><br =
class=3D""></div><div class=3D"">Volume Name:
data2</div><div =
class=3D"">Type: Striped-Replicate</div><div
class=3D"">Volume ID: =
20f85c9a-541b-4df4-9dba-<wbr class=3D"">44c5179bbfb0</div><div =
class=3D"">Status: Started</div><div class=3D"">Number
of Bricks: 1 x 2 =
x 2 =3D 4</div><div class=3D"">Transport-type:
tcp</div><div =
class=3D"">Bricks:</div><div class=3D"">Brick1: <a
=
href=3D"http://fai-kvm-1-vmn.ravnalaska.net/" target=3D"_blank" =
class=3D"">fai-kvm-1-vmn.ravnalaska.net</a>:/<wbr =
class=3D"">kvm2/gluster/data2/brick</div><div
class=3D"">Brick2: <a =
href=3D"http://fai-kvm-2-vmn.ravnalaska.net/" target=3D"_blank" =
class=3D"">fai-kvm-2-vmn.ravnalaska.net</a>:/<wbr =
class=3D"">kvm2/gluster/data2/brick</div><div
class=3D"">Brick3: <a =
href=3D"http://fai-kvm-3-vmn.ravnalaska.net/" target=3D"_blank" =
class=3D"">fai-kvm-3-vmn.ravnalaska.net</a>:/<wbr =
class=3D"">kvm2/gluster/data2/brick</div><div
class=3D"">Brick4: <a =
href=3D"http://fai-kvm-4-vmn.ravnalaska.net/" target=3D"_blank" =
class=3D"">fai-kvm-4-vmn.ravnalaska.net</a>:/<wbr =
class=3D"">kvm2/gluster/data2/brick</div><div
class=3D"">Options =
Reconfigured:</div><div class=3D"">performance.readdir-ahead: =
on</div><div class=3D"">performance.quick-read:
off</div><div =
class=3D"">performance.read-ahead: off</div><div
class=3D""><a =
href=3D"http://performance.io/" target=3D"_blank" =
class=3D"">performance.io</a>-cache: off</div><div =
class=3D"">performance.stat-prefetch: off</div><div =
class=3D"">cluster.eager-lock: enable</div><div =
class=3D"">network.remote-dio: enable</div><div =
class=3D"">cluster.quorum-type: auto</div><div =
class=3D"">cluster.server-quorum-type: server</div><div =
class=3D"">storage.owner-uid: 36</div><div
class=3D"">storage.owner-gid: =
36</div></div><div class=3D""><br
class=3D""></div><div class=3D""><br =
class=3D""></div><div class=3D"">See attached file for
the mount =
log.</div></div></blockquote><div class=3D""><br
class=3D""><br =
class=3D""></div><div class=3D"">Striped-Replicate is
no longer =
supported in GlusterFS upstream. Instead, you should be using a =
Distribute-Replicate with sharding enabled. Also when using a gluster =
volume as storage domain, it is recommended to use replica 3.<br =
class=3D""><br class=3D""></div><div
class=3D"">=46rom the mount logs, =
there is no indication as to why the volume is unmounted frequently. =
Could you try again with a replica 3 volume that has sharding =
enabled?<br class=3D""> <br
class=3D""></div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px =
solid rgb(204,204,204);padding-left:1ex"><div =
style=3D"word-wrap:break-word" class=3D""><span
class=3D"HOEnZb"><font =
color=3D"#888888" class=3D""><div class=3D""><br
class=3D""></div><div =
class=3D"">Gary</div><div class=3D""><br
class=3D""></div><div =
class=3D""></div></font></span></div><br
class=3D""><div =
style=3D"word-wrap:break-word" class=3D""><div
class=3D""></div><div =
class=3D"">
<div style=3D"letter-spacing: normal; text-align: start; text-indent: =
0px; text-transform: none; white-space: normal; word-spacing: 0px; =
word-wrap: break-word;" class=3D""><div style=3D"letter-spacing:
normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; word-wrap: break-word;" class=3D""><div =
style=3D"text-align:start;text-indent:0px;word-wrap:break-word" =
class=3D""><div =
style=3D"text-align:start;text-indent:0px;word-wrap:break-word" =
class=3D""><div =
style=3D"text-align:start;text-indent:0px;word-wrap:break-word" =
class=3D""><div =
style=3D"text-align:start;text-indent:0px;word-wrap:break-word" =
class=3D""><div =
style=3D"text-align:start;text-indent:0px;word-wrap:break-word" =
class=3D""><div =
style=3D"text-align:start;text-indent:0px;word-wrap:break-word" =
class=3D""><div =
style=3D"text-align:start;text-indent:0px;word-wrap:break-word" =
class=3D""><font style=3D"font-size: 12px; letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px;" =
face=3D"Menlo" class=3D""><div =
class=3D"">------------------------------<wbr =
class=3D"">------------------------------<wbr =
class=3D"">------------</div><div class=3D"">Gary
Pedretty =
=
<a =
href=3D"mailto:gary@eraalaska.net" target=3D"_blank" =
class=3D"">gary(a)ravnalaska.net</a></div><div
class=3D"">Systems Manager =
=
=
<a href=3D"http://www.flyravn.com/" target=3D"_blank" =
class=3D"">www.flyravn.com</a></div><div
class=3D"">Ravn Alaska =
=
/\
=
907-450-7251</div><div class=3D"">5245 Airport
Industrial =
Road / \/\
=
907-450-7238 fax</div><div
class=3D"">Fairbanks, Alaska =
99709 /\ /
\ \ =
Second greatest commandment</div></font><font face=3D"Monaco" =
class=3D""><span style=3D"font-size:12px"
class=3D"">Serving All of =
Alaska / \/ /\
\ \/\ =
=E2=80=9CLove your neighbor as</span></font><br =
style=3D"font-family:Monaco" class=3D""><font
style=3D"letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px;" =
face=3D"Menlo" class=3D""><span
style=3D"font-size:12px" class=3D"">Really=
loving the record green up date! Summmer!! yourself=E2=80=9D =
Matt 22:39</span></font><div style=3D"letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; =
font-family: Menlo;" class=3D""></div><font
style=3D"font-size:12px" =
face=3D"Menlo" class=3D""></font><span
style=3D"letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; font-size: =
12px;" class=3D""><font face=3D"Menlo"
class=3D""><div =
class=3D"">------------------------------<wbr =
class=3D"">------------------------------<wbr =
class=3D"">------------</div></font></span><div
style=3D"letter-spacing: =
normal; text-transform: none; white-space: normal; word-spacing: 0px;" =
class=3D""><font style=3D"font-size:12px"
face=3D"Menlo" class=3D""><br =
class=3D""></font></div></div><span
style=3D"letter-spacing: normal; =
text-transform: none; white-space: normal; word-spacing: 0px; font-size: =
12px;" class=3D""><br =
class=3D"m_8434586956379529661Apple-interchange-newline"></span></div><spa=
n style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; font-size: 12px;" class=3D""><br =
class=3D"m_8434586956379529661Apple-interchange-newline"></span></div><spa=
n style=3D"letter-spacing: normal; text-transform: none; white-space: =
normal; word-spacing: 0px; font-size: 12px;" class=3D""><br =
class=3D"m_8434586956379529661Apple-interchange-newline"></span></div><br
=
class=3D"m_8434586956379529661Apple-interchange-newline"></div><br
=
class=3D"m_8434586956379529661Apple-interchange-newline"></div><br
=
class=3D"m_8434586956379529661Apple-interchange-newline"></div><br
=
class=3D"m_8434586956379529661Apple-interchange-newline"></div><br
=
class=3D"m_8434586956379529661Apple-interchange-newline"></div><br
=
class=3D"m_8434586956379529661Apple-interchange-newline"><br =
class=3D"m_8434586956379529661Apple-interchange-newline">
</div>
<br class=3D""><div class=3D""><blockquote
type=3D"cite" class=3D""><div =
class=3D"">On Nov 6, 2016, at 9:50 PM, Sahina Bose <<a =
href=3D"mailto:sabose@redhat.com" target=3D"_blank" =
class=3D"">sabose(a)redhat.com</a>&gt; wrote:</div><br =
class=3D"m_8434586956379529661Apple-interchange-newline"><div =
class=3D""><span =
style=3D"font-family:LucidaGrande;font-size:13px;font-style:normal;font-va=
riant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:star=
t;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;=
float:none;display:inline!important" class=3D"">However your volume =
configuration seems suspect -"stripe 2 replica 2". Can you provide =
gluster volume info of your second storage domain gluster volume? The =
mount logs of the volume (under /var/log/glusterfs/rhev-<wbr =
class=3D"">datacenter..<volname>.log) from the host where the
=
volume is being mounted will also help.</span><br =
style=3D"font-family:LucidaGrande;font-size:13px;font-style:normal;font-va=
riant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:star=
t;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"=
class=3D""><span =
style=3D"font-family:LucidaGrande;font-size:13px;font-style:normal;font-va=
riant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:star=
t;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;=
float:none;display:inline!important" =
class=3D""> </span></div></blockquote></div><br
class=3D""></div><br =
class=3D""></blockquote></div><br
class=3D""></div></div>
</div></blockquote></div><br
class=3D""></div></body></html>=
--Apple-Mail=_5640DDB5-E478-43B7-AB26-61A2B8B4688F--
2734
days inactive
2749
days old
6 comments
3 participants
participants (3)
-
Gary Pedretty -
Maor Lipchuk -
Sahina Bose