<div dir="ltr">Hi Kanagaraj,<div><br></div><div>Please find the logs from here :- <a href="http://ur1.ca/jeszc">http://ur1.ca/jeszc</a></div><div><br></div><div><img src="cid:ii_14add9ba77567232" alt="Inline image 1" width="389" height="258"><br></div><div><br></div><div><img src="cid:ii_14add9c06888f54e" alt="Inline image 2" width="558" height="323"><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jan 12, 2015 at 1:02 PM, Kanagaraj <span dir="ltr"><<a href="mailto:kmayilsa@redhat.com" target="_blank">kmayilsa@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Looks like there are some failures in gluster. <br>
Can you send the log output from glusterd log file from the relevant
hosts?<br>
<br>
Thanks,<br>
Kanagaraj<div><div class="h5"><br>
<br>
<div>On 01/12/2015 10:24 AM, Punit Dambiwal
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi,
<div><br>
</div>
<div>Is there any one from gluster can help me here :- </div>
<div><br>
</div>
<div>Engine logs :- </div>
<div><br>
</div>
<div>
<div>2015-01-12 12:50:33,841 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:34,725 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:36,824 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:36,853 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:36,866 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:37,751 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:39,849 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:39,878 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:39,890 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:40,776 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:42,878 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:42,903 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:42,916 INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-12) Failed to acquire lock
and wait lock EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
<div>2015-01-12 12:50:43,771 INFO
[org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand]
(ajp--127.0.0.1-8702-1) [330ace48] FINISH,
CreateGlusterVolumeVDSCommand, log id: 303e70a4</div>
<div>2015-01-12 12:50:43,780 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ajp--127.0.0.1-8702-1) [330ace48] Correlation ID: 330ace48,
Job ID: 896a69b3-a678-40a7-bceb-3635e4062aa0, Call Stack:
null, Custom Event ID: -1, Message: Creation of Gluster
Volume vol01 failed.</div>
<div>2015-01-12 12:50:43,785 INFO
[org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand]
(ajp--127.0.0.1-8702-1) [330ace48] Lock freed to object
EngineLock [exclusiveLocks= key:
00000001-0001-0001-0001-000000000300 value: GLUSTER</div>
<div>, sharedLocks= ]</div>
</div>
<div><br>
</div>
<div><img src="cid:part1.02060703.06090409@redhat.com" alt="Inline image 2" height="456" width="558"><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, Jan 11, 2015 at 6:48 PM, Martin
Pavlík <span dir="ltr"><<a href="mailto:mpavlik@redhat.com" target="_blank">mpavlik@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">Hi Punit,
<div><br>
</div>
<div>unfortunately I’am not that good with the gluster, I
was just following the obvious clue from the log. Could
you try on the nodes if the packages are even available
for installation </div>
<span>
<div><br>
</div>
<div>yum install <span style="white-space:pre-wrap">gluster-swift
</span><span style="white-space:pre-wrap">gluster-swift-object
</span><span style="white-space:pre-wrap">gluster-swift-plugin
</span><span style="white-space:pre-wrap">gluster-swift-account
</span><span style="white-space:pre-wrap">gluster-swift-proxy
</span><span style="white-space:pre-wrap">gluster-swift-doc
</span><span style="white-space:pre-wrap">gluster-swift-container
</span><span style="white-space:pre-wrap">glusterfs-geo-replication</span></div>
<div><span style="white-space:pre-wrap"><br>
</span></div>
<div><span style="white-space:pre-wrap">if not you could
try to get them in official gluster repo. </span><span style="font-family:Verdana,Geneva,sans-serif;font-size:x-small;line-height:14px;background-color:rgb(255,255,255)"><a href="http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo" target="_blank">http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo</a></span></div>
<div><span style="white-space:pre-wrap"><br>
</span></div>
<div><span style="white-space:pre-wrap">HTH</span></div>
<div><span style="white-space:pre-wrap"><br>
</span></div>
</span>
<div><span><font color="#888888"><span style="white-space:pre-wrap">M.</span></font></span>
<div>
<div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>
<blockquote type="cite">
<div>On 10 Jan 2015, at 04:35, Punit Dambiwal
<<a href="mailto:hypunit@gmail.com" target="_blank">hypunit@gmail.com</a>>
wrote:</div>
<br>
<div>
<div dir="ltr">Hi Martin,
<div><br>
</div>
<div>I installed gluster from ovirt
repo....is it require to install those
packages manually ??</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Jan 9, 2015
at 7:19 PM, Martin Pavlík <span dir="ltr"><<a href="mailto:mpavlik@redhat.com" target="_blank">mpavlik@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div style="word-wrap:break-word">
<pre style="word-wrap:break-word;white-space:pre-wrap">Hi Punit,</pre>
<pre style="word-wrap:break-word;white-space:pre-wrap">can you verify that nodes contain cluster packages from the following log?</pre>
<pre style="word-wrap:break-word;white-space:pre-wrap">Thread-14::DEBUG::2015-01-09 18:06:28,823::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift',) not found
Thread-14::DEBUG::2015-01-09 18:06:28,825::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-object',) not found
Thread-14::DEBUG::2015-01-09 18:06:28,826::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-plugin',) not found
Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-account',) not found
Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-proxy',) not found
Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-doc',) not found
Thread-14::DEBUG::2015-01-09 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-container',) not found
Thread-14::DEBUG::2015-01-09 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package ('glusterfs-geo-replication',) not found</pre>
<div><br>
</div>
<div>M.</div>
<div><br>
</div>
<div>
<blockquote type="cite">
<div>
<div>
<div>On 09 Jan 2015, at 11:13,
Punit Dambiwal <<a href="mailto:hypunit@gmail.com" target="_blank">hypunit@gmail.com</a>>
wrote:</div>
<br>
</div>
</div>
<div>
<div>
<div>
<div dir="ltr">Hi Kanagaraj,
<div><br>
</div>
<div>Please find the
attached logs :- </div>
<div><br>
</div>
<div>Engine Logs :- <a href="http://ur1.ca/jdopt" target="_blank">http://ur1.ca/jdopt</a></div>
<div>VDSM Logs :- <a href="http://ur1.ca/jdoq9" target="_blank">http://ur1.ca/jdoq9</a></div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Thu, Jan 8, 2015 at 6:05
PM, Kanagaraj <span dir="ltr"><<a href="mailto:kmayilsa@redhat.com" target="_blank">kmayilsa@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Do
you see any errors
in the UI?<br>
<br>
Also please provide
the engine.log and
vdsm.log when the
failure occured.<br>
<br>
Thanks,<br>
Kanagaraj
<div>
<div><br>
<br>
<div>On
01/08/2015
02:25 PM,
Punit Dambiwal
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Hi
Martin,
<div><br>
</div>
<div>The steps
are below :- </div>
<div><br>
</div>
<div>1. Step
the ovirt
engine on the
one server...</div>
<div>2.
Installed
centos 7 on 4
host node
servers..</div>
<div>3. I am
using host
node
(compute+storage)....now
i have added
all 4 nodes to
engine...</div>
<div>4. Create
the gluster
volume from
GUI...</div>
<div><br>
</div>
<div>Network
:- </div>
<div>eth0 :-
public network
(1G)</div>
<div>eth1+eth2=bond0=
VM public
network (1G)</div>
<div>eth3+eth4=bond1=ovirtmgmt+storage
(10G private
network)</div>
<div><br>
</div>
<div>every
hostnode has
24
bricks=24*4(distributed
replicated)</div>
<div><br>
</div>
<div>Thanks,</div>
<div>Punit</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On
Thu, Jan 8,
2015 at 3:20
PM, Martin
Pavlík <span dir="ltr"><<a href="mailto:mpavlik@redhat.com" target="_blank">mpavlik@redhat.com</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi
Punit,<br>
<br>
can you please
provide also
errors from
/var/log/vdsm/vdsm.log
and
/var/log/vdsm/vdsmd.log<br>
<br>
it would be
really helpful
if you
provided exact
steps how to
reproduce the
problem.<br>
<br>
regards<br>
<br>
Martin Pavlik
- rhev QE<br>
<div>
<div>> On
08 Jan 2015,
at 03:06,
Punit Dambiwal
<<a href="mailto:hypunit@gmail.com" target="_blank">hypunit@gmail.com</a>>
wrote:<br>
><br>
> Hi,<br>
><br>
> I try to
add gluster
volume but it
failed...<br>
><br>
> Ovirt :-
3.5<br>
> VDSM :-
vdsm-4.16.7-1.gitdb83943.el7<br>
> KVM :-
1.5.3 -
60.el7_0.2<br>
>
libvirt-1.1.1-29.el7_0.4<br>
> Glusterfs
:-
glusterfs-3.5.3-1.el7<br>
><br>
> Engine
Logs :-<br>
><br>
>
2015-01-08
09:57:52,569
INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-16)
Failed to
acquire lock
and wait lock
EngineLock
[exclusiveLocks=
key:
00000001-0001-0001-0001-000000000300
value: GLUSTER<br>
> ,
sharedLocks= ]<br>
>
2015-01-08
09:57:52,609
INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-16)
Failed to
acquire lock
and wait lock
EngineLock
[exclusiveLocks=
key:
00000001-0001-0001-0001-000000000300
value: GLUSTER<br>
> ,
sharedLocks= ]<br>
>
2015-01-08
09:57:55,582
INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-16)
Failed to
acquire lock
and wait lock
EngineLock
[exclusiveLocks=
key:
00000001-0001-0001-0001-000000000300
value: GLUSTER<br>
> ,
sharedLocks= ]<br>
>
2015-01-08
09:57:55,591
INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-16)
Failed to
acquire lock
and wait lock
EngineLock
[exclusiveLocks=
key:
00000001-0001-0001-0001-000000000300
value: GLUSTER<br>
> ,
sharedLocks= ]<br>
>
2015-01-08
09:57:55,596
INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-16)
Failed to
acquire lock
and wait lock
EngineLock
[exclusiveLocks=
key:
00000001-0001-0001-0001-000000000300
value: GLUSTER<br>
> ,
sharedLocks= ]<br>
>
2015-01-08
09:57:55,633
INFO
[org.ovirt.engine.core.bll.lock.InMemoryLockManager]
(DefaultQuartzScheduler_Worker-16)
Failed to
acquire lock
and wait lock
EngineLock
[exclusiveLocks=
key:
00000001-0001-0001-0001-000000000300
value: GLUSTER<br>
> ,
sharedLocks= ]<br>
> ^C<br>
><br>
><br>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
<span><216 09-Jan-15.jpg></span><span><217
09-Jan-15.jpg></span></div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div>