
From: "Punit Dambiwal" <hypunit@gmail.com> To: "Kanagaraj" <kmayilsa@redhat.com> Cc: "Martin Pavl=C3=ADk" <mpavlik@redhat.com>, "Vijay Bellur" = <vbellur@redhat.com>, "Kaushal M" <kshlmster@gmail.com>, users@ovirt.org, gluster-users@gluster.org Sent: Monday, January 12, 2015 3:36:43 PM Subject: Re: Failed to create volume in OVirt with gluster
Hi Kanagaraj,
Please find the logs from here :- http://ur1.ca/jeszc
[image: Inline image 1]
[image: Inline image 2]
On Mon, Jan 12, 2015 at 1:02 PM, Kanagaraj <kmayilsa@redhat.com> = wrote:
Looks like there are some failures in gluster. Can you send the log output from glusterd log file from the relevant = hosts?
Thanks, Kanagaraj
On 01/12/2015 10:24 AM, Punit Dambiwal wrote:
Hi,
Is there any one from gluster can help me here :-
Engine logs :-
2015-01-12 12:50:33,841 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:34,725 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:36,824 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:36,853 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:36,866 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:37,751 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:39,849 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:39,878 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:39,890 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:40,776 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:42,878 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:42,903 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:42,916 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks=3D ] 2015-01-12 12:50:43,771 INFO = [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp--127.0.0.1-8702-1) [330ace48] FINISH, = CreateGlusterVolumeVDSCommand, log id: 303e70a4 2015-01-12 12:50:43,780 ERROR = [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-1) [330ace48] Correlation ID: 330ace48, Job ID: 896a69b3-a678-40a7-bceb-3635e4062aa0, Call Stack: null, Custom Event = ID: -1, Message: Creation of Gluster Volume vol01 failed. 2015-01-12 12:50:43,785 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp--127.0.0.1-8702-1) [330ace48] Lock freed to object EngineLock [exclusiveLocks=3D key: 00000001-0001-0001-0001-000000000300 value: = GLUSTER , sharedLocks=3D ]
[image: Inline image 2]
On Sun, Jan 11, 2015 at 6:48 PM, Martin Pavl=C3=ADk = <mpavlik@redhat.com> wrote:
Hi Punit,
unfortunately I=E2=80=99am not that good with the gluster, I was = just following the obvious clue from the log. Could you try on the nodes if the =
are even available for installation
yum install gluster-swift gluster-swift-object = gluster-swift-plugin gluster-swift-account gluster-swift-proxy gluster-swift-doc gluster-swift-container glusterfs-geo-replication
if not you could try to get them in official gluster repo. = http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs= -epel.repo
HTH
M.
On 10 Jan 2015, at 04:35, Punit Dambiwal <hypunit@gmail.com> = wrote:
Hi Martin,
I installed gluster from ovirt repo....is it require to install =
This is a multipart message in MIME format. ------=_NextPart_000_00B5_01D02F1E.B7FDC3D0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Are you using ctdb??? and did you specify Lock=3DFalse in = /etc/nfsmount.conf =20 Can you give a full run down of topology, and has this ever been working = or is it a new deployment? =20 Donny D =20 From: users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] On Behalf = Of Punit Dambiwal Sent: Monday, January 12, 2015 7:18 PM To: Kanagaraj Mayilsamy Cc: gluster-users@gluster.org; Kaushal M; users@ovirt.org Subject: Re: [ovirt-users] Failed to create volume in OVirt with gluster =20 Hi, =20 Please find the more details on this ....can anybody from gluster will = help me here :-=20 =20 =20 Gluster CLI Logs :- /var/log/glusterfs/cli.log =20 [2015-01-13 02:06:23.071969] T [cli.c:264:cli_rpc_notify] 0-glusterfs: = got RPC_CLNT_CONNECT [2015-01-13 02:06:23.072012] T = [cli-quotad-client.c:94:cli_quotad_notify] 0-glusterfs: got = RPC_CLNT_CONNECT [2015-01-13 02:06:23.072024] I [socket.c:2344:socket_event_handler] = 0-transport: disconnecting now [2015-01-13 02:06:23.072055] T = [cli-quotad-client.c:100:cli_quotad_notify] 0-glusterfs: got = RPC_CLNT_DISCONNECT [2015-01-13 02:06:23.072131] T [rpc-clnt.c:1381:rpc_clnt_record] = 0-glusterfs: Auth Info: pid: 0, uid: 0, gid: 0, owner: [2015-01-13 02:06:23.072176] T = [rpc-clnt.c:1238:rpc_clnt_record_build_header] 0-rpc-clnt: Request = fraglen 128, payload: 64, rpc hdr: 64 [2015-01-13 02:06:23.072572] T [socket.c:2863:socket_connect] (--> = /usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1e0)[0x7fed02f15420] = (--> = /usr/lib64/glusterfs/3.6.1/rpc-transport/socket.so(+0x7293)[0x7fed001a429= 3] (--> /usr/lib64/libgfrpc.so.0(rpc_clnt_submit+0x468)[0x7fed0266df98] = (--> /usr/sbin/gluster(cli_submit_request+0xdb)[0x40a9bb] (--> = /usr/sbin/gluster(cli_cmd_submit+0x8e)[0x40b7be] ))))) 0-glusterfs: = connect () called on transport already connected [2015-01-13 02:06:23.072616] T [rpc-clnt.c:1573:rpc_clnt_submit] = 0-rpc-clnt: submitted request (XID: 0x1 Program: Gluster CLI, ProgVers: = 2, Proc: 27) to rpc-transport (glusterfs) [2015-01-13 02:06:23.072633] D [rpc-clnt-ping.c:231:rpc_clnt_start_ping] = 0-glusterfs: ping timeout is 0, returning [2015-01-13 02:06:23.075930] T [rpc-clnt.c:660:rpc_clnt_reply_init] = 0-glusterfs: received rpc message (RPC XID: 0x1 Program: Gluster CLI, = ProgVers: 2, Proc: 27) from rpc-transport (glusterfs) [2015-01-13 02:06:23.075976] D [cli-rpc-ops.c:6548:gf_cli_status_cbk] = 0-cli: Received response to status cmd [2015-01-13 02:06:23.076025] D [cli-cmd.c:384:cli_cmd_submit] 0-cli: = Returning 0 [2015-01-13 02:06:23.076049] D [cli-rpc-ops.c:6811:gf_cli_status_volume] = 0-cli: Returning: 0 [2015-01-13 02:06:23.076192] D = [cli-xml-output.c:84:cli_begin_xml_output] 0-cli: Returning 0 [2015-01-13 02:06:23.076244] D = [cli-xml-output.c:131:cli_xml_output_common] 0-cli: Returning 0 [2015-01-13 02:06:23.076256] D = [cli-xml-output.c:1375:cli_xml_output_vol_status_begin] 0-cli: Returning = 0 [2015-01-13 02:06:23.076437] D [cli-xml-output.c:108:cli_end_xml_output] = 0-cli: Returning 0 [2015-01-13 02:06:23.076459] D = [cli-xml-output.c:1398:cli_xml_output_vol_status_end] 0-cli: Returning 0 [2015-01-13 02:06:23.076490] I [input.c:36:cli_batch] 0-: Exiting with: = 0 =20 Command log :- /var/log/glusterfs/.cmd_log_history =20 Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:10:35.836676] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:16:25.956514] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:17:36.977833] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:21:07.048053] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:26:57.168661] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:28:07.194428] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:30:27.256667] : volume status vol01 : FAILED : Locking = failed on cpu02.zne01.hkg1.stack.com. Please check log file for details. Locking failed on cpu03.zne01.hkg1.stack.com. Please check log file for = details. Locking failed on cpu04.zne01.hkg1.stack.com. Please check log file for = details. [2015-01-13 01:34:58.350748] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:36:08.375326] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:36:08.386470] : volume status vol01 : FAILED : Locking = failed on cpu02.zne01.hkg1.stack.com. Please check log file for details. Locking failed on cpu03.zne01.hkg1.stack.com. Please check log file for = details. Locking failed on cpu04.zne01.hkg1.stack.com. Please check log file for = details. [2015-01-13 01:42:59.524215] : volume stop vol01 : FAILED : Locking = failed on cpu02.zne01.hkg1.stack.com. Please check log file for details. Locking failed on cpu03.zne01.hkg1.stack.com. Please check log file for = details. Locking failed on cpu04.zne01.hkg1.stack.com. Please check log file for = details. [2015-01-13 01:45:10.550659] : volume status all tasks : FAILED : = Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. Staging failed on 00000000-0000-0000-0000-000000000000. Please check log = file for details. [2015-01-13 01:46:10.656802] : volume status all tasks : SUCCESS [2015-01-13 01:51:02.796031] : volume status all tasks : SUCCESS [2015-01-13 01:52:02.897804] : volume status all tasks : SUCCESS [2015-01-13 01:55:25.841070] : system:: uuid get : SUCCESS [2015-01-13 01:55:26.752084] : system:: uuid get : SUCCESS [2015-01-13 01:55:32.499049] : system:: uuid get : SUCCESS [2015-01-13 01:55:38.716907] : system:: uuid get : SUCCESS [2015-01-13 01:56:52.905899] : volume status all tasks : SUCCESS [2015-01-13 01:58:53.109613] : volume status all tasks : SUCCESS [2015-01-13 02:03:26.769430] : system:: uuid get : SUCCESS [2015-01-13 02:04:22.859213] : volume status all tasks : SUCCESS [2015-01-13 02:05:22.970393] : volume status all tasks : SUCCESS [2015-01-13 02:06:23.075823] : volume status all tasks : SUCCESS =20 =20 On Mon, Jan 12, 2015 at 10:53 PM, Kanagaraj Mayilsamy = <kmayilsa@redhat.com> wrote: I can see the failures in glusterd log. Can someone from glusterfs dev pls help on this? Thanks, Kanagaraj ----- Original Message ----- packages those
packages manually ??
On Fri, Jan 9, 2015 at 7:19 PM, Martin Pavl=C3=ADk = <mpavlik@redhat.com> wrote:
Hi Punit,
can you verify that nodes contain cluster packages from the = following log?
Thread-14::DEBUG::2015-01-09 18:06:28,823::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift',) not found Thread-14::DEBUG::2015-01-09 18:06:28,825::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-object',) not found Thread-14::DEBUG::2015-01-09 18:06:28,826::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-plugin',) not found Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-account',) not found Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-proxy',) not found Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-doc',) not found Thread-14::DEBUG::2015-01-09 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-container',) not found Thread-14::DEBUG::2015-01-09 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package ('glusterfs-geo-replication',) not found
M.
On 09 Jan 2015, at 11:13, Punit Dambiwal <hypunit@gmail.com> = wrote:
Hi Kanagaraj,
Please find the attached logs :-
Engine Logs :- http://ur1.ca/jdopt VDSM Logs :- http://ur1.ca/jdoq9
On Thu, Jan 8, 2015 at 6:05 PM, Kanagaraj <kmayilsa@redhat.com> = wrote:
Do you see any errors in the UI?
Also please provide the engine.log and vdsm.log when the failure occured.
Thanks, Kanagaraj
On 01/08/2015 02:25 PM, Punit Dambiwal wrote:
Hi Martin,
The steps are below :-
1. Step the ovirt engine on the one server... 2. Installed centos 7 on 4 host node servers.. 3. I am using host node (compute+storage)....now i have added all = 4 nodes to engine... 4. Create the gluster volume from GUI...
Network :- eth0 :- public network (1G) eth1+eth2=3Dbond0=3D VM public network (1G) eth3+eth4=3Dbond1=3Dovirtmgmt+storage (10G private network)
every hostnode has 24 bricks=3D24*4(distributed replicated)
Thanks, Punit
On Thu, Jan 8, 2015 at 3:20 PM, Martin Pavl=C3=ADk = <mpavlik@redhat.com> wrote:
Hi Punit,
can you please provide also errors from /var/log/vdsm/vdsm.log = and /var/log/vdsm/vdsmd.log
it would be really helpful if you provided exact steps how to reproduce the problem.
regards
Martin Pavlik - rhev QE > On 08 Jan 2015, at 03:06, Punit Dambiwal <hypunit@gmail.com> = wrote: > > Hi, > > I try to add gluster volume but it failed... > > Ovirt :- 3.5 > VDSM :- vdsm-4.16.7-1.gitdb83943.el7 > KVM :- 1.5.3 - 60.el7_0.2 > libvirt-1.1.1-29.el7_0.4 > Glusterfs :- glusterfs-3.5.3-1.el7 > > Engine Logs :- > > 2015-01-08 09:57:52,569 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and = wait lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER > , sharedLocks=3D ] > 2015-01-08 09:57:52,609 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and = wait lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER > , sharedLocks=3D ] > 2015-01-08 09:57:55,582 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and = wait lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER > , sharedLocks=3D ] > 2015-01-08 09:57:55,591 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and = wait lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER > , sharedLocks=3D ] > 2015-01-08 09:57:55,596 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and = wait lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER > , sharedLocks=3D ] > 2015-01-08 09:57:55,633 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and = wait lock EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER > , sharedLocks=3D ] > ^C > >
<216 09-Jan-15.jpg><217 09-Jan-15.jpg>
> > (ajp--127.0.0.1-8702-1) [330ace48] Correlation ID: 330ace48, = Job ID:<br>> > 896a69b3-a678-40a7-bceb-3635e4062aa0, Call Stack: = null, Custom Event ID:<br>> > -1, Message: Creation of Gluster = Volume vol01 failed.<br>> > 2015-01-12 12:50:43,785 INFO<br>> = > = [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand]<br>> = > (ajp--127.0.0.1-8702-1) [330ace48] Lock freed to object = EngineLock<br>> > [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300 value: GLUSTER<br>> > , = sharedLocks=3D ]<br>> ><br>> > [image: Inline image = 2]<br>> ><br>> ><br>> > On Sun, Jan 11, 2015 at 6:48 = PM, Martin Pavl=C3=ADk <<a =
=20 ------=_NextPart_000_00B5_01D02F1E.B7FDC3D0 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable <html xmlns:v=3D"urn:schemas-microsoft-com:vml" = xmlns:o=3D"urn:schemas-microsoft-com:office:office" = xmlns:w=3D"urn:schemas-microsoft-com:office:word" = xmlns:m=3D"http://schemas.microsoft.com/office/2004/12/omml" = xmlns=3D"http://www.w3.org/TR/REC-html40"><head><meta = http-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8"><meta = name=3DGenerator content=3D"Microsoft Word 14 (filtered = medium)"><style><!-- /* Font Definitions */ @font-face {font-family:Calibri; panose-1:2 15 5 2 2 2 4 3 2 4;} @font-face {font-family:Tahoma; panose-1:2 11 6 4 3 5 4 4 2 4;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {margin:0in; margin-bottom:.0001pt; font-size:12.0pt; font-family:"Times New Roman","serif";} a:link, span.MsoHyperlink {mso-style-priority:99; color:blue; text-decoration:underline;} a:visited, span.MsoHyperlinkFollowed {mso-style-priority:99; color:purple; text-decoration:underline;} span.EmailStyle17 {mso-style-type:personal-reply; font-family:"Calibri","sans-serif"; color:#1F497D;} .MsoChpDefault {mso-style-type:export-only; font-family:"Calibri","sans-serif";} @page WordSection1 {size:8.5in 11.0in; margin:1.0in 1.0in 1.0in 1.0in;} div.WordSection1 {page:WordSection1;} --></style><!--[if gte mso 9]><xml> <o:shapedefaults v:ext=3D"edit" spidmax=3D"1026" /> </xml><![endif]--><!--[if gte mso 9]><xml> <o:shapelayout v:ext=3D"edit"> <o:idmap v:ext=3D"edit" data=3D"1" /> </o:shapelayout></xml><![endif]--></head><body lang=3DEN-US link=3Dblue = vlink=3Dpurple><div class=3DWordSection1><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497= D'>Are you using ctdb??? and did you specify Lock=3DFalse in = /etc/nfsmount.conf<o:p></o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497= D'><o:p> </o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497= D'>Can you give a full run down of topology, and has this ever been = working or is it a new deployment?<o:p></o:p></span></p><p = class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497= D'><o:p> </o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497= D'>Donny D<o:p></o:p></span></p><p class=3DMsoNormal><span = style=3D'font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497= D'><o:p> </o:p></span></p><p class=3DMsoNormal><b><span = style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'>From:</span>= </b><span style=3D'font-size:10.0pt;font-family:"Tahoma","sans-serif"'> = users-bounces@ovirt.org [mailto:users-bounces@ovirt.org] <b>On Behalf Of = </b>Punit Dambiwal<br><b>Sent:</b> Monday, January 12, 2015 7:18 = PM<br><b>To:</b> Kanagaraj Mayilsamy<br><b>Cc:</b> = gluster-users@gluster.org; Kaushal M; users@ovirt.org<br><b>Subject:</b> = Re: [ovirt-users] Failed to create volume in OVirt with = gluster<o:p></o:p></span></p><p = class=3DMsoNormal><o:p> </o:p></p><div><p = class=3DMsoNormal>Hi,<o:p></o:p></p><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p = class=3DMsoNormal>Please find the more details on this ....can anybody = from gluster will help me here :- <o:p></o:p></p><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><p = class=3DMsoNormal>Gluster CLI Logs = :- /var/log/glusterfs/cli.log<o:p></o:p></p></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.071969] T = [cli.c:264:cli_rpc_notify] 0-glusterfs: got = RPC_CLNT_CONNECT<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.072012] T = [cli-quotad-client.c:94:cli_quotad_notify] 0-glusterfs: got = RPC_CLNT_CONNECT<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.072024] I = [socket.c:2344:socket_event_handler] 0-transport: disconnecting = now<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.072055] T [cli-quotad-client.c:100:cli_quotad_notify] = 0-glusterfs: got RPC_CLNT_DISCONNECT<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.072131] T = [rpc-clnt.c:1381:rpc_clnt_record] 0-glusterfs: Auth Info: pid: 0, uid: = 0, gid: 0, owner:<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.072176] T = [rpc-clnt.c:1238:rpc_clnt_record_build_header] 0-rpc-clnt: Request = fraglen 128, payload: 64, rpc hdr: 64<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.072572] T = [socket.c:2863:socket_connect] (--> = /usr/lib64/libglusterfs.so.0(_gf_log_callingfn+0x1e0)[0x7fed02f15420] = (--> = /usr/lib64/glusterfs/3.6.1/rpc-transport/socket.so(+0x7293)[0x7fed001a429= 3] (--> = /usr/lib64/libgfrpc.so.0(rpc_clnt_submit+0x468)[0x7fed0266df98] (--> = /usr/sbin/gluster(cli_submit_request+0xdb)[0x40a9bb] (--> = /usr/sbin/gluster(cli_cmd_submit+0x8e)[0x40b7be] ))))) 0-glusterfs: = connect () called on transport already = connected<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.072616] T [rpc-clnt.c:1573:rpc_clnt_submit] 0-rpc-clnt: = submitted request (XID: 0x1 Program: Gluster CLI, ProgVers: 2, Proc: 27) = to rpc-transport (glusterfs)<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.072633] D = [rpc-clnt-ping.c:231:rpc_clnt_start_ping] 0-glusterfs: ping timeout is = 0, returning<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.075930] T [rpc-clnt.c:660:rpc_clnt_reply_init] 0-glusterfs: = received rpc message (RPC XID: 0x1 Program: Gluster CLI, ProgVers: 2, = Proc: 27) from rpc-transport (glusterfs)<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.075976] D = [cli-rpc-ops.c:6548:gf_cli_status_cbk] 0-cli: Received response to = status cmd<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.076025] D [cli-cmd.c:384:cli_cmd_submit] 0-cli: Returning = 0<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.076049] D [cli-rpc-ops.c:6811:gf_cli_status_volume] 0-cli: = Returning: 0<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.076192] D [cli-xml-output.c:84:cli_begin_xml_output] 0-cli: = Returning 0<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.076244] D [cli-xml-output.c:131:cli_xml_output_common] 0-cli: = Returning 0<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.076256] D = [cli-xml-output.c:1375:cli_xml_output_vol_status_begin] 0-cli: Returning = 0<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.076437] D [cli-xml-output.c:108:cli_end_xml_output] 0-cli: = Returning 0<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.076459] D [cli-xml-output.c:1398:cli_xml_output_vol_status_end] = 0-cli: Returning 0<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 02:06:23.076490] I [input.c:36:cli_batch] = 0-: Exiting with: 0<o:p></o:p></p></div></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div></div><div><p = class=3DMsoNormal>Command log = :- /var/log/glusterfs/.cmd_log_history<o:p></o:p></p></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div><div><div><p = class=3DMsoNormal>Staging failed on = 00000000-0000-0000-0000-000000000000. Please check log file for = details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging failed on = 00000000-0000-0000-0000-000000000000. Please check log file for = details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:10:35.836676] : volume status all tasks : FAILED : Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:16:25.956514] : volume status all tasks : FAILED : Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:17:36.977833] : volume status all tasks : FAILED : Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:21:07.048053] : volume status all tasks : FAILED : Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:26:57.168661] : volume status all tasks : FAILED : Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:28:07.194428] : volume status all tasks : FAILED : Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:30:27.256667] : volume status vol01 : FAILED : Locking failed = on <a = href=3D"http://cpu02.zne01.hkg1.stack.com">cpu02.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>Locking failed on <a = href=3D"http://cpu03.zne01.hkg1.stack.com">cpu03.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>Locking failed on <a = href=3D"http://cpu04.zne01.hkg1.stack.com">cpu04.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 01:34:58.350748] : volume status all = tasks : FAILED : Staging failed on 00000000-0000-0000-0000-000000000000. = Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>Staging failed on = 00000000-0000-0000-0000-000000000000. Please check log file for = details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging failed on = 00000000-0000-0000-0000-000000000000. Please check log file for = details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:36:08.375326] : volume status all tasks : FAILED : Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging = failed on 00000000-0000-0000-0000-000000000000. Please check log file = for details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:36:08.386470] : volume status vol01 : FAILED : Locking failed = on <a = href=3D"http://cpu02.zne01.hkg1.stack.com">cpu02.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>Locking failed on <a = href=3D"http://cpu03.zne01.hkg1.stack.com">cpu03.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>Locking failed on <a = href=3D"http://cpu04.zne01.hkg1.stack.com">cpu04.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 01:42:59.524215] : volume stop vol01 = : FAILED : Locking failed on <a = href=3D"http://cpu02.zne01.hkg1.stack.com">cpu02.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>Locking failed on <a = href=3D"http://cpu03.zne01.hkg1.stack.com">cpu03.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>Locking failed on <a = href=3D"http://cpu04.zne01.hkg1.stack.com">cpu04.zne01.hkg1.stack.com</a>= . Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>[2015-01-13 01:45:10.550659] : volume status all = tasks : FAILED : Staging failed on 00000000-0000-0000-0000-000000000000. = Please check log file for details.<o:p></o:p></p></div><div><p = class=3DMsoNormal>Staging failed on = 00000000-0000-0000-0000-000000000000. Please check log file for = details.<o:p></o:p></p></div><div><p class=3DMsoNormal>Staging failed on = 00000000-0000-0000-0000-000000000000. Please check log file for = details.<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:46:10.656802] : volume status all tasks : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:51:02.796031] : volume status all tasks : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:52:02.897804] : volume status all tasks : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:55:25.841070] : system:: uuid get : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:55:26.752084] : system:: uuid get : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:55:32.499049] : system:: uuid get : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:55:38.716907] : system:: uuid get : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:56:52.905899] : volume status all tasks : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 01:58:53.109613] : volume status all tasks : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:03:26.769430] : system:: uuid get : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:04:22.859213] : volume status all tasks : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:05:22.970393] : volume status all tasks : = SUCCESS<o:p></o:p></p></div><div><p class=3DMsoNormal>[2015-01-13 = 02:06:23.075823] : volume status all tasks : = SUCCESS<o:p></o:p></p></div></div><div><p = class=3DMsoNormal><o:p> </o:p></p></div></div><div><p = class=3DMsoNormal><o:p> </o:p></p><div><p class=3DMsoNormal>On Mon, = Jan 12, 2015 at 10:53 PM, Kanagaraj Mayilsamy <<a = href=3D"mailto:kmayilsa@redhat.com" = target=3D"_blank">kmayilsa@redhat.com</a>> wrote:<o:p></o:p></p><p = class=3DMsoNormal>I can see the failures in glusterd log.<br><br>Can = someone from glusterfs dev pls help on = this?<br><br>Thanks,<br>Kanagaraj<o:p></o:p></p><div><div><p = class=3DMsoNormal><br>----- Original Message -----<br>> From: = "Punit Dambiwal" <<a = href=3D"mailto:hypunit@gmail.com">hypunit@gmail.com</a>><br>> To: = "Kanagaraj" <<a = href=3D"mailto:kmayilsa@redhat.com">kmayilsa@redhat.com</a>><br>> = Cc: "Martin Pavl=C3=ADk" <<a = href=3D"mailto:mpavlik@redhat.com">mpavlik@redhat.com</a>>, = "Vijay Bellur" <<a = href=3D"mailto:vbellur@redhat.com">vbellur@redhat.com</a>>, = "Kaushal M" <<a = href=3D"mailto:kshlmster@gmail.com">kshlmster@gmail.com</a>>,<br>> = <a href=3D"mailto:users@ovirt.org">users@ovirt.org</a>, <a = href=3D"mailto:gluster-users@gluster.org">gluster-users@gluster.org</a><b= r>> Sent: Monday, January 12, 2015 3:36:43 PM<br>> Subject: Re: = Failed to create volume in OVirt with gluster<br>><br>> Hi = Kanagaraj,<br>><br>> Please find the logs from here :- <a = href=3D"http://ur1.ca/jeszc" = target=3D"_blank">http://ur1.ca/jeszc</a><br>><br>> [image: Inline = image 1]<br>><br>> [image: Inline image 2]<br>><br>> On Mon, = Jan 12, 2015 at 1:02 PM, Kanagaraj <<a = href=3D"mailto:kmayilsa@redhat.com">kmayilsa@redhat.com</a>> = wrote:<br>><br>> > Looks like there are some failures in = gluster.<br>> > Can you send the log output from glusterd log file = from the relevant hosts?<br>> ><br>> > Thanks,<br>> > = Kanagaraj<br>> ><br>> ><br>> > On 01/12/2015 10:24 AM, = Punit Dambiwal wrote:<br>> ><br>> > Hi,<br>> ><br>> = > Is there any one from gluster can help me here :-<br>> = ><br>> > Engine logs :-<br>> ><br>> > = 2015-01-12 12:50:33,841 INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:34,725 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:36,824 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:36,853 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:36,866 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:37,751 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:39,849 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:39,878 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:39,890 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:40,776 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:42,878 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:42,903 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:42,916 = INFO<br>> > = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> > = (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait = lock<br>> > EngineLock [exclusiveLocks=3D key: = 00000001-0001-0001-0001-000000000300<br>> > value: GLUSTER<br>> = > , sharedLocks=3D ]<br>> > 2015-01-12 12:50:43,771 = INFO<br>> > = [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand]<b= r>> > (ajp--127.0.0.1-8702-1) [330ace48] FINISH, = CreateGlusterVolumeVDSCommand,<br>> > log id: 303e70a4<br>> = > 2015-01-12 12:50:43,780 ERROR<br>> > = [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]<br= href=3D"mailto:mpavlik@redhat.com">mpavlik@redhat.com</a>> = wrote:<br>> ><br>> >> Hi Punit,<br>> >><br>> = >> unfortunately I=E2=80=99am not that good with the = gluster, I was just following<br>> >> the obvious clue from the = log. Could you try on the nodes if the packages<br>> >> are = even available for installation<br>> >><br>> >> = yum install gluster-swift gluster-swift-object = gluster-swift-plugin<br>> >> = gluster-swift-account<br>> >> gluster-swift-proxy = gluster-swift-doc gluster-swift-container<br>> >> = glusterfs-geo-replication<br>> >><br>> >> if not = you could try to get them in official gluster repo.<br>> >> <a = href=3D"http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/g= lusterfs-epel.repo" = target=3D"_blank">http://download.gluster.org/pub/gluster/glusterfs/LATES= T/CentOS/glusterfs-epel.repo</a><br>> >><br>> >> = HTH<br>> >><br>> >> M.<br>> >><br>> = >><br>> >><br>> >><br>> >> = On 10 Jan 2015, at 04:35, Punit Dambiwal <<a = href=3D"mailto:hypunit@gmail.com">hypunit@gmail.com</a>> = wrote:<br>> >><br>> >> Hi Martin,<br>> = >><br>> >> I installed gluster from ovirt = repo....is it require to install those<br>> >> packages = manually ??<br>> >><br>> >> On Fri, Jan 9, 2015 at = 7:19 PM, Martin Pavl=C3=ADk <<a = href=3D"mailto:mpavlik@redhat.com">mpavlik@redhat.com</a>> = wrote:<br>> >><br>> >>> Hi Punit,<br>> = >>><br>> >>> can you verify that nodes contain = cluster packages from the following<br>> >>> log?<br>> = >>><br>> >>> Thread-14::DEBUG::2015-01-09<br>> = >>> 18:06:28,823::caps::716::root::(_getKeyPackages) rpm = package<br>> >>> ('gluster-swift',) not found<br>> = >>> Thread-14::DEBUG::2015-01-09<br>> >>> = 18:06:28,825::caps::716::root::(_getKeyPackages) rpm package<br>> = >>> ('gluster-swift-object',) not found<br>> >>> = Thread-14::DEBUG::2015-01-09<br>> >>> = 18:06:28,826::caps::716::root::(_getKeyPackages) rpm package<br>> = >>> ('gluster-swift-plugin',) not found<br>> >>> = Thread-14::DEBUG::2015-01-09<br>> >>> = 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package<br>> = >>> ('gluster-swift-account',) not found<br>> >>> = Thread-14::DEBUG::2015-01-09<br>> >>> = 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package<br>> = >>> ('gluster-swift-proxy',) not found<br>> >>> = Thread-14::DEBUG::2015-01-09<br>> >>> = 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package<br>> = >>> ('gluster-swift-doc',) not found<br>> >>> = Thread-14::DEBUG::2015-01-09<br>> >>> = 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package<br>> = >>> ('gluster-swift-container',) not found<br>> >>> = Thread-14::DEBUG::2015-01-09<br>> >>> = 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package<br>> = >>> ('glusterfs-geo-replication',) not found<br>> = >>><br>> >>><br>> >>> M.<br>> = >>><br>> >>> On 09 Jan 2015, at 11:13, = Punit Dambiwal <<a = href=3D"mailto:hypunit@gmail.com">hypunit@gmail.com</a>> = wrote:<br>> >>><br>> >>> Hi = Kanagaraj,<br>> >>><br>> >>> Please find = the attached logs :-<br>> >>><br>> >>> = Engine Logs :- <a href=3D"http://ur1.ca/jdopt" = target=3D"_blank">http://ur1.ca/jdopt</a><br>> >>> VDSM Logs = :- <a href=3D"http://ur1.ca/jdoq9" = target=3D"_blank">http://ur1.ca/jdoq9</a><br>> >>><br>> = >>><br>> >>><br>> >>> On Thu, Jan 8, = 2015 at 6:05 PM, Kanagaraj <<a = href=3D"mailto:kmayilsa@redhat.com">kmayilsa@redhat.com</a>> = wrote:<br>> >>><br>> >>>> Do you see = any errors in the UI?<br>> >>>><br>> >>>> = Also please provide the engine.log and vdsm.log when the failure<br>> = >>>> occured.<br>> >>>><br>> = >>>> Thanks,<br>> >>>> Kanagaraj<br>> = >>>><br>> >>>><br>> >>>> On = 01/08/2015 02:25 PM, Punit Dambiwal wrote:<br>> = >>>><br>> >>>> Hi Martin,<br>> = >>>><br>> >>>> The steps are below = :-<br>> >>>><br>> >>>> 1. Step the = ovirt engine on the one server...<br>> >>>> 2. Installed = centos 7 on 4 host node servers..<br>> >>>> 3. I am using = host node (compute+storage)....now i have added all 4<br>> = >>>> nodes to engine...<br>> >>>> 4. Create = the gluster volume from GUI...<br>> >>>><br>> = >>>> Network :-<br>> >>>> eth0 :- = public network (1G)<br>> >>>> eth1+eth2=3Dbond0=3D VM = public network (1G)<br>> >>>> = eth3+eth4=3Dbond1=3Dovirtmgmt+storage (10G private network)<br>> = >>>><br>> >>>> every hostnode has 24 = bricks=3D24*4(distributed replicated)<br>> >>>><br>> = >>>> Thanks,<br>> >>>> Punit<br>> = >>>><br>> >>>><br>> >>>> On = Thu, Jan 8, 2015 at 3:20 PM, Martin Pavl=C3=ADk <<a = href=3D"mailto:mpavlik@redhat.com">mpavlik@redhat.com</a>><br>> = >>>> wrote:<br>> >>>><br>> = >>>>> Hi Punit,<br>> >>>>><br>> = >>>>> can you please provide also errors from = /var/log/vdsm/vdsm.log and<br>> >>>>> = /var/log/vdsm/vdsmd.log<br>> >>>>><br>> = >>>>> it would be really helpful if you provided exact = steps how to<br>> >>>>> reproduce the problem.<br>> = >>>>><br>> >>>>> regards<br>> = >>>>><br>> >>>>> Martin Pavlik - rhev = QE<br>> >>>>> > On 08 Jan 2015, at 03:06, = Punit Dambiwal <<a = href=3D"mailto:hypunit@gmail.com">hypunit@gmail.com</a>> = wrote:<br>> >>>>> ><br>> >>>>> = > Hi,<br>> >>>>> ><br>> >>>>> = > I try to add gluster volume but it failed...<br>> = >>>>> ><br>> >>>>> > Ovirt :- = 3.5<br>> >>>>> > VDSM :- = vdsm-4.16.7-1.gitdb83943.el7<br>> >>>>> > KVM :- = 1.5.3 - 60.el7_0.2<br>> >>>>> > = libvirt-1.1.1-29.el7_0.4<br>> >>>>> > Glusterfs :- = glusterfs-3.5.3-1.el7<br>> >>>>> ><br>> = >>>>> > Engine Logs :-<br>> >>>>> = ><br>> >>>>> > 2015-01-08 09:57:52,569 = INFO<br>> >>>>> = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> = >>>>> (DefaultQuartzScheduler_Worker-16) Failed to = acquire lock and wait lock<br>> >>>>> EngineLock = [exclusiveLocks=3D key: 00000001-0001-0001-0001-000000000300<br>> = >>>>> value: GLUSTER<br>> >>>>> > , = sharedLocks=3D ]<br>> >>>>> > 2015-01-08 = 09:57:52,609 INFO<br>> >>>>> = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> = >>>>> (DefaultQuartzScheduler_Worker-16) Failed to = acquire lock and wait lock<br>> >>>>> EngineLock = [exclusiveLocks=3D key: 00000001-0001-0001-0001-000000000300<br>> = >>>>> value: GLUSTER<br>> >>>>> > , = sharedLocks=3D ]<br>> >>>>> > 2015-01-08 = 09:57:55,582 INFO<br>> >>>>> = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> = >>>>> (DefaultQuartzScheduler_Worker-16) Failed to = acquire lock and wait lock<br>> >>>>> EngineLock = [exclusiveLocks=3D key: 00000001-0001-0001-0001-000000000300<br>> = >>>>> value: GLUSTER<br>> >>>>> > , = sharedLocks=3D ]<br>> >>>>> > 2015-01-08 = 09:57:55,591 INFO<br>> >>>>> = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> = >>>>> (DefaultQuartzScheduler_Worker-16) Failed to = acquire lock and wait lock<br>> >>>>> EngineLock = [exclusiveLocks=3D key: 00000001-0001-0001-0001-000000000300<br>> = >>>>> value: GLUSTER<br>> >>>>> > , = sharedLocks=3D ]<br>> >>>>> > 2015-01-08 = 09:57:55,596 INFO<br>> >>>>> = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> = >>>>> (DefaultQuartzScheduler_Worker-16) Failed to = acquire lock and wait lock<br>> >>>>> EngineLock = [exclusiveLocks=3D key: 00000001-0001-0001-0001-000000000300<br>> = >>>>> value: GLUSTER<br>> >>>>> > , = sharedLocks=3D ]<br>> >>>>> > 2015-01-08 = 09:57:55,633 INFO<br>> >>>>> = [org.ovirt.engine.core.bll.lock.InMemoryLockManager]<br>> = >>>>> (DefaultQuartzScheduler_Worker-16) Failed to = acquire lock and wait lock<br>> >>>>> EngineLock = [exclusiveLocks=3D key: 00000001-0001-0001-0001-000000000300<br>> = >>>>> value: GLUSTER<br>> >>>>> > , = sharedLocks=3D ]<br>> >>>>> > ^C<br>> = >>>>> ><br>> >>>>> ><br>> = >>>>><br>> >>>>><br>> = >>>><br>> >>>><br>> >>> = <216 09-Jan-15.jpg><217 09-Jan-15.jpg><br>> = >>><br>> >>><br>> >>><br>> = >><br>> >><br>> ><br>> = ><br>><o:p></o:p></p></div></div></div><p = class=3DMsoNormal><o:p> </o:p></p></div></div></body></html> ------=_NextPart_000_00B5_01D02F1E.B7FDC3D0--