Re: [ovirt-users] Failed to create volume in OVirt with gluster

Looks like there are some failures in gluster. Can you send the log output from glusterd log file from the relevant hosts? Thanks, Kanagaraj On 01/12/2015 10:24 AM, Punit Dambiwal wrote: Hi, Is there any one from gluster can help me here :- Engine logs :- 2015-01-12 12:50:33,841 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:34,725 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:36,824 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:36,853 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:36,866 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:37,751 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:39,849 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:39,878 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:39,890 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:40,776 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:42,878 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:42,903 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:42,916 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] 2015-01-12 12:50:43,771 INFO [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand] (ajp--127.0.0.1-8702-1) [330ace48] FINISH, CreateGlusterVolumeVDSCommand, log id: 303e70a4 2015-01-12 12:50:43,780 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ajp--127.0.0.1-8702-1) [330ace48] Correlation ID: 330ace48, Job ID: 896a69b3-a678-40a7-bceb-3635e4062aa0, Call Stack: null, Custom Event ID: -1, Message: Creation of Gluster Volume vol01 failed. 2015-01-12 12:50:43,785 INFO [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand] (ajp--127.0.0.1-8702-1) [330ace48] Lock freed to object EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER , sharedLocks= ] [image: Inline image 2] On Sun, Jan 11, 2015 at 6:48 PM, Martin Pavlík <mpavlik(a)redhat.com&gt; wrote: > Hi Punit, > > unfortunately I’am not that good with the gluster, I was just following > the obvious clue from the log. Could you try on the nodes if the packages > are even available for installation > > yum install gluster-swift gluster-swift-object gluster-swift-plugin gluster-swift-account > gluster-swift-proxy gluster-swift-doc gluster-swift-container > glusterfs-geo-replication > > if not you could try to get them in official gluster repo. > http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs... > > HTH > > M. > > > > > On 10 Jan 2015, at 04:35, Punit Dambiwal <hypunit(a)gmail.com&gt; wrote: > > Hi Martin, > > I installed gluster from ovirt repo....is it require to install those > packages manually ?? > > On Fri, Jan 9, 2015 at 7:19 PM, Martin Pavlík <mpavlik(a)redhat.com&gt; wrote: > >> Hi Punit, >> >> can you verify that nodes contain cluster packages from the following log? >> >> Thread-14::DEBUG::2015-01-09 18:06:28,823::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift',) not found >> Thread-14::DEBUG::2015-01-09 18:06:28,825::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-object',) not found >> Thread-14::DEBUG::2015-01-09 18:06:28,826::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-plugin',) not found >> Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-account',) not found >> Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-proxy',) not found >> Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-doc',) not found >> Thread-14::DEBUG::2015-01-09 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-container',) not found >> Thread-14::DEBUG::2015-01-09 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package ('glusterfs-geo-replication',) not found >> >> >> M. >> >> On 09 Jan 2015, at 11:13, Punit Dambiwal <hypunit(a)gmail.com&gt; wrote: >> >> Hi Kanagaraj, >> >> Please find the attached logs :- >> >> Engine Logs :- http://ur1.ca/jdopt >> VDSM Logs :- http://ur1.ca/jdoq9 >> >> >> >> On Thu, Jan 8, 2015 at 6:05 PM, Kanagaraj <kmayilsa(a)redhat.com&gt; wrote: >> >>> Do you see any errors in the UI? >>> >>> Also please provide the engine.log and vdsm.log when the failure >>> occured. >>> >>> Thanks, >>> Kanagaraj >>> >>> >>> On 01/08/2015 02:25 PM, Punit Dambiwal wrote: >>> >>> Hi Martin, >>> >>> The steps are below :- >>> >>> 1. Step the ovirt engine on the one server... >>> 2. Installed centos 7 on 4 host node servers.. >>> 3. I am using host node (compute+storage)....now i have added all 4 >>> nodes to engine... >>> 4. Create the gluster volume from GUI... >>> >>> Network :- >>> eth0 :- public network (1G) >>> eth1+eth2=bond0= VM public network (1G) >>> eth3+eth4=bond1=ovirtmgmt+storage (10G private network) >>> >>> every hostnode has 24 bricks=24*4(distributed replicated) >>> >>> Thanks, >>> Punit >>> >>> >>> On Thu, Jan 8, 2015 at 3:20 PM, Martin Pavlík <mpavlik(a)redhat.com&gt; >>> wrote: >>> >>>> Hi Punit, >>>> >>>> can you please provide also errors from /var/log/vdsm/vdsm.log and >>>> /var/log/vdsm/vdsmd.log >>>> >>>> it would be really helpful if you provided exact steps how to >>>> reproduce the problem. >>>> >>>> regards >>>> >>>> Martin Pavlik - rhev QE >>>> > On 08 Jan 2015, at 03:06, Punit Dambiwal <hypunit(a)gmail.com&gt; wrote: >>>> > >>>> > Hi, >>>> > >>>> > I try to add gluster volume but it failed... >>>> > >>>> > Ovirt :- 3.5 >>>> > VDSM :- vdsm-4.16.7-1.gitdb83943.el7 >>>> > KVM :- 1.5.3 - 60.el7_0.2 >>>> > libvirt-1.1.1-29.el7_0.4 >>>> > Glusterfs :- glusterfs-3.5.3-1.el7 >>>> > >>>> > Engine Logs :- >>>> > >>>> > 2015-01-08 09:57:52,569 INFO >>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager] >>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock >>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 >>>> value: GLUSTER >>>> > , sharedLocks= ] >>>> > 2015-01-08 09:57:52,609 INFO >>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager] >>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock >>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 >>>> value: GLUSTER >>>> > , sharedLocks= ] >>>> > 2015-01-08 09:57:55,582 INFO >>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager] >>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock >>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 >>>> value: GLUSTER >>>> > , sharedLocks= ] >>>> > 2015-01-08 09:57:55,591 INFO >>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager] >>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock >>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 >>>> value: GLUSTER >>>> > , sharedLocks= ] >>>> > 2015-01-08 09:57:55,596 INFO >>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager] >>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock >>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 >>>> value: GLUSTER >>>> > , sharedLocks= ] >>>> > 2015-01-08 09:57:55,633 INFO >>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager] >>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock >>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 >>>> value: GLUSTER >>>> > , sharedLocks= ] >>>> > ^C >>>> > >>>> > >>>> >>>> >>> >>> >> <216 09-Jan-15.jpg><217 09-Jan-15.jpg> >> >> >> > >