[ovirt-users] Failed to create volume in OVirt with gluster

Mon Jan 12 05:06:43 EST 2015

Hi Kanagaraj,

Please find the logs from here :- http://ur1.ca/jeszc

[image: Inline image 1]

[image: Inline image 2]

On Mon, Jan 12, 2015 at 1:02 PM, Kanagaraj <kmayilsa at redhat.com> wrote:

>  Looks like there are some failures in gluster.
> Can you send the log output from glusterd log file from the relevant hosts?
>
> Thanks,
> Kanagaraj
>
>
> On 01/12/2015 10:24 AM, Punit Dambiwal wrote:
>
> Hi,
>
>  Is there any one from gluster can help me here :-
>
>  Engine logs :-
>
>  2015-01-12 12:50:33,841 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:34,725 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:36,824 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:36,853 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:36,866 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:37,751 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:39,849 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:39,878 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:39,890 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:40,776 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:42,878 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:42,903 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:42,916 INFO
>  [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (DefaultQuartzScheduler_Worker-12) Failed to acquire lock and wait lock
> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
> value: GLUSTER
> , sharedLocks= ]
> 2015-01-12 12:50:43,771 INFO
>  [org.ovirt.engine.core.vdsbroker.gluster.CreateGlusterVolumeVDSCommand]
> (ajp--127.0.0.1-8702-1) [330ace48] FINISH, CreateGlusterVolumeVDSCommand,
> log id: 303e70a4
> 2015-01-12 12:50:43,780 ERROR
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
> (ajp--127.0.0.1-8702-1) [330ace48] Correlation ID: 330ace48, Job ID:
> 896a69b3-a678-40a7-bceb-3635e4062aa0, Call Stack: null, Custom Event ID:
> -1, Message: Creation of Gluster Volume vol01 failed.
> 2015-01-12 12:50:43,785 INFO
>  [org.ovirt.engine.core.bll.gluster.CreateGlusterVolumeCommand]
> (ajp--127.0.0.1-8702-1) [330ace48] Lock freed to object EngineLock
> [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300 value: GLUSTER
> , sharedLocks= ]
>
>  [image: Inline image 2]
>
>
> On Sun, Jan 11, 2015 at 6:48 PM, Martin Pavlík <mpavlik at redhat.com> wrote:
>
>> Hi Punit,
>>
>>  unfortunately I’am not that good with the gluster, I was just following
>> the obvious clue from the log. Could you try on the nodes if the packages
>> are even available for installation
>>
>>  yum install gluster-swift gluster-swift-object gluster-swift-plugin gluster-swift-account
>> gluster-swift-proxy gluster-swift-doc gluster-swift-container
>> glusterfs-geo-replication
>>
>>  if not you could try to get them in official gluster repo.
>> http://download.gluster.org/pub/gluster/glusterfs/LATEST/CentOS/glusterfs-epel.repo
>>
>>  HTH
>>
>>  M.
>>
>>
>>
>>
>>   On 10 Jan 2015, at 04:35, Punit Dambiwal <hypunit at gmail.com> wrote:
>>
>>  Hi Martin,
>>
>>  I installed gluster from ovirt repo....is it require to install those
>> packages manually ??
>>
>> On Fri, Jan 9, 2015 at 7:19 PM, Martin Pavlík <mpavlik at redhat.com> wrote:
>>
>>>  Hi Punit,
>>>
>>> can you verify that nodes contain cluster packages from the following log?
>>>
>>> Thread-14::DEBUG::2015-01-09 18:06:28,823::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift',) not found
>>> Thread-14::DEBUG::2015-01-09 18:06:28,825::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-object',) not found
>>> Thread-14::DEBUG::2015-01-09 18:06:28,826::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-plugin',) not found
>>> Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-account',) not found
>>> Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-proxy',) not found
>>> Thread-14::DEBUG::2015-01-09 18:06:28,829::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-doc',) not found
>>> Thread-14::DEBUG::2015-01-09 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package ('gluster-swift-container',) not found
>>> Thread-14::DEBUG::2015-01-09 18:06:28,830::caps::716::root::(_getKeyPackages) rpm package ('glusterfs-geo-replication',) not found
>>>
>>>
>>>  M.
>>>
>>>    On 09 Jan 2015, at 11:13, Punit Dambiwal <hypunit at gmail.com> wrote:
>>>
>>>    Hi Kanagaraj,
>>>
>>>  Please find the attached logs :-
>>>
>>>  Engine Logs :- http://ur1.ca/jdopt
>>> VDSM Logs :- http://ur1.ca/jdoq9
>>>
>>>
>>>
>>> On Thu, Jan 8, 2015 at 6:05 PM, Kanagaraj <kmayilsa at redhat.com> wrote:
>>>
>>>>  Do you see any errors in the UI?
>>>>
>>>> Also please provide the engine.log and vdsm.log when the failure
>>>> occured.
>>>>
>>>> Thanks,
>>>> Kanagaraj
>>>>
>>>>
>>>> On 01/08/2015 02:25 PM, Punit Dambiwal wrote:
>>>>
>>>> Hi Martin,
>>>>
>>>>  The steps are below :-
>>>>
>>>>  1. Step the ovirt engine on the one server...
>>>> 2. Installed centos 7 on 4 host node servers..
>>>> 3. I am using host node (compute+storage)....now i have added all 4
>>>> nodes to engine...
>>>> 4. Create the gluster volume from GUI...
>>>>
>>>>  Network :-
>>>> eth0 :- public network (1G)
>>>> eth1+eth2=bond0= VM public network (1G)
>>>> eth3+eth4=bond1=ovirtmgmt+storage (10G private network)
>>>>
>>>>  every hostnode has 24 bricks=24*4(distributed replicated)
>>>>
>>>>  Thanks,
>>>> Punit
>>>>
>>>>
>>>> On Thu, Jan 8, 2015 at 3:20 PM, Martin Pavlík <mpavlik at redhat.com>
>>>> wrote:
>>>>
>>>>> Hi Punit,
>>>>>
>>>>> can you please provide also errors from /var/log/vdsm/vdsm.log and
>>>>> /var/log/vdsm/vdsmd.log
>>>>>
>>>>> it would be really helpful if you provided exact steps how to
>>>>> reproduce the problem.
>>>>>
>>>>> regards
>>>>>
>>>>> Martin Pavlik - rhev QE
>>>>>  > On 08 Jan 2015, at 03:06, Punit Dambiwal <hypunit at gmail.com> wrote:
>>>>> >
>>>>> > Hi,
>>>>> >
>>>>> > I try to add gluster volume but it failed...
>>>>> >
>>>>> > Ovirt :- 3.5
>>>>> > VDSM :- vdsm-4.16.7-1.gitdb83943.el7
>>>>> > KVM :- 1.5.3 - 60.el7_0.2
>>>>> > libvirt-1.1.1-29.el7_0.4
>>>>> > Glusterfs :- glusterfs-3.5.3-1.el7
>>>>> >
>>>>> > Engine Logs :-
>>>>> >
>>>>> > 2015-01-08 09:57:52,569 INFO
>>>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
>>>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock
>>>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
>>>>> value: GLUSTER
>>>>> > , sharedLocks= ]
>>>>> > 2015-01-08 09:57:52,609 INFO
>>>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
>>>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock
>>>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
>>>>> value: GLUSTER
>>>>> > , sharedLocks= ]
>>>>> > 2015-01-08 09:57:55,582 INFO
>>>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
>>>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock
>>>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
>>>>> value: GLUSTER
>>>>> > , sharedLocks= ]
>>>>> > 2015-01-08 09:57:55,591 INFO
>>>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
>>>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock
>>>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
>>>>> value: GLUSTER
>>>>> > , sharedLocks= ]
>>>>> > 2015-01-08 09:57:55,596 INFO
>>>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
>>>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock
>>>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
>>>>> value: GLUSTER
>>>>> > , sharedLocks= ]
>>>>> > 2015-01-08 09:57:55,633 INFO
>>>>> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
>>>>> (DefaultQuartzScheduler_Worker-16) Failed to acquire lock and wait lock
>>>>> EngineLock [exclusiveLocks= key: 00000001-0001-0001-0001-000000000300
>>>>> value: GLUSTER
>>>>> > , sharedLocks= ]
>>>>> > ^C
>>>>> >
>>>>> >
>>>>>
>>>>>
>>>>
>>>>
>>>   <216 09-Jan-15.jpg><217 09-Jan-15.jpg>
>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150112/3ec00dc0/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 144424 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150112/3ec00dc0/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/png
Size: 92270 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150112/3ec00dc0/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 167891 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20150112/3ec00dc0/attachment-0005.png>