[ovirt-devel] dynamic ownership changes

Sun Apr 22 22:29:43 UTC 2018

Sorry, this is the new execution link:
https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-storage/1048/testReport/

On Mon, Apr 23, 2018 at 1:23 AM, Elad Ben Aharon <ebenahar at redhat.com>
wrote:

> Hi, I've triggered another execution [1] due to some issues I saw in the
> first which are not related to the patch.
>
> The success rate is 78% which is low comparing to tier1 executions with
> code from downstream builds (95-100% success rates) [2].
>
> From what I could see so far, there is an issue with move and copy
> operations to and from Gluster domains. For example [3].
>
> The logs are attached.
>
>
> [1]
> *https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-tier1-after-upgrade/7/testReport/
> <https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-4.2-ge-runner-tier1-after-upgrade/7/testReport/>*
>
>
>
> [2]
> https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv-
> 4.2-ge-runner-tier1-after-upgrade/7/
>
>
>
> [3]
> 2018-04-22 13:06:28,316+0300 INFO  (jsonrpc/7) [vdsm.api] FINISH
> deleteImage error=Image does not exist in domain:
> 'image=cabb8846-7a4b-4244-9835-5f603e682f33,
> domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4' from=:
> :ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935,
> task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51)
> 2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task]
> (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875)
> Traceback (most recent call last):
>  File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,
> in _run
>    return fn(*args, **kargs)
>  File "<string>", line 2, in deleteImage
>  File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in
> method
>    ret = func(*args, **kwargs)
>  File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503,
> in deleteImage
>    raise se.ImageDoesNotExistInSD(imgUUID, sdUUID)
> ImageDoesNotExistInSD: Image does not exist in domain:
> 'image=cabb8846-7a4b-4244-9835-5f603e682f33,
> domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'
> 2018-04-22 13:06:28,317+0300 INFO  (jsonrpc/7) [storage.TaskManager.Task]
> (Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted:
> "Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-
> 5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268
> (task:1181)
> 2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH
> deleteImage error=Image does not exist in domain:
> 'image=cabb8846-7a4b-4244-9835-5f603e682f33,
> domain=e5fd29c8-52ba-467e-be09-ca40ff054d
> d4' (dispatcher:82)
>
>
>
> On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar at redhat.com>
> wrote:
>
>> Triggered a sanity tier1 execution [1] using [2], which covers all the
>> requested areas, on iSCSI, NFS and Gluster.
>> I'll update with the results.
>>
>> [1]
>> https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2
>> _dev/job/rhv-4.2-ge-flow-storage/1161/
>>
>> [2]
>> https://gerrit.ovirt.org/#/c/89830/
>> vdsm-4.30.0-291.git77aef9a.el7.x86_64
>>
>>
>>
>> On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik at redhat.com>
>> wrote:
>>
>>> On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:
>>>
>>>> Hi Martin,
>>>>
>>>> I see [1] requires a rebase, can you please take care?
>>>>
>>>
>>> Should be rebased.
>>>
>>> At the moment, our automation is stable only on iSCSI, NFS, Gluster and
>>>> FC.
>>>> Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's
>>>> not
>>>> stable enough at the moment.
>>>>
>>>
>>> That is still pretty good.
>>>
>>>
>>> [1] https://gerrit.ovirt.org/#/c/89830/
>>>>
>>>>
>>>> Thanks
>>>>
>>>> On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik at redhat.com>
>>>> wrote:
>>>>
>>>> On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:
>>>>>
>>>>> Hi, sorry if I misunderstood, I waited for more input regarding what
>>>>>> areas
>>>>>> have to be tested here.
>>>>>>
>>>>>>
>>>>> I'd say that you have quite a bit of freedom in this regard. GlusterFS
>>>>> should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite
>>>>> that covers basic operations (start & stop VM, migrate it), snapshots
>>>>> and merging them, and whatever else would be important for storage
>>>>> sanity.
>>>>>
>>>>> mpolednik
>>>>>
>>>>>
>>>>> On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <
>>>>> mpolednik at redhat.com>
>>>>>
>>>>>> wrote:
>>>>>>
>>>>>> On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:
>>>>>>
>>>>>>>
>>>>>>> We can test this on iSCSI, NFS and GlusterFS. As for ceph and cinder,
>>>>>>>
>>>>>>>> will
>>>>>>>> have to check, since usually, we don't execute our automation on
>>>>>>>> them.
>>>>>>>>
>>>>>>>>
>>>>>>>> Any update on this? I believe the gluster tests were successful, OST
>>>>>>> passes fine and unit tests pass fine, that makes the storage backends
>>>>>>> test the last required piece.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir at redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>> +Elad
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken at redhat.com
>>>>>>>>> >
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer at redhat.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri at redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Please make sure to run as much OST suites on this patch as
>>>>>>>>>>> possible
>>>>>>>>>>>
>>>>>>>>>>> before merging ( using 'ci please build' )
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> But note that OST is not a way to verify the patch.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Such changes require testing with all storage types we support.
>>>>>>>>>>>
>>>>>>>>>>> Nir
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <
>>>>>>>>>>> mpolednik at redhat.com
>>>>>>>>>>> >
>>>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Hey,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> I've created a patch[0] that is finally able to activate
>>>>>>>>>>>>> libvirt's
>>>>>>>>>>>>> dynamic_ownership for VDSM while not negatively affecting
>>>>>>>>>>>>> functionality of our storage code.
>>>>>>>>>>>>>
>>>>>>>>>>>>> That of course comes with quite a bit of code removal, mostly
>>>>>>>>>>>>> in
>>>>>>>>>>>>> the
>>>>>>>>>>>>> area of host devices, hwrng and anything that touches devices;
>>>>>>>>>>>>> bunch
>>>>>>>>>>>>> of test changes and one XML generation caveat (storage is
>>>>>>>>>>>>> handled
>>>>>>>>>>>>> by
>>>>>>>>>>>>> VDSM, therefore disk relabelling needs to be disabled on the
>>>>>>>>>>>>> VDSM
>>>>>>>>>>>>> level).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Because of the scope of the patch, I welcome
>>>>>>>>>>>>> storage/virt/network
>>>>>>>>>>>>> people to review the code and consider the implication this
>>>>>>>>>>>>> change
>>>>>>>>>>>>> has
>>>>>>>>>>>>> on current/future features.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [0] https://gerrit.ovirt.org/#/c/89830/
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> In particular:  dynamic_ownership was set to 0 prehistorically
>>>>>>>>>>>>> (as
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> part
>>>>>>>>>>>
>>>>>>>>>> of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because
>>>>>>>>>> libvirt,
>>>>>>>>>> running as root, was not able to play properly with root-squash
>>>>>>>>>> nfs
>>>>>>>>>> mounts.
>>>>>>>>>>
>>>>>>>>>> Have you attempted this use case?
>>>>>>>>>>
>>>>>>>>>> I join to Nir's request to run this with storage QE.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Raz Tamir
>>>>>>>>> Manager, RHV QE
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/devel/attachments/20180423/9ada0230/attachment.html>