<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Elad,<div class="">to make it easier to compare, Martin backported the change to 4.2 so it is actually comparable with a run without that patch. Would you please try that out? </div><div class="">It would be best to have 4.2 upstream and this[1] run to really minimize the noise.</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">michal</div><div class=""><br class=""></div><div class="">[1] <a href="http://jenkins.ovirt.org/job/vdsm_4.2_build-artifacts-on-demand-el7-x86_64/28/" class="">http://jenkins.ovirt.org/job/vdsm_4.2_build-artifacts-on-demand-el7-x86_64/28/</a></div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 27 Apr 2018, at 09:23, Martin Polednik <<a href="mailto:mpolednik@redhat.com" class="">mpolednik@redhat.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">On 24/04/18 00:37 +0300, Elad Ben Aharon wrote:<br class=""><blockquote type="cite" class="">I will update with the results of the next tier1 execution on latest 4.2.3<br class=""></blockquote><br class="">That isn't master but old branch though. Could you run it against<br class="">*current* VDSM master?<br class=""><br class=""><blockquote type="cite" class="">On Mon, Apr 23, 2018 at 3:56 PM, Martin Polednik <<a href="mailto:mpolednik@redhat.com" class="">mpolednik@redhat.com</a>><br class="">wrote:<br class=""><br class=""><blockquote type="cite" class="">On 23/04/18 01:23 +0300, Elad Ben Aharon wrote:<br class=""><br class=""><blockquote type="cite" class="">Hi, I've triggered another execution [1] due to some issues I saw in the<br class="">first which are not related to the patch.<br class=""><br class="">The success rate is 78% which is low comparing to tier1 executions with<br class="">code from downstream builds (95-100% success rates) [2].<br class=""><br class=""></blockquote><br class="">Could you run the current master (without the dynamic_ownership patch)<br class="">so that we have viable comparision?<br class=""><br class="">From what I could see so far, there is an issue with move and copy<br class=""><blockquote type="cite" class="">operations to and from Gluster domains. For example [3].<br class=""><br class="">The logs are attached.<br class=""><br class=""><br class="">[1]<br class="">*<a href="https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv" class="">https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv</a><br class="">-4.2-ge-runner-tier1-after-upgrade/7/testReport/<br class=""><<a href="https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv" class="">https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/rhv</a><br class="">-4.2-ge-runner-tier1-after-upgrade/7/testReport/>*<br class=""><br class=""><br class=""><br class="">[2]<br class=""><a href="https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/" class="">https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/job/</a><br class=""><br class="">rhv-4.2-ge-runner-tier1-after-upgrade/7/<br class=""><br class=""><br class=""><br class="">[3]<br class="">2018-04-22 13:06:28,316+0300 INFO (jsonrpc/7) [vdsm.api] FINISH<br class="">deleteImage error=Image does not exist in domain:<br class="">'image=cabb8846-7a4b-4244-9835-5f603e682f33,<br class="">domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'<br class="">from=:<br class="">:ffff:10.35.161.182,40936, flow_id=disks_syncAction_ba6b2630-5976-4935,<br class="">task_id=3d5f2a8a-881c-409e-93e9-aaa643c10e42 (api:51)<br class="">2018-04-22 13:06:28,317+0300 ERROR (jsonrpc/7) [storage.TaskManager.Task]<br class="">(Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') Unexpected error (task:875)<br class="">Traceback (most recent call last):<br class="">File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882,<br class="">in<br class="">_run<br class=""> return fn(*args, **kargs)<br class="">File "<string>", line 2, in deleteImage<br class="">File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 49, in<br class="">method<br class=""> ret = func(*args, **kwargs)<br class="">File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1503,<br class="">in<br class="">deleteImage<br class=""> raise se.ImageDoesNotExistInSD(imgUUID, sdUUID)<br class="">ImageDoesNotExistInSD: Image does not exist in domain:<br class="">'image=cabb8846-7a4b-4244-9835-5f603e682f33,<br class="">domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'<br class=""><br class="">2018-04-22 13:06:28,317+0300 INFO (jsonrpc/7) [storage.TaskManager.Task]<br class="">(Task='3d5f2a8a-881c-409e-93e9-aaa643c10e42') aborting: Task is aborted:<br class="">"Image does not exist in domain: 'image=cabb8846-7a4b-4244-9835-<br class="">5f603e682f33, domain=e5fd29c8-52ba-467e-be09-ca40ff054dd4'" - code 268<br class="">(task:1181)<br class="">2018-04-22 13:06:28,318+0300 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH<br class="">deleteImage error=Image does not exist in domain:<br class="">'image=cabb8846-7a4b-4244-9835-5f603e682f33,<br class="">domain=e5fd29c8-52ba-467e-be09<br class="">-ca40ff054d<br class="">d4' (dispatcher:82)<br class=""><br class=""><br class=""><br class="">On Thu, Apr 19, 2018 at 5:34 PM, Elad Ben Aharon <ebenahar@redhat.com><br class="">wrote:<br class=""><br class="">Triggered a sanity tier1 execution [1] using [2], which covers all the<br class=""><blockquote type="cite" class="">requested areas, on iSCSI, NFS and Gluster.<br class="">I'll update with the results.<br class=""><br class="">[1]<br class="">https://rhv-jenkins.rhev-ci-vms.eng.rdu2.redhat.com/view/4.2<br class="">_dev/job/rhv-4.2-ge-flow-storage/1161/<br class=""><br class="">[2]<br class="">https://gerrit.ovirt.org/#/c/89830/<br class="">vdsm-4.30.0-291.git77aef9a.el7.x86_64<br class=""><br class=""><br class=""><br class="">On Thu, Apr 19, 2018 at 3:07 PM, Martin Polednik <mpolednik@redhat.com><br class="">wrote:<br class=""><br class="">On 19/04/18 14:54 +0300, Elad Ben Aharon wrote:<br class=""><blockquote type="cite" class=""><br class="">Hi Martin,<br class=""><blockquote type="cite" class=""><br class="">I see [1] requires a rebase, can you please take care?<br class=""><br class=""><br class=""></blockquote>Should be rebased.<br class=""><br class="">At the moment, our automation is stable only on iSCSI, NFS, Gluster and<br class=""><br class=""><blockquote type="cite" class="">FC.<br class="">Ceph is not supported and Cinder will be stabilized soon, AFAIR, it's<br class="">not<br class="">stable enough at the moment.<br class=""><br class=""><br class=""></blockquote>That is still pretty good.<br class=""><br class=""><br class="">[1] https://gerrit.ovirt.org/#/c/89830/<br class=""><br class=""><blockquote type="cite" class=""><br class=""><br class="">Thanks<br class=""><br class="">On Wed, Apr 18, 2018 at 2:17 PM, Martin Polednik <mpolednik@redhat.com<br class="">><br class="">wrote:<br class=""><br class="">On 18/04/18 11:37 +0300, Elad Ben Aharon wrote:<br class=""><br class=""><blockquote type="cite" class=""><br class="">Hi, sorry if I misunderstood, I waited for more input regarding what<br class=""><br class=""><blockquote type="cite" class="">areas<br class="">have to be tested here.<br class=""><br class=""><br class="">I'd say that you have quite a bit of freedom in this regard.<br class=""></blockquote>GlusterFS<br class="">should be covered by Dennis, so iSCSI/NFS/ceph/cinder with some suite<br class="">that covers basic operations (start & stop VM, migrate it), snapshots<br class="">and merging them, and whatever else would be important for storage<br class="">sanity.<br class=""><br class="">mpolednik<br class=""><br class=""><br class="">On Wed, Apr 18, 2018 at 11:16 AM, Martin Polednik <<br class="">mpolednik@redhat.com<br class="">><br class=""><br class="">wrote:<br class=""><blockquote type="cite" class=""><br class="">On 11/04/18 16:52 +0300, Elad Ben Aharon wrote:<br class=""><br class=""><br class=""><blockquote type="cite" class="">We can test this on iSCSI, NFS and GlusterFS. As for ceph and<br class="">cinder,<br class=""><br class="">will<br class=""><blockquote type="cite" class="">have to check, since usually, we don't execute our automation on<br class="">them.<br class=""><br class=""><br class="">Any update on this? I believe the gluster tests were successful,<br class="">OST<br class=""><br class=""></blockquote>passes fine and unit tests pass fine, that makes the storage<br class="">backends<br class="">test the last required piece.<br class=""><br class=""><br class="">On Wed, Apr 11, 2018 at 4:38 PM, Raz Tamir <ratamir@redhat.com><br class="">wrote:<br class=""><br class=""><br class="">+Elad<br class=""><blockquote type="cite" class=""><br class=""><br class="">On Wed, Apr 11, 2018 at 4:28 PM, Dan Kenigsberg <danken@redhat.com<br class=""><blockquote type="cite" class="">><br class="">wrote:<br class=""><br class="">On Wed, Apr 11, 2018 at 12:34 PM, Nir Soffer <nsoffer@redhat.com><br class="">wrote:<br class=""><br class=""><br class="">On Wed, Apr 11, 2018 at 12:31 PM Eyal Edri <eedri@redhat.com><br class=""><blockquote type="cite" class="">wrote:<br class=""><br class=""><br class="">Please make sure to run as much OST suites on this patch as<br class=""><blockquote type="cite" class="">possible<br class=""><br class="">before merging ( using 'ci please build' )<br class=""><br class=""><blockquote type="cite" class=""><br class=""><br class="">But note that OST is not a way to verify the patch.<br class=""><br class=""><br class=""></blockquote>Such changes require testing with all storage types we support.<br class=""><br class="">Nir<br class=""><br class="">On Tue, Apr 10, 2018 at 4:09 PM, Martin Polednik <<br class="">mpolednik@redhat.com<br class="">><br class=""><br class="">wrote:<br class=""><br class=""><br class=""><blockquote type="cite" class="">Hey,<br class=""><br class=""><br class="">I've created a patch[0] that is finally able to activate<br class=""><blockquote type="cite" class="">libvirt's<br class="">dynamic_ownership for VDSM while not negatively affecting<br class="">functionality of our storage code.<br class=""><br class="">That of course comes with quite a bit of code removal, mostly<br class="">in<br class="">the<br class="">area of host devices, hwrng and anything that touches devices;<br class="">bunch<br class="">of test changes and one XML generation caveat (storage is<br class="">handled<br class="">by<br class="">VDSM, therefore disk relabelling needs to be disabled on the<br class="">VDSM<br class="">level).<br class=""><br class="">Because of the scope of the patch, I welcome<br class="">storage/virt/network<br class="">people to review the code and consider the implication this<br class="">change<br class="">has<br class="">on current/future features.<br class=""><br class="">[0] https://gerrit.ovirt.org/#/c/89830/<br class=""><br class=""><br class="">In particular: dynamic_ownership was set to 0 prehistorically<br class="">(as<br class=""><br class=""><br class=""></blockquote>part<br class=""><br class=""></blockquote><br class="">of https://bugzilla.redhat.com/show_bug.cgi?id=554961 ) because<br class=""></blockquote>libvirt,<br class="">running as root, was not able to play properly with root-squash<br class="">nfs<br class="">mounts.<br class=""><br class="">Have you attempted this use case?<br class=""><br class="">I join to Nir's request to run this with storage QE.<br class=""><br class=""><br class=""><br class=""><br class="">--<br class=""></blockquote><br class=""><br class="">Raz Tamir<br class="">Manager, RHV QE<br class=""><br class=""><br class=""><br class=""><br class=""><br class=""></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote></blockquote><br class=""></blockquote></blockquote><br class=""><br class=""></blockquote></blockquote>_______________________________________________<br class="">Devel mailing list<br class=""><a href="mailto:Devel@ovirt.org" class="">Devel@ovirt.org</a><br class="">http://lists.ovirt.org/mailman/listinfo/devel<br class=""><br class=""><br class=""></div></div></blockquote></div><br class=""></div></body></html>