[ovirt-devel] Re: CI: vdsm-standard-check-patch fails

Amit Bawer <abawer(a)redhat.com&gt; writes: > On Thu, Aug 8, 2019 at 2:50 PM Marcin Sobczyk <msobczyk(a)redhat.com&gt; wrote: > >> >> On 8/8/19 1:44 PM, Amit Bawer wrote: >> >> >> >> On Thu, Aug 8, 2019 at 12:48 PM Milan Zamazal <mzamazal(a)redhat.com&gt; wrote: >> >>> Amit Bawer <abawer(a)redhat.com&gt; writes: >>> >>> > On Wed, Aug 7, 2019 at 3:14 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >>> > >>> >> On Wed, Aug 7, 2019 at 3:06 PM Amit Bawer <abawer(a)redhat.com&gt; wrote: >>> >> >>> >>> >>> >>> >>> >>> On Wed, Aug 7, 2019 at 2:53 PM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: >>> >>> >>> >>>> On Wed, Aug 7, 2019 at 1:23 PM Amit Bawer <abawer(a)redhat.com&gt; wrote: >>> >>>> >>> >>>>> >>> >>>>> >>> >>>>> On Wed, Aug 7, 2019 at 11:19 AM Amit Bawer <abawer(a)redhat.com&gt; >>> wrote: >>> >>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> On Tue, Aug 6, 2019 at 5:07 PM Nir Soffer <nsoffer(a)redhat.com&gt; >>> wrote: >>> >>>>>> >>> >>>>>>> On Tue, Aug 6, 2019 at 5:01 PM Amit Bawer <abawer(a)redhat.com&gt; >>> wrote: >>> >>>>>>> >>> >>>>>>>> >>> >>>>>>>> >>> >>>>>>>> On Tue, Aug 6, 2019 at 4:58 PM Nir Soffer <nsoffer(a)redhat.com > >>> >>>>>>>> wrote: >>> >>>>>>>> >>> >>>>>>>>> On Tue, Aug 6, 2019 at 11:27 AM Amit Bawer < abawer(a)redhat.com&gt; >>> >>>>>>>>> wrote: >>> >>>>>>>>> >>> >>>>>>>>>> I have seen some improvement: when I re-trigger the CI per >>> patch I >>> >>>>>>>>>> am able to pass or get the actual test errors if any (if not >>> on first try, >>> >>>>>>>>>> then on second one). >>> >>>>>>>>>> Probably not a very useful information, but I have noticed that >>> >>>>>>>>>> when I push 30+ patches at the same >>> >>>>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>> Do not do that, jenkins cannot handle 30 concurrent builds, and >>> is >>> >>>>>>>>> it also bad for reviewers, >>> >>>>>>>>> getting several mails about every patch in your chain, for every >>> >>>>>>>>> rebase. >>> >>>>>>>>> >>> >>>>>>>> >>> >>>>>>>> Is there is a way to prevent CI from running per gerrit push >>> >>>>>>>> (without working on 30 different branches) ? >>> >>>>>>>> >>> >>>>>>> >>> >>>>>>> I don't know about such way. >>> >>>>>>> >>> >>>>>> >>> >>>>>> A legit option could be adding the Skip CI plugin to jenkins >>> plugins >>> >>>>>> [1]; with that devs can add "[skip ci]" to their commit messages >>> to prevent >>> >>>>>> jenkins from automatically launching CI upon push. >>> >>>>>> >>> >>>>> >>> >>>> Do you want to modify the commit message for every patch to decide >>> if ci >>> >>>> should run or not? >>> >>>> >>> >>> >>> >>> I think that having the option to knowingly disable automated CI in >>> some >>> >>> cases is useful. We could always re-trigger when time is right [3]. >>> >>> [3] https://jenkins.ovirt.org/login?from=%2Fgerrit_manual_trigger%2F >>> >>> >>> >> >>> >> This is too much work, but I think today we can add a comment to gerrit >>> >> like >>> >> >>> >> ci please test >>> >> >>> >> That will trigger a build of this patch. >>> >> >>> > >>> > Indeed, but it leaves the "Continuous-Integration" mark untouched in >>> > gerrit, giving the wrong impression this patch is still CI failed. >>> >>> No, it updates CI score. I use it routinely with falsely failed tests. >>> >>> In my experience, CI score may not get updated if there are concurrent >>> builds, such as when you upload a new version of a patch while CI is >>> still running on the previous version. >>> >> >> I may have missed something, but i tried "ci build" gerrit comment on one >> of the CI failed patches https://gerrit.ovirt.org/#/c/101357/ >> the CI build passed, but CI indicator is still -1. AFAICT I had no other >> CI jobs running at the time. >> >> "ci build" runs only the "build-artifacts" stage. To affect the score (and >> run the tests as a matter of fact) you should use "ci test". >> > Thanks for the clarification, good to know. > So that only leaves the "how do disable automated CI upon gerrit push" > issue. One solution of the problem might be to have smaller batches of unmerged patches. If we could review and merge patches faster, it would be less burden for everybody, including CI infrastructure. >>> > The re-trigger UI takes care for that as well. >>> > >>> > >>> >> >>> >> >>> >>> >>> >>> >>> >>>> >>> >>>>> Another option is to emulate the behaviour in the existing gerrit >>> >>>>>> plugin (guess there is already such one in ovirt's jenkins), for >>> example >>> >>>>>> skipping by a topic regex [2]. >>> >>>>>> >>> >>>>> >>> >>>> Not clear how this will help. >>> >>>> >>> >>> >>> >>> If I make a gerrit topic with some name like "my_feature_skip_ci" I >>> can >>> >>> control whether I want to have automated CI for its patches. >>> >>> When I want to go back to normal I can rename it to "my_feature" and >>> have >>> >>> CI per push as usual. >>> >>> >>> >>> >>> >>>> I think a possible solution can be running only the top patch in a >>> >>>> changeset, same way we have in travis, >>> >>>> and the same way systems that grab patches from mailing lists work. >>> >>>> Every post to gerrit will trigger one >>> >>>> build, instead of one build per patch in the chain. >>> >>>> >>> >>> >>> >>> That could do as well. >>> >>> >>> >>> >>> >>>> Of course this will allow merging broken patches that are fixed by a >>> >>>> later patch in the chain, which >>> >>>> is also not ideal, but it is better given our restricted resources. >>> >>>> >>> >>> >>> >>> We can re-trigger CI manually in this case as part of the verification >>> >>> process. >>> >>> >>> >> >>> >>> >>> >>>> +Anton Marchukov <amarchuk(a)redhat.com&gt; I have been told you might >>> be >>> >>>>> familiar with a similar solution. >>> >>>>> >>> >>>>>> >>> >>>>>> [1] https://plugins.jenkins.io/ci-skip >>> >>>>>> [2] >>> >>>>>> >>> https://stackoverflow.com/questions/37807941/how-can-i-get-jenkins-gerrit... >>> >>>>>> >>> >>>>>> >>> >>>>>>> >>> >>>>>>> I'm using keeping several small active branches. While you wait >>> for >>> >>>>>>> reviews on one topic >>> >>>>>>> you can work on the other branches. >>> >>>>>>> >>> >>>>>>> >>> >>>>>>> >>> >>>>>>>> >>> >>>>>>>>> >>> >>>>>>>>>> time the AWS connection issue arises constantly. >>> >>>>>>>>>> >>> >>>>>>>>>> On Sun, Aug 4, 2019 at 4:49 PM Eyal Edri <eedri(a)redhat.com&gt; >>> wrote: >>> >>>>>>>>>> >>> >>>>>>>>>>> This was reported already and AFAIK its a network issue >>> between >>> >>>>>>>>>>> AWS and PHX which is still being investigated. >>> >>>>>>>>>>> Evgheni, any insights or update on the issue? should we >>> involve >>> >>>>>>>>>>> debugging from amazon side? >>> >>>>>>>>>>> >>> >>>>>>>>>>> On Sun, Aug 4, 2019 at 4:46 PM Amit Bawer < abawer(a)redhat.com&gt; >>> >>>>>>>>>>> wrote: >>> >>>>>>>>>>> >>> >>>>>>>>>>>> Hi, >>> >>>>>>>>>>>> CI seems to fail constantly for unavailable remote gerrit >>> >>>>>>>>>>>> repository. >>> >>>>>>>>>>>> Example can be seen here: >>> >>>>>>>>>>>> >>> >>>>>>>>>>>> >>> https://jenkins.ovirt.org/job/vdsm_standard-check-patch/9415/console >>> >>>>>>>>>>>> _______________________________________________ >>> >>>>>>>>>>>> Devel mailing list -- devel(a)ovirt.org >>> >>>>>>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> >>>>>>>>>>>> Privacy Statement: >>> https://www.ovirt.org/site/privacy-policy/ >>> >>>>>>>>>>>> oVirt Code of Conduct: >>> >>>>>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>> >>>>>>>>>>>> List Archives: >>> >>>>>>>>>>>> >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/AHPHUZAABAQ... >>> >>>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> -- >>> >>>>>>>>>>> >>> >>>>>>>>>>> Eyal edri >>> >>>>>>>>>>> >>> >>>>>>>>>>> He / Him / His >>> >>>>>>>>>>> >>> >>>>>>>>>>> >>> >>>>>>>>>>> MANAGER >>> >>>>>>>>>>> >>> >>>>>>>>>>> CONTINUOUS PRODUCTIZATION >>> >>>>>>>>>>> >>> >>>>>>>>>>> SYSTEM ENGINEERING >>> >>>>>>>>>>> >>> >>>>>>>>>>> Red Hat <https://www.redhat.com/> >>> >>>>>>>>>>> <https://red.ht/sig> >>> >>>>>>>>>>> phone: +972-9-7692018 >>> >>>>>>>>>>> irc: eedri (on #tlv #rhev-dev #rhev-integ #cp-devel) >>> >>>>>>>>>>> >>> >>>>>>>>>> _______________________________________________ >>> >>>>>>>>>> Devel mailing list -- devel(a)ovirt.org >>> >>>>>>>>>> To unsubscribe send an email to devel-leave(a)ovirt.org >>> >>>>>>>>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> >>>>>>>>>> oVirt Code of Conduct: >>> >>>>>>>>>> https://www.ovirt.org/community/about/community-guidelines/ >>> >>>>>>>>>> List Archives: >>> >>>>>>>>>> >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/W6DUMIUSN5D... >>> >>>>>>>>>> >>> >>>>>>>>> >>> > _______________________________________________ >>> > Devel mailing list -- devel(a)ovirt.org >>> > To unsubscribe send an email to devel-leave(a)ovirt.org >>> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> > oVirt Code of Conduct: >>> https://www.ovirt.org/community/about/community-guidelines/ >>> > List Archives: >>> > >>> https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MMXGDBHHLMF... >>> >> >> _______________________________________________ >> Infra mailing list -- infra(a)ovirt.org >> To unsubscribe send an email to infra-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZUT3AJSGY5L... >> >>