From: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt; To: "Eyal Edri" <eedri(a)redhat.com&gt; Cc: "Juan Hernandez" <jhernand(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt;, "devel" <devel(a)ovirt.org&gt; Sent: Monday, September 19, 2016 8:41:12 AM Subject: Re: [ovirt-devel] Dropping rpm build from ovirt-engine check-merged.sh On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri(a)redhat.com > wrote: Hi, Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now). The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3]. So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'. Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests. This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically. If there's not gating process performed by check-merge then I agree in dropping rpm build.

[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/... [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com _______________________________________________ Devel mailing list Devel(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/devel

----- Original Message ----- > From: "Sandro Bonazzola" <sbonazzo(a)redhat.com&gt; > To: "Eyal Edri" <eedri(a)redhat.com&gt; > Cc: "Juan Hernandez" <jhernand(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt;, "devel" <devel(a)ovirt.org&gt; > Sent: Monday, September 19, 2016 8:41:12 AM > Subject: Re: [ovirt-devel] Dropping rpm build from ovirt-engine check-merged.sh > > > > On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri(a)redhat.com > wrote: > > > > Hi, > > Following [1] I'd like to propose to remove rpm building from the > 'check-merged.sh' script from ovirt-engine (master for now). > > The job [2] takes on avg 15 min while actually the rpms are built already in > check-patch > (with gwt draft mode if needed) and runs exactly the same build rpm command > as check-patch [3]. > > So there isn't real value in running exactly the same rpm build post merge, > and we already build full permutation mode in 'build-artifacts.sh'. > > Any reason to keep it? > We can cut down valuable time in CI if we drop it and vacant more time for > more meaningful tests. > > > This depends on the flow: if we make check_merge gating to the merge and to > the build we should keep the rpm build becuase at merge a rebase is done > automatically. > If there's not gating process performed by check-merge then I agree in > dropping rpm build. First of all, for oVirt Engine snapshot (non-release) builds, we should avoid doing a full GWT compilation [all browsers x all locales]. That's simply because the full GWT compilation is too costly for CI to run on a regular basis.

Doing [Firefox + Chrome x English = 2 permutations] && [draft GWT build option enabled] for snapshot builds should be enough. The *only* downside of not doing a full GWT build is that problems with localization resources [e.g. non-English .properties files] will not be reported by the GWT compiler. But we have Java unit tests to cover this, so it should be OK. (And if not, we must improve those tests.) In general, we should do a full GWT compilation only for release builds. Eyal mentioned at [1] that his team is working on 'build official manual' option to standard CI

so that would be a great place to document & perform the full GWT build. As for build jobs: if `check-merged` is indeed acting as gate for merge [fail of `check-merged` => patch not merged

, `build-artifacts` does not execute], then it actually makes sense to: - keep `check-merged` doing a (draft) GWT build + Engine RPM build - maybe skip GWT build in `check-patch` [right now, there's detection logic => if frontend files were changed, do GWT build] Otherwise, if `check-merged` doesn't act as gate for merge, we can just skip GWT build / Engine RPM build for that script.

> > > > > > > [1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 > [2] > http://jenkins.ovirt.org/job/ovirt-engine_master_check- merged-el7-x86_64/buildTimeTrend > [3] > rpmbuild \ > -D "_rpmdir $PWD/output" \ > -D "_topmdir $PWD/rpmbuild" \ > -D "release_suffix ${SUFFIX}" \ > -D "ovirt_build_ut $BUILD_UT" \ > -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ > -D "ovirt_build_draft 1" \ > --rebuild output/*.src.rpm > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com > > > _______________________________________________ > Devel mailing list > Devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel

On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri <eedri(a)redhat.com&gt; wrote: > > > On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; > wrote: > >> >> >> On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri <eedri(a)redhat.com&gt; wrote: >> >>> Hi, >>> >>> Following [1] I'd like to propose to remove rpm building from the >>> 'check-merged.sh' script from ovirt-engine (master for now). >>> >>> The job [2] takes on avg 15 min while actually the rpms are built >>> already in check-patch >>> (with gwt draft mode if needed) and runs exactly the same build rpm >>> command as check-patch [3]. >>> >>> So there isn't real value in running exactly the same rpm build post >>> merge, and we already build full permutation mode in 'build-artifacts.sh'. >>> >>> Any reason to keep it? >>> We can cut down valuable time in CI if we drop it and vacant more time >>> for more meaningful tests. >>> >> >> >> This depends on the flow: if we make check_merge gating to the merge and >> to the build we should keep the rpm build becuase at merge a rebase is done >> automatically. >> > > What do you mean by 'gating to the merge'? I'm not sure I understand what > it means. > Isn't check-patch.sh does the gating? check-merge runs post merge so its > already too late to gate the code ... > And I think check-merge and check-patch currently runs the same rpmbuild > command, so I don't see how check-merged has any value over check-patch. > when merge command is issued a rebase is done as well. We still need a check-merged job because the code checked by check-patch is not the same anymore when check-merged runs.

In original desing of stdci, check-merged was supposed to become a gating test for build-artifacts.

> > >> If there's not gating process performed by check-merge then I agree in >> dropping rpm build. >> >> >> >>> >>> >>> [1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 >>> [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-m >>> erged-el7-x86_64/buildTimeTrend >>> [3] >>> rpmbuild \ >>> -D "_rpmdir $PWD/output" \ >>> -D "_topmdir $PWD/rpmbuild" \ >>> -D "release_suffix ${SUFFIX}" \ >>> -D "ovirt_build_ut $BUILD_UT" \ >>> -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ >>> -D "ovirt_build_draft 1" \ >>> --rebuild output/*.src.rpm >>> >>> >>> -- >>> Eyal Edri >>> Associate Manager >>> RHV DevOps >>> EMEA ENG Virtualization R&D >>> Red Hat Israel >>> >>> phone: +972-9-7692018 >>> irc: eedri (on #tlv #rhev-dev #rhev-integ) >>> >> >> >> >> -- >> Sandro Bonazzola >> Better technology. Faster innovation. Powered by community collaboration. >> See how it works at redhat.com >> <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016> >> > > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) > -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>

From: "Nir Soffer" <nsoffer(a)redhat.com&gt; To: "Eyal Edri" <eedri(a)redhat.com&gt; Cc: "Juan Hernandez" <jhernand(a)redhat.com&gt;, "devel" <devel(a)ovirt.org&gt;, "Martin Perina" <mperina(a)redhat.com&gt;, "Tal Nisan" <tnisan(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt; Sent: Tuesday, September 20, 2016 5:38:08 PM Subject: Re: Dropping rpm build from ovirt-engine check-merged.sh On Tue, Sep 20, 2016 at 11:27 AM, Eyal Edri < eedri(a)redhat.com > wrote: On Tue, Sep 20, 2016 at 9:34 AM, Sandro Bonazzola < sbonazzo(a)redhat.com > wrote: On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri < eedri(a)redhat.com > wrote: On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola < sbonazzo(a)redhat.com > wrote: On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri(a)redhat.com > wrote: Hi, Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now). The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3]. So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'. Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests. This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically. What do you mean by 'gating to the merge'? I'm not sure I understand what it means. Isn't check-patch.sh does the gating? check-merge runs post merge so its already too late to gate the code ... And I think check-merge and check-patch currently runs the same rpmbuild command, so I don't see how check-merged has any value over check-patch. when merge command is issued a rebase is done as well. We still need a check-merged job because the code checked by check-patch is not the same anymore when check-merged runs. OK, now I understand, so indeed check-merge can potentially run on different code than check-patch and possibly fail due to it. If we require only fast-forward merges, there is no way to merge patch before a rebase. Once you rebase a patch, check-patch runs... So check-merge may be unneeded in this case. In original desing of stdci, check-merged was supposed to become a gating test for build-artifacts. We have it in our backlog, i.e installing Zuul and adding gating for the check-merged jobs, its mostly relevant for system jobs, but we can defiently do it first for simple 'check-merged.sh' jobs as part of standard CI. Opened a ticket for it [1] [1] https://ovirt-jira.atlassian.net/browse/OVIRT-734 If there's not gating process performed by check-merge then I agree in dropping rpm build. [1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/... [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Infra mailing list Infra(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/infra _______________________________________________ Infra mailing list Infra(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/infra

On Mon, Sep 26, 2016 at 7:37 PM, Vojtech Szocs <vszocs(a)redhat.com&gt; wrote: > Hi, > > to me, it seems that with current CI flow, we can do following: > > - in build-artifacts.sh, build UI for English (only) & all supported > browsers (IE10+, Firefox, Chrome), by using `ovirt_build_locales 0` > as the default fallback > +1 from my side. Sandro - any objections or thoughts?

> > Note: this will cut down the # of permutations from 3x9 to just 3 > (currently we have 3 supported browser types, 9 supported locales). > > - ensure that release builds still target all supported UI locales, > by using `ovirt_build_locales 1` (override the default fallback) > You can make sure in the automation/ dir in the 'manual*' script has it. > > - it would be nice if the Jenkins job for build-artifacts.sh would > set some `RELEASE=1` flag (or similar) when doing a release build > (is this possible with current standard CI?) > In planning, what I want to do is to move the 'manual official' [1] to be automatic official builds, as for RELEASE flag, we'll need to discuss on changing how we manage ovirt-engine versioning, I think we talked about 4.1 or later to try to use mvn release plugin or an alternate way to avoid the manual patches for updating POM files. > > Eyal/others, does above make sense? > Yea, see my comments. > > Also, as Nir mentioned, if `ovirt-engine` repo was configured with > Submit Type == Fast Forward Only, we could drop RPM / GWT build in > check-merged.sh (which was Eyal's original suggestion). > Its currently on 'Rebase if necessary', if there is an agreement I can change it to 'Fast forward only'.

> > Regards, > Vojtech > > > ----- Original Message ----- > > From: "Nir Soffer" <nsoffer(a)redhat.com&gt; > > To: "Eyal Edri" <eedri(a)redhat.com&gt; > > Cc: "Juan Hernandez" <jhernand(a)redhat.com&gt;, "devel" <devel(a)ovirt.org&gt;, > "Martin Perina" <mperina(a)redhat.com&gt;, "Tal > > Nisan" <tnisan(a)redhat.com&gt;, "infra" <infra(a)ovirt.org&gt; > > Sent: Tuesday, September 20, 2016 5:38:08 PM > > Subject: Re: Dropping rpm build from ovirt-engine check-merged.sh > > > > On Tue, Sep 20, 2016 at 11:27 AM, Eyal Edri < eedri(a)redhat.com > wrote: > > > > > > > > > > > > On Tue, Sep 20, 2016 at 9:34 AM, Sandro Bonazzola < sbonazzo(a)redhat.com > > > > wrote: > > > > > > > > > > > > On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri < eedri(a)redhat.com > wrote: > > > > > > > > > > > > On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola < sbonazzo(a)redhat.com > > > > wrote: > > > > > > > > > > > > On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri(a)redhat.com > wrote: > > > > > > > > Hi, > > > > Following [1] I'd like to propose to remove rpm building from the > > 'check-merged.sh' script from ovirt-engine (master for now). > > > > The job [2] takes on avg 15 min while actually the rpms are built > already in > > check-patch > > (with gwt draft mode if needed) and runs exactly the same build rpm > command > > as check-patch [3]. > > > > So there isn't real value in running exactly the same rpm build post > merge, > > and we already build full permutation mode in 'build-artifacts.sh'. > > > > Any reason to keep it? > > We can cut down valuable time in CI if we drop it and vacant more time > for > > more meaningful tests. > > > > > > This depends on the flow: if we make check_merge gating to the merge > and to > > the build we should keep the rpm build becuase at merge a rebase is done > > automatically. > > > > What do you mean by 'gating to the merge'? I'm not sure I understand > what it > > means. > > Isn't check-patch.sh does the gating? check-merge runs post merge so its > > already too late to gate the code ... > > And I think check-merge and check-patch currently runs the same rpmbuild > > command, so I don't see how check-merged has any value over check-patch. > > > > when merge command is issued a rebase is done as well. We still need a > > check-merged job because the code checked by check-patch is not the same > > anymore when check-merged runs. > > > > OK, now I understand, so indeed check-merge can potentially run on > different > > code than check-patch and possibly fail due to it. > > > > If we require only fast-forward merges, there is no way to merge patch > > before a rebase. Once you rebase a patch, check-patch runs... > > > > So check-merge may be unneeded in this case. > > > > > > > > > > > > > > In original desing of stdci, check-merged was supposed to become a > gating > > test for build-artifacts. > > > > We have it in our backlog, i.e installing Zuul and adding gating for the > > check-merged jobs, its mostly relevant for system jobs, but we can > defiently > > do it first for simple 'check-merged.sh' jobs > > as part of standard CI. > > > > Opened a ticket for it [1] > > > > [1] https://ovirt-jira.atlassian.net/browse/OVIRT-734 > > > > > > > > > > > > > > > > > > > > > > > > If there's not gating process performed by check-merge then I agree in > > dropping rpm build. > > > > > > > > > > > > > > [1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 > > [2] > > http://jenkins.ovirt.org/job/ovirt-engine_master_check-merge > d-el7-x86_64/buildTimeTrend > > [3] > > rpmbuild \ > > -D "_rpmdir $PWD/output" \ > > -D "_topmdir $PWD/rpmbuild" \ > > -D "release_suffix ${SUFFIX}" \ > > -D "ovirt_build_ut $BUILD_UT" \ > > -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ > > -D "ovirt_build_draft 1" \ > > --rebuild output/*.src.rpm > > > > > > -- > > Eyal Edri > > Associate Manager > > RHV DevOps > > EMEA ENG Virtualization R&D > > Red Hat Israel > > > > phone: +972-9-7692018 > > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > > > > > -- > > Sandro Bonazzola > > Better technology. Faster innovation. Powered by community > collaboration. > > See how it works at redhat.com > > > > > > > > > > -- > > Eyal Edri > > Associate Manager > > RHV DevOps > > EMEA ENG Virtualization R&D > > Red Hat Israel > > > > phone: +972-9-7692018 > > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > > > > > -- > > Sandro Bonazzola > > Better technology. Faster innovation. Powered by community > collaboration. > > See how it works at redhat.com > > > > > > > > > > -- > > Eyal Edri > > Associate Manager > > RHV DevOps > > EMEA ENG Virtualization R&D > > Red Hat Israel > > > > phone: +972-9-7692018 > > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > _______________________________________________ > > Infra mailing list > > Infra(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/infra > > > > > > > > _______________________________________________ > > Infra mailing list > > Infra(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/infra > > > -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)