Dropping rpm build from ovirt-engine check-merged.sh
Hi, Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now). The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3]. So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'. Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests. [1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/bui... [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm -- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri <eedri@redhat.com> wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically. If there's not gating process performed by check-merge then I agree in dropping rpm build.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check- merged-el7-x86_64/buildTimeTrend [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
----- Original Message -----
From: "Sandro Bonazzola" <sbonazzo@redhat.com> To: "Eyal Edri" <eedri@redhat.com> Cc: "Juan Hernandez" <jhernand@redhat.com>, "infra" <infra@ovirt.org>, "devel" <devel@ovirt.org> Sent: Monday, September 19, 2016 8:41:12 AM Subject: Re: [ovirt-devel] Dropping rpm build from ovirt-engine check-merged.sh
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri@redhat.com > wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically. If there's not gating process performed by check-merge then I agree in dropping rpm build.
First of all, for oVirt Engine snapshot (non-release) builds, we should avoid doing a full GWT compilation [all browsers x all locales]. That's simply because the full GWT compilation is too costly for CI to run on a regular basis. Doing [Firefox + Chrome x English = 2 permutations] && [draft GWT build option enabled] for snapshot builds should be enough. The *only* downside of not doing a full GWT build is that problems with localization resources [e.g. non-English .properties files] will not be reported by the GWT compiler. But we have Java unit tests to cover this, so it should be OK. (And if not, we must improve those tests.) In general, we should do a full GWT compilation only for release builds. Eyal mentioned at [1] that his team is working on 'build official manual' option to standard CI so that would be a great place to document & perform the full GWT build. As for build jobs: if `check-merged` is indeed acting as gate for merge [fail of `check-merged` => patch not merged, `build-artifacts` does not execute], then it actually makes sense to: - keep `check-merged` doing a (draft) GWT build + Engine RPM build - maybe skip GWT build in `check-patch` [right now, there's detection logic => if frontend files were changed, do GWT build] Otherwise, if `check-merged` doesn't act as gate for merge, we can just skip GWT build / Engine RPM build for that script.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/bui... [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Mon, Sep 19, 2016 at 6:49 PM, Vojtech Szocs <vszocs@redhat.com> wrote:
----- Original Message -----
From: "Sandro Bonazzola" <sbonazzo@redhat.com> To: "Eyal Edri" <eedri@redhat.com> Cc: "Juan Hernandez" <jhernand@redhat.com>, "infra" <infra@ovirt.org>, "devel" <devel@ovirt.org> Sent: Monday, September 19, 2016 8:41:12 AM Subject: Re: [ovirt-devel] Dropping rpm build from ovirt-engine check-merged.sh
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri@redhat.com > wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically. If there's not gating process performed by check-merge then I agree in dropping rpm build.
First of all, for oVirt Engine snapshot (non-release) builds, we should avoid doing a full GWT compilation [all browsers x all locales]. That's simply because the full GWT compilation is too costly for CI to run on a regular basis.
+1, It also makes us find regressions slower because it takes more than 1 hour to build + consume very valuable resource which is BM slave.
Doing [Firefox + Chrome x English = 2 permutations] && [draft GWT build option enabled] for snapshot builds should be enough.
The *only* downside of not doing a full GWT build is that problems with localization resources [e.g. non-English .properties files] will not be reported by the GWT compiler. But we have Java unit tests to cover this, so it should be OK. (And if not, we must improve those tests.)
In general, we should do a full GWT compilation only for release builds. Eyal mentioned at [1] that his team is working on
'build official manual' option to standard CI
Already merged and working for most projects.who requested it, also part of standard CI.
so that would be a great place to document & perform the full GWT build.
As for build jobs: if `check-merged` is indeed acting as gate for merge [fail of `check-merged` => patch not merged
its not the case today, we'll need to install Zuul for that, which is in the plan (including system tests).
, `build-artifacts` does not execute], then it actually makes sense to:
- keep `check-merged` doing a (draft) GWT build + Engine RPM build - maybe skip GWT build in `check-patch` [right now, there's detection logic => if frontend files were changed, do GWT build]
Otherwise, if `check-merged` doesn't act as gate for merge, we can just skip GWT build / Engine RPM build for that script.
Yea, doesn't act as gate, it just run post merge.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-
merged-el7-x86_64/buildTimeTrend
[3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri <eedri@redhat.com> wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically.
What do you mean by 'gating to the merge'? I'm not sure I understand what it means. Isn't check-patch.sh does the gating? check-merge runs post merge so its already too late to gate the code ... And I think check-merge and check-patch currently runs the same rpmbuild command, so I don't see how check-merged has any value over check-patch.
If there's not gating process performed by check-merge then I agree in dropping rpm build.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-m erged-el7-x86_64/buildTimeTrend [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri <eedri@redhat.com> wrote:
On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri <eedri@redhat.com> wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically.
What do you mean by 'gating to the merge'? I'm not sure I understand what it means. Isn't check-patch.sh does the gating? check-merge runs post merge so its already too late to gate the code ... And I think check-merge and check-patch currently runs the same rpmbuild command, so I don't see how check-merged has any value over check-patch.
when merge command is issued a rebase is done as well. We still need a check-merged job because the code checked by check-patch is not the same anymore when check-merged runs. In original desing of stdci, check-merged was supposed to become a gating test for build-artifacts.
If there's not gating process performed by check-merge then I agree in dropping rpm build.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-m erged-el7-x86_64/buildTimeTrend [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
On Tue, Sep 20, 2016 at 9:34 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri <eedri@redhat.com> wrote:
On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri <eedri@redhat.com> wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically.
What do you mean by 'gating to the merge'? I'm not sure I understand what it means. Isn't check-patch.sh does the gating? check-merge runs post merge so its already too late to gate the code ... And I think check-merge and check-patch currently runs the same rpmbuild command, so I don't see how check-merged has any value over check-patch.
when merge command is issued a rebase is done as well. We still need a check-merged job because the code checked by check-patch is not the same anymore when check-merged runs.
OK, now I understand, so indeed check-merge can potentially run on different code than check-patch and possibly fail due to it.
In original desing of stdci, check-merged was supposed to become a gating test for build-artifacts.
We have it in our backlog, i.e installing Zuul and adding gating for the check-merged jobs, its mostly relevant for system jobs, but we can defiently do it first for simple 'check-merged.sh' jobs as part of standard CI. Opened a ticket for it [1] [1] https://ovirt-jira.atlassian.net/browse/OVIRT-734
If there's not gating process performed by check-merge then I agree in dropping rpm build.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-m erged-el7-x86_64/buildTimeTrend [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Tue, Sep 20, 2016 at 11:27 AM, Eyal Edri <eedri@redhat.com> wrote:
On Tue, Sep 20, 2016 at 9:34 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri <eedri@redhat.com> wrote:
On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri <eedri@redhat.com> wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically.
What do you mean by 'gating to the merge'? I'm not sure I understand what it means. Isn't check-patch.sh does the gating? check-merge runs post merge so its already too late to gate the code ... And I think check-merge and check-patch currently runs the same rpmbuild command, so I don't see how check-merged has any value over check-patch.
when merge command is issued a rebase is done as well. We still need a check-merged job because the code checked by check-patch is not the same anymore when check-merged runs.
OK, now I understand, so indeed check-merge can potentially run on different code than check-patch and possibly fail due to it.
If we require only fast-forward merges, there is no way to merge patch before a rebase. Once you rebase a patch, check-patch runs... So check-merge may be unneeded in this case.
In original desing of stdci, check-merged was supposed to become a gating test for build-artifacts.
We have it in our backlog, i.e installing Zuul and adding gating for the check-merged jobs, its mostly relevant for system jobs, but we can defiently do it first for simple 'check-merged.sh' jobs as part of standard CI.
Opened a ticket for it [1]
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-734
If there's not gating process performed by check-merge then I agree in dropping rpm build.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-m erged-el7-x86_64/buildTimeTrend [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Hi, to me, it seems that with current CI flow, we can do following: - in build-artifacts.sh, build UI for English (only) & all supported browsers (IE10+, Firefox, Chrome), by using `ovirt_build_locales 0` as the default fallback Note: this will cut down the # of permutations from 3x9 to just 3 (currently we have 3 supported browser types, 9 supported locales). - ensure that release builds still target all supported UI locales, by using `ovirt_build_locales 1` (override the default fallback) - it would be nice if the Jenkins job for build-artifacts.sh would set some `RELEASE=1` flag (or similar) when doing a release build (is this possible with current standard CI?) Eyal/others, does above make sense? Also, as Nir mentioned, if `ovirt-engine` repo was configured with Submit Type == Fast Forward Only, we could drop RPM / GWT build in check-merged.sh (which was Eyal's original suggestion). Regards, Vojtech ----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Eyal Edri" <eedri@redhat.com> Cc: "Juan Hernandez" <jhernand@redhat.com>, "devel" <devel@ovirt.org>, "Martin Perina" <mperina@redhat.com>, "Tal Nisan" <tnisan@redhat.com>, "infra" <infra@ovirt.org> Sent: Tuesday, September 20, 2016 5:38:08 PM Subject: Re: Dropping rpm build from ovirt-engine check-merged.sh
On Tue, Sep 20, 2016 at 11:27 AM, Eyal Edri < eedri@redhat.com > wrote:
On Tue, Sep 20, 2016 at 9:34 AM, Sandro Bonazzola < sbonazzo@redhat.com > wrote:
On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri < eedri@redhat.com > wrote:
On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola < sbonazzo@redhat.com > wrote:
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri@redhat.com > wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically.
What do you mean by 'gating to the merge'? I'm not sure I understand what it means. Isn't check-patch.sh does the gating? check-merge runs post merge so its already too late to gate the code ... And I think check-merge and check-patch currently runs the same rpmbuild command, so I don't see how check-merged has any value over check-patch.
when merge command is issued a rebase is done as well. We still need a check-merged job because the code checked by check-patch is not the same anymore when check-merged runs.
OK, now I understand, so indeed check-merge can potentially run on different code than check-patch and possibly fail due to it.
If we require only fast-forward merges, there is no way to merge patch before a rebase. Once you rebase a patch, check-patch runs...
So check-merge may be unneeded in this case.
In original desing of stdci, check-merged was supposed to become a gating test for build-artifacts.
We have it in our backlog, i.e installing Zuul and adding gating for the check-merged jobs, its mostly relevant for system jobs, but we can defiently do it first for simple 'check-merged.sh' jobs as part of standard CI.
Opened a ticket for it [1]
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-734
If there's not gating process performed by check-merge then I agree in dropping rpm build.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/bui... [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
On Mon, Sep 26, 2016 at 7:37 PM, Vojtech Szocs <vszocs@redhat.com> wrote:
Hi,
to me, it seems that with current CI flow, we can do following:
- in build-artifacts.sh, build UI for English (only) & all supported browsers (IE10+, Firefox, Chrome), by using `ovirt_build_locales 0` as the default fallback
+1 from my side. Sandro - any objections or thoughts?
Note: this will cut down the # of permutations from 3x9 to just 3 (currently we have 3 supported browser types, 9 supported locales).
- ensure that release builds still target all supported UI locales, by using `ovirt_build_locales 1` (override the default fallback)
You can make sure in the automation/ dir in the 'manual*' script has it.
- it would be nice if the Jenkins job for build-artifacts.sh would set some `RELEASE=1` flag (or similar) when doing a release build (is this possible with current standard CI?)
In planning, what I want to do is to move the 'manual official' [1] to be automatic official builds, as for RELEASE flag, we'll need to discuss on changing how we manage ovirt-engine versioning, I think we talked about 4.1 or later to try to use mvn release plugin or an alternate way to avoid the manual patches for updating POM files.
Eyal/others, does above make sense?
Yea, see my comments.
Also, as Nir mentioned, if `ovirt-engine` repo was configured with Submit Type == Fast Forward Only, we could drop RPM / GWT build in check-merged.sh (which was Eyal's original suggestion).
Its currently on 'Rebase if necessary', if there is an agreement I can change it to 'Fast forward only'.
Regards, Vojtech
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Eyal Edri" <eedri@redhat.com> Cc: "Juan Hernandez" <jhernand@redhat.com>, "devel" <devel@ovirt.org>, "Martin Perina" <mperina@redhat.com>, "Tal Nisan" <tnisan@redhat.com>, "infra" <infra@ovirt.org> Sent: Tuesday, September 20, 2016 5:38:08 PM Subject: Re: Dropping rpm build from ovirt-engine check-merged.sh
On Tue, Sep 20, 2016 at 11:27 AM, Eyal Edri < eedri@redhat.com > wrote:
On Tue, Sep 20, 2016 at 9:34 AM, Sandro Bonazzola < sbonazzo@redhat.com
wrote:
On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri < eedri@redhat.com > wrote:
On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola < sbonazzo@redhat.com
wrote:
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri@redhat.com > wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically.
What do you mean by 'gating to the merge'? I'm not sure I understand what it means. Isn't check-patch.sh does the gating? check-merge runs post merge so its already too late to gate the code ... And I think check-merge and check-patch currently runs the same rpmbuild command, so I don't see how check-merged has any value over check-patch.
when merge command is issued a rebase is done as well. We still need a check-merged job because the code checked by check-patch is not the same anymore when check-merged runs.
OK, now I understand, so indeed check-merge can potentially run on different code than check-patch and possibly fail due to it.
If we require only fast-forward merges, there is no way to merge patch before a rebase. Once you rebase a patch, check-patch runs...
So check-merge may be unneeded in this case.
In original desing of stdci, check-merged was supposed to become a gating test for build-artifacts.
We have it in our backlog, i.e installing Zuul and adding gating for the check-merged jobs, its mostly relevant for system jobs, but we can defiently do it first for simple 'check-merged.sh' jobs as part of standard CI.
Opened a ticket for it [1]
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-734
If there's not gating process performed by check-merge then I agree in dropping rpm build.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check- merged-el7-x86_64/buildTimeTrend [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Tue, Sep 27, 2016 at 2:40 PM, Eyal Edri <eedri@redhat.com> wrote:
On Mon, Sep 26, 2016 at 7:37 PM, Vojtech Szocs <vszocs@redhat.com> wrote:
Hi,
to me, it seems that with current CI flow, we can do following:
- in build-artifacts.sh, build UI for English (only) & all supported browsers (IE10+, Firefox, Chrome), by using `ovirt_build_locales 0` as the default fallback
+1 from my side. Sandro - any objections or thoughts?
No objections provided that for release we can fix it.
Note: this will cut down the # of permutations from 3x9 to just 3 (currently we have 3 supported browser types, 9 supported locales).
- ensure that release builds still target all supported UI locales, by using `ovirt_build_locales 1` (override the default fallback)
You can make sure in the automation/ dir in the 'manual*' script has it.
- it would be nice if the Jenkins job for build-artifacts.sh would set some `RELEASE=1` flag (or similar) when doing a release build (is this possible with current standard CI?)
In planning, what I want to do is to move the 'manual official' [1] to be automatic official builds, as for RELEASE flag, we'll need to discuss on changing how we manage ovirt-engine versioning, I think we talked about 4.1 or later to try to use mvn release plugin or an alternate way to avoid the manual patches for updating POM files.
Eyal/others, does above make sense?
Yea, see my comments.
Also, as Nir mentioned, if `ovirt-engine` repo was configured with Submit Type == Fast Forward Only, we could drop RPM / GWT build in check-merged.sh (which was Eyal's original suggestion).
Its currently on 'Rebase if necessary', if there is an agreement I can change it to 'Fast forward only'.
Fast forward only will require lot of manual rebase. That's the reason we dropped it in the past
Regards, Vojtech
----- Original Message -----
From: "Nir Soffer" <nsoffer@redhat.com> To: "Eyal Edri" <eedri@redhat.com> Cc: "Juan Hernandez" <jhernand@redhat.com>, "devel" <devel@ovirt.org>, "Martin Perina" <mperina@redhat.com>, "Tal Nisan" <tnisan@redhat.com>, "infra" <infra@ovirt.org> Sent: Tuesday, September 20, 2016 5:38:08 PM Subject: Re: Dropping rpm build from ovirt-engine check-merged.sh
On Tue, Sep 20, 2016 at 11:27 AM, Eyal Edri < eedri@redhat.com > wrote:
On Tue, Sep 20, 2016 at 9:34 AM, Sandro Bonazzola < sbonazzo@redhat.com
wrote:
On Mon, Sep 19, 2016 at 7:56 PM, Eyal Edri < eedri@redhat.com > wrote:
On Mon, Sep 19, 2016 at 9:41 AM, Sandro Bonazzola < sbonazzo@redhat.com
wrote:
On Sun, Sep 18, 2016 at 4:18 PM, Eyal Edri < eedri@redhat.com > wrote:
Hi,
Following [1] I'd like to propose to remove rpm building from the 'check-merged.sh' script from ovirt-engine (master for now).
The job [2] takes on avg 15 min while actually the rpms are built already in check-patch (with gwt draft mode if needed) and runs exactly the same build rpm command as check-patch [3].
So there isn't real value in running exactly the same rpm build post merge, and we already build full permutation mode in 'build-artifacts.sh'.
Any reason to keep it? We can cut down valuable time in CI if we drop it and vacant more time for more meaningful tests.
This depends on the flow: if we make check_merge gating to the merge and to the build we should keep the rpm build becuase at merge a rebase is done automatically.
What do you mean by 'gating to the merge'? I'm not sure I understand what it means. Isn't check-patch.sh does the gating? check-merge runs post merge so its already too late to gate the code ... And I think check-merge and check-patch currently runs the same rpmbuild command, so I don't see how check-merged has any value over check-patch.
when merge command is issued a rebase is done as well. We still need a check-merged job because the code checked by check-patch is not the same anymore when check-merged runs.
OK, now I understand, so indeed check-merge can potentially run on different code than check-patch and possibly fail due to it.
If we require only fast-forward merges, there is no way to merge patch before a rebase. Once you rebase a patch, check-patch runs...
So check-merge may be unneeded in this case.
In original desing of stdci, check-merged was supposed to become a gating test for build-artifacts.
We have it in our backlog, i.e installing Zuul and adding gating for the check-merged jobs, its mostly relevant for system jobs, but we can defiently do it first for simple 'check-merged.sh' jobs as part of standard CI.
Opened a ticket for it [1]
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-734
If there's not gating process performed by check-merge then I agree in dropping rpm build.
[1] https://ovirt-jira.atlassian.net/browse/OVIRT-416 [2] http://jenkins.ovirt.org/job/ovirt-engine_master_check-merge d-el7-x86_64/buildTimeTrend [3] rpmbuild \ -D "_rpmdir $PWD/output" \ -D "_topmdir $PWD/rpmbuild" \ -D "release_suffix ${SUFFIX}" \ -D "ovirt_build_ut $BUILD_UT" \ -D "ovirt_build_extra_flags $EXTRA_BUILD_FLAGS" \ -D "ovirt_build_draft 1" \ --rebuild output/*.src.rpm
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community
collaboration. > See how it works at redhat.com > > > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > > > -- > Sandro Bonazzola > Better technology. Faster innovation. Powered by community collaboration. > See how it works at redhat.com > > > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) > > _______________________________________________ > Infra mailing list > Infra@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra > > > > _______________________________________________ > Infra mailing list > Infra@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra >
-- Eyal Edri Associate Manager RHV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com <https://www.redhat.com/it/about/events/red-hat-open-source-day-2016>
participants (4)
-
Eyal Edri -
Nir Soffer -
Sandro Bonazzola -
Vojtech Szocs