speed up engine development with SKIP_CHECKS
Hi, I've added a SKIP_CHECKS flag to the ovirt-engine build here [1]. Thanks to Roy for the idea [2]. TL;DR: you'll save roughly 2 minutes 10 seconds per full compile with SKIP_CHECKS. You'll save more (3 min 10 sec) if you don't need a 'clean'. You'll save the most (about 3 min 20 sec) if you can skip rebuilding GWT. Here are some statistics (3 runs per command, for averages sake) My test machine is a Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz 256GB Samsung SSD (old, using SATA) 32GB DDR3 @ 1333 MT/s --- fresh master checkout no clean, no gwt, skip checks: make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 1:04.38 total 1:06.04 total 1:04.41 total same as above, but now change one non-GWT class in the utils project make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 1:05.49 total Ok, pretend we recompiled and our utils class change had an error. Same as above, change the one non-GWT class in the utils project again, and use the resume (-pl) functionality: make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 EXTRA_BUILD_FLAGS="-pl org.ovirt.engine.core:utils" 17.958 total (yes, 18 seconds, obviously a world record for any engine build) [The resume piece is EXTRA_BUILD_FLAGS="-pl org.ovirt.engine.ui:webadmin" -- I personally need to use this more often!] --- no clean, no gwt, run checks (but skip unit tests): make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 BUILD_UT=0 3:07.66 total 3:16.69 total 3:14.28 total clean, no gwt, skip checks: make clean install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 4:17.74 total 4:15.76 total 4:13.90 total no clean, gwt (1 permutation for chrome), skip checks: make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=1 SKIP_CHECKS=1 DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari" 4:25.63 total 4:26.71 total 4:30.69 total no clean, gwt (1 permutation for chrome), run checks but skip unit tests: make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=1 BUILD_UT=0 DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari" 6:44.23 total And the whole enchilada [3] ... clean, gwt (1 permutation for chrome), run checks, run unit tests: 11:52.25 total ... wow :) Hope this helps you have a faster change-compile-check cycle :) Best wishes, Greg [1] https://gerrit.ovirt.org/#/c/83961/ [2] https://www.mail-archive.com/devel@ovirt.org/msg10377.html [3] https://idioms.thefreedictionary.com/whole+enchilada -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA <https://www.redhat.com/> gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
On Sun, 12 Nov 2017 at 19:29 Greg Sheremeta <gshereme@redhat.com> wrote:
Hi,
I've added a SKIP_CHECKS flag to the ovirt-engine build here [1]. Thanks to Roy for the idea [2].
TL;DR: you'll save roughly 2 minutes 10 seconds per full compile with SKIP_CHECKS. You'll save more (3 min 10 sec) if you don't need a 'clean'. You'll save the most (about 3 min 20 sec) if you can skip rebuilding GWT.
Here are some statistics (3 runs per command, for averages sake)
My test machine is a Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz 256GB Samsung SSD (old, using SATA) 32GB DDR3 @ 1333 MT/s
---
fresh master checkout
no clean, no gwt, skip checks:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 1:04.38 total 1:06.04 total 1:04.41 total
same as above, but now change one non-GWT class in the utils project
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 1:05.49 total
Ok, pretend we recompiled and our utils class change had an error. Same as above, change the one non-GWT class in the utils project again, and use the resume (-pl) functionality:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 EXTRA_BUILD_FLAGS="-pl org.ovirt.engine.core:utils" 17.958 total (yes, 18 seconds, obviously a world record for any engine build)
"-pl" is very useful mvn directive. It compiles only the *p*roject-*l*ist supplied. hence "pl". Add "-am" (*a*lso *m*ake) when you want to compile what utils depends on (like common) or "amd" (*a*lso *m*ake *d*ependents) if you want to compile also projectd depends on utils (like vdsbroker) [The resume piece is EXTRA_BUILD_FLAGS="-pl org.ovirt.engine.ui:webadmin"
-- I personally need to use this more often!]
---
no clean, no gwt, run checks (but skip unit tests):
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 BUILD_UT=0 3:07.66 total 3:16.69 total 3:14.28 total
clean, no gwt, skip checks:
make clean install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1
4:17.74 total 4:15.76 total 4:13.90 total
no clean, gwt (1 permutation for chrome), skip checks:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=1 SKIP_CHECKS=1 DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari"
4:25.63 total 4:26.71 total 4:30.69 total
no clean, gwt (1 permutation for chrome), run checks but skip unit tests:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=1 BUILD_UT=0 DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari"
6:44.23 total
And the whole enchilada [3] ... clean, gwt (1 permutation for chrome), run checks, run unit tests:
11:52.25 total ... wow :)
Nice one Greg! Now can we have "make WHOLE_ENCHILADA=1" please? :)
Hope this helps you have a faster change-compile-check cycle :)
Best wishes, Greg
[1] https://gerrit.ovirt.org/#/c/83961/ [2] https://www.mail-archive.com/devel@ovirt.org/msg10377.html [3] https://idioms.thefreedictionary.com/whole+enchilada
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
gshereme@redhat.com IRC: gshereme <https://red.ht/sig> _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
for the below scenario of making a change limited to one module, where you want a really fast install but do not want to forgo unit tests or checks, you can run a direct install of just that module like so: cd $DEV_ENGINE/backend/manager/modules/utils; mvn clean install && cp target/utils-$ENGINE_VERSION-SNAPSHOT.jar $DEP_ENGINE/share/ovirt-engine/modules/common/org/ovirt/engine/core/utils/main/utils.jar; cd - #install utils-manager where: - $DEV_ENGINE is your source code home dir - $DEP_ENGINE is your prefix dir - $ENGINE_VERSION is the current snapshot version. e.g. 4.2.0 - using a hash at the end, allows me to quickly call this command with reverse search on bash BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 2.699s :) On Sun, Nov 12, 2017 at 7:28 PM, Greg Sheremeta <gshereme@redhat.com> wrote:
Hi,
I've added a SKIP_CHECKS flag to the ovirt-engine build here [1]. Thanks to Roy for the idea [2].
TL;DR: you'll save roughly 2 minutes 10 seconds per full compile with SKIP_CHECKS. You'll save more (3 min 10 sec) if you don't need a 'clean'. You'll save the most (about 3 min 20 sec) if you can skip rebuilding GWT.
Here are some statistics (3 runs per command, for averages sake)
My test machine is a Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz 256GB Samsung SSD (old, using SATA) 32GB DDR3 @ 1333 MT/s
---
fresh master checkout
no clean, no gwt, skip checks:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 1:04.38 total 1:06.04 total 1:04.41 total
same as above, but now change one non-GWT class in the utils project
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 1:05.49 total
Ok, pretend we recompiled and our utils class change had an error. Same as above, change the one non-GWT class in the utils project again, and use the resume (-pl) functionality:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1 EXTRA_BUILD_FLAGS="-pl org.ovirt.engine.core:utils" 17.958 total (yes, 18 seconds, obviously a world record for any engine build)
[The resume piece is EXTRA_BUILD_FLAGS="-pl org.ovirt.engine.ui:webadmin" -- I personally need to use this more often!]
---
no clean, no gwt, run checks (but skip unit tests):
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 BUILD_UT=0 3:07.66 total 3:16.69 total 3:14.28 total
clean, no gwt, skip checks:
make clean install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=0 SKIP_CHECKS=1
4:17.74 total 4:15.76 total 4:13.90 total
no clean, gwt (1 permutation for chrome), skip checks:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=1 SKIP_CHECKS=1 DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari"
4:25.63 total 4:26.71 total 4:30.69 total
no clean, gwt (1 permutation for chrome), run checks but skip unit tests:
make install-dev PREFIX=/home/greg/ovirt-engine BUILD_GWT=1 BUILD_UT=0 DEV_EXTRA_BUILD_FLAGS_GWT_DEFAULTS="-Dgwt.userAgent=safari"
6:44.23 total
And the whole enchilada [3] ... clean, gwt (1 permutation for chrome), run checks, run unit tests:
11:52.25 total ... wow :)
Hope this helps you have a faster change-compile-check cycle :)
Best wishes, Greg
[1] https://gerrit.ovirt.org/#/c/83961/ [2] https://www.mail-archive.com/devel@ovirt.org/msg10377.html [3] https://idioms.thefreedictionary.com/whole+enchilada
--
GREG SHEREMETA
SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX
Red Hat NA
gshereme@redhat.com IRC: gshereme <https://red.ht/sig>
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- IRC: erav
participants (3)
-
Eitan Raviv -
Greg Sheremeta -
Roy Golan