[JIRA] (OVIRT-1745) Make upstream source collection and polling code usable for pipelines
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1515572840-15436-331
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1745?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1745:
--------------------------------
Component/s: (was: oVirt CI)
usrc.py
pusher.py
> Make upstream source collection and polling code usable for pipelines
> ---------------------------------------------------------------------
>
> Key: OVIRT-1745
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1745
> Project: oVirt - virtualization made easy
> Issue Type: New Feature
> Components: pusher.py, usrc.py
> Reporter: Barak Korren
> Assignee: infra
> Labels: poll-upstream-sources, upstream-source-collector
>
> The upstream source collection and polling code is currently only suitable for use in Free-Style jobs. This is because:
> # We support having 'jenkins' repositories (Where jobs expect STDCI code to be found) use the upstream sources mechanism to get the actual code from the main 'jenkins' repo in gerrit.ovirt.org
> # This creates a situation where in order to get the STDCI code we need to run the upstream sources code
> # This creates a chicken and and egg problem because the upstream sources code is part of the STDCI code
> # To solve this issue we simply embedded the upstream sources code into the job code by using '{{#include}}' in JJB rather then running it as a stand-alone script.
> # This technique is not usable for pipelines, so we need to come with a different solution and make the required adjustments to the source collection and polling code.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100075)
------------=_1515572840-15436-331
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1745?page=com.atlassian.jir... ]</pre>
<h3>Barak Korren updated OVIRT-1745:</h3>
<pre> Component/s: (was: oVirt CI)
usrc.py
pusher.py</pre>
<blockquote><h3>Make upstream source collection and polling code usable for pipelines</h3>
<pre> Key: OVIRT-1745
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1745
Project: oVirt - virtualization made easy
Issue Type: New Feature
Components: pusher.py, usrc.py
Reporter: Barak Korren
Assignee: infra
Labels: poll-upstream-sources, upstream-source-collector</pre>
<p>The upstream source collection and polling code is currently only suitable for use in Free-Style jobs. This is because: # We support having ‘jenkins’ repositories (Where jobs expect STDCI code to be found) use the upstream sources mechanism to get the actual code from the main ‘jenkins’ repo in gerrit.ovirt.org # This creates a situation where in order to get the STDCI code we need to run the upstream sources code # This creates a chicken and and egg problem because the upstream sources code is part of the STDCI code # To solve this issue we simply embedded the upstream sources code into the job code by using ‘{{#include}}’ in JJB rather then running it as a stand-alone script. # This technique is not usable for pipelines, so we need to come with a different solution and make the required adjustments to the source collection and polling code.</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100075)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BS..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1515572840-15436-331--
6 years, 11 months
[JIRA] (OVIRT-1774) Upstream source collector can fail to push on multi-branch projects
by Barak Korren (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1774?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1774:
--------------------------------
Component/s: (was: oVirt CI)
pusher.py
> Upstream source collector can fail to push on multi-branch projects
> -------------------------------------------------------------------
>
> Key: OVIRT-1774
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1774
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Components: pusher.py
> Reporter: Barak Korren
> Assignee: infra
> Labels: poll-upstream-sources, upstream-source-collector
>
> When the upstream source collection code looks for similar patches in order to avoid pushing a new patch, it lookes for patches that contain the same change to the '{{upstream-sources.yaml}}' file, regardless of the branch they may belong to.
> The code currently ignores the possibility that similar changes might be required for different branches, in which case different patches may need to be pushed.
> The way of detecting similar patches should be changed so that either the branch name is included in the checksum that are used to identify a patch, or the query is limited to patches of the branch being handled.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100075)
6 years, 11 months
[JIRA] (OVIRT-1836) Optimize options normalization methods in stdci_dsl
by Barak Korren (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1836?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1836:
--------------------------------
Component/s: (was: oVirt CI)
STDCI DSL
> Optimize options normalization methods in stdci_dsl
> ---------------------------------------------------
>
> Key: OVIRT-1836
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1836
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: STDCI DSL
> Reporter: Daniel Belenky
> Assignee: infra
>
> Generally, we have 4 kinds of options: script, globals, options that affect host selection and options to configure the runner.
> We need to take care of global options first since they affect everything, then take care of script because it tells which vectors to throw away.
> Since runners and pipelines care about different sets of options, we need to split the options handling functions.
> This will enable us to compose much more efficient pipelines in the API.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100075)
6 years, 11 months
[JIRA] (OVIRT-1817) package included in .packages file, missing in check-patch.sh - caching issue?
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1515571559-19030-310
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1817?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1817:
--------------------------------
Component/s: (was: oVirt CI)
mock_runner
> package included in .packages file, missing in check-patch.sh - caching issue?
> ------------------------------------------------------------------------------
>
> Key: OVIRT-1817
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1817
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: mock_runner
> Reporter: Nir Soffer
> Assignee: infra
> Fix For: OVIRT-INFRA-DEC-2017
>
>
> I added 2 packages to ovirt-imageio packages files:
> https://gerrit.ovirt.org/#/c/85639/
> The next patch using these packages
> https://gerrit.ovirt.org/#/c/85245/
> fail to import the package on some slaves:
> *00:02:34.093* ==================================== ERRORS
> ====================================*00:02:34.093*
> ____________________ ERROR collecting test/qemuimg_test.py
> _____________________*00:02:34.093* ImportError while importing test
> module '/home/jenkins/workspace/ovirt-imageio_master_check-patch-el7-x86_64/ovirt-imageio/common/test/qemuimg_test.py'.*00:02:34.094*
> Hint: make sure your test modules/packages have valid Python
> names.*00:02:34.094* Traceback:*00:02:34.094* test/qemuimg_test.py:16:
> in <module>*00:02:34.094* from ovirt_imageio_common import
> qemuimg*00:02:34.094* ovirt_imageio_common/qemuimg.py:11: in
> <module>*00:02:34.094* from . compat import
> subprocess*00:02:34.095* ovirt_imageio_common/compat.py:12: in
> <module>*00:02:34.095* import subprocess32 as subprocess # NOQA:
> F401 (unused import)*00:02:34.095* E ImportError: No module named
> subprocess32*00:02:34.095* !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors
> during collection !!!!!!!!!!!!!!!!!!!!*00:02:34.096*
> =========================== 1 error in 1.06 seconds
> ============================
> Do we have a way to invalidate mock cache manually?
> Nir
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100075)
------------=_1515571559-19030-310
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1817?page=com.atlassian.jir... ]</pre>
<h3>Barak Korren updated OVIRT-1817:</h3>
<pre> Component/s: (was: oVirt CI)
mock_runner</pre>
<blockquote><h3>package included in .packages file, missing in check-patch.sh – caching issue?</h3>
<pre> Key: OVIRT-1817
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1817
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: mock_runner
Reporter: Nir Soffer
Assignee: infra
Fix For: OVIRT-INFRA-DEC-2017</pre>
<p>I added 2 packages to ovirt-imageio packages files: <a href="https://gerrit.ovirt.org/#/c/85639/">https://gerrit.ovirt.org/#/c/85639/</a> The next patch using these packages <a href="https://gerrit.ovirt.org/#/c/85245/">https://gerrit.ovirt.org/#/c/85245/</a> fail to import the package on some slaves: <strong>00:02:34.093</strong> ==================================== ERRORS ====================================*00:02:34.093* <em>_________________</em>_ ERROR collecting test/qemuimg_test.py _____________________*00:02:34.093* ImportError while importing test module '/home/jenkins/workspace/ovirt-imageio_master_check-patch-el7-x86_64/ovirt-imageio/common/test/qemuimg_test.py'.*00:02:34.094* Hint: make sure your test modules/packages have valid Python names.*00:02:34.094* Traceback:*00:02:34.094* test/qemuimg_test.py:16: in <module>*00:02:34.094* from ovirt_imageio_common import qemuimg*00:02:34.094* ovirt_imageio_common/qemuimg.py:11: in <module>*00:02:34.094* from . compat import subprocess*00:02:34.095* ovirt_imageio_common/compat.py:12: in <module>*00:02:34.095* import subprocess32 as subprocess # NOQA: F401 (unused import)*00:02:34.095* E ImportError: No module named subprocess32*00:02:34.095* !!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!*00:02:34.096* =========================== 1 error in 1.06 seconds ============================ Do we have a way to invalidate mock cache manually? Nir</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100075)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BS..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1515571559-19030-310--
6 years, 11 months
[JIRA] (OVIRT-1818) ioprocess build fail on ppc64le
by Barak Korren (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1818?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1818:
--------------------------------
Component/s: (was: oVirt CI)
mock_runner
> ioprocess build fail on ppc64le
> -------------------------------
>
> Key: OVIRT-1818
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1818
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Components: mock_runner
> Reporter: Nir Soffer
> Assignee: Barak Korren
> Labels: ppc64le, standard-ci
>
> The new builds for ppc64le fail with the errors below.
> 1.
> http://jenkins.ovirt.org/job/ioprocess_master_check-patch-fc26-ppc64le/1/...
> *00:00:42.499* ERROR: Command failed: *00:00:42.500* # /usr/bin/yum
> --installroot /var/lib/mock/fedora-26-ppc64le-881f89337ffa51754418efa2ad261881-16706/root/
> --releasever 26 install @buildsys-build autoconf automake git
> glib2-devel python-pip python-six python-subprocess32 python2-devel
> python3-devel python3-six util-linux yajl-devel
> --setopt=tsflags=nocontexts*00:00:42.500* Failed to set locale,
> defaulting to C*00:00:42.500* No package git available.*00:00:42.501*
> No package glib2-devel available.*00:00:42.501* No package
> python-subprocess32 available.*00:00:42.501* No package python2-devel
> available.*00:00:42.501* No package python3-devel
> available.*00:00:42.501* No package util-linux
> available.*00:00:42.501* No package yajl-devel available.
> 2.
> http://jenkins.ovirt.org/job/ioprocess_master_check-patch-fc27-ppc64le/1/...
> *00:00:18.335* +
> /home/jenkins/workspace/ioprocess_master_check-patch-fc27-ppc64le/jenkins/mock_configs/mock_runner.sh
> --patch-only --mock-confs-dir
> /home/jenkins/workspace/ioprocess_master_check-patch-fc27-ppc64le/jenkins/mock_configs
> --secrets-file /home/jenkins/workspace/ioprocess_master_check-patch-fc27-ppc64le/std_ci_secrets.yaml
> --try-proxy --try-mirrors
> http://mirrors.phx.ovirt.org/repos/yum/all_latest.json
> 'fc27.*ppc64le'*00:00:18.347* Unable to find mock env
> fc27.*ppc64le*00:00:18.351* Collecting mock logs*00:00:18.352* mv:
> missing destination file operand after 'exported-artifacts/mock_logs'
> 3.
> http://jenkins.ovirt.org/job/ioprocess_master_check-patch-fcraw-ppc64le/1...
> *00:00:26.701* +
> /home/jenkins/workspace/ioprocess_master_check-patch-fcraw-ppc64le/jenkins/mock_configs/mock_runner.sh
> --patch-only --mock-confs-dir
> /home/jenkins/workspace/ioprocess_master_check-patch-fcraw-ppc64le/jenkins/mock_configs
> --secrets-file /home/jenkins/workspace/ioprocess_master_check-patch-fcraw-ppc64le/std_ci_secrets.yaml
> --try-proxy --try-mirrors
> http://mirrors.phx.ovirt.org/repos/yum/all_latest.json
> 'fcraw.*ppc64le'*00:00:26.702* Unable to find mock env
> fcraw.*ppc64le*00:00:26.703* Collecting mock logs
> Nir
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100075)
6 years, 11 months