June 2018 - Infra - oVirt List Archives

[JIRA] (OVIRT-682) Improve CI logging and reporting to Gerrit
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-682?page=com.atlassian.jira.p… ] Barak Korren updated OVIRT-682: ------------------------------- Resolution: Won't Fix Status: Done (was: To Do) This ticket is overly broad and generic, closing it as we now have more specific tickets for specific aspects of this. > Improve CI logging and reporting to Gerrit > ------------------------------------------ > > Key: OVIRT-682 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-682 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: eyal edri > Assignee: infra > Priority: Low > Labels: standard-ci > > On Fri, Aug 12, 2016 at 8:08 PM, Nir Soffer <nsoffer(a)redhat.com> wrote: > > Hi all, > > > > Lately vdsm builds are failing in the install packages stage with > > this unhelpful error: > > > > 13:59:42 INFO: installing package(s): autoconf automake gdb ... > > 13:59:53 ERROR: Command failed. See logs for output. > > > > I downloaded the logs.tgz file from > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24- > > x86_64/1154/artifact/exported-artifacts/logs.tgz > > > > And indeed in /./vdsm/logs/mocker-fedora-24-x86_64.fc24.install_packages/ > > root.log > > I found found this error: > > > > DEBUG util.py:421: Error: Package: > > python-ioprocess-0.17.0-1.201608111609.gitbd272f2.fc24.noarch > > (ovirt-snapshot) > > DEBUG util.py:421: Requires: ioprocess = > > 0.17.0-1.201608111609.gitbd272f2.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.15.1-1.fc24.x86_64 (fedora) > > DEBUG util.py:421: ioprocess = 0.15.1-1.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.16.1-1.fc24.x86_64 (updates) > > DEBUG util.py:421: ioprocess = 0.16.1-1.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.17.0-1.201607121058.gitbd272f2.fc24.x86_64 > > (ovirt-snapshot) > > DEBUG util.py:421: ioprocess = > > 0.17.0-1.201607121058.gitbd272f2.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.17.0-1.201608111129.gitbd272f2.fc24.x86_64 > > (ovirt-snapshot) > > DEBUG util.py:421: ioprocess = > > 0.17.0-1.201608111129.gitbd272f2.fc24 > > > > So we have 2 issues here: > > > > 1. We need *all* errors in the console > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24- > > x86_64/1154/console > Not sure its possible or easy to do, these errors usually come from mock > and are not relevant to the code running on the console. > I've cc'd infra-support and updated topic to track it and check our options. > Going foward, we might want to install a log collector like logstash and > then searching errors there should be much easier. > > > > > > 2. Someone need to fix the ovirt-snapshot repository - it should have > > the missing ioprocess > > package. > > Maybe the project xml is not correct? > > > > Nir > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-682) Improve CI logging and reporting to Gerrit
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-682?page=com.atlassian.jira.p… ] Barak Korren updated OVIRT-682: ------------------------------- Summary: Improve CI logging and reporting to Gerrit (was: Improve CI logging and reporting to Gerrit (v2)) > Improve CI logging and reporting to Gerrit > ------------------------------------------ > > Key: OVIRT-682 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-682 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: eyal edri > Assignee: infra > Priority: Low > Labels: standard-ci > > On Fri, Aug 12, 2016 at 8:08 PM, Nir Soffer <nsoffer(a)redhat.com> wrote: > > Hi all, > > > > Lately vdsm builds are failing in the install packages stage with > > this unhelpful error: > > > > 13:59:42 INFO: installing package(s): autoconf automake gdb ... > > 13:59:53 ERROR: Command failed. See logs for output. > > > > I downloaded the logs.tgz file from > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24- > > x86_64/1154/artifact/exported-artifacts/logs.tgz > > > > And indeed in /./vdsm/logs/mocker-fedora-24-x86_64.fc24.install_packages/ > > root.log > > I found found this error: > > > > DEBUG util.py:421: Error: Package: > > python-ioprocess-0.17.0-1.201608111609.gitbd272f2.fc24.noarch > > (ovirt-snapshot) > > DEBUG util.py:421: Requires: ioprocess = > > 0.17.0-1.201608111609.gitbd272f2.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.15.1-1.fc24.x86_64 (fedora) > > DEBUG util.py:421: ioprocess = 0.15.1-1.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.16.1-1.fc24.x86_64 (updates) > > DEBUG util.py:421: ioprocess = 0.16.1-1.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.17.0-1.201607121058.gitbd272f2.fc24.x86_64 > > (ovirt-snapshot) > > DEBUG util.py:421: ioprocess = > > 0.17.0-1.201607121058.gitbd272f2.fc24 > > DEBUG util.py:421: Available: > > ioprocess-0.17.0-1.201608111129.gitbd272f2.fc24.x86_64 > > (ovirt-snapshot) > > DEBUG util.py:421: ioprocess = > > 0.17.0-1.201608111129.gitbd272f2.fc24 > > > > So we have 2 issues here: > > > > 1. We need *all* errors in the console > > http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc24- > > x86_64/1154/console > Not sure its possible or easy to do, these errors usually come from mock > and are not relevant to the code running on the console. > I've cc'd infra-support and updated topic to track it and check our options. > Going foward, we might want to install a log collector like logstash and > then searching errors there should be much easier. > > > > > > 2. Someone need to fix the ovirt-snapshot repository - it should have > > the missing ioprocess > > package. > > Maybe the project xml is not correct? > > > > Nir > > > -- > Eyal Edri > Associate Manager > RHV DevOps > EMEA ENG Virtualization R&D > Red Hat Israel > phone: +972-9-7692018 > irc: eedri (on #tlv #rhev-dev #rhev-integ) -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1197) Re-factor the standard-CI publishers file
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1197?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1197: -------------------------------- Resolution: Won't Fix Status: Done (was: To Do) Since STDCI had been re-implemented with pipeline jobs it no longer makes sense to re-factor the code for FreeStyle jobs. > Re-factor the standard-CI publishers file > ----------------------------------------- > > Key: OVIRT-1197 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1197 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Labels: standard-ci > > Thi has already been improved as part of OVIRT-988 (in [Gerrit 69194|https://gerrit.ovirt.org/#/c/69194] But there is more work to do. > We can probably have the same publishers macro for all Standard-CI jobs, and add conditional steps to: > # Collect test results if any (OVIRT-449) > # Improve output to Gerrit (OVIRT-682) > # Launch deploy-to-experimental if this is a build-artifacts job and the patch was merged > This way we can finally stop having to have a separate entry for the build-artifacts jobs and simplify the project YAML files. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-2183) Automate creation of "poll" jobs
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2183?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2183: -------------------------------- Epic Link: OVIRT-1013 > Automate creation of "poll" jobs > -------------------------------- > > Key: OVIRT-2183 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2183 > Project: oVirt - virtualization made easy > Issue Type: New Feature > Components: Standard CI (Pipelines) > Reporter: Barak Korren > Assignee: infra > > As a developer I would like to be able to configure upstream source polling jobs (In other works jobs that execute the "poll-upstream-sources" STDCI stage) from the STDCI YAML file in my project so that I never have to deal with JJB or the 'jenkins' repository. > h3. Acceptance Criteria > # To get "poll-upstream-sources" jobs working for a certain branch developers should only configure some setting in the STDCI YAML file of that project. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-2183) Automate creation of "poll" jobs
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

Barak Korren created OVIRT-2183: ----------------------------------- Summary: Automate creation of "poll" jobs Key: OVIRT-2183 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2183 Project: oVirt - virtualization made easy Issue Type: New Feature Components: Standard CI (Pipelines) Reporter: Barak Korren Assignee: infra As a developer I would like to be able to configure upstream source polling jobs (In other works jobs that execute the "poll-upstream-sources" STDCI stage) from the STDCI YAML file in my project so that I never have to deal with JJB or the 'jenkins' repository. h3. Acceptance Criteria # To get "poll-upstream-sources" jobs working for a certain branch developers should only configure some setting in the STDCI YAML file of that project. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1013) Automate standard-CI job creation
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1013?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1013: -------------------------------- Description: The premise of the CI standard is simple: The developers place simple script in the 'automation' directory, and The infa team take care of making Jenkins run it when it should. But we haven't been able to fully deliver on this premise yet. Getting a project to work with the CI standard also takes writing some YAML in the 'Jenkins' repo. Even worse, this YAML needs to be maintained over time as new project branches get created, new platforms get targeted, etc. The core reason behind having to write YAML, is that there are two technical details that we need to know in order to run the CI jobs, but are not specified in a way that allows detecting them automatically. Those details are: 1. The platforms a certain project needs to be built and tested on. 2. The branches of the project that CI needs to look at, and how do they map to an oVirt releases. We need to specify a way to specify the details above in a way that will allow the CI system to automatically detect them. h3. Acceptance criteria: # The only JJB YAML needed to be written on a per-project basis for full support of all STDCI features is few details to basically enable STDCI support for the project # For most projects the code to be added should be one line with just the project name was: The premise of the CI standard is simple: The developers place simple script in the 'automation' directory, and The infa team take care of making Jenkins run it when it should. But we haven't been able to fully deliver on this premise yet. Getting a project to work with the CI standard also takes writing some YAML in the 'Jenkins' repo. Even worse, this YAML needs to be maintained over time as new project branches get created, new platforms get targeted, etc. The core reason behind having to write YAML, is that there are two technical details that we need to know in order to run the CI jobs, but are not specified in a way that allows detecting them automatically. Those details are: 1. The platforms a certain project needs to be built and tested on. 2. The branches of the project that CI needs to look at, and how do they map to an oVirt releases. We need to specify a way to specify the details above in a way that will allow the CI system to automatically detect them. > Automate standard-CI job creation > --------------------------------- > > Key: OVIRT-1013 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1013 > Project: oVirt - virtualization made easy > Issue Type: Epic > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Labels: standard-ci > > The premise of the CI standard is simple: The developers place simple script in the 'automation' directory, and The infa team take care of making Jenkins run it when it should. > But we haven't been able to fully deliver on this premise yet. Getting a project to work with the CI standard also takes writing some YAML in the 'Jenkins' repo. Even worse, this YAML needs to be maintained over time as new project branches get created, new platforms get targeted, etc. > The core reason behind having to write YAML, is that there are two technical details that we need to know in order to run the CI jobs, but are not specified in a way that allows detecting them automatically. Those details are: > 1. The platforms a certain project needs to be built and tested on. > 2. The branches of the project that CI needs to look at, and how do they map to an oVirt releases. > We need to specify a way to specify the details above in a way that will allow the CI system to automatically detect them. > h3. Acceptance criteria: > # The only JJB YAML needed to be written on a per-project basis for full support of all STDCI features is few details to basically enable STDCI support for the project > # For most projects the code to be added should be one line with just the project name -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1013) Automate standard-CI job creation
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1013?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1013: -------------------------------- Epic Name: Automate standard-CI job creation > Automate standard-CI job creation > --------------------------------- > > Key: OVIRT-1013 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1013 > Project: oVirt - virtualization made easy > Issue Type: Epic > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Labels: standard-ci > > The premise of the CI standard is simple: The developers place simple script in the 'automation' directory, and The infa team take care of making Jenkins run it when it should. > But we haven't been able to fully deliver on this premise yet. Getting a project to work with the CI standard also takes writing some YAML in the 'Jenkins' repo. Even worse, this YAML needs to be maintained over time as new project branches get created, new platforms get targeted, etc. > The core reason behind having to write YAML, is that there are two technical details that we need to know in order to run the CI jobs, but are not specified in a way that allows detecting them automatically. Those details are: > 1. The platforms a certain project needs to be built and tested on. > 2. The branches of the project that CI needs to look at, and how do they map to an oVirt releases. > We need to specify a way to specify the details above in a way that will allow the CI system to automatically detect them. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1013) Automate standard-CI job creation
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1013?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1013: -------------------------------- Issue Type: Epic (was: Improvement) > Automate standard-CI job creation > --------------------------------- > > Key: OVIRT-1013 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1013 > Project: oVirt - virtualization made easy > Issue Type: Epic > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Labels: standard-ci > > The premise of the CI standard is simple: The developers place simple script in the 'automation' directory, and The infa team take care of making Jenkins run it when it should. > But we haven't been able to fully deliver on this premise yet. Getting a project to work with the CI standard also takes writing some YAML in the 'Jenkins' repo. Even worse, this YAML needs to be maintained over time as new project branches get created, new platforms get targeted, etc. > The core reason behind having to write YAML, is that there are two technical details that we need to know in order to run the CI jobs, but are not specified in a way that allows detecting them automatically. Those details are: > 1. The platforms a certain project needs to be built and tested on. > 2. The branches of the project that CI needs to look at, and how do they map to an oVirt releases. > We need to specify a way to specify the details above in a way that will allow the CI system to automatically detect them. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1013) Automate standard-CI job creation
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1013?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1013: -------------------------------- Epic Link: (was: OVIRT-400) > Automate standard-CI job creation > --------------------------------- > > Key: OVIRT-1013 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1013 > Project: oVirt - virtualization made easy > Issue Type: Epic > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Labels: standard-ci > > The premise of the CI standard is simple: The developers place simple script in the 'automation' directory, and The infa team take care of making Jenkins run it when it should. > But we haven't been able to fully deliver on this premise yet. Getting a project to work with the CI standard also takes writing some YAML in the 'Jenkins' repo. Even worse, this YAML needs to be maintained over time as new project branches get created, new platforms get targeted, etc. > The core reason behind having to write YAML, is that there are two technical details that we need to know in order to run the CI jobs, but are not specified in a way that allows detecting them automatically. Those details are: > 1. The platforms a certain project needs to be built and tested on. > 2. The branches of the project that CI needs to look at, and how do they map to an oVirt releases. > We need to specify a way to specify the details above in a way that will allow the CI system to automatically detect them. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1013) Automate standard-CI job creation
by Barak Korren (oVirt JIRA) 10 Jun '18

10 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1013?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1013: -------------------------------- Summary: Automate standard-CI job creation (was: Automate standard-CI job creation (V2)) > Automate standard-CI job creation > --------------------------------- > > Key: OVIRT-1013 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1013 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: oVirt CI > Reporter: Barak Korren > Assignee: infra > Labels: standard-ci > > The premise of the CI standard is simple: The developers place simple script in the 'automation' directory, and The infa team take care of making Jenkins run it when it should. > But we haven't been able to fully deliver on this premise yet. Getting a project to work with the CI standard also takes writing some YAML in the 'Jenkins' repo. Even worse, this YAML needs to be maintained over time as new project branches get created, new platforms get targeted, etc. > The core reason behind having to write YAML, is that there are two technical details that we need to know in order to run the CI jobs, but are not specified in a way that allows detecting them automatically. Those details are: > 1. The platforms a certain project needs to be built and tested on. > 2. The branches of the project that CI needs to look at, and how do they map to an oVirt releases. > We need to specify a way to specify the details above in a way that will allow the CI system to automatically detect them. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0