- Infra - oVirt List Archives

[JIRA] (OVIRT-1867) Allow embedded secrets inside the source repo for CI
by Daniel Belenky (oVirt JIRA) 30 Jan '18

30 Jan '18

This is a multi-part message in MIME format... ------------=_1517312708-12381-241 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1867?page=com.atlassian.jira.… ] Daniel Belenky updated OVIRT-1867: ---------------------------------- Component/s: STDCI DSL Standard CI (Pipelines) > Allow embedded secrets inside the source repo for CI > ---------------------------------------------------- > > Key: OVIRT-1867 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1867 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Components: Standard CI (Pipelines), STDCI DSL > Reporter: Roman Mohr > Assignee: infra > > In order to improve the self-service capabilities of standard-ci it is > important for projects, that they can add their own secrets to projects (to > reach external services, e.g. docker hub, ...). > Travis has a very nice system which helps engineers there: > https://docs.travis-ci.com/user/encryption-keys/ > Basically the CI system needs to generate a public/private key pair for > every enabled git repo. The engineer simply fetches the public key via a > well know URL and encrypts the secrets. Then the encrypted secret can be > made part of the source repo. Before the tests are run the CI system > decrypts the secrets. Than can play together pretty well with Jenkinsfiles > too. > Benefit: > * Less manual intervention from CI team to add secrets to jobs > * Strengthen the config-in-code thinking -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077) ------------=_1517312708-12381-241 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1867?page=com.atlassian.jira.… ]</pre> <h3>Daniel Belenky updated OVIRT-1867:</h3> <pre> Component/s: STDCI DSL Standard CI (Pipelines)</pre> <blockquote><h3>Allow embedded secrets inside the source repo for CI</h3> <pre> Key: OVIRT-1867 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1867 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Components: Standard CI (Pipelines), STDCI DSL Reporter: Roman Mohr Assignee: infra</pre> <p>In order to improve the self-service capabilities of standard-ci it is important for projects, that they can add their own secrets to projects (to reach external services, e.g. docker hub, …). Travis has a very nice system which helps engineers there: <a href="https://docs.travis-ci.com/user/encryption-keys/">https://docs.travis-ci.com/user/encryption-keys/</a> Basically the CI system needs to generate a public/private key pair for every enabled git repo. The engineer simply fetches the public key via a well know URL and encrypts the secrets. Then the encrypted secret can be made part of the source repo. Before the tests are run the CI system decrypts the secrets. Than can play together pretty well with Jenkinsfiles too. Benefit:</p> <pre>* Less manual intervention from CI team to add secrets to jobs * Strengthen the config-in-code thinking</pre></blockquote> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1517312708-12381-241--

1 0

[JIRA] (OVIRT-1867) Allow embedded secrets inside the source repo for CI
by Daniel Belenky (oVirt JIRA) 30 Jan '18

30 Jan '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1867?page=com.atlassian.jira.… ] Daniel Belenky commented on OVIRT-1867: --------------------------------------- We actually have a mechanism for secrets as environmental variables in stdci. Currently, we hold the secrets in a single secret file on our Jenkins master, and engineers can request those secrets as environmental variables via *.environment.yaml. The syntax is the same as for Kubernetes: {code:java} --- - name: MY_VAR valueFrom: secretKeyRef: name: MySecret1 value: password {code} The example above will bind the password field from MySecret1 to $MY_VAR inside the chroot created by STDCI. The one part which is still missing though is a common interface to allow projects to add the secrets to this file automatically. > Allow embedded secrets inside the source repo for CI > ---------------------------------------------------- > > Key: OVIRT-1867 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1867 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Roman Mohr > Assignee: infra > > In order to improve the self-service capabilities of standard-ci it is > important for projects, that they can add their own secrets to projects (to > reach external services, e.g. docker hub, ...). > Travis has a very nice system which helps engineers there: > https://docs.travis-ci.com/user/encryption-keys/ > Basically the CI system needs to generate a public/private key pair for > every enabled git repo. The engineer simply fetches the public key via a > well know URL and encrypts the secrets. Then the encrypted secret can be > made part of the source repo. Before the tests are run the CI system > decrypts the secrets. Than can play together pretty well with Jenkinsfiles > too. > Benefit: > * Less manual intervention from CI team to add secrets to jobs > * Strengthen the config-in-code thinking -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077)

1 0

[JIRA] (OVIRT-1868) Allow engineers to write Jenkinsfiles
by Roman Mohr (oVirt JIRA) 30 Jan '18

30 Jan '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1868?page=com.atlassian.jira.… ] Roman Mohr updated OVIRT-1868: ------------------------------ Description: Looks like standard-ci switched internally to use Jenkinsfiles. However it would be very valuable for engineers, if they could just write their Jenkinsfile, instead of all the usual standard-ci yamls/scripts. With the Jenkinsfile the chroot based approach seems to be pretty obsolete, if you allow people to use the docker agent for the Jenkinsfile. For KubeVirt it would make standard-ci finally really valuable. was: Looks like standard-ci switched internally to use Jenkinsfiles. However it would be very valuable for engineers, if they could just write their Jenkinsfile, instead of all the usual standard-ci yamls/scripts. With the Jenkinsfile the chroot based approach seems to be pretty obsolate, if you allow people to use the docker agent for the Jenkinsfile. For KubeVirt it would make standard-ci finally really valuable. > Allow engineers to write Jenkinsfiles > ------------------------------------- > > Key: OVIRT-1868 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1868 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Roman Mohr > Assignee: infra > > Looks like standard-ci switched internally to use Jenkinsfiles. However it > would be very valuable for engineers, if they could just write their > Jenkinsfile, instead of all the usual standard-ci yamls/scripts. > With the Jenkinsfile the chroot based approach seems to be pretty obsolete, > if you allow people to use the docker agent for the Jenkinsfile. For > KubeVirt it would make standard-ci finally really valuable. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077)

1 0

[JIRA] (OVIRT-1868) Allow engineers to write Jenkinsfiles
by Roman Mohr (oVirt JIRA) 30 Jan '18

30 Jan '18

Roman Mohr created OVIRT-1868: --------------------------------- Summary: Allow engineers to write Jenkinsfiles Key: OVIRT-1868 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1868 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Roman Mohr Assignee: infra Looks like standard-ci switched internally to use Jenkinsfiles. However it would be very valuable for engineers, if they could just write their Jenkinsfile, instead of all the usual standard-ci yamls/scripts. With the Jenkinsfile the chroot based approach seems to be pretty obsolate, if you allow people to use the docker agent for the Jenkinsfile. For KubeVirt it would make standard-ci finally really valuable. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077)

1 0

[JIRA] (OVIRT-1867) Allow embedded secrets inside the source repo for CI
by Roman Mohr (oVirt JIRA) 30 Jan '18

30 Jan '18

This is a multi-part message in MIME format... ------------=_1517308592-17438-377 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Roman Mohr created OVIRT-1867: --------------------------------- Summary: Allow embedded secrets inside the source repo for CI Key: OVIRT-1867 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1867 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Roman Mohr Assignee: infra In order to improve the self-service capabilities of standard-ci it is important for projects, that they can add their own secrets to projects (to reach external services, e.g. docker hub, ...). Travis has a very nice system which helps engineers there: https://docs.travis-ci.com/user/encryption-keys/ Basically the CI system needs to generate a public/private key pair for every enabled git repo. The engineer simply fetches the public key via a well know URL and encrypts the secrets. Then the encrypted secret can be made part of the source repo. Before the tests are run the CI system decrypts the secrets. Than can play together pretty well with Jenkinsfiles too. Benefit: * Less manual intervention from CI team to add secrets to jobs * Strengthen the config-in-code thinking -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077) ------------=_1517308592-17438-377 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <h3>Roman Mohr created OVIRT-1867:</h3> <pre> Summary: Allow embedded secrets inside the source repo for CI Key: OVIRT-1867 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1867 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Roman Mohr Assignee: infra</pre> <p>In order to improve the self-service capabilities of standard-ci it is important for projects, that they can add their own secrets to projects (to reach external services, e.g. docker hub, …).</p> <p>Travis has a very nice system which helps engineers there: <a href="https://docs.travis-ci.com/user/encryption-keys/">https://docs.travis-ci.com/user/encryption-keys/</a></p> <p>Basically the CI system needs to generate a public/private key pair for every enabled git repo. The engineer simply fetches the public key via a well know URL and encrypts the secrets. Then the encrypted secret can be made part of the source repo. Before the tests are run the CI system decrypts the secrets. Than can play together pretty well with Jenkinsfiles too.</p> <p>Benefit:</p> <pre>* Less manual intervention from CI team to add secrets to jobs * Strengthen the config-in-code thinking</pre> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1517308592-17438-377--

1 0

[CQ]: 86910, 1 (ovirt-hosted-engine-setup) failed "ovirt-4.2" system tests
by oVirt Jenkins 30 Jan '18

30 Jan '18

Change 86910,1 (ovirt-hosted-engine-setup) is probably the reason behind recent system test failures in the "ovirt-4.2" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/86910/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/370/

1 0

[CQ]: e74f3bb (stage-gh-stdci2) failed "dummy-master" system tests, but isn't the failure root cause
by oVirt Jenkins 30 Jan '18

30 Jan '18

A system test invoked by the "dummy-master" change queue including change e74f3bb (stage-gh-stdci2) failed. However, this change seems not to be the root cause for this failure. Change 2ff79b3 (stage-gh-stdci2) that this change depends on or is based on, was detected as the cause of the testing failures. This change had been removed from the testing queue. Artifacts built from this change will not be released until either change 2ff79b3 (stage-gh-stdci2) is fixed and this change is updated to refer to or rebased on the fixed version, or this change is modified to no longer depend on it. For further details about the change see: https://github.com/oVirt/stage-gh-stdci2/commit/e74f3bbc48cc8b76c599ec9c56f… For further details about the change that seems to be the root cause behind the testing failures see: https://github.com/oVirt/stage-gh-stdci2/commit/2ff79b3f37717e65ab84a11c167… For failed test results see: http://jenkins-staging.phx.ovirt.org/job/dummy-master_change-queue-tester/1…

1 0

[JIRA] (OVIRT-1866) Use PERFORMANCE_OPTIMIZED pipelines instead of MAX_SURVIVABILITY
by Daniel Belenky (oVirt JIRA) 30 Jan '18

30 Jan '18

This is a multi-part message in MIME format... ------------=_1517299177-24378-150 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1866?page=com.atlassian.jira.… ] Daniel Belenky updated OVIRT-1866: ---------------------------------- Labels: jenkins (was: ) > Use PERFORMANCE_OPTIMIZED pipelines instead of MAX_SURVIVABILITY > ---------------------------------------------------------------- > > Key: OVIRT-1866 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1866 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Jenkins Master > Reporter: Daniel Belenky > Assignee: infra > Labels: jenkins > > It seems that pipelines now have support to configure their behavior of serialization of objects, and since we know that this serialization never worked for us as we'd expect it to, maybe we should consider using PERFORMANCE_OPTIMIZED mode which will avoid serializing and maybe boost the performance. > [Link to subject in jenkincci-dev google|https://groups.google.com/forum/#!msg/jenkinsci-dev/F4zcOWLI32w/51qi…] -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077) ------------=_1517299177-24378-150 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1866?page=com.atlassian.jira.… ]</pre> <h3>Daniel Belenky updated OVIRT-1866:</h3> <pre>Labels: jenkins (was: )</pre> <blockquote><h3>Use PERFORMANCE_OPTIMIZED pipelines instead of MAX_SURVIVABILITY</h3> <pre> Key: OVIRT-1866 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1866 Project: oVirt - virtualization made easy Issue Type: Improvement Components: Jenkins Master Reporter: Daniel Belenky Assignee: infra Labels: jenkins</pre> <p>It seems that pipelines now have support to configure their behavior of serialization of objects, and since we know that this serialization never worked for us as we'd expect it to, maybe we should consider using PERFORMANCE_OPTIMIZED mode which will avoid serializing and maybe boost the performance. [Link to subject in jenkincci-dev google|<a href="https://groups.google.com/forum/#!msg/jenkinsci-dev/F4zcOWLI32w/51qixjepCgAJ">https://groups.google.com/forum/#!msg/jenkinsci-dev/F4zcOWLI32w/51qixjepCgAJ</a>]</p></blockquote> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1517299177-24378-150--

1 0

[JIRA] (OVIRT-1866) Use PERFORMANCE_OPTIMIZED pipelines instead of MAX_SURVIVABILITY
by Barak Korren (oVirt JIRA) 30 Jan '18

30 Jan '18

This is a multi-part message in MIME format... ------------=_1517298659-23592-267 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1866?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-1866: -------------------------------- Component/s: Jenkins Master > Use PERFORMANCE_OPTIMIZED pipelines instead of MAX_SURVIVABILITY > ---------------------------------------------------------------- > > Key: OVIRT-1866 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1866 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: Jenkins Master > Reporter: Daniel Belenky > Assignee: infra > > It seems that pipelines now have support to configure their behavior of serialization of objects, and since we know that this serialization never worked for us as we'd expect it to, maybe we should consider using PERFORMANCE_OPTIMIZED mode which will avoid serializing and maybe boost the performance. > [Link to subject in jenkincci-dev google|https://groups.google.com/forum/#!msg/jenkinsci-dev/F4zcOWLI32w/51qi…] -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077) ------------=_1517298659-23592-267 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1866?page=com.atlassian.jira.… ]</pre> <h3>Barak Korren updated OVIRT-1866:</h3> <pre>Component/s: Jenkins Master</pre> <blockquote><h3>Use PERFORMANCE_OPTIMIZED pipelines instead of MAX_SURVIVABILITY</h3> <pre> Key: OVIRT-1866 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1866 Project: oVirt - virtualization made easy Issue Type: Improvement Components: Jenkins Master Reporter: Daniel Belenky Assignee: infra</pre> <p>It seems that pipelines now have support to configure their behavior of serialization of objects, and since we know that this serialization never worked for us as we'd expect it to, maybe we should consider using PERFORMANCE_OPTIMIZED mode which will avoid serializing and maybe boost the performance. [Link to subject in jenkincci-dev google|<a href="https://groups.google.com/forum/#!msg/jenkinsci-dev/F4zcOWLI32w/51qixjepCgAJ">https://groups.google.com/forum/#!msg/jenkinsci-dev/F4zcOWLI32w/51qixjepCgAJ</a>]</p></blockquote> <p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BJ33BSM3…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1517298659-23592-267--

1 0

[JIRA] (OVIRT-1866) Use PERFORMANCE_OPTIMIZED pipelines instead of MAX_SURVIVABILITY
by Daniel Belenky (oVirt JIRA) 30 Jan '18

30 Jan '18

Daniel Belenky created OVIRT-1866: ------------------------------------- Summary: Use PERFORMANCE_OPTIMIZED pipelines instead of MAX_SURVIVABILITY Key: OVIRT-1866 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1866 Project: oVirt - virtualization made easy Issue Type: Improvement Reporter: Daniel Belenky Assignee: infra It seems that pipelines now have support to configure their behavior of serialization of objects, and since we know that this serialization never worked for us as we'd expect it to, maybe we should consider using PERFORMANCE_OPTIMIZED mode which will avoid serializing and maybe boost the performance. [Link to subject in jenkincci-dev google|https://groups.google.com/forum/#!msg/jenkinsci-dev/F4zcOWLI32w/51qi…] -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100077)

1 0