- Infra - oVirt List Archives

[JIRA] (OVIRT-2192) remove separate initialization function from mock_runner.sh
by Barak Korren (oVirt JIRA) 12 Jun '18

12 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2192?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2192: -------------------------------- Epic Link: OVIRT-2189 > remove separate initialization function from mock_runner.sh > ----------------------------------------------------------- > > Key: OVIRT-2192 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2192 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: mock_runner > Reporter: Barak Korren > Assignee: infra > > Mock runner includes a function that runs mock to initialize the mock environment before it actually gets used. > Theoretically the initialization could have been done in the same {{mock}} invocation that makes use of it. It seems the reason for the separate initialization was that it could be done without including bind mount configuration, because there was a need to pre-create the mount points inside the chroot before bind mounts could be done. > Since mock 1.4.10, it can now create the mount points for bind mounts on its own, a separate hase for creating mount points is no longer needed and OVIRT-2191 tracks the work to remove it. > It seems that once the mount point creation phase is removed, the separate initialization could be removed as well. Preliminary testing showed that this may be more tricky then anticipated though, as the way 'chroothome' mock configuration option is configured by mock_runner causes issues for mock when bootstrap chroots are used (The reason this is not causing issue for the separate initialization is that this option only gets configured along with the bind mount options only when mock is invoked to do actual work). > To remove the separate initialization phase and reduce the amount of times mock is called from 3 to 2, we need to work around the issue above by either submitting a patch to mock itself or finding a work-around that could be used from {{mock_runner.sh}}. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-2192) remove separate initialization function from mock_runner.sh
by Barak Korren (oVirt JIRA) 12 Jun '18

12 Jun '18

Barak Korren created OVIRT-2192: ----------------------------------- Summary: remove separate initialization function from mock_runner.sh Key: OVIRT-2192 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2192 Project: oVirt - virtualization made easy Issue Type: Improvement Components: mock_runner Reporter: Barak Korren Assignee: infra Mock runner includes a function that runs mock to initialize the mock environment before it actually gets used. Theoretically the initialization could have been done in the same {{mock}} invocation that makes use of it. It seems the reason for the separate initialization was that it could be done without including bind mount configuration, because there was a need to pre-create the mount points inside the chroot before bind mounts could be done. Since mock 1.4.10, it can now create the mount points for bind mounts on its own, a separate hase for creating mount points is no longer needed and OVIRT-2191 tracks the work to remove it. It seems that once the mount point creation phase is removed, the separate initialization could be removed as well. Preliminary testing showed that this may be more tricky then anticipated though, as the way 'chroothome' mock configuration option is configured by mock_runner causes issues for mock when bootstrap chroots are used (The reason this is not causing issue for the separate initialization is that this option only gets configured along with the bind mount options only when mock is invoked to do actual work). To remove the separate initialization phase and reduce the amount of times mock is called from 3 to 2, we need to work around the issue above by either submitting a patch to mock itself or finding a work-around that could be used from {{mock_runner.sh}}. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-2189) Imrpove mock_runner.sh speed, containment and maintainability
by Barak Korren (oVirt JIRA) 12 Jun '18

12 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2189?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2189: -------------------------------- Summary: Imrpove mock_runner.sh speed, containment and maintainability (was: Imrpove mock_runner.sh speed and containment) > Imrpove mock_runner.sh speed, containment and maintainability > ------------------------------------------------------------- > > Key: OVIRT-2189 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2189 > Project: oVirt - virtualization made easy > Issue Type: Epic > Components: mock_runner > Reporter: Barak Korren > Assignee: infra > > {{mock_runner.sh}} is the core of all STDCI, yet it has multiple issues and shortcomings that make it less the ideal. Many of the ways {{mock_runner.sh}} does things have to do with the way {{mock}} worked a few years ago when the script was first written. However, {{mock}} had been improved over time, and the new features it includes allow for making improvements to the way {{mock_runner.sh}} works without a very big time investment. > h3. Acceptance Criteria > # {{mock_runner.sh}} uses systemsd-nspawn for containment rather then simple chroot > # {{mock_runner.sh}} can start a chroot from cache in less then 10 seconds. > # {{mock_runner.sh}} does not reinstall the whole chroot if just a few packages in the *.packages file were changed. > # {{mock_runner.sh}} invokes {{mock}} only once when it runs > # {{mock_runner.sh}} can use the mock config files that are shipped with {{mock}} directly without the CI team having to maintain their own set of files. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-2189) Imrpove mock_runner.sh speed and containment
by Barak Korren (oVirt JIRA) 12 Jun '18

12 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2189?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2189: -------------------------------- Description: {{mock_runner.sh}} is the core of all STDCI, yet it has multiple issues and shortcomings that make it less the ideal. Many of the ways {{mock_runner.sh}} does things have to do with the way {{mock}} worked a few years ago when the script was first written. However, {{mock}} had been improved over time, and the new features it includes allow for making improvements to the way {{mock_runner.sh}} works without a very big time investment. h3. Acceptance Criteria # {{mock_runner.sh}} uses systemsd-nspawn for containment rather then simple chroot # {{mock_runner.sh}} can start a chroot from cache in less then 10 seconds. # {{mock_runner.sh}} does not reinstall the whole chroot if just a few packages in the *.packages file were changed. # {{mock_runner.sh}} invokes {{mock}} only once when it runs # {{mock_runner.sh}} can use the mock config files that are shipped with {{mock}} directly without the CI team having to maintain their own set of files. was: {{mock_runner.sh}} is the core of all STDCI, yet it has multiple issues and shortcomings that make it less the ideal. Many of the ways {{mock_runner.sh}} does things have to do with the way {{mock}} worked a few years ago when the script was first written. However, {{mock}} had been improved over time, and the new features it includes allow for making improvements to the way {{mock_runner.sh}} works without a very big time investment. h3. Acceptance Criteria # {{mock_runner.sh}} uses systemsd-nspawn for containment rather then simple chroot # {{mock_runner.sh}} can start a chroot from cache in less then 10 seconds. # {{mock_runner.sh}} does not reinstall the whole chroot if just a few packages in the *.packages file were changed. # {{mock_runner.sh}} invokes {{mock}} only once when it runs > Imrpove mock_runner.sh speed and containment > -------------------------------------------- > > Key: OVIRT-2189 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2189 > Project: oVirt - virtualization made easy > Issue Type: Epic > Components: mock_runner > Reporter: Barak Korren > Assignee: infra > > {{mock_runner.sh}} is the core of all STDCI, yet it has multiple issues and shortcomings that make it less the ideal. Many of the ways {{mock_runner.sh}} does things have to do with the way {{mock}} worked a few years ago when the script was first written. However, {{mock}} had been improved over time, and the new features it includes allow for making improvements to the way {{mock_runner.sh}} works without a very big time investment. > h3. Acceptance Criteria > # {{mock_runner.sh}} uses systemsd-nspawn for containment rather then simple chroot > # {{mock_runner.sh}} can start a chroot from cache in less then 10 seconds. > # {{mock_runner.sh}} does not reinstall the whole chroot if just a few packages in the *.packages file were changed. > # {{mock_runner.sh}} invokes {{mock}} only once when it runs > # {{mock_runner.sh}} can use the mock config files that are shipped with {{mock}} directly without the CI team having to maintain their own set of files. -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-2191) remove the mountpoint creation phase from mock_runner
by Barak Korren (oVirt JIRA) 12 Jun '18

12 Jun '18

Barak Korren created OVIRT-2191: ----------------------------------- Summary: remove the mountpoint creation phase from mock_runner Key: OVIRT-2191 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2191 Project: oVirt - virtualization made easy Issue Type: Improvement Components: mock_runner Reporter: Barak Korren Assignee: infra There is a part in mock_runner code that runs mock in order to create mount points for things that are going to be bind mounted into it. Since mock 1.4.10 can create its own mount point we can remove that code from mock runner. This should reduce the amount of times mock is called from mock_runner -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-2191) remove the mountpoint creation phase from mock_runner
by Barak Korren (oVirt JIRA) 12 Jun '18

12 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2191?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2191: -------------------------------- Epic Link: OVIRT-2189 > remove the mountpoint creation phase from mock_runner > ----------------------------------------------------- > > Key: OVIRT-2191 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2191 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: mock_runner > Reporter: Barak Korren > Assignee: infra > > There is a part in mock_runner code that runs mock in order to create mount points for things that are going to be bind mounted into it. Since mock 1.4.10 can create its own mount point we can remove that code from mock runner. > This should reduce the amount of times mock is called from mock_runner -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-2032) Mounting device files prevents using systemd-nspawn in mock
by Barak Korren (oVirt JIRA) 12 Jun '18

12 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2032?page=com.atlassian.jira.… ] Barak Korren updated OVIRT-2032: -------------------------------- Assignee: Barak Korren (was: infra) Status: In Progress (was: To Do) > Mounting device files prevents using systemd-nspawn in mock > ----------------------------------------------------------- > > Key: OVIRT-2032 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2032 > Project: oVirt - virtualization made easy > Issue Type: Bug > Components: mock_runner > Reporter: Barak Korren > Assignee: Barak Korren > Labels: upstream-issue > > The systemd-nspawn functionality in mock is introduced in OVIRT-2031. > When using systemd-nspawn mock uses some kind of a layered FS that is created when it starts and removed when it exits. This is different from the way it works when using chroot where the directory that the chroot is configured in stays around until it is explicitly removed. > mock had an issue where if you tried to bind-mount something into the mock environment, you had to have the mount point ready for it. We worked around this in {{mock_runner}} by using the fact the chroot was persistent, going into it and setting up the mount points as needed before actually starting to use it to run the STDCI script. > Since the mock authors were aware of the issue when the implemented the systemd-nspawn functionality, they made mock pre-create mount point in the container as needed. However, they only supported directory mount points, so if we tried to bind-mount a socket we would get en error message failing to mount a socket file on a directory. > We reported this issue to the mock developers as issue [#87|https://github.com/rpm-software-management/mock/issues/87] -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1678) Move gerrit-admin project to be standard CI
by eyal edri (oVirt JIRA) 12 Jun '18

12 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1678?page=com.atlassian.jira.… ] eyal edri updated OVIRT-1678: ----------------------------- Epic Link: (was: OVIRT-411) > Move gerrit-admin project to be standard CI > ------------------------------------------- > > Key: OVIRT-1678 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1678 > Project: oVirt - virtualization made easy > Issue Type: Task > Components: Gerrit/git, Gerrit Hooks > Reporter: eyal edri > Assignee: infra > > Since we're moving all projects on CI to be a std-ci project ( i.e have automation/ dir with all the relevant scripts inside ), we should also move the gerrit-admin project which holds all the code for the Gerrit hooks. > This will help lay the ground for adding unit-tests and deployments for the hooks in a standard way. > I would start with adding a 'check-patch.sh' script which will do basic sanity on the code, maybe using partner-bugizlla instance and a test bug for testing the hooks. > It's also a great way to learn more about how to create a stanard CI project. > More info on it can be found on the infra docs page: > http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Build_and_test_standard… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1678) Move gerrit-admin project to be standard CI
by eyal edri (oVirt JIRA) 12 Jun '18

12 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1678?page=com.atlassian.jira.… ] eyal edri updated OVIRT-1678: ----------------------------- Epic Link: (was: OVIRT-411) > Move gerrit-admin project to be standard CI > ------------------------------------------- > > Key: OVIRT-1678 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1678 > Project: oVirt - virtualization made easy > Issue Type: Task > Components: Gerrit/git, Gerrit Hooks > Reporter: eyal edri > Assignee: infra > > Since we're moving all projects on CI to be a std-ci project ( i.e have automation/ dir with all the relevant scripts inside ), we should also move the gerrit-admin project which holds all the code for the Gerrit hooks. > This will help lay the ground for adding unit-tests and deployments for the hooks in a standard way. > I would start with adding a 'check-patch.sh' script which will do basic sanity on the code, maybe using partner-bugizlla instance and a test bug for testing the hooks. > It's also a great way to learn more about how to create a stanard CI project. > More info on it can be found on the infra docs page: > http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Build_and_test_standard… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0

[JIRA] (OVIRT-1678) Move gerrit-admin project to be standard CI
by eyal edri (oVirt JIRA) 12 Jun '18

12 Jun '18

[ https://ovirt-jira.atlassian.net/browse/OVIRT-1678?page=com.atlassian.jira.… ] eyal edri updated OVIRT-1678: ----------------------------- Component/s: Gerrit Hooks > Move gerrit-admin project to be standard CI > ------------------------------------------- > > Key: OVIRT-1678 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1678 > Project: oVirt - virtualization made easy > Issue Type: Task > Components: Gerrit/git, Gerrit Hooks > Reporter: eyal edri > Assignee: infra > > Since we're moving all projects on CI to be a std-ci project ( i.e have automation/ dir with all the relevant scripts inside ), we should also move the gerrit-admin project which holds all the code for the Gerrit hooks. > This will help lay the ground for adding unit-tests and deployments for the hooks in a standard way. > I would start with adding a 'check-patch.sh' script which will do basic sanity on the code, maybe using partner-bugizlla instance and a test bug for testing the hooks. > It's also a great way to learn more about how to create a stanard CI project. > More info on it can be found on the infra docs page: > http://ovirt-infra-docs.readthedocs.io/en/latest/CI/Build_and_test_standard… -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100087)

1 0