[CQ]: 95559,13 (vdsm) failed "ovirt-master" system tests
Change 95559,13 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/95559/13 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/11714/
On Wed, Nov 28, 2018 at 9:14 PM oVirt Jenkins <jenkins@ovirt.org> wrote:
Change 95559,13 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed.
This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed.
For further details about the change see: https://gerrit.ovirt.org/#/c/95559/13
This change added stdci v2. How can it break OST? Nir
1. it did not break ost. one failed CQ run on for one project does not mean that ost is broken :) 2. the build was reported as failed even though there was no actual failure as CQ did not even start to run. if you look at the error, you can see that CQ actually exited because the package was failed to be build. so no package -> no CQ run. *18:25:48* vdsm_standard-on-merge (33) failed building The vdsm build is failing with this error; *20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_nic_hotplug/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_device_create/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:32* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_vm_start/50_fileinject from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually! On Wed, Nov 28, 2018 at 7:52 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 9:14 PM oVirt Jenkins <jenkins@ovirt.org> wrote:
Change 95559,13 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed.
This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed.
For further details about the change see: https://gerrit.ovirt.org/#/c/95559/13
This change added stdci v2. How can it break OST?
Nir _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/OBJ3SGQBYT2GVC...
On Wed, Nov 28, 2018 at 10:32 PM Dafna Ron <dron@redhat.com> wrote:
1. it did not break ost. one failed CQ run on for one project does not mean that ost is broken :)
Ok, broke the change queue :-)
2. the build was reported as failed even though there was no actual failure as CQ did not even start to run. if you look at the error, you can see that CQ actually exited because the package was failed to be build. so no package -> no CQ run.
*18:25:48* vdsm_standard-on-merge (33) failed building
The vdsm build is failing with this error;
*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_nic_hotplug/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_device_create/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:32* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_vm_start/50_fileinject from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
This looks like a warning not an error.
Maybe the stdci change added fedora 28 build on s390x, that was not enabled before the patch? Can we test this using manual build-artifacts job? Or maybe we need to add build-artifacts job to check-patch, to make sure that every patch can be built on all platforms?
On Wed, Nov 28, 2018 at 7:52 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 9:14 PM oVirt Jenkins <jenkins@ovirt.org> wrote:
Change 95559,13 (vdsm) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed.
This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed.
For further details about the change see: https://gerrit.ovirt.org/#/c/95559/13
This change added stdci v2. How can it break OST?
Nir _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/OBJ3SGQBYT2GVC...
On Wed, Nov 28, 2018 at 10:47 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:32 PM Dafna Ron <dron@redhat.com> wrote:
1. it did not break ost. one failed CQ run on for one project does not mean that ost is broken :)
Ok, broke the change queue :-)
2. the build was reported as failed even though there was no actual failure as CQ did not even start to run. if you look at the error, you can see that CQ actually exited because the package was failed to be build. so no package -> no CQ run.
*18:25:48* vdsm_standard-on-merge (33) failed building
The vdsm build is failing with this error;
*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_nic_hotplug/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_device_create/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:32* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_vm_start/50_fileinject from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
This looks like a warning not an error.
Maybe the stdci change added fedora 28 build on s390x, that was not enabled before the patch?
Checking gerrit, we see: http://jenkins.ovirt.org/job/vdsm_standard-on-merge/33/ : FAILURE http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/3939/ : FAILURE check merged is failing most of the time, we can safely ignore it. http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-x86_64/343/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/4507/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-ppc64le/3925/ : SUCCESS http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-s390x/208/ : SUCCESS So build artifacts succeeded with all platform/distros. http://jenkins.ovirt.org/job/standard-enqueue/17729/ : This change was successfully submitted to the change queue(s) for system testing. And the patch was added to the change queue. Based on this I expect OST to use the packages built in gerrit. We need more info about this failure.
On Wed, Nov 28, 2018 at 9:01 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:47 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:32 PM Dafna Ron <dron@redhat.com> wrote:
1. it did not break ost. one failed CQ run on for one project does not mean that ost is broken :)
Ok, broke the change queue :-)
2. the build was reported as failed even though there was no actual failure as CQ did not even start to run. if you look at the error, you can see that CQ actually exited because the package was failed to be build. so no package -> no CQ run.
*18:25:48* vdsm_standard-on-merge (33) failed building
The vdsm build is failing with this error;
*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_nic_hotplug/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_device_create/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:32* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_vm_start/50_fileinject from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
This looks like a warning not an error.
Maybe the stdci change added fedora 28 build on s390x, that was not enabled before the patch?
Checking gerrit, we see:
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/33/ : FAILURE
This is what CQ is waiting on so as long as it fails CQ will not run.
http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/3939/ : FAILURE check merged is failing most of the time, we can safely ignore it.
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-x86_64/343/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/4507/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-ppc64le/3925/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-s390x/208/ : SUCCESS So build artifacts succeeded with all platform/distros.
http://jenkins.ovirt.org/job/standard-enqueue/17729/ : This change was successfully submitted to the change queue(s) for system testing. And the patch was added to the change queue. Based on this I expect OST to use the packages built in gerrit.
As I said, and as you can see in the logs, the build-artifacts is not what CQ is expecting for in stdci v2 so it will not run as long as standard on merge fails. looking again using blue ocean we can see the cause of failure is check-merged: https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-on-merge/... As long as it fails the build will fail and CQ will no be able to run.
We need more info about this failure.
בתאריך יום ד׳, 28 בנוב׳ 2018, 23:12, מאת Dafna Ron <dron@redhat.com>:
On Wed, Nov 28, 2018 at 9:01 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:47 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:32 PM Dafna Ron <dron@redhat.com> wrote:
1. it did not break ost. one failed CQ run on for one project does not mean that ost is broken :)
Ok, broke the change queue :-)
2. the build was reported as failed even though there was no actual failure as CQ did not even start to run. if you look at the error, you can see that CQ actually exited because the package was failed to be build. so no package -> no CQ run.
*18:25:48* vdsm_standard-on-merge (33) failed building
The vdsm build is failing with this error;
*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_nic_hotplug/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_device_create/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:32* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_vm_start/50_fileinject from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
This looks like a warning not an error.
Maybe the stdci change added fedora 28 build on s390x, that was not enabled before the patch?
Checking gerrit, we see:
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/33/ : FAILURE
This is what CQ is waiting on so as long as it fails CQ will not run.
http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/3939/ : FAILURE check merged is failing most of the time, we can safely ignore it.
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-x86_64/343/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/4507/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-ppc64le/3925/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-s390x/208/ : SUCCESS So build artifacts succeeded with all platform/distros.
http://jenkins.ovirt.org/job/standard-enqueue/17729/ : This change was successfully submitted to the change queue(s) for system testing. And the patch was added to the change queue. Based on this I expect OST to use the packages built in gerrit.
As I said, and as you can see in the logs, the build-artifacts is not what CQ is expecting for in stdci v2 so it will not run as long as standard on merge fails. looking again using blue ocean we can see the cause of failure is check-merged: https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-on-merge/... As long as it fails the build will fail and CQ will no be able to run.
We need more info about this failure.
Please note that vdsm currently has both v1 and v2 jobs, so changes are being submitted to the CQ twice. Its perfectly possible that the change made it through the CQ via the V1 jobs. It check-merged is an issue for now, you can just exclude it from running in stdci.yaml.
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/ZPAZ7LNENB4B46...
On Wed, Nov 28, 2018 at 11:11 PM Dafna Ron <dron@redhat.com> wrote:
On Wed, Nov 28, 2018 at 9:01 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:47 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:32 PM Dafna Ron <dron@redhat.com> wrote:
1. it did not break ost. one failed CQ run on for one project does not mean that ost is broken :)
Ok, broke the change queue :-)
2. the build was reported as failed even though there was no actual failure as CQ did not even start to run. if you look at the error, you can see that CQ actually exited because the package was failed to be build. so no package -> no CQ run.
*18:25:48* vdsm_standard-on-merge (33) failed building
The vdsm build is failing with this error;
*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_nic_hotplug/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:31* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_device_create/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!*20:04:32* [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_vm_start/50_fileinject from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
This looks like a warning not an error.
Maybe the stdci change added fedora 28 build on s390x, that was not enabled before the patch?
Checking gerrit, we see:
http://jenkins.ovirt.org/job/vdsm_standard-on-merge/33/ : FAILURE
This is what CQ is waiting on so as long as it fails CQ will not run.
http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/3939/ : FAILURE check merged is failing most of the time, we can safely ignore it.
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-x86_64/343/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/4507/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-ppc64le/3925/ : SUCCESS
http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-s390x/208/ : SUCCESS So build artifacts succeeded with all platform/distros.
http://jenkins.ovirt.org/job/standard-enqueue/17729/ : This change was successfully submitted to the change queue(s) for system testing. And the patch was added to the change queue. Based on this I expect OST to use the packages built in gerrit.
As I said, and as you can see in the logs, the build-artifacts is not what CQ is expecting for in stdci v2 so it will not run as long as standard on merge fails. looking again using blue ocean we can see the cause of failure is check-merged: https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-on-merge/... As long as it fails the build will fail and CQ will no be able to run.
Thanks! I posted this to fix this issue: https://gerrit.ovirt.org/c/95845/ Please review. Nir
participants (4)
-
Barak Korren -
Dafna Ron -
Nir Soffer -
oVirt Jenkins