On Wed, Nov 28, 2018 at 9:01 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:47 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Nov 28, 2018 at 10:32 PM Dafna Ron <dron@redhat.com> wrote:
1. it did not break ost. one failed CQ run on for one project does not mean that ost is broken :)

Ok, broke the change queue :-)
 
2. the build was reported as failed even though there was no actual failure as CQ did not even start to run. if you look at the error, you can see that CQ actually exited because the package was failed to be build. so no package -> no CQ run. 
18:25:48 vdsm_standard-on-merge (33) failed building

The vdsm build is failing with this error;

20:04:31 [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_nic_hotplug/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
20:04:31 [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_device_create/50_macspoof from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
20:04:32 [build-artifacts.fc28.s390x] *** WARNING: mangling shebang in /usr/libexec/vdsm/hooks/before_vm_start/50_fileinject from #!/usr/bin/python to #!/usr/bin/python2. This will become an ERROR, fix it manually!
This looks like a warning not an error.

Maybe the stdci change added fedora 28 build on s390x, that was not
enabled before the patch?

Checking gerrit, we see:


This is what CQ is waiting on so as long as it fails CQ will not run.
 

http://jenkins.ovirt.org/job/vdsm_master_check-merged-el7-x86_64/3939/ : FAILURE
check merged is failing most of the time, we can safely ignore it.

http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-x86_64/343/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-x86_64/4507/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-el7-ppc64le/3925/ : SUCCESS

http://jenkins.ovirt.org/job/vdsm_master_build-artifacts-fc28-s390x/208/ : SUCCESS
So build artifacts succeeded with all platform/distros.

http://jenkins.ovirt.org/job/standard-enqueue/17729/ : This change was successfully submitted to the change queue(s) for system testing.
And the patch was added to the change queue.
Based on this I expect OST to use the packages built in gerrit.

As I said, and as you can see in the logs, the build-artifacts is not what CQ is expecting for in stdci v2 so it will not run as long as standard on merge fails.
looking again using blue ocean we can see the cause of failure is check-merged: https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-on-merge/detail/vdsm_standard-on-merge/35/pipeline/132
As long as it fails the build will fail and CQ will no be able to run.


We need more info about this failure.