Sorry, but I don't see any improvements since morning, all engine check-patch jobs are still failing:

17:50:07 Initialized empty Git repository in /home/jenkins/agent/workspace/ovirt-engine_standard-check-patch/ovirt-engine/.git/
17:50:51 stdci-production/jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48 Container jnlp was terminated (Exit Code: 137, Reason: OOMKilled)
17:50:53 Cannot contact jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48: hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@55ad1988:JNLP4-connect connection from a2.15.3ea9.ip4.static.sl-reverse.com/169.62.21.162:1024": Remote call on JNLP4-connect connection from a2.15.3ea9.ip4.static.sl-reverse.com/169.62.21.162:1024 failed. The channel is closing down or has closed down
17:51:41 stdci-production/jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48 Container jnlp was terminated (Exit Code: 137, Reason: OOMKilled)
17:52:07 stdci-production/jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48 Pod just failed (Reason: null, Message: null)
17:52:07 stdci-production/jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48 Pod just failed (Reason: null, Message: null)
17:57:07 Agent jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48 was deleted; cancelling node body
17:57:07 Agent jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48 was deleted; cancelling node body
17:57:07 Could not connect to jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48 to send interrupt signal to process
17:57:07 Could not connect to jenkins-ovirt-engine-standard-check-patch-15330-61vhd-63q48 to send interrupt signal to process




On Fri, Nov 26, 2021 at 12:00 PM Ehud Yonasi <eyonasi@redhat.com> wrote:
Hi Nir,

There was a problem in jenkins to switch to the new k8s stack on IBM cloud, a restart was required and now it’s solved.


On 25 Nov 2021, at 21:53, Nir Soffer <nsoffer@redhat.com> wrote:

On Thu, Nov 25, 2021 at 9:15 PM Michal Skrivanek
<michal.skrivanek@redhat.com> wrote:



On 25. 11. 2021, at 16:45, Evgheni Dereveanchin <ederevea@redhat.com> wrote:

Hi Milan,

Indeed, this is related to the Jenkins move: all pre-existing nodes are no longer usable so new ones have been created.
We're looking into the reasons new nodes fail to initialize and bringing up new ones in parallel to help with processing the build queue.

Sorry for the inconvenience. Please report any other issues if you see them as there may be quite some instability due to the rebuild.


Hi Evgheni,
the queue is now 53 builds long, and it seems nothing is building, perhaps there's no worker or the labelling is wrong?

Looks like jobs are stuck for several hours
https://jenkins.ovirt.org/job/vdsm_standard-check-patch/31036/

at the "loading code" stage:
https://jenkins.ovirt.org/blue/organizations/jenkins/vdsm_standard-check-patch/detail/vdsm_standard-check-patch/31036/pipeline

With this error:

   There are no nodes with the label ‘jenkins-vdsm_standard-check-patch-31036’

The label looks wrong. vdsm requires nodes with "el8" label.


Thanks,
michal


Regards.
Evgheni

On Thu, Nov 25, 2021 at 4:24 PM Nir Soffer <nsoffer@redhat.com> wrote:

On Thu, Nov 25, 2021 at 3:48 PM Milan Zamazal <mzamazal@redhat.com> wrote:

Hi,

all patches uploaded today I've seen (for Engine and Vdsm) fail due to
problems with availability of packages when preparing the el7
environment.  For example:
https://jenkins.ovirt.org/job/ovirt-engine_standard-check-patch/15291/

Additionally, Vdsm tests are not run on PSI.

Does anybody know what's wrong and how to fix it?  Can it be related to
the Jenkins move?

Looking at
https://jenkins.ovirt.org/

There are 33 jobs in the queue, and only 3 jobs running.

Maybe we did not restore all the nodes?


_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ZLH3ILKHC3YPZWQFA2SG4GS3QSIALUXZ/


_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/RZ7H4L26DU6LAKFQXJE4W3D4C5HP76CX/
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/UK4KVTPITCAWHWUR5QPGN3RSNFYQSH34/

_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/APB6NEWFR2RN7WWOLWTG5HHJF25X4BL5/


--
Martin Perina
Manager, Software Engineering
Red Hat Czech s.r.o.