[ATN] - on going outage in phx lab infra

fyi, there is an issue with the ovirt-engine server on phx lab, which causes some of the jenkins vms to be inaccessible. we've contacted the local team to help, but it might take some time since they are in the US. till then, jobs might be queued for a long time and possibly failing. If you have an urgent patch to commit and can't wait, please verify it locally (maybe bit more than you are used to) before merging. the jenkins won't block you on if it's an infra issue. we'll update once the outage is over. Eyal.

--=-3H2uT/4HK5/dUj6dH8to Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le lundi 17 novembre 2014 =C3=A0 00:53 -0500, Eyal Edri a =C3=A9crit :
fyi, =20 there is an issue with the ovirt-engine server on phx lab, which causes s= ome=20 of the jenkins vms to be inaccessible.
we've contacted the local team to help, but it might take some time since= =20 they are in the US.
What kind of issue do we have ?
till then, jobs might be queued for a long time and possibly failing. =20 If you have an urgent patch to commit and can't wait, please verify it= =20 locally (maybe bit more than you are used to) before merging. =20 the jenkins won't block you on if it's an infra issue. =20 we'll update once the outage is over.
--=20 Michael Scherer Open Source and Standards, Sysadmin --=-3H2uT/4HK5/dUj6dH8to Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUae6KAAoJEE89Wa+PrSK9e5gP/2vx9pxFWIM3IWhUZeW/VycO kX1fdEs8X0ElXQfUe+Da7LtF+Sw5OmqAFrtvoR2f2xikDz7wNQ3HTp2gzjNkvhoS bBLfTdl3H9baRHCprjQGllO/C/qrG5sVVwjfRqOFupUcI6Qk289kFoSvxYfKEE2R E8E7qVTbRiAPyvrnAtwP/Es5wDGyd/W63xlu8XwdgB6w59mKFQKsUalTihLcSpDy yVMH6SeI1pEegceQOLprwOnX88eUoMvnnw4Y1K1X10uH96mraSwFdPkHCg4EvRAO qNuW6hGQVsRheAhDFTGZNsuZviJ51wwtqsnTs7SBJ7ZZMLHnjiK+zrfxS6XilLpo cG3NCIhECqjAxamJy0yRfsxLPcvY3hDn51qRO3MQxlMSPovH8O1lgO7qkUTv0zll ESSf3BBzvm8LJOsoipXVQokUBdBqICNE3mkzax2/v8YJxGvQWRwVOCtB7ip/w+tu hHCN6KVNxP2O5taLdmaNxjPDM5IykLv8o3+1ywL8DnyKOkEE75vkmVY46ir9ICpF WNNwFC4gZbOIIkTTGhT6EHWmfUlBUi9OOIdsFev30XLv9aBHsSR9CTb6nrtTD4Sj h2Cuuw3b/SB+h6WlWNCSCXFTxf47ZhG2VWstyYQWQAzTjVGs004idf8HbFOcm7Q/ RWDURQrSFxGcbr+eWP1Y =QndH -----END PGP SIGNATURE----- --=-3H2uT/4HK5/dUj6dH8to--

--BXVAT5kNtrzKuDFl Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 11/17, Michael Scherer wrote:
Le lundi 17 novembre 2014 =E0 00:53 -0500, Eyal Edri a =E9crit :
fyi, =20 there is an issue with the ovirt-engine server on phx lab, which causes= some=20 of the jenkins vms to be inaccessible.
we've contacted the local team to help, but it might take some time sin= ce=20 they are in the US. =20 What kind of issue do we have ?
duting the upgrade to oVirt 3.5 on one of the hosts vdsm misocnfigured the network and we are having issues connecting to the console. The problem is that the host that went down was the SPM and also was the on running the forman-proxy (dhcp). So the first means that we can't start any vm on the cluster and the second that all the slaves lost their ips (and connectivity, I configured statically a few of them and will configure more during the day if the issue persists).
=20
till then, jobs might be queued for a long time and possibly failing. =20 If you have an urgent patch to commit and can't wait, please verify it= =20 locally (maybe bit more than you are used to) before merging. =20 the jenkins won't block you on if it's an infra issue. =20 we'll update once the outage is over. =20 --=20 Michael Scherer Open Source and Standards, Sysadmin
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
--=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --BXVAT5kNtrzKuDFl Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJUae9vAAoJEEBxx+HSYmnD8a8H/Atv69eOSlHQHy8EX8kw9iQg dhnRPUr9aU9vQnqzV/Huy3ihe4Sa8BDY2uOnfh5lBdvBYyUDhwYTTYdTqCu1zDLw U0ioVQC6/aPJbJ/sQGnDY3KTvl87970Xkom2w5qhICIeMxKa+MDFchKuoSdtFb/+ JChYfKLB1GmXJeNYEA2/9jxRS39TKRspBVW/4f2naBfgvST67LmwBN3gliJD1o6z upqWOAGkxhcUnFBPPg6jRard0C1JHMj1MFIbC1uKHL7E+GxRgZY9YGUOnu1APPd7 o61LLHFTZnAnxUar7F6N2G4NWFIeMpP3clsz3N9ZN9OUjc9of25DCScVPutzLJA= =II45 -----END PGP SIGNATURE----- --BXVAT5kNtrzKuDFl--
participants (3)
-
David Caro
-
Eyal Edri
-
Michael Scherer