2:33 a.m.
Hello. Today we have found that the infrastructure of oVirt, Kubevirt
projects running at PHX datacenter is not accessible. It is not accessible
from both Red Hat networks and from the internet. Looks to be network
related as I did not manage to access anything there.
Few main hosts affected are jenkins.ovirt.org (66.187.230.92) and
resources.ovirt.org (66.187.230.28). Here is also traceroute to jenkins
from internet: # traceroute jenkins.ovirt.org traceroute to
jenkins.ovirt.org (66.187.230.92), 30 hops max, 60 byte packets 1 _gateway
(10.0.0.138) 11.635 ms 11.614 ms 11.554 ms 2 bzq-179-37-1.cust.bezeqint.net
(212.179.37.1) 15.719 ms 17.011 ms 17.012 ms 3 * * * 4
bzq-25-77-10.cust.bezeqint.net (212.25.77.10) 16.951 ms 17.556 ms 17.545 ms
5 bzq-219-189-2.cablep.bezeqint.net (62.219.189.2) 18.053 ms 18.044 ms
73.674 ms 6 bzq-179-124-62.cust.bezeqint.net (212.179.124.62) 85.149 ms *
bzq-219-189-73.dsl.bezeqint.net (62.219.189.73) 77.685 ms 7
bzq-179-72-242.cust.bezeqint.net (212.179.72.242) 70.555 ms
ge-2-1-0.mpr1.lhr2.uk.above.net (195.66.224.76) 75.340 ms
ae11.mpr2.lhr2.uk.zip.zayo.com (64.125.30.52) 70.436 ms 8
ge-2-1-0.mpr1.lhr2.uk.above.net (195.66.224.76) 76.982 ms
ae11.mpr2.lhr2.uk.zip.zayo.com (64.125.30.52) 76.960 ms
ge-2-1-0.mpr1.lhr2.uk.above.net (195.66.224.76) 71.310 ms 9 * * * 10 * * *
11 ae0.cs2.lga5.us.eth.zayo.com (64.125.29.187) 208.885 ms *
ae4.cs2.dca2.us.eth.zayo.com (64.125.29.31) 195.693 ms 12
ae4.cs2.dca2.us.eth.zayo.com (64.125.29.31) 232.666 ms 200.411 ms 212.885
ms 13 ae26.mpr4.phx2.us.zip.zayo.com (64.125.31.255) 200.021 ms
ae3.cs2.iah1.us.eth.zayo.com (64.125.29.45) 200.916 ms
ae4.cs2.dca2.us.eth.zayo.com (64.125.29.31) 196.332 ms 14
ae26.mpr4.phx2.us.zip.zayo.com (64.125.31.255) 206.193 ms 206.258 ms
ae3.cs2.iah1.us.eth.zayo.com (64.125.29.45) 197.296 ms 15
ae0.mpr3.phx2.us.zip.zayo.com (64.125.27.12) 207.483 ms
ae26.mpr4.phx2.us.zip.zayo.com (64.125.31.255) 202.373 ms
ae0.mpr3.phx2.us.zip.zayo.com (64.125.27.12) 208.078 ms 16
ae0.mpr3.phx2.us.zip.zayo.com (64.125.27.12) 204.214 ms
border5.ae2-bbnet2.phx010.pnap.net (69.25.168.65) 200.320 ms
208.184.108.38.IPYX-072053-007-ZYO.above.net (208.184.108.38) 203.315 ms 17
redhat-2.edge1.phx004.pnap.net (69.25.121.26) 200.961 ms
208.184.108.38.IPYX-072053-007-ZYO.above.net (208.184.108.38) 198.863 ms
border5.ae2-bbnet2.phx010.pnap.net (69.25.168.65) 206.668 ms 18
redhat-2.edge1.phx004.pnap.net (69.25.121.26) 206.648 ms unused
(66.187.228.249) 227.743 ms unused (66.187.228.248) 228.747 ms 19
redhatphx-1.phx004.pnap.net (69.25.120.242) 203.819 ms
redhat-2.edge1.phx004.pnap.net (69.25.121.26) 199.135 ms * 20 * unused
(66.187.228.248) 241.699 ms * 21 * * * 22 * * * The services running there
such as jenkins are business critical for oVirt (and RHV) development. Also
resources is used to publish oVirt externally.