7:13 a.m.
This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
--UjerjdpPI1W3Jwd0dWEnDKWKtGMXxXMg7
Content-Type: multipart/mixed; boundary="5rcCa7NeC9kJsmU0XEABtc3HH2nIqASqS"
From: Mikey Ariel <mariel(a)redhat.com>
To: infra(a)ovirt.org
Message-ID: <56DD7E86.8070202(a)redhat.com>
Subject: [SOLVED] Temporary Outage - ovirt.org Website
--5rcCa7NeC9kJsmU0XEABtc3HH2nIqASqS
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi folks,
It seems that our logs on OpenShift filled up and the ovirt.org website
was unavailable for a few hours this morning.
The issue was resolved and the site is back online.
Cheers,
Mikey
--=20
Mikey Ariel
Community Lead, oVirt
www.ovirt.org
"To be is to do" (Socrates)
"To do is to be" (Jean-Paul Sartre)
"Do be do be do" (Frank Sinatra)
Mobile: +420-702-131-141
IRC: mariel / thatdocslady
Twitter: @ThatDocsLady
--5rcCa7NeC9kJsmU0XEABtc3HH2nIqASqS--
--UjerjdpPI1W3Jwd0dWEnDKWKtGMXxXMg7
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: OpenPGP digital signature
Content-Disposition: attachment; filename="signature.asc"
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQEcBAEBCAAGBQJW3X6GAAoJEHYPPTOszxHogaoH/iLn4bXy0uQDjusJidUwI1/l
pVkGfPrgbgnwCDnVqNasipbRzCPHZ/xVEQV63xG/0KlFrm1yeiZ2HTrufkRYdR2d
s6tRPcBCzGvPDygeM/pQySo/ozXMkccE4l1+U2vUjCV5N1K11B5E42AZ2SnVP0t7
CfRZGgxyYhjWcKJ6bWhn72UdiGmUc9x9LSsUDgAkez74Pk7++TUeQHt8Bv4tzOZq
j9mpGis5e2uAMdwY6W+Xd4l0qSTrp3ammRI3QzApxN0ZdKTirgkdgtxaEXjtpq7N
PX++sTZr6qm8I1xqnCDsNbxhlTZZe64l+FkzCd6DeXdI11vTyLyyIZ70WnqGeCY=
=1ciD
-----END PGP SIGNATURE-----
--UjerjdpPI1W3Jwd0dWEnDKWKtGMXxXMg7--
7:26 a.m.
--=-P+jMSFUV/o8IeH24nhkp
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Le lundi 07 mars 2016 =C3=A0 14:13 +0100, Mikey Ariel a =C3=A9crit :
Hi folks,
=20
It seems that our logs on OpenShift filled up and the ovirt.org website
was unavailable for a few hours this morning.
=20
The issue was resolved and the site is back online.
So to complete what Mikey said:
- we have increased the partition from 1G to 3G (this was done
automatically with previous openshift version, so insert my usual rant
about changes in *aaS), so we would have more room next time.
- a RCA (root cause analysis) show that we got 250 000 requests today
from 1 single IP at OVH, around 100 000 from bing bot reindexing the
website (on 4 or 5 ip ), and 30 000 requests from a ip of bezeqint.net,
with a nagios check_http signature.=20
While we can't do much for the first or the 2nd (besides blocking), I
would suggest that the 3rd ip owner decide to reduce the frequency of
check from 1 every 2 seconds to 1 every 5 minutes.
--=20
Michael Scherer
Sysadmin, Community Infrastructure and Platform, OSAS
--=-P+jMSFUV/o8IeH24nhkp
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAABAgAGBQJW3YFzAAoJEE89Wa+PrSK92FcP/RU8Zdkkbi0qIvU9FodGP/dR
3yKtDHIp5Djxan1SWtppT+HlAPtp6q+YRA2rDovHKvfsrOjyzR9Ui4qd46S6yIv9
V2XV3Hw9vO50isV3vZEmEQ/8T/+vm/B/aZ1pi9c3ZvK7UlZdNw6qZzmYJLqxfpHh
8KbCa6iaCPe3hsrv9vZnCyu3bQo6zHNakb9DJx0KaScnDZcuqFDRa5eTk8JIBLh5
e+CkMJ0wrkLj8wg0o04Ybfc1/ptGE+MI0XXV7HNGIXULE7vRi8Q2GS8ZDhuRd0qf
GLSncZ966owKkaSwRkftzxFbdjBLPHSn7tOXIcuJtacE81KDrs2JAG9ogVPwGM7Q
XLuLIj1Ac6H/Zn6hPLSMXTf1xI3rpoojZ+OMTuXpFeoQJ2Jpt85gKntJGDSczitA
ZuneSBmgKGd0blSj/Wc3JW5hxCownPBJJVrr4Rkf39izCT+HlsWkp4Q9bU8lr+SZ
kXlnhEU42wQdy3aHrBrMp5RBUmsWNjqFiXB8LF0QX8T9OXK6RI3spGyeSIPHRtA6
ZfR5/SMYJS0U3xwTF4u3Y+lsFoCti16Hp/7Frr5e/Ro0kq1CJ9sl2iU2fswori6u
nlJtZ4miVYVZLsTH8yhehm4HE8rK8q185UteK+WQBXwxSzFRawNZPVTOt3bC8Wrs
jP+wKbaVubfNcwDA4Gxr
=UhSQ
-----END PGP SIGNATURE-----
--=-P+jMSFUV/o8IeH24nhkp--
7:50 a.m.
While we can't do much for the first or the 2nd (besides blocking), I
would suggest that the 3rd ip owner decide to reduce the frequency of
check from 1 every 2 seconds to 1 every 5 minutes.
Sure, that was us, I just removed the whole thing, I guess you have
your own and better monitoring now.
--
Barak Korren
bkorren(a)redhat.com
RHEV-CI Team
7:51 a.m.
--=-gtIJFMhA5RKT2NfX6Qzy
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Le lundi 07 mars 2016 =C3=A0 15:50 +0200, Barak Korren a =C3=A9crit :
>
> While we can't do much for the first or the 2nd (besides blocking), I
> would suggest that the 3rd ip owner decide to reduce the frequency of
> check from 1 every 2 seconds to 1 every 5 minutes.
>
=20
Sure, that was us, I just removed the whole thing, I guess you have
your own and better monitoring now.
Nope, we don't. nagios is fine, but well, every second is excessive :/
--=20
Michael Scherer
Sysadmin, Community Infrastructure and Platform, OSAS
--=-gtIJFMhA5RKT2NfX6Qzy
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAABAgAGBQJW3Yd9AAoJEE89Wa+PrSK91OAP/31c3MqKucXvQHByhmeYDLUb
Y89YUVfJmoqc9Ys8FqUZwT+0bDYFOOjTW9fySxDDEgjlrCIOXrFc847fl7OVtGS0
wIQ6ssY97pyf2DY4RxJizwoa8wZugDHlsO3meCbBmJPDg8jI59jwY2+RbvPpxuIC
VQuCcrZdkFGbDaOx2Is1ZFb/GTj8eOM5v42b2U0P6/eV6+kq1KCP7njCsfzcEzYM
SFPwgqvMS5/M4mqmr3fQAcvcZfWV1zd3j0nIi220wvbJe3txogTEHwSbuxHBLTCX
maL7d3IVx1XIADcFR2JGqQZdtB8x/OgtVOYJquY9lgG8e7XEIoyoRb+IyGLiKfth
x8HVEw8oFTxiE6y1K79PX36I74rq53pvWIswAHG1X3ELOqX7Rvw/f6LPNzvxGpTr
fD9WYM31FE9ubNJ3q9d/oakdFooQO+m3BDMgy0DgHizE4HZpoRCTL+hWBB0x05wc
mqMkslBQEAtIEutcwfXZLt8Nt6UFqhvwKxufx/A9sxZChmmyritggtpzqeJGsrdo
fJl12oRiLj2mY4nWqe5tMO4g81UEityqM5ZF4ynNPgzK3p8ppgjpVOqoKf5UB151
mrpSMstyLNWIvjX8dSqsDrjYIY5KIQdn0hy0ll7iECvmEgBjFNUNd3LcqCmv6Rtg
CVo3/F3p0KII3jnAX3gm
=ynG3
-----END PGP SIGNATURE-----
--=-gtIJFMhA5RKT2NfX6Qzy--
2981
days inactive
2981
days old
3 comments
3 participants
participants (3)
-
Barak Korren -
Michael Scherer -
Mikey Ariel