[OUTAGE] ovirt.org wiki outage report [07/12/14]
fyi, ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST. REASON: ======= after logging into the openshift gear, seems like there was 99.9% space taken. RESOLUTION: =========== deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql using 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon how to resolve this. wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or extend again the existing one. Eyal.
fyi, =20 ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST. =20 REASON: =3D=3D=3D=3D=3D=3D=3D after logging into the openshift gear, seems like there was 99.9% space t= aken. =20 =20 RESOLUTION: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql u= sing 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon how to res=
--=-oU9cwyL9Z523xHYR89Fk Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri a =C3=A9cri= t : olve this.
=20 =20 =20 wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or extend agai= n the existing one.
Happened again today. I still maintain that we should move to a dedicated hosting on the phx2 cluster :) ( or request quota extensions ).=20 --=20 Michael Scherer Open Source and Standards, Sysadmin --=-oU9cwyL9Z523xHYR89Fk Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUhYVlAAoJEE89Wa+PrSK9Rh0P/3L058LYqQs6w6+QoMi0Kwmt oi6CKzELQ27l0uJjCbgOuLJioIcSWQ9NeeFlV6tqw36C8oCIZx08u1aNfTQEgWDk 8347ipv/TG93SSHzO0rtcU+nsdTGYasplHZRk3RKchYP6cGBfYZVsuSDhZavY8tk imz+36AFnp2/59za66jHjm9M9FcX+YVVLLNbLcBijFRBpEhWtiLkaMXKC9cfjet4 org4EHe/YbiifsFFNQ4hqK0HIE8ZfpeMATt8/MD0G9xP+EcwLv+0mkH5IpV3T1p1 +6L6yMukj/RxZ9cKmR5K4eJm7DnwgnFWm5biLbwsQLVSDRqYTX/66iz8KAXB/gAc KqUB/o0OjekuBGvZW9BmsmcgPAaMjU+ZKQGFn9qrnvle0YpoD1zHJR2gcSE35s23 ZGRJkaSLPlBUk2O1b6Dq0Lh3y6oRamQtbgsac5mPEVXeYlBc5I4WbQ1QK8dCbG+2 T+oCvc8hnGmV00rW/o3kcZxHx7yHq8xosxvWCdYgz+pHFHNllufjenIpM44fa6ez ogDdDyS1vVPGpen/Y1JS/JtFzZWLnAHNfjQN/FZpIpki3h+Tw2YR2p9VZi/zrZMB ZwJui+YUaxvC/1qJ8MhudMTEV/NI9qhvUrkMD7LsUsCzw0StR4x98fg5B6gYMe95 9fCH+ymaGWWeYWZt8zI+ =XU16 -----END PGP SIGNATURE----- --=-oU9cwyL9Z523xHYR89Fk--
----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 1:03:01 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le dimanche 07 décembre 2014 à 03:25 -0500, Eyal Edri a écrit :
fyi,
ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
REASON: ======= after logging into the openshift gear, seems like there was 99.9% space taken.
RESOLUTION: =========== deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql using 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon how to resolve this.
wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or extend again the existing one.
Happened again today.
I still maintain that we should move to a dedicated hosting on the phx2 cluster :) ( or request quota extensions ).
whether if we'll migrate or not, that will take time and won't happen until the ILO issue will be resolved. for now we need to extend our volume there ASAP, cause it will happen again soon, not too many log files are there to delete/gzip. e.
-- Michael Scherer Open Source and Standards, Sysadmin
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
--=-1kQcmNimmI7H3ptkbG5x Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le lundi 08 d=C3=A9cembre 2014 =C3=A0 06:23 -0500, Eyal Edri a =C3=A9crit :
=20 ----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 1:03:01 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14] =20 Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri a =C3= =A9crit :
fyi, =20 ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST. =20 REASON: =3D=3D=3D=3D=3D=3D=3D after logging into the openshift gear, seems like there was 99.9% spa= ce taken. =20 =20 RESOLUTION: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mys= ql using 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon how to resolve this. =20 =20 =20 wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or extend = again the existing one. =20 Happened again today. =20 I still maintain that we should move to a dedicated hosting on the phx2 cluster :) ( or request quota extensions ). =20 whether if we'll migrate or not, that will take time and won't happen unt= il the ILO issue will be resolved. for now we need to extend our volume there ASAP, cause it will happen aga= in soon, not too many log files are there to delete/gzip.
Agreed. We did request already a quota extension, IIRC, so I guess that was not sufficient ? --=20 Michael Scherer Open Source and Standards, Sysadmin --=-1kQcmNimmI7H3ptkbG5x Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUhZl+AAoJEE89Wa+PrSK9WYwP/jn5Nu3O9qDynnn+XnBEcXam ZQ4I/h6NL5ctxl4Yev0lqGmI50IPnq7rpfhEhWmKKjHjSwS1tDFPxuijljVzLqQd ojsA2aYfIlpDMA1GdJbLI2/EoaOoCuGL7TCcbbROfjzkZqKZIXQTG04mpgFQRhPX gQiaFOiz4B8M3iNpfAhwdlo77x8J1xtIHlX082fbRA2ulepvb7rmaNUFHC7OyInJ +STcR5v556Eb1IME1aP47Q4JjbsJFsvJR67tqvlHK4J+tKVLWS+O60aHghAdO05P DgQU03JsNo9Le6Lx0c7mEvWktymgm6KE3qXeQNFdH7pteVxR3h2rEb+77e2aGKdK PYp64ow/YxiVZ3vLNuRgbyY4q2M3WLNzhs8mPNYgMveskUAouj8hOkR371RC9PV7 TE4kLIGxF5MH3tTn6iCq0x8TjgA2tdenK9t+cK5tjoFvWs63T+84iTF9i0Em1LFK Ap0OcSa1hvGSA3PesAyKfUZ+dvqOWfmny7c73HmUWkRDXGO6Ne1aZZcG+hnTZ18R WwoklvGjH4TnZcR+x6teSxAjE/xp1SVnwkM2iyuQEjTLwGX/fexC2e54KWTAa7IZ Xgjo9QZZUjGJw+1K7cNlO45kSEZgWe6v3y903pFj54ggRcmvarhRIJEuq9+5tieL mUkZ/sWf9Y0AJ6ryUj+h =Erck -----END PGP SIGNATURE----- --=-1kQcmNimmI7H3ptkbG5x--
----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: "Eyal Edri" <eedri@redhat.com> Cc: infra@ovirt.org Sent: Monday, December 8, 2014 2:28:46 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le lundi 08 décembre 2014 à 06:23 -0500, Eyal Edri a écrit :
----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 1:03:01 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le dimanche 07 décembre 2014 à 03:25 -0500, Eyal Edri a écrit :
fyi,
ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
REASON: ======= after logging into the openshift gear, seems like there was 99.9% space taken.
RESOLUTION: =========== deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql using 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon how to resolve this.
wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or extend again the existing one.
Happened again today.
I still maintain that we should move to a dedicated hosting on the phx2 cluster :) ( or request quota extensions ).
whether if we'll migrate or not, that will take time and won't happen until the ILO issue will be resolved. for now we need to extend our volume there ASAP, cause it will happen again soon, not too many log files are there to delete/gzip.
Agreed. We did request already a quota extension, IIRC, so I guess that was not sufficient ?
looking at the mysql db, it seems to be taking most of the space (9.8GB data file). so i guess no choice but to extend it more.
-- Michael Scherer Open Source and Standards, Sysadmin
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Le lundi 08 d=C3=A9cembre 2014 =C3=A0 06:23 -0500, Eyal Edri a =C3=A9crit= :
=20 ----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 1:03:01 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14] =20 Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri a =C3= =A9crit :
fyi, =20 ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST= . =20 REASON: =3D=3D=3D=3D=3D=3D=3D after logging into the openshift gear, seems like there was 99.9% s=
--=-xEIq5DPZS/xUU1TKd3vy Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le lundi 08 d=C3=A9cembre 2014 =C3=A0 13:28 +0100, Michael Scherer a =C3=A9= crit : pace
taken. =20 =20 RESOLUTION: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted m= ysql using 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon how = to resolve this. =20 =20 =20 wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or exten= d again the existing one. =20 Happened again today. =20 I still maintain that we should move to a dedicated hosting on the ph= x2 cluster :) ( or request quota extensions ). =20 whether if we'll migrate or not, that will take time and won't happen u= ntil the ILO issue will be resolved. for now we need to extend our volume there ASAP, cause it will happen a= gain soon, not too many log files are there to delete/gzip. =20 Agreed. We did request already a quota extension, IIRC, so I guess that was not sufficient ?
Filled the form to get more, will tell you when I have a response. --=20 Michael Scherer Open Source and Standards, Sysadmin --=-xEIq5DPZS/xUU1TKd3vy Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUhaKVAAoJEE89Wa+PrSK9HB8P/3+2WYWt5RH5b/wHojev6JrL AQGe/sTGsWGI4o/HJ5yOqqFwz8nJGFr3t2s2J0QF6vsq6+VaPgEGDrd5F/O3F/V6 RSVO31TIE0VPNdwJXDnLilDYqmKFpagbDecbXPBVjeeZuyudpqSQ8rEn7OgcQahJ cTQBuVlsOMFqdvsK3hpwpw6DG+Yi39OuBPb/t02UvJhlgJ4VACTWjW7N1q7HLpjz 7Ld30HjxmCIFJAiUT+aHLGibrSrTbps91/itt6dZG5KsBCXeiRAXW2lprPr3472+ OLUcI+nXs1t0c6xQKsijESHD1SRQ0s0n3l9QbkWa1cfaHk/FOzSM+Cuuqc/8ANUX gQeNzTS6rxsi+Z+pDYsr4t0DHZRxDLLcOSMJSlO2TGRt8HRiXopQc0xrjNQG8unP 5cpFXx7QWNFNs7hLQ+961mCGbdWpViHUCqldF8c337XA/Lq+DjmGEuYjipCYPR1J 4amAo4Omge6qU4PHL9f3svEhCVrsZzFb7FOoL29ooh19CF92cW2hVvAclGHYkSWi /b9fTgVSNyP72Qhe6HkoVAbSv9gXLUS7NAgLIhXcwRgTEcyIfAmTWTrDYA3EIi5+ rSAMEU/BIcGkBQ2MpB+nbknbnmVxo9KAP5MfxitAXfwwE9HLhWzRq3RJtUbUm7VL iuVQW/tgt0nPjcr33tK4 =oROx -----END PGP SIGNATURE----- --=-xEIq5DPZS/xUU1TKd3vy--
----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 8:07:33 AM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le lundi 08 décembre 2014 à 13:28 +0100, Michael Scherer a écrit :
Le lundi 08 décembre 2014 à 06:23 -0500, Eyal Edri a écrit :
----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 1:03:01 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le dimanche 07 décembre 2014 à 03:25 -0500, Eyal Edri a écrit :
fyi,
ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
REASON: ======= after logging into the openshift gear, seems like there was 99.9% space taken.
RESOLUTION: =========== deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql using 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon how to resolve this.
wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or extend again the existing one.
Happened again today.
I still maintain that we should move to a dedicated hosting on the phx2 cluster :) ( or request quota extensions ).
whether if we'll migrate or not, that will take time and won't happen until the ILO issue will be resolved. for now we need to extend our volume there ASAP, cause it will happen again soon, not too many log files are there to delete/gzip.
Agreed. We did request already a quota extension, IIRC, so I guess that was not sufficient ?
Filled the form to get more, will tell you when I have a response. -- Michael Scherer Open Source and Standards, Sysadmin
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Went out again this afternoon (12/9), but managed to squeeze out a little more disk space running rhc app tidy. (Thanks to Jason and Justin for the assist!) If there is any way I can help expedite more disk quota, please let me know! BKP -- Brian Proffitt Community Liaison oVirt Open Source and Standards, Red Hat - http://community.redhat.com Phone: +1 574 383 9BKP IRC: bkp @ OFTC
----- Original Message -----
From: "Brian Proffitt" <bproffit@redhat.com> To: infra@ovirt.org Sent: Wednesday, December 10, 2014 1:00:48 AM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 8:07:33 AM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le lundi 08 décembre 2014 à 13:28 +0100, Michael Scherer a écrit :
Le lundi 08 décembre 2014 à 06:23 -0500, Eyal Edri a écrit :
----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 1:03:01 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le dimanche 07 décembre 2014 à 03:25 -0500, Eyal Edri a écrit :
fyi,
ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
REASON: ======= after logging into the openshift gear, seems like there was 99.9% space taken.
RESOLUTION: =========== deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql using 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon how to resolve this.
wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or extend again the existing one.
Happened again today.
I still maintain that we should move to a dedicated hosting on the phx2 cluster :) ( or request quota extensions ).
whether if we'll migrate or not, that will take time and won't happen until the ILO issue will be resolved. for now we need to extend our volume there ASAP, cause it will happen again soon, not too many log files are there to delete/gzip.
Agreed. We did request already a quota extension, IIRC, so I guess that was not sufficient ?
Filled the form to get more, will tell you when I have a response. -- Michael Scherer Open Source and Standards, Sysadmin
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Went out again this afternoon (12/9), but managed to squeeze out a little more disk space running rhc app tidy. (Thanks to Jason and Justin for the assist!)
If there is any way I can help expedite more disk quota, please let me know!
i think the best short term solution will be to get more space from openshift, right now i think we stand at less than 10G, so doubling it will give us some air to think how we best want to progress with the wiki.
BKP
-- Brian Proffitt
Community Liaison oVirt Open Source and Standards, Red Hat - http://community.redhat.com Phone: +1 574 383 9BKP IRC: bkp @ OFTC _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
--=-KwVlcyHAzz14qMuwuiIh Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le mardi 09 d=C3=A9cembre 2014 =C3=A0 18:00 -0500, Brian Proffitt a =C3=A9c= rit :
=20 ----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 8:07:33 AM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14] =20 Le lundi 08 d=C3=A9cembre 2014 =C3=A0 13:28 +0100, Michael Scherer a = =C3=A9crit :
Le lundi 08 d=C3=A9cembre 2014 =C3=A0 06:23 -0500, Eyal Edri a =C3=A9= crit :
=20 ----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 1:03:01 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14] =20 Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri a= =C3=A9crit :
fyi, =20 ovirt.org wiki page was down today from 01:33:07 IST till 10:22= IST. =20 REASON: =3D=3D=3D=3D=3D=3D=3D after logging into the openshift gear, seems like there was 99.= 9% space taken. =20 =20 RESOLUTION: =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restart= ed mysql using 'ctl_app restart'. space is still not great - at 84%, so we'll need to think soon = how to resolve this. =20 =20 =20 wiki is up now and seems to working well. infra - we should look into migrating to a new gear maybe? or e= xtend again the existing one. =20 Happened again today. =20 I still maintain that we should move to a dedicated hosting on th= e phx2 cluster :) ( or request quota extensions ). =20 whether if we'll migrate or not, that will take time and won't happ= en until the ILO issue will be resolved. for now we need to extend our volume there ASAP, cause it will happ= en again soon, not too many log files are there to delete/gzip. =20 Agreed. We did request already a quota extension, IIRC, so I guess th= at was not sufficient ? =20 Filled the form to get more, will tell you when I have a response. -- Michael Scherer Open Source and Standards, Sysadmin =20
Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra =20 =20 =20 Went out again this afternoon (12/9), but managed to squeeze out a little= more disk space running rhc app tidy. (Thanks to Jason and Justin for the = assist!) =20 If there is any way I can help expedite more disk quota, please let me kn= ow!
It take a few days. Last time we did ask, it was request =3D> go to marketing for approval =3D> go to the sysadmins team for action.=20 As long as this in the marketing side, I have no visibility ( but can ask ). On the openshift sysadmin side, I know where to ping :) --=20 Michael Scherer Open Source and Standards, Sysadmin --=-KwVlcyHAzz14qMuwuiIh Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUiBVEAAoJEE89Wa+PrSK9szsP/2BjlWzPlpU3QAQOa9VKNm+/ xhsLKrSeL0+81RuXL+tC8Br9e/nodNuLpNAssZBPrWQ7v/OCqAIAJq44l6r8m91K M9OK0g/niPIDNbJVMl73qJEVWUcVBelL/EKN+TRn6yuRJP3ItQE8W/NuTZURMHPw TjfLbjHJvHTLJ3YCDneZXKHO4JZzdwLRO+f63JQ94MskCrN2JFw71OPaOKMIUOgO tGiZFb8+oJlle5Qmr/11J4Ppk8FM6q58pPCwFY04bVrC9z8gEQNdYgTie642E3/a tzRZf4Xn9od5ZbfJLMLw4lq+NB+JGwsp9ODILAWUsWq/01IBrI6lqEDKHLI+65kC dBMT+auJPt5n1GOjnheQ98GPC9bvsg1w4XUn/2nGOssRhKtA70B8ugLrTMJPNoFQ h41BNzOPOA4plO5eTVzTNWjnsapBkIOpY2OsAfM9tyw46TM/pmDCtUx4apXrtXKS /XwCdCch35PgyY/5i2jdsAcrzBc603E5h4pLpi8hD5HaQMxK/95bJOBcWRRr5mZK dTcE1kv0RSp5ceTWh9bVfDZBbsTyioNNhJB6zLNyEd+o7pEbBxZygjHVP+eDNWxN yAIFlcMZTI5L78T2pHMRG6NC9f0LWXsJwSDXq65438kHJSeALarqkCPtSisqkBmm pn7lSFlLYsVzO2L8Ubzo =WkEn -----END PGP SIGNATURE----- --=-KwVlcyHAzz14qMuwuiIh--
Le mardi 09 d=C3=A9cembre 2014 =C3=A0 18:00 -0500, Brian Proffitt a =C3= =A9crit :
=20 ----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 8:07:33 AM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14] =20 Le lundi 08 d=C3=A9cembre 2014 =C3=A0 13:28 +0100, Michael Scherer a = =C3=A9crit :
Le lundi 08 d=C3=A9cembre 2014 =C3=A0 06:23 -0500, Eyal Edri a =C3= =A9crit :
=20 ----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Monday, December 8, 2014 1:03:01 PM Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14] =20 Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri= a =C3=A9crit : > fyi, >=20 > ovirt.org wiki page was down today from 01:33:07 IST till 10:= 22 IST. >=20 > REASON: > =3D=3D=3D=3D=3D=3D=3D > after logging into the openshift gear, seems like there was 9= 9.9% > space > taken. >=20 >=20 > RESOLUTION: > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and resta= rted > mysql > using 'ctl_app restart'. > space is still not great - at 84%, so we'll need to think soo= n how to > resolve this. >=20 >=20 >=20 > wiki is up now and seems to working well. > infra - we should look into migrating to a new gear maybe? or= extend > again > the existing one. =20 Happened again today. =20 I still maintain that we should move to a dedicated hosting on =
cluster :) ( or request quota extensions ). =20 whether if we'll migrate or not, that will take time and won't ha=
until the ILO issue will be resolved. for now we need to extend our volume there ASAP, cause it will ha=
again soon, not too many log files are there to delete/gzip. =20 Agreed. We did request already a quota extension, IIRC, so I guess =
--=-cLaelN2dV7AgGqf5GLNT Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le mercredi 10 d=C3=A9cembre 2014 =C3=A0 10:41 +0100, Michael Scherer a =C3= =A9crit : the phx2 ppen ppen that
was not sufficient ? =20 Filled the form to get more, will tell you when I have a response. -- Michael Scherer Open Source and Standards, Sysadmin =20
Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra =20 =20 =20 Went out again this afternoon (12/9), but managed to squeeze out a litt= le more disk space running rhc app tidy. (Thanks to Jason and Justin for th= e assist!) =20 If there is any way I can help expedite more disk quota, please let me = know! =20 It take a few days. Last time we did ask, it was request =3D> go to marketing for approval =3D> go to the sysadmins team for action.=20 =20 As long as this in the marketing side, I have no visibility ( but can ask ). On the openshift sysadmin side, I know where to ping :)
So, today it was full again, and openshift people did contact me. We increased the quota, and while on it, I did the dump/restore needed to clean the ever increasing ibdata1 file ( http://www.percona.com/blog/2013/08/20/why-is-the-ibdata1-file-continuous= ly-growing-in-mysql/ ). It took a few hours to process, and since things would not be fun without it, my internet connexion started to behave badly. But now, we are down to 6G out of 10G, so wait and see. --=20 Michael Scherer Open Source and Standards, Sysadmin --=-cLaelN2dV7AgGqf5GLNT Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJUicFOAAoJEE89Wa+PrSK9kuYP/j6YuUHq47zIdwY2lyHZlV6U 6XY8pibh6R/RrhJG7ZcRPyZIzwduNMof2UnRt0HHH/iLvQuqAt4MzPWjm5pMEja0 EufHaxsBOJZdBL5IW0S5vSuGZLSelPPPAwkoBYovg74iSHRRKMFyvmo4/o+sGadw teVwVPRO5cNMuSL1u7w4EbJN3aSZpFBdnUyd9ICldycSdWRw88qOZLIYFYmNO0ES ZzlnPm/4YnvW39Ch2G1KIAmmJv0DJwFRYRq4PNKaCTR4MZYne22u+8MqjDI7nLDE 9lTmLd57aduVj/cHOCT/GUKmFQOmo5L93/BvSOWtpb9v/vkljbpfQ7WjaAkkYt/L TTk12PjDnZHq/b9/zNeJhorqMIakJfYvZ3KS3sInJLtTimdWP3zXi80mFt1FYrbq iPD66thS3t6yp0qBktX4tv+XtmbE15Bc3HpfOkb8sqHwBok6PRAJQA4mIZUpFyVS NK+E010qUxmzBwxjeon7ja2GSBEMJOfEtlDKKaLX96IoaL3bxOGaIInWEELKshwA X7XpOb+nizMBQO8EYE80dvtV8sSFPsBNVc5G+l+8taeoeOp7iytNa2j5ub0Z1Ad/ 1CVsCcNTPt43MTP6staQC2R21TMVW8jLYg1XMrIlucheulzrzum51pWJOjqEBxTQ E4N+eFkrDVoF09aU2vll =FxDo -----END PGP SIGNATURE----- --=-cLaelN2dV7AgGqf5GLNT--
participants (3)
-
Brian Proffitt -
Eyal Edri -
Michael Scherer