2:25 a.m.
fyi,
ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
REASON:
=======
after logging into the openshift gear, seems like there was 99.9% space taken.
RESOLUTION:
===========
deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql using 'ctl_app
restart'.
space is still not great - at 84%, so we'll need to think soon how to resolve this.
wiki is up now and seems to working well.
infra - we should look into migrating to a new gear maybe? or extend again the existing
one.
Eyal.
5:03 a.m.
--=-oU9cwyL9Z523xHYR89Fk
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri a =C3=A9cri=
t :
fyi,
=20
ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
=20
REASON:
=3D=3D=3D=3D=3D=3D=3D
after logging into the openshift gear, seems like there was 99.9% space t=
aken.
=20
=20
RESOLUTION:
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql u=
sing
'ctl_app restart'.
space is still not great - at 84%, so we'll need to think soon
how to res=
olve this.
=20
=20
=20
wiki is up now and seems to working well.
infra - we should look into migrating to a new gear maybe? or extend agai=
n the
existing one.
Happened again today.
I still maintain that we should move to a dedicated hosting on the phx2
cluster :)
( or request quota extensions ).=20
--=20
Michael Scherer
Open Source and Standards, Sysadmin
--=-oU9cwyL9Z523xHYR89Fk
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAABAgAGBQJUhYVlAAoJEE89Wa+PrSK9Rh0P/3L058LYqQs6w6+QoMi0Kwmt
oi6CKzELQ27l0uJjCbgOuLJioIcSWQ9NeeFlV6tqw36C8oCIZx08u1aNfTQEgWDk
8347ipv/TG93SSHzO0rtcU+nsdTGYasplHZRk3RKchYP6cGBfYZVsuSDhZavY8tk
imz+36AFnp2/59za66jHjm9M9FcX+YVVLLNbLcBijFRBpEhWtiLkaMXKC9cfjet4
org4EHe/YbiifsFFNQ4hqK0HIE8ZfpeMATt8/MD0G9xP+EcwLv+0mkH5IpV3T1p1
+6L6yMukj/RxZ9cKmR5K4eJm7DnwgnFWm5biLbwsQLVSDRqYTX/66iz8KAXB/gAc
KqUB/o0OjekuBGvZW9BmsmcgPAaMjU+ZKQGFn9qrnvle0YpoD1zHJR2gcSE35s23
ZGRJkaSLPlBUk2O1b6Dq0Lh3y6oRamQtbgsac5mPEVXeYlBc5I4WbQ1QK8dCbG+2
T+oCvc8hnGmV00rW/o3kcZxHx7yHq8xosxvWCdYgz+pHFHNllufjenIpM44fa6ez
ogDdDyS1vVPGpen/Y1JS/JtFzZWLnAHNfjQN/FZpIpki3h+Tw2YR2p9VZi/zrZMB
ZwJui+YUaxvC/1qJ8MhudMTEV/NI9qhvUrkMD7LsUsCzw0StR4x98fg5B6gYMe95
9fCH+ymaGWWeYWZt8zI+
=XU16
-----END PGP SIGNATURE-----
--=-oU9cwyL9Z523xHYR89Fk--
5:23 a.m.
----- Original Message -----
From: "Michael Scherer" <mscherer(a)redhat.com>
To: infra(a)ovirt.org
Sent: Monday, December 8, 2014 1:03:01 PM
Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le dimanche 07 décembre 2014 à 03:25 -0500, Eyal Edri a écrit :
> fyi,
>
> ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
>
> REASON:
> =======
> after logging into the openshift gear, seems like there was 99.9% space
> taken.
>
>
> RESOLUTION:
> ===========
> deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql
> using 'ctl_app restart'.
> space is still not great - at 84%, so we'll need to think soon how to
> resolve this.
>
>
>
> wiki is up now and seems to working well.
> infra - we should look into migrating to a new gear maybe? or extend again
> the existing one.
Happened again today.
I still maintain that we should move to a dedicated hosting on the phx2
cluster :)
( or request quota extensions ).
whether if we'll migrate or not, that will take time and won't happen until the
ILO issue will be resolved.
for now we need to extend our volume there ASAP, cause it will happen again soon, not too
many log files
are there to delete/gzip.
e.
--
Michael Scherer
Open Source and Standards, Sysadmin
_______________________________________________
Infra mailing list
Infra(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra
6:28 a.m.
--=-1kQcmNimmI7H3ptkbG5x
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Le lundi 08 d=C3=A9cembre 2014 =C3=A0 06:23 -0500, Eyal Edri a =C3=A9crit :
=20
----- Original Message -----
> From: "Michael Scherer" <mscherer(a)redhat.com>
> To: infra(a)ovirt.org
> Sent: Monday, December 8, 2014 1:03:01 PM
> Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
>=20
> Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri a =C3=
=A9crit
:
> > fyi,
> >=20
> > ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
> >=20
> > REASON:
> > =3D=3D=3D=3D=3D=3D=3D
> > after logging into the openshift gear, seems like there was 99.9% spa=
ce
> > taken.
> >=20
> >=20
> > RESOLUTION:
> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mys=
ql
> > using 'ctl_app restart'.
> > space is still not great - at 84%, so we'll need to think soon how to
> > resolve this.
> >=20
> >=20
> >=20
> > wiki is up now and seems to working well.
> > infra - we should look into migrating to a new gear maybe? or extend =
again
> > the existing one.
>=20
> Happened again today.
>=20
> I still maintain that we should move to a dedicated hosting on the phx2
> cluster :)
> ( or request quota extensions ).
=20
whether if we'll migrate or not, that will take time and won't happen unt=
il the ILO issue will be resolved.
for now we need to extend our volume there ASAP, cause it will happen
aga=
in soon, not too many log files
are there to delete/gzip.
Agreed. We did request already a quota extension, IIRC, so I guess that
was not sufficient ?
--=20
Michael Scherer
Open Source and Standards, Sysadmin
--=-1kQcmNimmI7H3ptkbG5x
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAABAgAGBQJUhZl+AAoJEE89Wa+PrSK9WYwP/jn5Nu3O9qDynnn+XnBEcXam
ZQ4I/h6NL5ctxl4Yev0lqGmI50IPnq7rpfhEhWmKKjHjSwS1tDFPxuijljVzLqQd
ojsA2aYfIlpDMA1GdJbLI2/EoaOoCuGL7TCcbbROfjzkZqKZIXQTG04mpgFQRhPX
gQiaFOiz4B8M3iNpfAhwdlo77x8J1xtIHlX082fbRA2ulepvb7rmaNUFHC7OyInJ
+STcR5v556Eb1IME1aP47Q4JjbsJFsvJR67tqvlHK4J+tKVLWS+O60aHghAdO05P
DgQU03JsNo9Le6Lx0c7mEvWktymgm6KE3qXeQNFdH7pteVxR3h2rEb+77e2aGKdK
PYp64ow/YxiVZ3vLNuRgbyY4q2M3WLNzhs8mPNYgMveskUAouj8hOkR371RC9PV7
TE4kLIGxF5MH3tTn6iCq0x8TjgA2tdenK9t+cK5tjoFvWs63T+84iTF9i0Em1LFK
Ap0OcSa1hvGSA3PesAyKfUZ+dvqOWfmny7c73HmUWkRDXGO6Ne1aZZcG+hnTZ18R
WwoklvGjH4TnZcR+x6teSxAjE/xp1SVnwkM2iyuQEjTLwGX/fexC2e54KWTAa7IZ
Xgjo9QZZUjGJw+1K7cNlO45kSEZgWe6v3y903pFj54ggRcmvarhRIJEuq9+5tieL
mUkZ/sWf9Y0AJ6ryUj+h
=Erck
-----END PGP SIGNATURE-----
--=-1kQcmNimmI7H3ptkbG5x--
6:53 a.m.
----- Original Message -----
From: "Michael Scherer" <mscherer(a)redhat.com>
To: "Eyal Edri" <eedri(a)redhat.com>
Cc: infra(a)ovirt.org
Sent: Monday, December 8, 2014 2:28:46 PM
Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le lundi 08 décembre 2014 à 06:23 -0500, Eyal Edri a écrit :
>
> ----- Original Message -----
> > From: "Michael Scherer" <mscherer(a)redhat.com>
> > To: infra(a)ovirt.org
> > Sent: Monday, December 8, 2014 1:03:01 PM
> > Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
> >
> > Le dimanche 07 décembre 2014 à 03:25 -0500, Eyal Edri a écrit :
> > > fyi,
> > >
> > > ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
> > >
> > > REASON:
> > > =======
> > > after logging into the openshift gear, seems like there was 99.9% space
> > > taken.
> > >
> > >
> > > RESOLUTION:
> > > ===========
> > > deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted mysql
> > > using 'ctl_app restart'.
> > > space is still not great - at 84%, so we'll need to think soon how to
> > > resolve this.
> > >
> > >
> > >
> > > wiki is up now and seems to working well.
> > > infra - we should look into migrating to a new gear maybe? or extend
> > > again
> > > the existing one.
> >
> > Happened again today.
> >
> > I still maintain that we should move to a dedicated hosting on the phx2
> > cluster :)
> > ( or request quota extensions ).
>
> whether if we'll migrate or not, that will take time and won't happen until
> the ILO issue will be resolved.
> for now we need to extend our volume there ASAP, cause it will happen again
> soon, not too many log files
> are there to delete/gzip.
Agreed. We did request already a quota extension, IIRC, so I guess that
was not sufficient ?
looking at the mysql db, it seems to be taking most of the space (9.8GB data file).
so i guess no choice but to extend it more.
--
Michael Scherer
Open Source and Standards, Sysadmin
_______________________________________________
Infra mailing list
Infra(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra
7:07 a.m.
--=-xEIq5DPZS/xUU1TKd3vy
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Le lundi 08 d=C3=A9cembre 2014 =C3=A0 13:28 +0100, Michael Scherer a =C3=A9=
crit :
Le lundi 08 d=C3=A9cembre 2014 =C3=A0 06:23 -0500, Eyal Edri a
=C3=A9crit=
:
>=20
> ----- Original Message -----
> > From: "Michael Scherer" <mscherer(a)redhat.com>
> > To: infra(a)ovirt.org
> > Sent: Monday, December 8, 2014 1:03:01 PM
> > Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
> >=20
> > Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri a =C3=
=A9crit :
> > > fyi,
> > >=20
> > > ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST=
.
> > >=20
> > > REASON:
> > > =3D=3D=3D=3D=3D=3D=3D
> > > after logging into the openshift gear, seems like there was 99.9% s=
pace
> > > taken.
> > >=20
> > >=20
> > > RESOLUTION:
> > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > > deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted m=
ysql
> > > using 'ctl_app restart'.
> > > space is still not great - at 84%, so we'll need to think soon how =
to
> > > resolve this.
> > >=20
> > >=20
> > >=20
> > > wiki is up now and seems to working well.
> > > infra - we should look into migrating to a new gear maybe? or exten=
d again
> > > the existing one.
> >=20
> > Happened again today.
> >=20
> > I still maintain that we should move to a dedicated hosting on the ph=
x2
> > cluster :)
> > ( or request quota extensions ).
>=20
> whether if we'll migrate or not, that will take time and won't happen u=
ntil the ILO issue will be resolved.
> for now we need to extend our volume there ASAP, cause it will
happen a=
gain soon, not too many log files
> are there to delete/gzip.
=20
Agreed. We did request already a quota extension, IIRC, so I guess that
was not sufficient ?
Filled the form to get more, will tell you when I have a response.
--=20
Michael Scherer
Open Source and Standards, Sysadmin
--=-xEIq5DPZS/xUU1TKd3vy
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAABAgAGBQJUhaKVAAoJEE89Wa+PrSK9HB8P/3+2WYWt5RH5b/wHojev6JrL
AQGe/sTGsWGI4o/HJ5yOqqFwz8nJGFr3t2s2J0QF6vsq6+VaPgEGDrd5F/O3F/V6
RSVO31TIE0VPNdwJXDnLilDYqmKFpagbDecbXPBVjeeZuyudpqSQ8rEn7OgcQahJ
cTQBuVlsOMFqdvsK3hpwpw6DG+Yi39OuBPb/t02UvJhlgJ4VACTWjW7N1q7HLpjz
7Ld30HjxmCIFJAiUT+aHLGibrSrTbps91/itt6dZG5KsBCXeiRAXW2lprPr3472+
OLUcI+nXs1t0c6xQKsijESHD1SRQ0s0n3l9QbkWa1cfaHk/FOzSM+Cuuqc/8ANUX
gQeNzTS6rxsi+Z+pDYsr4t0DHZRxDLLcOSMJSlO2TGRt8HRiXopQc0xrjNQG8unP
5cpFXx7QWNFNs7hLQ+961mCGbdWpViHUCqldF8c337XA/Lq+DjmGEuYjipCYPR1J
4amAo4Omge6qU4PHL9f3svEhCVrsZzFb7FOoL29ooh19CF92cW2hVvAclGHYkSWi
/b9fTgVSNyP72Qhe6HkoVAbSv9gXLUS7NAgLIhXcwRgTEcyIfAmTWTrDYA3EIi5+
rSAMEU/BIcGkBQ2MpB+nbknbnmVxo9KAP5MfxitAXfwwE9HLhWzRq3RJtUbUm7VL
iuVQW/tgt0nPjcr33tK4
=oROx
-----END PGP SIGNATURE-----
--=-xEIq5DPZS/xUU1TKd3vy--
5 p.m.
----- Original Message -----
From: "Michael Scherer" <mscherer(a)redhat.com>
To: infra(a)ovirt.org
Sent: Monday, December 8, 2014 8:07:33 AM
Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
Le lundi 08 décembre 2014 à 13:28 +0100, Michael Scherer a écrit :
> Le lundi 08 décembre 2014 à 06:23 -0500, Eyal Edri a écrit :
> >
> > ----- Original Message -----
> > > From: "Michael Scherer" <mscherer(a)redhat.com>
> > > To: infra(a)ovirt.org
> > > Sent: Monday, December 8, 2014 1:03:01 PM
> > > Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
> > >
> > > Le dimanche 07 décembre 2014 à 03:25 -0500, Eyal Edri a écrit :
> > > > fyi,
> > > >
> > > > ovirt.org wiki page was down today from 01:33:07 IST till 10:22 IST.
> > > >
> > > > REASON:
> > > > =======
> > > > after logging into the openshift gear, seems like there was 99.9%
> > > > space
> > > > taken.
> > > >
> > > >
> > > > RESOLUTION:
> > > > ===========
> > > > deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restarted
> > > > mysql
> > > > using 'ctl_app restart'.
> > > > space is still not great - at 84%, so we'll need to think soon
how to
> > > > resolve this.
> > > >
> > > >
> > > >
> > > > wiki is up now and seems to working well.
> > > > infra - we should look into migrating to a new gear maybe? or extend
> > > > again
> > > > the existing one.
> > >
> > > Happened again today.
> > >
> > > I still maintain that we should move to a dedicated hosting on the phx2
> > > cluster :)
> > > ( or request quota extensions ).
> >
> > whether if we'll migrate or not, that will take time and won't happen
> > until the ILO issue will be resolved.
> > for now we need to extend our volume there ASAP, cause it will happen
> > again soon, not too many log files
> > are there to delete/gzip.
>
> Agreed. We did request already a quota extension, IIRC, so I guess that
> was not sufficient ?
Filled the form to get more, will tell you when I have a response.
--
Michael Scherer
Open Source and Standards, Sysadmin
_______________________________________________
Infra mailing list
Infra(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra
Went out again this afternoon (12/9), but managed to squeeze out a little more disk space
running rhc app tidy. (Thanks to Jason and Justin for the assist!)
If there is any way I can help expedite more disk quota, please let me know!
BKP
--
Brian Proffitt
Community Liaison
oVirt
Open Source and Standards, Red Hat - http://community.redhat.com
Phone: +1 574 383 9BKP
IRC: bkp @ OFTC
2:15 a.m.
----- Original Message -----
From: "Brian Proffitt" <bproffit(a)redhat.com>
To: infra(a)ovirt.org
Sent: Wednesday, December 10, 2014 1:00:48 AM
Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
----- Original Message -----
> From: "Michael Scherer" <mscherer(a)redhat.com>
> To: infra(a)ovirt.org
> Sent: Monday, December 8, 2014 8:07:33 AM
> Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
>
> Le lundi 08 décembre 2014 à 13:28 +0100, Michael Scherer a écrit :
> > Le lundi 08 décembre 2014 à 06:23 -0500, Eyal Edri a écrit :
> > >
> > > ----- Original Message -----
> > > > From: "Michael Scherer" <mscherer(a)redhat.com>
> > > > To: infra(a)ovirt.org
> > > > Sent: Monday, December 8, 2014 1:03:01 PM
> > > > Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
> > > >
> > > > Le dimanche 07 décembre 2014 à 03:25 -0500, Eyal Edri a écrit :
> > > > > fyi,
> > > > >
> > > > > ovirt.org wiki page was down today from 01:33:07 IST till 10:22
> > > > > IST.
> > > > >
> > > > > REASON:
> > > > > =======
> > > > > after logging into the openshift gear, seems like there was
99.9%
> > > > > space
> > > > > taken.
> > > > >
> > > > >
> > > > > RESOLUTION:
> > > > > ===========
> > > > > deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and
restarted
> > > > > mysql
> > > > > using 'ctl_app restart'.
> > > > > space is still not great - at 84%, so we'll need to think
soon how
> > > > > to
> > > > > resolve this.
> > > > >
> > > > >
> > > > >
> > > > > wiki is up now and seems to working well.
> > > > > infra - we should look into migrating to a new gear maybe? or
> > > > > extend
> > > > > again
> > > > > the existing one.
> > > >
> > > > Happened again today.
> > > >
> > > > I still maintain that we should move to a dedicated hosting on the
> > > > phx2
> > > > cluster :)
> > > > ( or request quota extensions ).
> > >
> > > whether if we'll migrate or not, that will take time and won't
happen
> > > until the ILO issue will be resolved.
> > > for now we need to extend our volume there ASAP, cause it will happen
> > > again soon, not too many log files
> > > are there to delete/gzip.
> >
> > Agreed. We did request already a quota extension, IIRC, so I guess that
> > was not sufficient ?
>
> Filled the form to get more, will tell you when I have a response.
> --
> Michael Scherer
> Open Source and Standards, Sysadmin
>
> _______________________________________________
> Infra mailing list
> Infra(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>
Went out again this afternoon (12/9), but managed to squeeze out a little
more disk space running rhc app tidy. (Thanks to Jason and Justin for the
assist!)
If there is any way I can help expedite more disk quota, please let me know!
i think the best short term solution will be to get more space from openshift,
right now i think we stand at less than 10G, so doubling it will give us some air to
think
how we best want to progress with the wiki.
BKP
--
Brian Proffitt
Community Liaison
oVirt
Open Source and Standards, Red Hat - http://community.redhat.com
Phone: +1 574 383 9BKP
IRC: bkp @ OFTC
_______________________________________________
Infra mailing list
Infra(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra
3:41 a.m.
--=-KwVlcyHAzz14qMuwuiIh
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Le mardi 09 d=C3=A9cembre 2014 =C3=A0 18:00 -0500, Brian Proffitt a =C3=A9c=
rit :
=20
----- Original Message -----
> From: "Michael Scherer" <mscherer(a)redhat.com>
> To: infra(a)ovirt.org
> Sent: Monday, December 8, 2014 8:07:33 AM
> Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
>=20
> Le lundi 08 d=C3=A9cembre 2014 =C3=A0 13:28 +0100, Michael Scherer a =
=C3=A9crit :
> > Le lundi 08 d=C3=A9cembre 2014 =C3=A0 06:23 -0500, Eyal
Edri a =C3=A9=
crit :
> > >=20
> > > ----- Original Message -----
> > > > From: "Michael Scherer" <mscherer(a)redhat.com>
> > > > To: infra(a)ovirt.org
> > > > Sent: Monday, December 8, 2014 1:03:01 PM
> > > > Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
> > > >=20
> > > > Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri a=
=C3=A9crit :
> > > > > fyi,
> > > > >=20
> > > > > ovirt.org wiki page was down today from 01:33:07 IST till 10:22=
IST.
> > > > >=20
> > > > > REASON:
> > > > > =3D=3D=3D=3D=3D=3D=3D
> > > > > after logging into the openshift gear, seems like there was 99.=
9%
> > > > > space
> > > > > taken.
> > > > >=20
> > > > >=20
> > > > > RESOLUTION:
> > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > > > > deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and restart=
ed
> > > > > mysql
> > > > > using 'ctl_app restart'.
> > > > > space is still not great - at 84%, so we'll need to think
soon =
how to
> > > > > resolve this.
> > > > >=20
> > > > >=20
> > > > >=20
> > > > > wiki is up now and seems to working well.
> > > > > infra - we should look into migrating to a new gear maybe? or e=
xtend
> > > > > again
> > > > > the existing one.
> > > >=20
> > > > Happened again today.
> > > >=20
> > > > I still maintain that we should move to a dedicated hosting on th=
e phx2
> > > > cluster :)
> > > > ( or request quota extensions ).
> > >=20
> > > whether if we'll migrate or not, that will take time and won't
happ=
en
> > > until the ILO issue will be resolved.
> > > for now we need to extend our volume there ASAP, cause it will happ=
en
> > > again soon, not too many log files
> > > are there to delete/gzip.
> >=20
> > Agreed. We did request already a quota extension, IIRC, so I guess th=
at
> > was not sufficient ?
>=20
> Filled the form to get more, will tell you when I have a response.
> --
> Michael Scherer
> Open Source and Standards, Sysadmin
>=20
> _______________________________________________
> Infra mailing list
> Infra(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>=20
=20
=20
Went out again this afternoon (12/9), but managed to squeeze out a little=
more
disk space running rhc app tidy. (Thanks to Jason and Justin for the =
assist!)
=20
If there is any way I can help expedite more disk quota, please let me kn=
ow!
It take a few days. Last time we did ask, it was request =3D> go to
marketing for approval =3D> go to the sysadmins team for action.=20
As long as this in the marketing side, I have no visibility ( but can
ask ). On the openshift sysadmin side, I know where to ping :)
--=20
Michael Scherer
Open Source and Standards, Sysadmin
--=-KwVlcyHAzz14qMuwuiIh
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAABAgAGBQJUiBVEAAoJEE89Wa+PrSK9szsP/2BjlWzPlpU3QAQOa9VKNm+/
xhsLKrSeL0+81RuXL+tC8Br9e/nodNuLpNAssZBPrWQ7v/OCqAIAJq44l6r8m91K
M9OK0g/niPIDNbJVMl73qJEVWUcVBelL/EKN+TRn6yuRJP3ItQE8W/NuTZURMHPw
TjfLbjHJvHTLJ3YCDneZXKHO4JZzdwLRO+f63JQ94MskCrN2JFw71OPaOKMIUOgO
tGiZFb8+oJlle5Qmr/11J4Ppk8FM6q58pPCwFY04bVrC9z8gEQNdYgTie642E3/a
tzRZf4Xn9od5ZbfJLMLw4lq+NB+JGwsp9ODILAWUsWq/01IBrI6lqEDKHLI+65kC
dBMT+auJPt5n1GOjnheQ98GPC9bvsg1w4XUn/2nGOssRhKtA70B8ugLrTMJPNoFQ
h41BNzOPOA4plO5eTVzTNWjnsapBkIOpY2OsAfM9tyw46TM/pmDCtUx4apXrtXKS
/XwCdCch35PgyY/5i2jdsAcrzBc603E5h4pLpi8hD5HaQMxK/95bJOBcWRRr5mZK
dTcE1kv0RSp5ceTWh9bVfDZBbsTyioNNhJB6zLNyEd+o7pEbBxZygjHVP+eDNWxN
yAIFlcMZTI5L78T2pHMRG6NC9f0LWXsJwSDXq65438kHJSeALarqkCPtSisqkBmm
pn7lSFlLYsVzO2L8Ubzo
=WkEn
-----END PGP SIGNATURE-----
--=-KwVlcyHAzz14qMuwuiIh--
10:07 a.m.
--=-cLaelN2dV7AgGqf5GLNT
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Le mercredi 10 d=C3=A9cembre 2014 =C3=A0 10:41 +0100, Michael Scherer a =C3=
=A9crit :
Le mardi 09 d=C3=A9cembre 2014 =C3=A0 18:00 -0500, Brian Proffitt a
=C3=
=A9crit :
>=20
> ----- Original Message -----
> > From: "Michael Scherer" <mscherer(a)redhat.com>
> > To: infra(a)ovirt.org
> > Sent: Monday, December 8, 2014 8:07:33 AM
> > Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
> >=20
> > Le lundi 08 d=C3=A9cembre 2014 =C3=A0 13:28 +0100, Michael Scherer a =
=C3=A9crit :
> > > Le lundi 08 d=C3=A9cembre 2014 =C3=A0 06:23 -0500,
Eyal Edri a =C3=
=A9crit :
> > > >=20
> > > > ----- Original Message -----
> > > > > From: "Michael Scherer" <mscherer(a)redhat.com>
> > > > > To: infra(a)ovirt.org
> > > > > Sent: Monday, December 8, 2014 1:03:01 PM
> > > > > Subject: Re: [OUTAGE] ovirt.org wiki outage report [07/12/14]
> > > > >=20
> > > > > Le dimanche 07 d=C3=A9cembre 2014 =C3=A0 03:25 -0500, Eyal Edri=
a =C3=A9crit :
> > > > > > fyi,
> > > > > >=20
> > > > > > ovirt.org wiki page was down today from 01:33:07 IST till
10:=
22 IST.
> > > > > >=20
> > > > > > REASON:
> > > > > > =3D=3D=3D=3D=3D=3D=3D
> > > > > > after logging into the openshift gear, seems like there was
9=
9.9%
> > > > > > space
> > > > > > taken.
> > > > > >=20
> > > > > >=20
> > > > > > RESOLUTION:
> > > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
> > > > > > deleted some logs from $OPENSHIFT_MYSQL_DB_LOG_DIR, and
resta=
rted
> > > > > > mysql
> > > > > > using 'ctl_app restart'.
> > > > > > space is still not great - at 84%, so we'll need to
think soo=
n how to
> > > > > > resolve this.
> > > > > >=20
> > > > > >=20
> > > > > >=20
> > > > > > wiki is up now and seems to working well.
> > > > > > infra - we should look into migrating to a new gear maybe?
or=
extend
> > > > > > again
> > > > > > the existing one.
> > > > >=20
> > > > > Happened again today.
> > > > >=20
> > > > > I still maintain that we should move to a dedicated hosting on =
the phx2
> > > > > cluster :)
> > > > > ( or request quota extensions ).
> > > >=20
> > > > whether if we'll migrate or not, that will take time and
won't ha=
ppen
> > > > until the ILO issue will be resolved.
> > > > for now we need to extend our volume there ASAP, cause it will ha=
ppen
> > > > again soon, not too many log files
> > > > are there to delete/gzip.
> > >=20
> > > Agreed. We did request already a quota extension, IIRC, so I guess =
that
> > > was not sufficient ?
> >=20
> > Filled the form to get more, will tell you when I have a response.
> > --
> > Michael Scherer
> > Open Source and Standards, Sysadmin
> >=20
> > _______________________________________________
> > Infra mailing list
> > Infra(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/infra
> >=20
>=20
>=20
> Went out again this afternoon (12/9), but managed to squeeze out a litt=
le
more disk space running rhc app tidy. (Thanks to Jason and Justin for th=
e assist!)
>=20
> If there is any way I can help expedite more disk quota, please let me =
know!
=20
It take a few days. Last time we did ask, it was request =3D> go to
marketing for approval =3D> go to the sysadmins team for action.=20
=20
As long as this in the marketing side, I have no visibility ( but can
ask ). On the openshift sysadmin side, I know where to ping :)
So, today it was full again, and openshift people did contact me. We
increased the quota, and while on it, I did the dump/restore needed to
clean the ever increasing ibdata1 file
( http://www.percona.com/blog/2013/08/20/why-is-the-ibdata1-file-continuous=
ly-growing-in-mysql/ ).
It took a few hours to process, and since things would not be fun
without it, my internet connexion started to behave badly.
But now, we are down to 6G out of 10G, so wait and see.
--=20
Michael Scherer
Open Source and Standards, Sysadmin
--=-cLaelN2dV7AgGqf5GLNT
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)
iQIcBAABAgAGBQJUicFOAAoJEE89Wa+PrSK9kuYP/j6YuUHq47zIdwY2lyHZlV6U
6XY8pibh6R/RrhJG7ZcRPyZIzwduNMof2UnRt0HHH/iLvQuqAt4MzPWjm5pMEja0
EufHaxsBOJZdBL5IW0S5vSuGZLSelPPPAwkoBYovg74iSHRRKMFyvmo4/o+sGadw
teVwVPRO5cNMuSL1u7w4EbJN3aSZpFBdnUyd9ICldycSdWRw88qOZLIYFYmNO0ES
ZzlnPm/4YnvW39Ch2G1KIAmmJv0DJwFRYRq4PNKaCTR4MZYne22u+8MqjDI7nLDE
9lTmLd57aduVj/cHOCT/GUKmFQOmo5L93/BvSOWtpb9v/vkljbpfQ7WjaAkkYt/L
TTk12PjDnZHq/b9/zNeJhorqMIakJfYvZ3KS3sInJLtTimdWP3zXi80mFt1FYrbq
iPD66thS3t6yp0qBktX4tv+XtmbE15Bc3HpfOkb8sqHwBok6PRAJQA4mIZUpFyVS
NK+E010qUxmzBwxjeon7ja2GSBEMJOfEtlDKKaLX96IoaL3bxOGaIInWEELKshwA
X7XpOb+nizMBQO8EYE80dvtV8sSFPsBNVc5G+l+8taeoeOp7iytNa2j5ub0Z1Ad/
1CVsCcNTPt43MTP6staQC2R21TMVW8jLYg1XMrIlucheulzrzum51pWJOjqEBxTQ
E4N+eFkrDVoF09aU2vll
=FxDo
-----END PGP SIGNATURE-----
--=-cLaelN2dV7AgGqf5GLNT--
3432
days inactive
3436
days old
9 comments
3 participants
participants (3)
-
Brian Proffitt -
Eyal Edri -
Michael Scherer