Wiki and Mailing Lists Outage -- 2012-11-14

[Doron Fediuck]

Thanks Mike!
I suggest to have a cron alerting for no-space issues.

----- Original Message -----
> From: "Mike Burns" <mburns at redhat.com>
> To: "board" <board at ovirt.org>, "infra" <infra at ovirt.org>, "users" <users at ovirt.org>
> Sent: Wednesday, November 14, 2012 3:31:11 PM
> Subject: Wiki and Mailing Lists Outage -- 2012-11-14
> 
> We experienced an outage today in both the wiki and the mailing
> lists.
> 
> * Wiki content was available throughout the outage, but attempts to
> login or edit received an error message about requiring cookies to be
> enabled.
> * All mails to the mailing  list failed to show up on the lists, but
> also did not return rejection messages.
> 
> Cause:
> 
> This was caused by an "Out of Space" error on the host running both
> of
> these services.  A temporary workaround was put in place to get both
> services up and running again.
> 
> 
> Action Taken:
> 
> Remove the oldest gerrit backup (600MB)
> Remove some older non-functional ovirt-node-iso images and rpms from
> the
> releases (source remains there)
> 
> Long term solution:
> 
> Migrating these services away from a single host onto hosted
> solutions
> (OpenShift, AlterWay).
> 
> Current Situation:
> 
> Wiki is back up and running, login works as expected
> Lists are processing the backlog of emails since the outage began.
> At this time, it does not appear that any mail was lost due to the
> outage.
> 
> 
> Thanks for the patience and understanding
> 
> Mike
> 
> _______________________________________________
> Infra mailing list
> Infra at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/infra
>