Wiki and Mailing Lists Outage -- 2012-11-14

[Mike Burns]

We experienced an outage today in both the wiki and the mailing lists.  

* Wiki content was available throughout the outage, but attempts to
login or edit received an error message about requiring cookies to be
enabled.  
* All mails to the mailing  list failed to show up on the lists, but
also did not return rejection messages.

Cause: 

This was caused by an "Out of Space" error on the host running both of
these services.  A temporary workaround was put in place to get both
services up and running again.  


Action Taken:

Remove the oldest gerrit backup (600MB)
Remove some older non-functional ovirt-node-iso images and rpms from the
releases (source remains there)

Long term solution:

Migrating these services away from a single host onto hosted solutions
(OpenShift, AlterWay).

Current Situation:

Wiki is back up and running, login works as expected
Lists are processing the backlog of emails since the outage began.
At this time, it does not appear that any mail was lost due to the
outage.


Thanks for the patience and understanding

Mike