Unplanned outage :: www.ovirt.org :: 2012-12-06

This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig47D4BE28EC6A9598175BB169 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable We just had an unplanned outage of the www.ovirt.org site. The site was unavailable for about three minutes. =3D=3D Details =3D=3D I got a new theme.css file from Garrett to make the topbar a bit more obvious to logged-in users. But I didn't want to push it and restart the app, which is the default OpenShift behavior. OpenShift has a hot deploy feature, set by putting a marker (touching a file) in .openshift/markers/hot_deploy. When the marker is in place during a 'git push', OpenShift is not supposed to restart the service. Ideally, just having the marker there is enough so that the very push itself doesn't restart anything. Unfortunately, things didn't work as expected. When I pushed the change, the output indicated that it stopped the service despite the hot_deploy marker, and then didn't restart the service because of the hot_deploy marker. I had to remove the marker and do another push to get it to start the service. git push Counting objects: 22, done. Delta compression using up to 4 threads. Compressing objects: 100% (14/14), done. Writing objects: 100% (14/14), 1.80 KiB, done. Total 14 (delta 10), reused 0 (delta 0) remote: restart_on_add=3Dfalse remote: Waiting for stop to finish remote: Done remote: restart_on_add=3Dfalse remote: ~/git/wiki.git ~/git/wiki.git remote: ~/git/wiki.git remote: Running .openshift/action_hooks/pre_build remote: Running .openshift/action_hooks/build remote: Running .openshift/action_hooks/deploy remote: Database found, skipping build remote: hot_deploy_added=3Dfalse remote: App will not be started due to presence of hot_deploy marker remote: Running .openshift/action_hooks/post_deploy To ssh://847edb45aea84198838f915be6faa066@wiki-ovirt.rhcloud.com/~/git/wiki.= git/ 85d7d77..d9a1670 master -> master =3D=3D Affected services =3D=3D=3D * www.ovirt.org =3D=3D=3D Not-affected services =3D=3D=3D * lists.ovirt.org * resources.ovirt.org =3D=3D Future plans =3D=3D Make the hot deploy work during a planned outage window rather than during the middle of the work day. When hot deploy is working, we should have fewer occasions of restarting the website just because we want to push a CSS, config, etc. change. --=20 Karsten 'quaid' Wade, Sr. Analyst - Community Growth http://TheOpenSourceWay.org .^\ http://community.redhat.com @quaid (identi.ca/twitter/IRC) \v' gpg: AD0E0C41 --------------enig47D4BE28EC6A9598175BB169 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" --------------enig47D4BE28EC6A9598175BB169--