--=-32WVZRI+BqMRGUDcK7Zk Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le dimanche 08 juin 2014 =C3=A0 02:55 -0400, Eyal Edri a =C3=A9crit :
=20 ----- Original Message -----
From: "Michael Scherer" <mscherer@redhat.com> To: infra@ovirt.org Sent: Friday, June 6, 2014 2:29:44 PM Subject: infra security update =20 Hi, =20 Due to CVE on openssl and on kernel, I did upgrade various piece of the infrastructure ( foreman, lists, stats, monitoring ), which implied a few reboots ( due to kernel lagging behind, which is not that great wit= h local root exploit ). As this is friday and I assumed most of the Tel Aviv office was not working, i hope this kept the disruption to a minimum. However, if something is broken, please tell it so we can fix. =20 This also got me thinking. In order to bring a bit more order, what about having a fixed schedule for upgrade ? =20 =20 =20 =20 In my previous position, we were doing that once per month ( except during end of quarter freeze ), with mandatory reboot ( cause if something do not boot, you want to know it when you have a planned outage, not when everyone is running around updating stuff ). Fedora ha= s a rather complex procedure to decide what to upgrade, hilighted on http://infrastructure.fedoraproject.org/infra/docs/massupgrade.txt =20 So we could adopt a schedule ( once per month, unless there is somethin= g critical, in which case we do it ASAP, with warning on the list and irc ). =20 =20 =20 =20 The schedule should of course take in account "business need", which is "release schedule of ovirt". =20 So what about "first friday of the month, unless exception" ? =20 And by update, i mean "yum upgrade -y". Cleaning the list of repo on various servers is also IMHO another task to discuss, to make sure the task can be safely executed. ( having something like mcollective/ansible/func is also needed, but that's more a convenience than a requirement at this stage ). =20 we use 'fabric' for these kind of stuff in redhat, so we might be able to= =20 use that for oVirt as well.
wel, IT use taboot/func, and Fedora use ansible ( some part of IT also do look at ansible ). Openshift use mcollective. basically, what we need is a tool to enforce configuration ( which is puppet ), and one to execute remote command with a specific schedule and orchestration.=20 For a simple task, they would likely all work fine, so maybe we should start to list what we want, and then see which one fullfill the need and some constraints. based on the issue I faced in the past, and my understanding of the sitaution, I would propose the following list of criteria: - be packaged for Fedora/Centos ( agent and server ) - packaged in epel/main repo (so we can be sure there is a proper update policy) - optionally, for Debian/Ubuntu (cf the need of having jenkins builder on others platform) - the simple, the better (I am not fond of having to deploy amqp for mcollective) - know by at least a few people in the current team (the more, the better) - able to orchestrate, ie, run tasks on X server on after the other - still maintained (ie, unlike func) - not moving too fast I guess we for now want to have it run the tasks of upgrades, but in the event of needing more ( regenerate ssl certificates, restart some specific service ), I guess we may need more. We should also take a look at the infra deployment, ie do we have stuff over ssh ( ansible, fabric ), which mean giving a lot of access (basically, root), with likely a need for using something like a restricted user and a bastion. Or there is mcollective, salt, func, where you have a 2nd communication channel and fine grained ACL
+1 for monthly maintenance window,Friday sounds good, since most of the u= sers are from tlv office. we can keep sunday as optional also, if a critical server should be up on= a certain friday. so either tlv office of non-tlv can performan the outages. =20 also, worth adding it to the calendar, as a monthly maintenance outage, where we can update servers like jenkins/gerrit/formean etc... we can use either ovirt cal [1] or open a new infra cal for that.=20
thought, we should map pkg we want to keep latest, and ensure them via pu=
Also, what happen if we forget. And how do we make sure that more than 1 people can do the job. ppet,
while the maintenance windows will be used for reboots and downtimes. =20 we also should update that info on the wiki once ready [2] =20 [1] https://www.google.com/calendar/ical/ppqtk46u9cglj7l987ruo2l0f8%40gro= up.calendar.google.com/public/basic.ics [2] http://www.ovirt.org/Infra
/me still have to enable https on the wiki --=20 Michael Scherer Open Source and Standards, Sysadmin --=-32WVZRI+BqMRGUDcK7Zk Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJTlaicAAoJEE89Wa+PrSK9lzAQAJBBaPD3q7dPfdTEhRkA5zz5 rpSdGl5bvhmqQkXSHFALuXUohVLkB6NJELNKYD9L/7WmngxJ+M1uuCtMMlnncOSa jsuvt7SyqhRg7uCAKyqjatch/8mFpFNwixU5WoBVndiAS1MUWazu5pctSx8Z0fCY aJwa2s4RW5GIBUSzZKagWoXsKKhkriqGSAPSi3L4ukGkQmPehvf/vCCPoHBKV+Ea +kpV6wCY/k/waOzJLG2NNIKDqk21CtZsbCTeI9LK+5OIGg55T0aO/dUHH4pxDTL7 gkJnL+SSRNVsOXME6KSTsujG5Ki+0smm3dPRBgHrdcrEhAMHJuGf9klfosgq/0FG n9pqfVcyTxrC2cs86zcNP8/6MngsBbYYCz2ZlD9+ztOvAPVPkOXi7+3730wH/z5g jnw/+9L/q869tdXuQwiCwhb3e4qykyeQ+4uMBXBnwgSx24I8LTWQhsW2vdVeAFMO xylaUaR1Pg7Y+3fYUGLQMl9Ykz+3CtYDhTAj+R61V7s64JIXSzilfdVQBGX1oYGv 5SD0yluiYp5iCuuVRDJ2o8Hogqjr37zHglsrSUx7TzfHQVNMfeHJ/ZtIfnIBvaoX DOEp2IkpGNBNkgd1uuYkWOGmaCNwJCmB2WiKZ1UOk1DkrYa7nI4UXR45CW0rRYxL Rs/bAXYUq0BRyLQKFSoP =zuJi -----END PGP SIGNATURE----- --=-32WVZRI+BqMRGUDcK7Zk--