migration of services from linode server to phx2 lab
Hi, We're finally able to start migrating services from linode01 to a production VMs on the phx lab. [1] I'd like to go over all the services and decide which ones we want to migrate and what else exists on linode: Dns entries: resources.ovirt.org, lists.ovirt.org Services: mailman ovirt irc bot spamassin postfix ( aliases for ovirt.org ) package repositories mirrorlist Storage: gerrit backups (still there?) download for rpm and code (repos) For each we should decide if we're migrating or just installing a new server with new configuration on phx. optimally we'd like to do it with relevant puppet classes per service. Please share your thoughts about each service and feel free to volunteer and help our with migrating one of these. [1] hopefully we'll also have the ilos working by next week, so another reason to migrate. -- Eyal Edri oVirt infra team irc: eedri
--4XBiSqiiHCIj6zoM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 04/28, Eyal Edri wrote:
Hi, =20 We're finally able to start migrating services from linode01 to a product= ion VMs on the phx lab. [1] I'd like to go over all the services and decide which ones we want to mig= rate and what else exists on linode: =20 Dns entries: resources.ovirt.org, lists.ovirt.org Services: mailman=20
I'm not familiar with mailman, so for me it might be the most complicated (email usually is also very sensitive to ip changes)
ovirt irc bot=20
This might be the simplest
spamassin =20 postfix ( aliases for ovirt.org ) package repositories
fairly simple to move too, but careful with all the release crons and setups around the publishers
mirrorlist
the mirrorlist is just a basic cgi script, also easy to move
=20 Storage: gerrit backups (still there?)
this was migrated to jenkins.ovirt.org due to space issues iirc
download for rpm and code (repos)
the code is on gerrit itself
=20 =20 For each we should decide if we're migrating or just installing a new ser= ver with new configuration on phx. optimally we'd like to do it with relevant puppet classes per ser= vice. =20 Please share your thoughts about each service and feel free to volunteer = and help our with migrating one of these. =20 =20 [1] hopefully we'll also have the ilos working by next week, so another r= eason to migrate. =20 --=20 Eyal Edri oVirt infra team irc: eedri =20 _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
--=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --4XBiSqiiHCIj6zoM Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVP0t5AAoJEEBxx+HSYmnDVJoH/A3ea8EkN9gET+y+9Xxc+2Bv L4t2HsHLATF9GgSZYL6t/S+/b8YTNUKwQzBSIliDF0wQpZP9UfjWLsyREysWeY3x 30bSEo6PVUkdb4M9LsCFrBIGvyyu0Du2QLayDTHVAjr+4uBFUfZrwXnRzsTR6mQR BulZTv2qWsbub4kswMphoxqQXlSqWzey2+EUemQj0tRKt/NIi2b1kM+/PYFA9fvn kSn/4uFkTqnsgGEji20zCx9lD4ILLhW+J/DaVj0Wd4Bg/gcmp4hlqps35Iz0kWfA 5XB1gACnyW5m3HClW0Kmb5v3s/MP3cnKEDi+sZ8Qyj5+bEKgNxOpGd7fYptIR4k= =ZgEl -----END PGP SIGNATURE----- --4XBiSqiiHCIj6zoM--
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, Between myself (original linode01.ovirt.org admin) and Michael (misc, aka knower-of-stuff), what can we do to get us off this Linode instance? From what I can tell, at least Mailman is running from there. If we still need service failover, can we switch to another Red Hat-provided service such as an OpenShift or OpenStack instance? Regards, - - Karsten On 04/28/2015 01:57 AM, David Caro wrote:
On 04/28, Eyal Edri wrotea cente:
Hi,
We're finally able to start migrating services from linode01 to a production VMs on the phx lab. [1] I'd like to go over all the services and decide which ones we want to migrate and what else exists on linode:
Dns entries: resources.ovirt.org, lists.ovirt.org Services: mailman
I'm not familiar with mailman, so for me it might be the most complicated (email usually is also very sensitive to ip changes)
ovirt irc bot
This might be the simplest
spamassin postfix ( aliases for ovirt.org ) package repositories
fairly simple to move too, but careful with all the release crons and setups around the publishers
mirrorlist
the mirrorlist is just a basic cgi script, also easy to move
tack Storage: gerrit backups (still there?)
this was migrated to jenkins.ovirt.org due to space issues iirc
download for rpm and code (repos)
the code is on gerrit itself
For each we should decide if we're migrating or just installing a new server with new configuration on phx. optimally we'd like to do it with relevant puppet classes per service.
Please share your thoughts about each service and feel free to volunteer and help our with migrating one of these.
[1] hopefully we'll also have the ilos working by next week, so another reason to migrate.
-- Eyal Edri oVirt infra team irc: eedri
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
- -- Karsten 'quaid' Wade .^\ CentOS Doer of Stuff http://TheOpenSourceWay.org \ http://community.redhat.com @quaid (identi.ca/twitter/IRC) \v' gpg: AD0E0C41 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iEYEARECAAYFAlV/mAAACgkQ2ZIOBq0ODEFuWACgtbBiG3nOOYjKWHME+b9rKsCn azIAoJ/fJFdIh/JWp58/rg/f5zOffDJS =y8aP -----END PGP SIGNATURE-----
----- Original Message -----
From: "Karsten Wade" <kwade@redhat.com> To: infra@ovirt.org Sent: Tuesday, June 16, 2015 6:29:04 AM Subject: Re: migration of services from linode server to phx2 lab
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Hi,
Between myself (original linode01.ovirt.org admin) and Michael (misc, aka knower-of-stuff), what can we do to get us off this Linode instance? From what I can tell, at least Mailman is running from there.
If we still need service failover, can we switch to another Red Hat-provided service such as an OpenShift or OpenStack instance?
Hi Karsten, I know this has been taking for too long, but unfortunately there are many reasons and obstacles that prevented us from moving, which i'll explain below, while there is a risk right now of moving services to PHX, but i think we can try. Blocking items: - were missing more hypervisors to the production DC ( we finally got them installed last week, and are now in final stages of bringing them up) - storage issue - currently NFS storage is quite slow, we are testing moving to a mixed mode of local + nfs for the jenkins slaves, though might be that the production service won't get affected too much - worth a try. - lack of monitoring for servers, which might add risk if we hit an issue. there are some other issues, but not blocking imo. Here is what needs to be done (per service to migrate off linode), and we'd appreciate any help given. 1. create new VM on production DC in PHX (assign dns/ip/etc.. ) 2. create puppet manifests to manage that vm, so it will be easy to reproduce it and maintain it 3. install the relevant software on it (for e.g mailman/ircbot/etc...) 4. test to see it works 5. do the actual migration (downtime of existing service, and bringing up the new one) (might have left out some of the steps, but thats in general) if you or Michael can take ownership on migrating one of the services off linode, with help from us, it will surely hasten the process. Eyal.
Regards,
- - Karsten
On 04/28/2015 01:57 AM, David Caro wrote:
On 04/28, Eyal Edri wrotea cente:
Hi,
We're finally able to start migrating services from linode01 to a production VMs on the phx lab. [1] I'd like to go over all the services and decide which ones we want to migrate and what else exists on linode:
Dns entries: resources.ovirt.org, lists.ovirt.org Services: mailman
I'm not familiar with mailman, so for me it might be the most complicated (email usually is also very sensitive to ip changes)
ovirt irc bot
This might be the simplest
spamassin postfix ( aliases for ovirt.org ) package repositories
fairly simple to move too, but careful with all the release crons and setups around the publishers
mirrorlist
the mirrorlist is just a basic cgi script, also easy to move
tack Storage: gerrit backups (still there?)
this was migrated to jenkins.ovirt.org due to space issues iirc
download for rpm and code (repos)
the code is on gerrit itself
For each we should decide if we're migrating or just installing a new server with new configuration on phx. optimally we'd like to do it with relevant puppet classes per service.
Please share your thoughts about each service and feel free to volunteer and help our with migrating one of these.
[1] hopefully we'll also have the ilos working by next week, so another reason to migrate.
-- Eyal Edri oVirt infra team irc: eedri
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
- -- Karsten 'quaid' Wade .^\ CentOS Doer of Stuff http://TheOpenSourceWay.org \ http://community.redhat.com @quaid (identi.ca/twitter/IRC) \v' gpg: AD0E0C41 -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux)
iEYEARECAAYFAlV/mAAACgkQ2ZIOBq0ODEFuWACgtbBiG3nOOYjKWHME+b9rKsCn azIAoJ/fJFdIh/JWp58/rg/f5zOffDJS =y8aP -----END PGP SIGNATURE----- _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Supervisor, RHEV CI EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
--=-Be7faPQe1vIfreVF1FcP Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Le mardi 16 juin 2015 =C3=A0 03:38 -0400, Eyal Edri a =C3=A9crit : >=20 > ----- Original Message ----- > > From: "Karsten Wade" <kwade@redhat.com> > > To: infra@ovirt.org > > Sent: Tuesday, June 16, 2015 6:29:04 AM > > Subject: Re: migration of services from linode server to phx2 lab > >=20 > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > >=20 > > Hi, > >=20 > > Between myself (original linode01.ovirt.org admin) and Michael (misc, > > aka knower-of-stuff), what can we do to get us off this Linode > > instance? From what I can tell, at least Mailman is running from there. > >=20 > > If we still need service failover, can we switch to another Red > > Hat-provided service such as an OpenShift or OpenStack instance? >=20 > Hi Karsten, > I know this has been taking for too long, but unfortunately there are man= y reasons > and obstacles that prevented us from moving, which i'll explain below, > while there is a risk right now of moving services to PHX, but i think we= can try. >=20 > Blocking items: > - were missing more hypervisors to the production DC ( we finally got th= em installed last week, and are now in final stages of bringing them up) > - storage issue - currently NFS storage is quite slow, we are testing mo= ving to a mixed mode of local + nfs for the jenkins slaves, > though might be that the production service won't get = affected too much - worth a try. > - lack of monitoring for servers, which might add risk if we hit an issu= e. >=20 > there are some other issues, but not blocking imo. >=20 > Here is what needs to be done (per service to migrate off linode), and we= 'd appreciate any help given. > 1. create new VM on production DC in PHX (assign dns/ip/etc.. ) > 2. create puppet manifests to manage that vm, so it will be easy to repro= duce it and maintain it > 3. install the relevant software on it (for e.g mailman/ircbot/etc...) > 4. test to see it works > 5. do the actual migration (downtime of existing service, and bringing up= the new one) So, last time I did look, the puppet setup was a bit "unfinished" rather overkill ( using r10k to deploy count as overkill when you do not have stage/prod/qa, at least for me ). Did stuff changed, or should first some effort be made to fix/automate/improve the puppet setup ? --=20 Michael Scherer Sysadmin, Community Infrastructure and Platform, OSAS --=-Be7faPQe1vIfreVF1FcP Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAABAgAGBQJVgMDXAAoJEE89Wa+PrSK9qo4P/jyKeWkR59sVLdz/CxcAzJXa U3DRgLcp3Ejesv06Jd6OGwWVtFJCxWZX83ME13nqIWUWABk+GIVFJn69qgb/sK+n CP41CUHlobTsQMndSeirdPUMQZ9TJLxPjxJyTEmiPYyGulM75dQ8pk/g71oNb8JR GwpHtyxp5Lr87pyezMrHUyA/mlVm9PBQrV2W/3b9YVo7hZjwK35k33wA46A05Een kTDN/PpW33j6w2giskUERG9PpaSUt5+vG2qTfMjc/37XDFm+nLpjmLdW7lZWJ6+H OwA0U21VydImSzn3o+erng4qPDtSCqB7S+wF0E+lxpBIjKAePI2u793t8YszYEh9 Y2g3oacS2/qQ7LYk+/VrM8fvn21IsHUkURl3aVfYsWBHv40KdCqufJ4lH72VPBVE 9ZrOkfwtlPnPqHvhh5c9BlpfGAGcKRir2nGQU1vVSS1Fk8bl21QAbDpFbKMfn82v unDNMLr4oT0It1ueAscupUaIE0jkVF0IJDV2eXeRhvTXFj9rN7BMuAdxI+bem3Z5 Ic71cxdH1xYNO6InboQTj93p6m0jYLzByIv8gjuEK4CcJybMk+ytNeLj7sCAlrzI lnJ7eJxGoW1kW/LdCIDWQFzKt+Lx+ImND/FuDNMs7p1E2K3iyAtDZwDL94uDDib+ bHHKmOKUt85nv/mJg+NU =aZNh -----END PGP SIGNATURE----- --=-Be7faPQe1vIfreVF1FcP--
--cz6wLo+OExbGG7q/ Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 06/17, Michael Scherer wrote: > Le mardi 16 juin 2015 =E0 03:38 -0400, Eyal Edri a =E9crit : > >=20 > > ----- Original Message ----- > > > From: "Karsten Wade" <kwade@redhat.com> > > > To: infra@ovirt.org > > > Sent: Tuesday, June 16, 2015 6:29:04 AM > > > Subject: Re: migration of services from linode server to phx2 lab > > >=20 > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA1 > > >=20 > > > Hi, > > >=20 > > > Between myself (original linode01.ovirt.org admin) and Michael (misc, > > > aka knower-of-stuff), what can we do to get us off this Linode > > > instance? From what I can tell, at least Mailman is running from ther= e. > > >=20 > > > If we still need service failover, can we switch to another Red > > > Hat-provided service such as an OpenShift or OpenStack instance? > >=20 > > Hi Karsten, > > I know this has been taking for too long, but unfortunately there are m= any reasons > > and obstacles that prevented us from moving, which i'll explain below, > > while there is a risk right now of moving services to PHX, but i think = we can try. > >=20 > > Blocking items: > > - were missing more hypervisors to the production DC ( we finally got = them installed last week, and are now in final stages of bringing them up) > > - storage issue - currently NFS storage is quite slow, we are testing = moving to a mixed mode of local + nfs for the jenkins slaves, > > though might be that the production service won't ge= t affected too much - worth a try. > > - lack of monitoring for servers, which might add risk if we hit an is= sue. > >=20 > > there are some other issues, but not blocking imo. > >=20 > > Here is what needs to be done (per service to migrate off linode), and = we'd appreciate any help given. > > 1. create new VM on production DC in PHX (assign dns/ip/etc.. ) > > 2. create puppet manifests to manage that vm, so it will be easy to rep= roduce it and maintain it > > 3. install the relevant software on it (for e.g mailman/ircbot/etc...) > > 4. test to see it works > > 5. do the actual migration (downtime of existing service, and bringing = up the new one) >=20 > So, last time I did look, the puppet setup was a bit "unfinished" rather > overkill ( using r10k to deploy count as overkill when you do not have > stage/prod/qa, at least for me ). Did stuff changed, or should first > some effort be made to fix/automate/improve the puppet setup ? Still using only one branch, but hiera is enabled and ready to start getting used. It still needs some refactor but if we don't use it we will never do it... so better using it imo >=20 >=20 > --=20 > Michael Scherer > Sysadmin, Community Infrastructure and Platform, OSAS >=20 > _______________________________________________ > Infra mailing list > Infra@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra --=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro@redhat.com Web: www.redhat.com RHT Global #: 82-62605 --cz6wLo+OExbGG7q/ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJVgSCjAAoJEEBxx+HSYmnD96QH/3WWTUFe7MF9iKqpMahKyJke QCz962SHCaGYdB1I53rYA9sSKF2LUzGhEOpg4VHVMn+FsrV70Ahlo/WLWXKJzkg3 xRmztYAE69xpAAPPbqhOSBJWPEjrQMOICdlFV4ChSLrVBJt2FN44m2GM1wMSYysW ZcSzGs379pOrFU88gSO+AC1gfUcbQdOKtdOZzMYIurt5spUVplhBQDOzTfUn3fFL VaydNokNNpMbqdTYnWn7i/XqF6o6zfmSeeMitexSfETn9B/kqQ5PzK4kpFYBHvor B4krtKbrt+dB9T+HjjV8eiU5z2uJSYuewSFWM+Y5rmwmov9VR2IYvgeSk3RfO5k= =wREK -----END PGP SIGNATURE----- --cz6wLo+OExbGG7q/--
participants (4)
-
David Caro -
Eyal Edri -
Karsten Wade -
Michael Scherer