PHX updates - hypervisors in Jenkins datacenter
Hi all, As part of the upgrade process to 3.6 I will be rebuilding all hypervisors to CentOS7, and I'm starting with the Jenkins Datacenter. For this, I am evacuating slave VMs by shutting them down and restarting in the new cluster with CentOS hosts. I'm currently working on ovirt-srv04 so please don't schedule any VMs on that host until it's rebuilt. I found several VMs running on that host which are not listed on Jenkins so I can't mark them offline there for the update period: fc21-vm01.phx.ovirt.org el6-vm06.phx.ovirt.org If anyone knows what these VMs are used for - please tell me, otherwise I will just restart them as well. Also this VM is present on this hypervisor: artifactory.phx.ovirt.org Is it used by the slaves? Can it be shut down for a few minutes? I'm not sure I will be able to live migrate it even as the F20 hosts we have in the old cluster have issues with live migration even to other F20 hosts. Regards, Evgheni Dereveanchin
--Y5rl02BVI9TCfPar Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 06/02 05:40, Evgheni Dereveanchin wrote:
Hi all, =20 As part of the upgrade process to 3.6 I will be rebuilding all hypervisors to CentOS7, and I'm starting with the Jenkins Datacenter. For this, I am evacuating slave VMs by shutting them down and restarting in the new cluster with CentOS hosts. =20 I'm currently working on ovirt-srv04 so please don't schedule any VMs on that host until it's rebuilt. =20 I found several VMs running on that host which are not listed on Jenkins so I can't mark them offline there for the update period: =20 fc21-vm01.phx.ovirt.org el6-vm06.phx.ovirt.org =20 If anyone knows what these VMs are used for - please tell me, otherwise I will just restart them as well. =20 Also this VM is present on this hypervisor: artifactory.phx.ovirt.org =20 Is it used by the slaves? Can it be shut down for a few minutes? I'm not sure I will be able to live migrate it even as the F20 hosts we have in the old cluster have issues with live migration even to other F20 hosts.
This vm is quite critical actually, I'd move it to the prod cluster. It's used by all the engine jobs, to pull the jar dependencies from, and al= so to publish some ovirt jars. The jobs should fallback to the official maven repos if it's not available,= but some might not use that fallback config. I'd say it's ok to take it out for= a few minutes, just send an email to the devel list so they are not surprised= if any jobs fail during that time.
=20 Regards,=20 Evgheni Dereveanchin=20 =20 _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
--=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro@redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605 --Y5rl02BVI9TCfPar Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJXT//rAAoJEEBxx+HSYmnDWdoH/R/Tg1nYTWBAVE1LgkG7xHdS IlLG+IrMNgnuaLV9KKiXE5+NhBQs9BHlTCtnnQyVJt4RkUwejMzNRY74AhxVobhg 3cZTHAuwljYG+aVREtJmU7nX40bFAy/uPqhHMpr/AfyKISzGypa/VYnsqjc4hMdY T1yH48yUwRKPcn8mRvumRcs/9/fyTGmMw3y0veWZBC4Ygk5EkY8Ry79Iz5zoYHHd T7x8PBPCYL0t9tHpT1UF+/hGg6oZwk97qr+KSquVVKGx61gM798vY3BvgNTGrBad nuxr91v2nes73kHQDX/8tJ+HIzwi/hb5Q4AZ0XRHKp5zco9rW5lFxw+Tx8uec2o= =CQUY -----END PGP SIGNATURE----- --Y5rl02BVI9TCfPar--
Thanks David and Nadav! Will move all remaining slaves from this host and if I won't manage to live migrate artifactory I'll announce devel-list and reboot it. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Nadav Goldin" <ngoldin@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Thursday, 2 June, 2016 12:01:20 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter On Thu, Jun 2, 2016 at 12:40 PM, Evgheni Dereveanchin <ederevea@redhat.com> wrote:
fc21-vm01.phx.ovirt.org el6-vm06.phx.ovirt.org
those are on jenkins-old.ovirt.org
I found several VMs running on that host which are not listed on Jenkins so I can't mark them offline there for the update period:
fc21-vm01.phx.ovirt.org el6-vm06.phx.ovirt.org
If anyone knows what these VMs are used for - please tell me, otherwise I will just restart them as well.
These VMs are used on the old Jenkins: http://jenkins-old.ovirt.org/computer/ Since there are a few jobs still running there, please make sure to also look at it when you're moving slaves around. Regards, Barak. -- Barak Korren bkorren@redhat.com RHEV-CI Team
Status update: ovirt-srv04 was rebuilt with EL6. It's added to the Jenkins DC and pending network bonding configuration, after which the process will be repeated for the remaining F20 hypervisors: ovirt-srv05 ovirt-srv06 ovirt-srv07 Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Thursday, 2 June, 2016 11:40:14 AM Subject: PHX updates - hypervisors in Jenkins datacenter Hi all, As part of the upgrade process to 3.6 I will be rebuilding all hypervisors to CentOS7, and I'm starting with the Jenkins Datacenter. For this, I am evacuating slave VMs by shutting them down and restarting in the new cluster with CentOS hosts. I'm currently working on ovirt-srv04 so please don't schedule any VMs on that host until it's rebuilt. I found several VMs running on that host which are not listed on Jenkins so I can't mark them offline there for the update period: fc21-vm01.phx.ovirt.org el6-vm06.phx.ovirt.org If anyone knows what these VMs are used for - please tell me, otherwise I will just restart them as well. Also this VM is present on this hypervisor: artifactory.phx.ovirt.org Is it used by the slaves? Can it be shut down for a few minutes? I'm not sure I will be able to live migrate it even as the F20 hosts we have in the old cluster have issues with live migration even to other F20 hosts. Regards, Evgheni Dereveanchin _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <ederevea@redhat.com> כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
Status update: ovirt-srv12 and ovirt-srv13 updated to CentOS 7.2, will now rebuild ovirt-srv05 which is currently in Maintenance and has no VMs running. Please ignore monitoring alerts for this host if any. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "Barak Korren" <bkorren@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:32:25 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <ederevea@redhat.com> כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Status update: ovirt-srv05 rebuilt and back in production. Network bonding fixed on ovirt-srv17 and ovirt-srv18. ovirt-srv11 updated to CentOS 7.2. Currently bonding is being fixed on ovirt-srv01, ovirt-srv11 and ovirt-srv14. After that is complete, will proceed with rebuilding ovirt-srv06 and ovirt-srv07 which are the last F20 hosts in the Jenkins cluster. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 4:42:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Status update: ovirt-srv12 and ovirt-srv13 updated to CentOS 7.2, will now rebuild ovirt-srv05 which is currently in Maintenance and has no VMs running. Please ignore monitoring alerts for this host if any. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "Barak Korren" <bkorren@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:32:25 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <ederevea@redhat.com> כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
Status update: All VMs in Jenkins DC now running on CentOS hosts, all hypervisors there were updated/rebuilt. NIC bonding was enabled across all hosts as well. The only three remaining F20 hypervisors are the ones in the Production cluster and they also are attached to the Hosted Engine DC. Move of production VMs to CentOS hosts is in progress, just 8 left to go (HostedEngine being one of them). Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Tuesday, 7 June, 2016 3:54:47 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Status update: ovirt-srv05 rebuilt and back in production. Network bonding fixed on ovirt-srv17 and ovirt-srv18. ovirt-srv11 updated to CentOS 7.2. Currently bonding is being fixed on ovirt-srv01, ovirt-srv11 and ovirt-srv14. After that is complete, will proceed with rebuilding ovirt-srv06 and ovirt-srv07 which are the last F20 hosts in the Jenkins cluster. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 4:42:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Status update: ovirt-srv12 and ovirt-srv13 updated to CentOS 7.2, will now rebuild ovirt-srv05 which is currently in Maintenance and has no VMs running. Please ignore monitoring alerts for this host if any. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "Barak Korren" <bkorren@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:32:25 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <ederevea@redhat.com> כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
I'm back to the office and moving forward with the upgrade. Will now re-provision ovirt-srv01 which is part of the HostedEngine cluster and was the first host of the environment. Now backing up its configs just to be sure. There is 6 VMs still running on F20 hosts. Once there's two EL7 hosts in the cluster we will plan downtime and move them one by one just as we did with the other pre-prod ones. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 10 June, 2016 4:16:46 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Status update: All VMs in Jenkins DC now running on CentOS hosts, all hypervisors there were updated/rebuilt. NIC bonding was enabled across all hosts as well. The only three remaining F20 hypervisors are the ones in the Production cluster and they also are attached to the Hosted Engine DC. Move of production VMs to CentOS hosts is in progress, just 8 left to go (HostedEngine being one of them). Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Tuesday, 7 June, 2016 3:54:47 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Status update: ovirt-srv05 rebuilt and back in production. Network bonding fixed on ovirt-srv17 and ovirt-srv18. ovirt-srv11 updated to CentOS 7.2. Currently bonding is being fixed on ovirt-srv01, ovirt-srv11 and ovirt-srv14. After that is complete, will proceed with rebuilding ovirt-srv06 and ovirt-srv07 which are the last F20 hosts in the Jenkins cluster. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 4:42:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Status update: ovirt-srv12 and ovirt-srv13 updated to CentOS 7.2, will now rebuild ovirt-srv05 which is currently in Maintenance and has no VMs running. Please ignore monitoring alerts for this host if any. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "Barak Korren" <bkorren@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:32:25 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <ederevea@redhat.com> כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
I think the next step after this will be to upgrade hypervisors to 3.6 while keeping the engine 3.5 with cluster level 3.5. Can you confirm the is the recommended upgrade flow to users as well? On Tue, Jun 21, 2016 at 5:13 PM, Evgheni Dereveanchin <ederevea@redhat.com> wrote:
I'm back to the office and moving forward with the upgrade. Will now re-provision ovirt-srv01 which is part of the HostedEngine cluster and was the first host of the environment. Now backing up its configs just to be sure.
There is 6 VMs still running on F20 hosts. Once there's two EL7 hosts in the cluster we will plan downtime and move them one by one just as we did with the other pre-prod ones.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 10 June, 2016 4:16:46 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: All VMs in Jenkins DC now running on CentOS hosts, all hypervisors there were updated/rebuilt. NIC bonding was enabled across all hosts as well.
The only three remaining F20 hypervisors are the ones in the Production cluster and they also are attached to the Hosted Engine DC.
Move of production VMs to CentOS hosts is in progress, just 8 left to go (HostedEngine being one of them).
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Tuesday, 7 June, 2016 3:54:47 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: ovirt-srv05 rebuilt and back in production.
Network bonding fixed on ovirt-srv17 and ovirt-srv18. ovirt-srv11 updated to CentOS 7.2.
Currently bonding is being fixed on ovirt-srv01, ovirt-srv11 and ovirt-srv14. After that is complete, will proceed with rebuilding ovirt-srv06 and ovirt-srv07 which are the last F20 hosts in the Jenkins cluster.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 4:42:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: ovirt-srv12 and ovirt-srv13 updated to CentOS 7.2, will now rebuild ovirt-srv05 which is currently in Maintenance and has no VMs running. Please ignore monitoring alerts for this host if any.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "Barak Korren" <bkorren@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:32:25 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <ederevea@redhat.com> כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
Hi Eyal, Indeed, all hosts have to have 3.6 level packages installed before the engine gets upgraded. In this particular case we have some catching up to do on the host OS side of the infra. After host re-deploy is done we'll update VDSM and friends to 3.6, upgrade the Engine and change the compatibility level to make use of the new features. Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Eyal Edri" <eedri@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Tuesday, 21 June, 2016 5:28:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter I think the next step after this will be to upgrade hypervisors to 3.6 while keeping the engine 3.5 with cluster level 3.5. Can you confirm the is the recommended upgrade flow to users as well? On Tue, Jun 21, 2016 at 5:13 PM, Evgheni Dereveanchin <ederevea@redhat.com> wrote:
I'm back to the office and moving forward with the upgrade. Will now re-provision ovirt-srv01 which is part of the HostedEngine cluster and was the first host of the environment. Now backing up its configs just to be sure.
There is 6 VMs still running on F20 hosts. Once there's two EL7 hosts in the cluster we will plan downtime and move them one by one just as we did with the other pre-prod ones.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 10 June, 2016 4:16:46 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: All VMs in Jenkins DC now running on CentOS hosts, all hypervisors there were updated/rebuilt. NIC bonding was enabled across all hosts as well.
The only three remaining F20 hypervisors are the ones in the Production cluster and they also are attached to the Hosted Engine DC.
Move of production VMs to CentOS hosts is in progress, just 8 left to go (HostedEngine being one of them).
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Tuesday, 7 June, 2016 3:54:47 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: ovirt-srv05 rebuilt and back in production.
Network bonding fixed on ovirt-srv17 and ovirt-srv18. ovirt-srv11 updated to CentOS 7.2.
Currently bonding is being fixed on ovirt-srv01, ovirt-srv11 and ovirt-srv14. After that is complete, will proceed with rebuilding ovirt-srv06 and ovirt-srv07 which are the last F20 hosts in the Jenkins cluster.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 4:42:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: ovirt-srv12 and ovirt-srv13 updated to CentOS 7.2, will now rebuild ovirt-srv05 which is currently in Maintenance and has no VMs running. Please ignore monitoring alerts for this host if any.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "Barak Korren" <bkorren@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:32:25 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <ederevea@redhat.com> כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
+1, You can also take advantage of this to install and mount the SSDs we have on 10 of the hypervisors, as preparation for the local disk hook usage. On Tue, Jun 21, 2016 at 6:41 PM, Evgheni Dereveanchin <ederevea@redhat.com> wrote:
Hi Eyal,
Indeed, all hosts have to have 3.6 level packages installed before the engine gets upgraded. In this particular case we have some catching up to do on the host OS side of the infra. After host re-deploy is done we'll update VDSM and friends to 3.6, upgrade the Engine and change the compatibility level to make use of the new features.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Eyal Edri" <eedri@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Tuesday, 21 June, 2016 5:28:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
I think the next step after this will be to upgrade hypervisors to 3.6 while keeping the engine 3.5 with cluster level 3.5. Can you confirm the is the recommended upgrade flow to users as well?
On Tue, Jun 21, 2016 at 5:13 PM, Evgheni Dereveanchin <ederevea@redhat.com
wrote:
I'm back to the office and moving forward with the upgrade. Will now re-provision ovirt-srv01 which is part of the HostedEngine cluster and was the first host of the environment. Now backing up its configs just to be sure.
There is 6 VMs still running on F20 hosts. Once there's two EL7 hosts in the cluster we will plan downtime and move them one by one just as we did with the other pre-prod ones.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 10 June, 2016 4:16:46 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: All VMs in Jenkins DC now running on CentOS hosts, all hypervisors there were updated/rebuilt. NIC bonding was enabled across all hosts as well.
The only three remaining F20 hypervisors are the ones in the Production cluster and they also are attached to the Hosted Engine DC.
Move of production VMs to CentOS hosts is in progress, just 8 left to go (HostedEngine being one of them).
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Tuesday, 7 June, 2016 3:54:47 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: ovirt-srv05 rebuilt and back in production.
Network bonding fixed on ovirt-srv17 and ovirt-srv18. ovirt-srv11 updated to CentOS 7.2.
Currently bonding is being fixed on ovirt-srv01, ovirt-srv11 and ovirt-srv14. After that is complete, will proceed with rebuilding ovirt-srv06 and ovirt-srv07 which are the last F20 hosts in the Jenkins cluster.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 4:42:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: ovirt-srv12 and ovirt-srv13 updated to CentOS 7.2, will now rebuild ovirt-srv05 which is currently in Maintenance and has no VMs running. Please ignore monitoring alerts for this host if any.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "Barak Korren" <bkorren@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:32:25 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <
ederevea@redhat.com>
כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
Will do! Logged a Jira task to track progress: https://ovirt-jira.atlassian.net/browse/OVIRT-604 Regards, Evgheni Dereveanchin ----- Original Message ----- From: "Eyal Edri" <eedri@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Tuesday, 21 June, 2016 5:42:54 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter +1, You can also take advantage of this to install and mount the SSDs we have on 10 of the hypervisors, as preparation for the local disk hook usage. On Tue, Jun 21, 2016 at 6:41 PM, Evgheni Dereveanchin <ederevea@redhat.com> wrote:
Hi Eyal,
Indeed, all hosts have to have 3.6 level packages installed before the engine gets upgraded. In this particular case we have some catching up to do on the host OS side of the infra. After host re-deploy is done we'll update VDSM and friends to 3.6, upgrade the Engine and change the compatibility level to make use of the new features.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Eyal Edri" <eedri@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Tuesday, 21 June, 2016 5:28:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
I think the next step after this will be to upgrade hypervisors to 3.6 while keeping the engine 3.5 with cluster level 3.5. Can you confirm the is the recommended upgrade flow to users as well?
On Tue, Jun 21, 2016 at 5:13 PM, Evgheni Dereveanchin <ederevea@redhat.com
wrote:
I'm back to the office and moving forward with the upgrade. Will now re-provision ovirt-srv01 which is part of the HostedEngine cluster and was the first host of the environment. Now backing up its configs just to be sure.
There is 6 VMs still running on F20 hosts. Once there's two EL7 hosts in the cluster we will plan downtime and move them one by one just as we did with the other pre-prod ones.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 10 June, 2016 4:16:46 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: All VMs in Jenkins DC now running on CentOS hosts, all hypervisors there were updated/rebuilt. NIC bonding was enabled across all hosts as well.
The only three remaining F20 hypervisors are the ones in the Production cluster and they also are attached to the Hosted Engine DC.
Move of production VMs to CentOS hosts is in progress, just 8 left to go (HostedEngine being one of them).
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Tuesday, 7 June, 2016 3:54:47 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: ovirt-srv05 rebuilt and back in production.
Network bonding fixed on ovirt-srv17 and ovirt-srv18. ovirt-srv11 updated to CentOS 7.2.
Currently bonding is being fixed on ovirt-srv01, ovirt-srv11 and ovirt-srv14. After that is complete, will proceed with rebuilding ovirt-srv06 and ovirt-srv07 which are the last F20 hosts in the Jenkins cluster.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 4:42:41 PM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Status update: ovirt-srv12 and ovirt-srv13 updated to CentOS 7.2, will now rebuild ovirt-srv05 which is currently in Maintenance and has no VMs running. Please ignore monitoring alerts for this host if any.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Evgheni Dereveanchin" <ederevea@redhat.com> To: "Barak Korren" <bkorren@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:32:25 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
Right, sorry for the typo :) It's up and running now with EL7.2 and vdsm 4.16.30. I will now update ovirt-srv12 and ovirt-srv13 from 7.1 to 7.2 and then proceed to reprovisioning the remaining F20 hosts in the old cluster.
Regards, Evgheni Dereveanchin
----- Original Message ----- From: "Barak Korren" <bkorren@redhat.com> To: "Evgheni Dereveanchin" <ederevea@redhat.com> Cc: "infra" <infra@ovirt.org> Sent: Friday, 3 June, 2016 11:29:02 AM Subject: Re: PHX updates - hypervisors in Jenkins datacenter
el7**** (i hate phone keyboards) בתאריך 3 ביוני 2016 12:28, "Barak Korren" <bkorren@redhat.com> כתב:
בתאריך 2 ביוני 2016 20:00, "Evgheni Dereveanchin" <
ederevea@redhat.com>
כתב:
Status update: ovirt-srv04 was rebuilt with EL6.
You meant elu right?
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
-- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
participants (5)
-
Barak Korren -
David Caro -
Evgheni Dereveanchin -
Eyal Edri -
Nadav Goldin