One patch - two failures

Below are the details of one my patches that failed in CI. How surprising the fact that it failed with two different reasons and both aren't patch related! ---------- Forwarded message ---------- From: "Jenkins CI" <gerrit2@gerrit.ovirt.org> Date: Jun 5, 2016 11:37 PM Subject: Change in ovirt-engine[master]: core: update vdsm-jsonrpc-java to verion 1.2.3 To: "Moti Asayag" <masayag@redhat.com>, "Yevgeny Zaspitsky" < yzaspits@redhat.com> Cc: Jenkins CI has posted comments on this change. Change subject: core: update vdsm-jsonrpc-java to verion 1.2.3 ...................................................................... Patch Set 2: Build Failed http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/534... : FAILURE http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-master_el7_mer... : FAILURE http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-fc23-x86_64/53... : SUCCESS http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-4.0_el7_merged... : SUCCESS -- To view, visit https://gerrit.ovirt.org/58584 To unsubscribe, visit https://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: Iae15520c831c0b8de36169f0287057684033d376 Gerrit-PatchSet: 2 Gerrit-Project: ovirt-engine Gerrit-Branch: master Gerrit-Owner: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: Alona Kaplan <alkaplan@redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Moti Asayag <masayag@redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski@gmail.com> Gerrit-Reviewer: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: gerrit-hooks <automation@ovirt.org> Gerrit-HasComments: No

On 6 June 2016 at 00:37, Yevgeny Zaspitsky <yzaspits@redhat.com> wrote:
Below are the details of one my patches that failed in CI. How surprising the fact that it failed with two different reasons and both aren't patch related!
---------- Forwarded message ---------- From: "Jenkins CI" <gerrit2@gerrit.ovirt.org> Date: Jun 5, 2016 11:37 PM Subject: Change in ovirt-engine[master]: core: update vdsm-jsonrpc-java to verion 1.2.3 To: "Moti Asayag" <masayag@redhat.com>, "Yevgeny Zaspitsky" <yzaspits@redhat.com> Cc:
Jenkins CI has posted comments on this change.
Change subject: core: update vdsm-jsonrpc-java to verion 1.2.3 ......................................................................
Patch Set 2:
Build Failed
http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/534... : FAILURE
It seems we had multiple similar failures on the node where this happened, so I took it offline for now
http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-master_el7_mer... : FAILURE
http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-fc23-x86_64/53... : SUCCESS
http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-4.0_el7_merged... : SUCCESS
-- To view, visit https://gerrit.ovirt.org/58584 To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: Iae15520c831c0b8de36169f0287057684033d376 Gerrit-PatchSet: 2 Gerrit-Project: ovirt-engine Gerrit-Branch: master Gerrit-Owner: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: Alona Kaplan <alkaplan@redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Moti Asayag <masayag@redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski@gmail.com> Gerrit-Reviewer: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: gerrit-hooks <automation@ovirt.org> Gerrit-HasComments: No
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Barak Korren bkorren@redhat.com RHEV-CI Team

--hHWLQfXTYDoKhP50 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 06/06 00:37, Yevgeny Zaspitsky wrote:
Below are the details of one my patches that failed in CI. How surprising the fact that it failed with two different reasons and both aren't patch related! ---------- Forwarded message ---------- From: "Jenkins CI" <gerrit2@gerrit.ovirt.org> Date: Jun 5, 2016 11:37 PM Subject: Change in ovirt-engine[master]: core: update vdsm-jsonrpc-java to verion 1.2.3 To: "Moti Asayag" <masayag@redhat.com>, "Yevgeny Zaspitsky" < yzaspits@redhat.com> Cc: =20 Jenkins CI has posted comments on this change. =20 Change subject: core: update vdsm-jsonrpc-java to verion 1.2.3 ...................................................................... =20 =20 Patch Set 2: =20 Build Failed =20 http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/= 534/ : FAILURE =20 http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-master_el7_= merged/547/ : FAILURE
This one looks like a real issue: 2016-06-05 20:29:27 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.= schema plugin.execute:926 execute-output: ['/usr/share/ovirt-engine/dbscrip= ts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engi= ne', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160605202840-w= 8dvo1.log', '-c', 'apply'] stderr: psql:/usr/share/ovirt-engine/dbscripts/upgrade/03_05_0580_add_default_insta= nce_types.sql:131: ERROR: duplicate key value violates unique constraint "= pk_permissions_id" DETAIL: Key (id)=3D(00000004-0004-0004-0004-000000000355) already exists. CONTEXT: SQL statement "INSERT INTO permissions(id, role_id, ad_element_id, object_id, object_type_id) SELECT uuid_generate_v1(), 'DEF00009-0000-0000-0000-DEF000000009', -- UserTemplateBasedVm 'EEE00000-0000-0000-0000-**FILTERED**789EEE', -- Everyone v_instance_type_id, 4 -- template" PL/pgSQL function do_insert_instance_type(character varying,character varyi= ng,integer,integer,integer) line 93 at SQL statement SQL statement "SELECT do_insert_instance_type('Tiny', 'Tiny instance type',= 512, 1, 1)" PL/pgSQL function insert_default_instance_types() line 5 at PERFORM FATAL: Cannot execute sql command: --file=3D/usr/share/ovirt-engine/dbscrip= ts/upgrade/03_05_0580_add_default_instance_types.sql Does not seem related to your patch though.
=20 http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-fc23-x86_64= /534/ : SUCCESS =20 http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-4.0_el7_mer= ged/78/ : SUCCESS =20 -- To view, visit https://gerrit.ovirt.org/58584 To unsubscribe, visit https://gerrit.ovirt.org/settings =20 Gerrit-MessageType: comment Gerrit-Change-Id: Iae15520c831c0b8de36169f0287057684033d376 Gerrit-PatchSet: 2 Gerrit-Project: ovirt-engine Gerrit-Branch: master Gerrit-Owner: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: Alona Kaplan <alkaplan@redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Moti Asayag <masayag@redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski@gmail.com> Gerrit-Reviewer: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: gerrit-hooks <automation@ovirt.org> Gerrit-HasComments: No
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
--=20 David Caro Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D Tel.: +420 532 294 605 Email: dcaro@redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605 --hHWLQfXTYDoKhP50 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJXVSpxAAoJEEBxx+HSYmnDuMkIAIgc0Y1jOzckVzzy3rTuX9dC 5hdkSGvNmqBRrdmK1cq4bVP2ScKi1uPYgBOQ9FR9WHSPTgo4/QvkILh6Dn0dgZWU GB0PWYqdFZYwd37K0XN+aqH25j1Xo8IMPqyhgcoMdm+flg5czlw0c96T9Bom2rYs 0WKmGn6b+qr5tjiVe8AzdOuS1l41D1Fnl84H3uMCB3sSAzpG8If8gl6uKgVO9ZBS 2oLicyjZzUs0s77EzBw2qHOrzMrxxsQnWtDtrm4negTbXkimZtDVBwcPwlONMggs DjCVCPJWmJeFli8jHLNaLGvUqw50FJBbx2RWAn8BZ8Z54OVdOcwOqc6oavlILkg= =cDyG -----END PGP SIGNATURE----- --hHWLQfXTYDoKhP50--

Martin, Maybe still related to the db issue we saw last week? On Mon, Jun 6, 2016 at 10:46 AM, David Caro <dcaro@redhat.com> wrote:
On 06/06 00:37, Yevgeny Zaspitsky wrote:
Below are the details of one my patches that failed in CI. How surprising the fact that it failed with two different reasons and both aren't patch related! ---------- Forwarded message ---------- From: "Jenkins CI" <gerrit2@gerrit.ovirt.org> Date: Jun 5, 2016 11:37 PM Subject: Change in ovirt-engine[master]: core: update vdsm-jsonrpc-java to verion 1.2.3 To: "Moti Asayag" <masayag@redhat.com>, "Yevgeny Zaspitsky" < yzaspits@redhat.com> Cc:
Jenkins CI has posted comments on this change.
Change subject: core: update vdsm-jsonrpc-java to verion 1.2.3 ......................................................................
Patch Set 2:
Build Failed
http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/534...
: FAILURE
http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-master_el7_mer...
: FAILURE
This one looks like a real issue:
2016-06-05 20:29:27 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.execute:926 execute-output: ['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engine', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160605202840-w8dvo1.log', '-c', 'apply'] stderr: psql:/usr/share/ovirt-engine/dbscripts/upgrade/03_05_0580_add_default_instance_types.sql:131: ERROR: duplicate key value violates unique constraint "pk_permissions_id" DETAIL: Key (id)=(00000004-0004-0004-0004-000000000355) already exists. CONTEXT: SQL statement "INSERT INTO permissions(id, role_id, ad_element_id, object_id, object_type_id) SELECT uuid_generate_v1(), 'DEF00009-0000-0000-0000-DEF000000009', -- UserTemplateBasedVm 'EEE00000-0000-0000-0000-**FILTERED**789EEE', -- Everyone v_instance_type_id, 4 -- template" PL/pgSQL function do_insert_instance_type(character varying,character varying,integer,integer,integer) line 93 at SQL statement SQL statement "SELECT do_insert_instance_type('Tiny', 'Tiny instance type', 512, 1, 1)" PL/pgSQL function insert_default_instance_types() line 5 at PERFORM FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/03_05_0580_add_default_instance_types.sql
Does not seem related to your patch though.
http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-fc23-x86_64/53...
: SUCCESS
http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-4.0_el7_merged...
: SUCCESS
-- To view, visit https://gerrit.ovirt.org/58584 To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: Iae15520c831c0b8de36169f0287057684033d376 Gerrit-PatchSet: 2 Gerrit-Project: ovirt-engine Gerrit-Branch: master Gerrit-Owner: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: Alona Kaplan <alkaplan@redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Moti Asayag <masayag@redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski@gmail.com> Gerrit-Reviewer: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: gerrit-hooks <automation@ovirt.org> Gerrit-HasComments: No
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- David Caro
Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D
Tel.: +420 532 294 605 Email: dcaro@redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

No it's not, this is some strange failure in 3.5 upgrade script which seems to me impossible. The patch [1] didn't contain any db related stuff, so upgrade from master before this patch to master with this patch shouldn't fail (and I just verified locally that this upgrade works fine). So I assume this was some kind of mess in db content prior the upgrade job. Martin [1] https://gerrit.ovirt.org/#/c/58584 On Mon, Jun 6, 2016 at 9:55 AM, Eyal Edri <eedri@redhat.com> wrote:
Martin, Maybe still related to the db issue we saw last week?
On Mon, Jun 6, 2016 at 10:46 AM, David Caro <dcaro@redhat.com> wrote:
On 06/06 00:37, Yevgeny Zaspitsky wrote:
Below are the details of one my patches that failed in CI. How surprising the fact that it failed with two different reasons and both aren't patch related! ---------- Forwarded message ---------- From: "Jenkins CI" <gerrit2@gerrit.ovirt.org> Date: Jun 5, 2016 11:37 PM Subject: Change in ovirt-engine[master]: core: update vdsm-jsonrpc-java to verion 1.2.3 To: "Moti Asayag" <masayag@redhat.com>, "Yevgeny Zaspitsky" < yzaspits@redhat.com> Cc:
Jenkins CI has posted comments on this change.
Change subject: core: update vdsm-jsonrpc-java to verion 1.2.3 ......................................................................
Patch Set 2:
Build Failed
http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-el7-x86_64/534...
: FAILURE
http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-master_el7_mer...
: FAILURE
This one looks like a real issue:
2016-06-05 20:29:27 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.execute:926 execute-output: ['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engine', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20160605202840-w8dvo1.log', '-c', 'apply'] stderr: psql:/usr/share/ovirt-engine/dbscripts/upgrade/03_05_0580_add_default_instance_types.sql:131: ERROR: duplicate key value violates unique constraint "pk_permissions_id" DETAIL: Key (id)=(00000004-0004-0004-0004-000000000355) already exists. CONTEXT: SQL statement "INSERT INTO permissions(id, role_id, ad_element_id, object_id, object_type_id) SELECT uuid_generate_v1(), 'DEF00009-0000-0000-0000-DEF000000009', -- UserTemplateBasedVm 'EEE00000-0000-0000-0000-**FILTERED**789EEE', -- Everyone v_instance_type_id, 4 -- template" PL/pgSQL function do_insert_instance_type(character varying,character varying,integer,integer,integer) line 93 at SQL statement SQL statement "SELECT do_insert_instance_type('Tiny', 'Tiny instance type', 512, 1, 1)" PL/pgSQL function insert_default_instance_types() line 5 at PERFORM FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/03_05_0580_add_default_instance_types.sql
Does not seem related to your patch though.
http://jenkins.ovirt.org/job/ovirt-engine_master_check-merged-fc23-x86_64/53...
: SUCCESS
http://jenkins.ovirt.org/job/ovirt-engine_master_upgrade-from-4.0_el7_merged...
: SUCCESS
-- To view, visit https://gerrit.ovirt.org/58584 To unsubscribe, visit https://gerrit.ovirt.org/settings
Gerrit-MessageType: comment Gerrit-Change-Id: Iae15520c831c0b8de36169f0287057684033d376 Gerrit-PatchSet: 2 Gerrit-Project: ovirt-engine Gerrit-Branch: master Gerrit-Owner: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: Alona Kaplan <alkaplan@redhat.com> Gerrit-Reviewer: Jenkins CI Gerrit-Reviewer: Moti Asayag <masayag@redhat.com> Gerrit-Reviewer: Piotr Kliczewski <piotr.kliczewski@gmail.com> Gerrit-Reviewer: Yevgeny Zaspitsky <yzaspits@redhat.com> Gerrit-Reviewer: gerrit-hooks <automation@ovirt.org> Gerrit-HasComments: No
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- David Caro
Red Hat S.L. Continuous Integration Engineer - EMEA ENG Virtualization R&D
Tel.: +420 532 294 605 Email: dcaro@redhat.com IRC: dcaro|dcaroest@{freenode|oftc|redhat} Web: www.redhat.com RHT Global #: 82-62605
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel
phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

On 6 June 2016 at 11:46, Martin Perina <mperina@redhat.com> wrote:
No it's not, this is some strange failure in 3.5 upgrade script which seems to me impossible. The patch [1] didn't contain any db related stuff, so upgrade from master before this patch to master with this patch shouldn't fail (and I just verified locally that this upgrade works fine). So I assume this was some kind of mess in db content prior the upgrade job.
This is close to impossible, the upgrade job removes the postgresql package and deletes "/var/lib/pgsql/data" as can be seen here: https://gerrit.ovirt.org/gitweb?p=jenkins.git;a=blob;f=jobs/ovirt-engine_upg...

Well, this is the only explanation I was able to think of as I haven't been able to reproduce it locally and definitely there was not any change in 3.5 upgrade scripts for several months (the 03_05_0580_add_default_instance_types.sql script was changed last time on Jun 2nd 2014). Anyway it happen again, we can investigate it more. Thanks Martin On Tue, Jun 7, 2016 at 8:37 AM, Barak Korren <bkorren@redhat.com> wrote:
On 6 June 2016 at 11:46, Martin Perina <mperina@redhat.com> wrote:
No it's not, this is some strange failure in 3.5 upgrade script which seems to me impossible. The patch [1] didn't contain any db related stuff, so upgrade from master before this patch to master with this patch shouldn't fail (and I just verified locally that this upgrade works fine). So I assume this was some kind of mess in db content prior the upgrade job.
This is close to impossible, the upgrade job removes the postgresql package and deletes "/var/lib/pgsql/data" as can be seen here:
https://gerrit.ovirt.org/gitweb?p=jenkins.git;a=blob;f=jobs/ovirt-engine_upg...
participants (5)
-
Barak Korren
-
David Caro
-
Eyal Edri
-
Martin Perina
-
Yevgeny Zaspitsky