https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/15... Fails on calling host.name <http://host.name>(), Help understanding why this is happening is welcome.

-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/> *<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> ha scritto: Hi, On 4/29/20 11:42 AM, Sandro Bonazzola wrote: > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/15... > > > Fails on calling host.name <http://host.name>(), > > Help understanding why this is happening is welcome. The error is not about calling 'host.name <http://host.name>()' - the stack trace only shows part of the code [1]. The actual problem seems to be during deployment process - please take a look at stderr of the dnf commands (both hosts and the engine seem to be affected): 2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command 1b6078d6 on lago-he-basic-suite-master-host-0 errors: warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY Importing GPG key 0x2F86D6A1: Userid : "Fedora EPEL (8)&lt;epel(a)fedoraproject.org&gt; <mailto:epel@fedoraproject.org>" Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 From :https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY Importing GPG key 0x119783D1: Userid : "sbonazzo_EL8_collection (None)&lt;sbonazzo#EL8_collection(a)copr.fedorahosted.org&gt; <mailto:sbonazzo#EL8_collection@copr.fedorahosted.org>" Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 From :https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubkey.gpg warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY Importing GPG key 0xB4B47538: Userid : "nmstate_nmstate-0.2 (None)&lt;nmstate#nmstate-0.2(a)copr.fedorahosted.org&gt; <mailto:nmstate#nmstate-0.2@copr.fedorahosted.org>" Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 From :https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.gpg above stderr looks ok, dnf is importing missing gpg keys to validate the rpm signing. I don't really see the error there.

Regards, Marcin [1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c... > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA <https://www.redhat.com/> > > sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> > > <https://www.redhat.com/> > > *<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* > *Red Hat respects your work life balance. Therefore there is no > need to answer this email out of your office hours. > <https://mojo.redhat.com/docs/DOC-1199578>* -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/> *<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

Il giorno mer 29 apr 2020 alle ore 15:18 Marcin Sobczyk <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> ha scritto: On 4/29/20 3:00 PM, Sandro Bonazzola wrote: > > > Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk > <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> ha scritto: > > Hi, > > On 4/29/20 11:42 AM, Sandro Bonazzola wrote: >> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/15... >> >> >> Fails on calling host.name <http://host.name>(), >> >> Help understanding why this is happening is welcome. > The error is not about calling 'host.name > <http://host.name>()' - the stack trace only shows part of > the code [1]. > The actual problem seems to be during deployment process - > please take a look at stderr > of the dnf commands (both hosts and the engine seem to be > affected): > > 2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command 1b6078d6 on lago-he-basic-suite-master-host-0 errors: > warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY > Importing GPG key 0x2F86D6A1: > Userid : "Fedora EPEL (8)&lt;epel(a)fedoraproject.org&gt; <mailto:epel@fedoraproject.org>" > Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 > From :https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 > warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY > Importing GPG key 0x119783D1: > Userid : "sbonazzo_EL8_collection (None)&lt;sbonazzo#EL8_collection(a)copr.fedorahosted.org&gt; <mailto:sbonazzo#EL8_collection@copr.fedorahosted.org>" > Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 > From :https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubkey.gpg > warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY > Importing GPG key 0xB4B47538: > Userid : "nmstate_nmstate-0.2 (None)&lt;nmstate#nmstate-0.2(a)copr.fedorahosted.org&gt; <mailto:nmstate#nmstate-0.2@copr.fedorahosted.org>" > Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 > From :https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.gpg > > > > above stderr looks ok, dnf is importing missing gpg keys to > validate the rpm signing. > I don't really see the error there. Are these warnings also ok? warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY yes, rpms are being installed from external repos and yum is importing gpg keys from those repos when it encounters a rpm signed with an unknown key for checking it.

> > Regards, Marcin > > [1] > https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c... > >> >> -- >> >> Sandro Bonazzola >> >> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >> >> Red Hat EMEA <https://www.redhat.com/> >> >> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> >> >> <https://www.redhat.com/> >> >> *<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* >> *Red Hat respects your work life balance. Therefore there is >> no need to answer this email out of your office hours. >> <https://mojo.redhat.com/docs/DOC-1199578>* > > > > -- > > Sandro Bonazzola > > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > > Red Hat EMEA <https://www.redhat.com/> > > sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> > > <https://www.redhat.com/> > > *<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* > *Red Hat respects your work life balance. Therefore there is no > need to answer this email out of your office hours. > <https://mojo.redhat.com/docs/DOC-1199578>* -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/> *<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

On 29 Apr 2020, at 17:45, Sandro Bonazzola <sbonazzo(a)redhat.com&gt; wrote: Il giorno mer 29 apr 2020 alle ore 17:13 Marcin Sobczyk <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> ha scritto: On 4/29/20 4:23 PM, Sandro Bonazzola wrote: > > > Il giorno mer 29 apr 2020 alle ore 15:18 Marcin Sobczyk <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> ha scritto: > > > On 4/29/20 3:00 PM, Sandro Bonazzola wrote: >> >> >> Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> ha scritto: >> Hi, >> >> On 4/29/20 11:42 AM, Sandro Bonazzola wrote: >>> https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/15... <https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/15... >>> >>> Fails on calling host.name <http://host.name/>(), >>> >>> Help understanding why this is happening is welcome. >> The error is not about calling 'host.name <http://host.name/>()' - the stack trace only shows part of the code [1]. >> The actual problem seems to be during deployment process - please take a look at stderr >> of the dnf commands (both hosts and the engine seem to be affected): >> >> 2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command <> 1b6078d6 on lago-he-basic-suite-master-host-0 errors: >> warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY >> Importing GPG key 0x2F86D6A1: >> Userid : "Fedora EPEL (8) <epel(a)fedoraproject.org&gt; <mailto:epel@fedoraproject.org>" >> Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 >> From : https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 <https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8> >> warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY >> Importing GPG key 0x119783D1: >> Userid : "sbonazzo_EL8_collection (None) <sbonazzo#EL8_collection(a)copr.fedorahosted.org&gt; <mailto:sbonazzo#EL8_collection@copr.fedorahosted.org>" >> Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 >> From : https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/p... <https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/p... >> warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY >> Importing GPG key 0xB4B47538: >> Userid : "nmstate_nmstate-0.2 (None) <nmstate#nmstate-0.2(a)copr.fedorahosted.org&gt; <mailto:nmstate#nmstate-0.2@copr.fedorahosted.org>" >> Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 >> From : https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubke... <https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubke... >> >> >> above stderr looks ok, dnf is importing missing gpg keys to validate the rpm signing. >> I don't really see the error there. > Are these warnings also ok? > > warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY > warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY > warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY > > yes, rpms are being installed from external repos and yum is importing gpg keys from those repos when it encounters a rpm signed with an unknown key for checking it. Ahhh, ok, right - my bad. Now I see the issue - storage setup is failing: 2020-04-29 02:15:13,885::ssh.py::ssh::81::lago.ssh::DEBUG::Command <> 1f4a643e on lago-he-basic-suite-master-storage returned with 1 The problematic part seems to be: + setup_iscsi ++ ip -4 addr show eth1 ++ grep -oP '(?<=inet ).*(?=/)' + IP= Related, and already discussed in https://gerrit.ovirt.org/#/c/108466/ <https://gerrit.ovirt.org/#/c/108466/> Thanks! +Evgeny Slutsky <mailto:eslutsky@redhat.com> , +Michal Skrivanek <mailto:mskrivan@redhat.com> can you please fix he-basic-suite-master adapting to the change introduced in https://gerrit.ovirt.org/108466 <https://gerrit.ovirt.org/108466> ?

> >> >> >> Regards, Marcin >> >> [1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c... <https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c... >> >>> >>> -- >>> Sandro Bonazzola >>> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >>> Red Hat EMEA <https://www.redhat.com/> >>> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> >>> <https://www.redhat.com/><rh-summit-email-assets-us226039-201909sw-v... <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK> >>> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578> >> >> >> -- >> Sandro Bonazzola >> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >> Red Hat EMEA <https://www.redhat.com/> >> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> >> <https://www.redhat.com/><rh-summit-email-assets-us226039-201909sw-v... <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK> >> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578> > > > -- > Sandro Bonazzola > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> > <https://www.redhat.com/><rh-summit-email-assets-us226039-201909sw-v... <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK> > Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578> -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/><rh-summit-email-assets-us226039-201909sw-v... <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>