ovirt-system-tests_he-basic-suite-master fails on host.name()
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/... Fails on calling host.name(), Help understanding why this is happening is welcome. -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
Hi, On 4/29/20 11:42 AM, Sandro Bonazzola wrote:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/...
Fails on calling host.name <http://host.name>(),
Help understanding why this is happening is welcome. The error is not about calling 'host.name()' - the stack trace only shows part of the code [1]. The actual problem seems to be during deployment process - please take a look at stderr of the dnf commands (both hosts and the engine seem to be affected):
2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command 1b6078d6 on lago-he-basic-suite-master-host-0 errors: warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY Importing GPG key 0x2F86D6A1: Userid : "Fedora EPEL (8) <epel@fedoraproject.org>" Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 From : https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY Importing GPG key 0x119783D1: Userid : "sbonazzo_EL8_collection (None) <sbonazzo#EL8_collection@copr.fedorahosted.org>" Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 From : https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubk... warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY Importing GPG key 0xB4B47538: Userid : "nmstate_nmstate-0.2 (None) <nmstate#nmstate-0.2@copr.fedorahosted.org>" Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 From : https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.g... Regards, Marcin [1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c53e...
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <mailto:sbonazzo@redhat.com>
*<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk <msobczyk@redhat.com> ha scritto:
Hi,
On 4/29/20 11:42 AM, Sandro Bonazzola wrote:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/...
Fails on calling host.name(),
Help understanding why this is happening is welcome.
The error is not about calling 'host.name()' - the stack trace only shows part of the code [1]. The actual problem seems to be during deployment process - please take a look at stderr of the dnf commands (both hosts and the engine seem to be affected):
2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command 1b6078d6 on lago-he-basic-suite-master-host-0 errors: warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY Importing GPG key 0x2F86D6A1: Userid : "Fedora EPEL (8) <epel@fedoraproject.org> <epel@fedoraproject.org>" Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 From : https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY Importing GPG key 0x119783D1: Userid : "sbonazzo_EL8_collection (None) <sbonazzo#EL8_collection@copr.fedorahosted.org> <sbonazzo#EL8_collection@copr.fedorahosted.org>" Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 From : https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubk... warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY Importing GPG key 0xB4B47538: Userid : "nmstate_nmstate-0.2 (None) <nmstate#nmstate-0.2@copr.fedorahosted.org> <nmstate#nmstate-0.2@copr.fedorahosted.org>" Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 From : https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.g...
above stderr looks ok, dnf is importing missing gpg keys to validate the rpm signing. I don't really see the error there.
Regards, Marcin
[1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c53e...
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/> * <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
On 4/29/20 3:00 PM, Sandro Bonazzola wrote:
Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk <msobczyk@redhat.com <mailto:msobczyk@redhat.com>> ha scritto:
Hi,
On 4/29/20 11:42 AM, Sandro Bonazzola wrote:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/...
Fails on calling host.name <http://host.name>(),
Help understanding why this is happening is welcome.
The error is not about calling 'host.name <http://host.name>()' - the stack trace only shows part of the code [1]. The actual problem seems to be during deployment process - please take a look at stderr of the dnf commands (both hosts and the engine seem to be affected):
2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command 1b6078d6 on lago-he-basic-suite-master-host-0 errors: warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY Importing GPG key 0x2F86D6A1: Userid : "Fedora EPEL (8)<epel@fedoraproject.org> <mailto:epel@fedoraproject.org>" Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 From :https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY Importing GPG key 0x119783D1: Userid : "sbonazzo_EL8_collection (None)<sbonazzo#EL8_collection@copr.fedorahosted.org> <mailto:sbonazzo#EL8_collection@copr.fedorahosted.org>" Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 From :https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubk... warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY Importing GPG key 0xB4B47538: Userid : "nmstate_nmstate-0.2 (None)<nmstate#nmstate-0.2@copr.fedorahosted.org> <mailto:nmstate#nmstate-0.2@copr.fedorahosted.org>" Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 From :https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.g...
above stderr looks ok, dnf is importing missing gpg keys to validate the rpm signing. I don't really see the error there.
Are these warnings also ok? warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY
Regards, Marcin
[1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c53e...
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <mailto:sbonazzo@redhat.com>
*<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <mailto:sbonazzo@redhat.com>
*<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
Il giorno mer 29 apr 2020 alle ore 15:18 Marcin Sobczyk <msobczyk@redhat.com> ha scritto:
On 4/29/20 3:00 PM, Sandro Bonazzola wrote:
Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk < msobczyk@redhat.com> ha scritto:
Hi,
On 4/29/20 11:42 AM, Sandro Bonazzola wrote:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/...
Fails on calling host.name(),
Help understanding why this is happening is welcome.
The error is not about calling 'host.name()' - the stack trace only shows part of the code [1]. The actual problem seems to be during deployment process - please take a look at stderr of the dnf commands (both hosts and the engine seem to be affected):
2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command 1b6078d6 on lago-he-basic-suite-master-host-0 errors: warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY Importing GPG key 0x2F86D6A1: Userid : "Fedora EPEL (8) <epel@fedoraproject.org> <epel@fedoraproject.org>" Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 From : https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY Importing GPG key 0x119783D1: Userid : "sbonazzo_EL8_collection (None) <sbonazzo#EL8_collection@copr.fedorahosted.org> <sbonazzo#EL8_collection@copr.fedorahosted.org>" Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 From : https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubk... warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY Importing GPG key 0xB4B47538: Userid : "nmstate_nmstate-0.2 (None) <nmstate#nmstate-0.2@copr.fedorahosted.org> <nmstate#nmstate-0.2@copr.fedorahosted.org>" Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 From : https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.g...
above stderr looks ok, dnf is importing missing gpg keys to validate the rpm signing. I don't really see the error there.
Are these warnings also ok?
warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY
warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY
warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY
yes, rpms are being installed from external repos and yum is importing gpg keys from those repos when it encounters a rpm signed with an unknown key for checking it.
Regards, Marcin
[1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c53e...
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/> * <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/> * <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
On 4/29/20 4:23 PM, Sandro Bonazzola wrote:
Il giorno mer 29 apr 2020 alle ore 15:18 Marcin Sobczyk <msobczyk@redhat.com <mailto:msobczyk@redhat.com>> ha scritto:
On 4/29/20 3:00 PM, Sandro Bonazzola wrote:
Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk <msobczyk@redhat.com <mailto:msobczyk@redhat.com>> ha scritto:
Hi,
On 4/29/20 11:42 AM, Sandro Bonazzola wrote:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/...
Fails on calling host.name <http://host.name>(),
Help understanding why this is happening is welcome.
The error is not about calling 'host.name <http://host.name>()' - the stack trace only shows part of the code [1]. The actual problem seems to be during deployment process - please take a look at stderr of the dnf commands (both hosts and the engine seem to be affected):
2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command 1b6078d6 on lago-he-basic-suite-master-host-0 errors: warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY Importing GPG key 0x2F86D6A1: Userid : "Fedora EPEL (8)<epel@fedoraproject.org> <mailto:epel@fedoraproject.org>" Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 From :https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY Importing GPG key 0x119783D1: Userid : "sbonazzo_EL8_collection (None)<sbonazzo#EL8_collection@copr.fedorahosted.org> <mailto:sbonazzo#EL8_collection@copr.fedorahosted.org>" Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 From :https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubk... warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY Importing GPG key 0xB4B47538: Userid : "nmstate_nmstate-0.2 (None)<nmstate#nmstate-0.2@copr.fedorahosted.org> <mailto:nmstate#nmstate-0.2@copr.fedorahosted.org>" Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 From :https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.g...
above stderr looks ok, dnf is importing missing gpg keys to validate the rpm signing. I don't really see the error there.
Are these warnings also ok?
warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY
warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY
warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY
yes, rpms are being installed from external repos and yum is importing gpg keys from those repos when it encounters a rpm signed with an unknown key for checking it.
Ahhh, ok, right - my bad. Now I see the issue - storage setup is failing: 2020-04-29 02:15:13,885::ssh.py::ssh::81::lago.ssh::DEBUG::Command 1f4a643e on lago-he-basic-suite-master-storage returned with 1 The problematic part seems to be: + setup_iscsi ++ ip -4 addr show eth1 ++ grep -oP '(?<=inet ).*(?=/)' + IP= Related, and already discussed in https://gerrit.ovirt.org/#/c/108466/
Regards, Marcin
[1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c53e...
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <mailto:sbonazzo@redhat.com>
*<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <mailto:sbonazzo@redhat.com>
*<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <mailto:sbonazzo@redhat.com>
*<https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
Il giorno mer 29 apr 2020 alle ore 17:13 Marcin Sobczyk <msobczyk@redhat.com> ha scritto:
On 4/29/20 4:23 PM, Sandro Bonazzola wrote:
Il giorno mer 29 apr 2020 alle ore 15:18 Marcin Sobczyk < msobczyk@redhat.com> ha scritto:
On 4/29/20 3:00 PM, Sandro Bonazzola wrote:
Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk < msobczyk@redhat.com> ha scritto:
Hi,
On 4/29/20 11:42 AM, Sandro Bonazzola wrote:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/...
Fails on calling host.name(),
Help understanding why this is happening is welcome.
The error is not about calling 'host.name()' - the stack trace only shows part of the code [1]. The actual problem seems to be during deployment process - please take a look at stderr of the dnf commands (both hosts and the engine seem to be affected):
2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command 1b6078d6 on lago-he-basic-suite-master-host-0 errors: warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY Importing GPG key 0x2F86D6A1: Userid : "Fedora EPEL (8) <epel@fedoraproject.org> <epel@fedoraproject.org>" Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 From : https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY Importing GPG key 0x119783D1: Userid : "sbonazzo_EL8_collection (None) <sbonazzo#EL8_collection@copr.fedorahosted.org> <sbonazzo#EL8_collection@copr.fedorahosted.org>" Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 From : https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubk... warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY Importing GPG key 0xB4B47538: Userid : "nmstate_nmstate-0.2 (None) <nmstate#nmstate-0.2@copr.fedorahosted.org> <nmstate#nmstate-0.2@copr.fedorahosted.org>" Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 From : https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.g...
above stderr looks ok, dnf is importing missing gpg keys to validate the rpm signing. I don't really see the error there.
Are these warnings also ok?
warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY
warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY
warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY
yes, rpms are being installed from external repos and yum is importing gpg keys from those repos when it encounters a rpm signed with an unknown key for checking it.
Ahhh, ok, right - my bad.
Now I see the issue - storage setup is failing:
2020-04-29 02:15:13,885::ssh.py::ssh::81::lago.ssh::DEBUG::Command 1f4a643e on lago-he-basic-suite-master-storage returned with 1
The problematic part seems to be:
+ setup_iscsi ++ ip -4 addr show eth1 ++ grep -oP '(?<=inet ).*(?=/)' + IP=
Related, and already discussed in https://gerrit.ovirt.org/#/c/108466/
Thanks! +Evgeny Slutsky <eslutsky@redhat.com> , +Michal Skrivanek <mskrivan@redhat.com> can you please fix he-basic-suite-master adapting to the change introduced in https://gerrit.ovirt.org/108466 ?
Regards, Marcin
[1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c53e...
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/> * <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/> * <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/> * <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/>* <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK>* *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
On 29 Apr 2020, at 17:45, Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno mer 29 apr 2020 alle ore 17:13 Marcin Sobczyk <msobczyk@redhat.com <mailto:msobczyk@redhat.com>> ha scritto:
On 4/29/20 4:23 PM, Sandro Bonazzola wrote:
Il giorno mer 29 apr 2020 alle ore 15:18 Marcin Sobczyk <msobczyk@redhat.com <mailto:msobczyk@redhat.com>> ha scritto:
On 4/29/20 3:00 PM, Sandro Bonazzola wrote:
Il giorno mer 29 apr 2020 alle ore 13:19 Marcin Sobczyk <msobczyk@redhat.com <mailto:msobczyk@redhat.com>> ha scritto: Hi,
On 4/29/20 11:42 AM, Sandro Bonazzola wrote:
https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/... <https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/1586/artifact/exported-artifacts/lago_logs/lago.log>
Fails on calling host.name <http://host.name/>(),
Help understanding why this is happening is welcome. The error is not about calling 'host.name <http://host.name/>()' - the stack trace only shows part of the code [1]. The actual problem seems to be during deployment process - please take a look at stderr of the dnf commands (both hosts and the engine seem to be affected):
2020-04-29 02:20:10,720::ssh.py::ssh::96::lago.ssh::DEBUG::Command <> 1b6078d6 on lago-he-basic-suite-master-host-0 errors: warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY Importing GPG key 0x2F86D6A1: Userid : "Fedora EPEL (8) <epel@fedoraproject.org> <mailto:epel@fedoraproject.org>" Fingerprint: 94E2 79EB 8D8F 25B2 1810 ADF1 21EA 45AB 2F86 D6A1 From : https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8 <https://dl.fedoraproject.org/pub/epel/RPM-GPG-KEY-EPEL-8> warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY Importing GPG key 0x119783D1: Userid : "sbonazzo_EL8_collection (None) <sbonazzo#EL8_collection@copr.fedorahosted.org> <mailto:sbonazzo#EL8_collection@copr.fedorahosted.org>" Fingerprint: 42A2 7AF4 DCA1 3DC7 02B2 E8BE 35CB 6F97 1197 83D1 From : https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubk... <https://copr-be.cloud.fedoraproject.org/results/sbonazzo/EL8_collection/pubkey.gpg> warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY Importing GPG key 0xB4B47538: Userid : "nmstate_nmstate-0.2 (None) <nmstate#nmstate-0.2@copr.fedorahosted.org> <mailto:nmstate#nmstate-0.2@copr.fedorahosted.org>" Fingerprint: 62CD E4D5 38ED F759 4A51 9AA3 2380 10C9 B4B4 7538 From : https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.g... <https://copr-be.cloud.fedoraproject.org/results/nmstate/nmstate-0.2/pubkey.gpg>
above stderr looks ok, dnf is importing missing gpg keys to validate the rpm signing. I don't really see the error there.
Are these warnings also ok?
warning: /var/cache/dnf/epel-8-3ed1b0328992983f/packages/libsodium-1.0.18-2.el8.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID 2f86d6a1: NOKEY warning: /var/cache/dnf/ovirt-master-copr-sbonazzo-collection-el8-21f13732bf45d2f7/packages/python3-xmltodict-0.12.0-4.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID 119783d1: NOKEY warning: /var/cache/dnf/ovirt-master-copr-nmstate-0.2-el8-d80c26aead58c45f/packages/nmstate-0.2.10-1.el8.noarch.rpm: Header V3 RSA/SHA1 Signature, key ID b4b47538: NOKEY
yes, rpms are being installed from external repos and yum is importing gpg keys from those repos when it encounters a rpm signed with an unknown key for checking it.
Ahhh, ok, right - my bad.
Now I see the issue - storage setup is failing:
2020-04-29 02:15:13,885::ssh.py::ssh::81::lago.ssh::DEBUG::Command <> 1f4a643e on lago-he-basic-suite-master-storage returned with 1 The problematic part seems to be:
+ setup_iscsi ++ ip -4 addr show eth1 ++ grep -oP '(?<=inet ).*(?=/)' + IP= Related, and already discussed in https://gerrit.ovirt.org/#/c/108466/ <https://gerrit.ovirt.org/#/c/108466/>
Thanks! +Evgeny Slutsky <mailto:eslutsky@redhat.com> , +Michal Skrivanek <mailto:mskrivan@redhat.com> can you please fix he-basic-suite-master adapting to the change introduced in https://gerrit.ovirt.org/108466 <https://gerrit.ovirt.org/108466> ?
Maybe https://gerrit.ovirt.org/108710 is good enough for now. We need to figure out with Ales how to properly modify this for ipv6-only suites. Thats’ a larger problem and I think we don’t really have that now so the patch is probably fine. Thanks, michal
Regards, Marcin
[1] https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c53e... <https://github.com/lago-project/lago/blob/00440d89832f62beb38414c6eb8b48c53ed82bca/lago/prefix.py#L1664>
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/><rh-summit-email-assets-us226039-201909sw-virtualevent-v2_sig-340x50-left-c.png> <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/><rh-summit-email-assets-us226039-201909sw-virtualevent-v2_sig-340x50-left-c.png> <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/><rh-summit-email-assets-us226039-201909sw-virtualevent-v2_sig-340x50-left-c.png> <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <mailto:sbonazzo@redhat.com> <https://www.redhat.com/><rh-summit-email-assets-us226039-201909sw-virtualevent-v2_sig-340x50-left-c.png> <https://www.redhat.com/en/summit?sc_cid=7013a000002D2QxAAK> Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>
participants (3)
-
Marcin Sobczyk -
Michal Skrivanek -
Sandro Bonazzola