Lago's local sync repo not available in OST
Hi, just seen: 2019-03-25 10:41:36,062-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-hostUpdatesChecker-Thread-2) [] EVENT_ID: HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates on host lago-basic-suite-master-host-1 with message 'Failed to run check-update of host 'lago-basic-suite-master-host-1'. Error: fatal: [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "msg": " http://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\n\n\n One of the configured repositories failed (Latest oVirt nightly),\n and yum doesn't have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work \"fix\" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=alocalsync ...\n\n 4. Disable the repository permanently, so yum won't use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable alocalsync\n or\n subscription-manager repos --disable=alocalsync\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=alocalsync.skip_if_unavailable=true\n\nfailure: repodata/repomd.xml from alocalsync: [Errno 256] No more mirrors to try.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nCannot upload enabled repos report, is this client registered?\n", "rc": 1, "results": []} while looking at https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/3867/a... for https://gerrit.ovirt.org/98862 Can you please have a look? -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://red.ht/sig>
from what I can see, the host is unreachable at the time this is running. The actual job failed due to a reported issue (which is the hotplug_cpu) in libvirt: 2019-03-25 14:28:38.324+0000: 8010: info : libvirt version: 4.5.0, package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>, 2019-01-29-17:31:22, x86-01.bsys.centos.org) 2019-03-25 14:28:38.324+0000: 8010: info : hostname: lago-basic-suite-master-host-0 2019-03-25 14:28:38.324+0000: 8010: error : virNetSocketReadWire:1806 : End of file while reading data: Input/output error 2019-03-25 14:42:54.353+0000: 22186: info : libvirt version: 4.5.0, package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>, 2019-01-29-17:31:22, x86-01.bsys.centos.org) 2019-03-25 14:42:54.353+0000: 22186: info : hostname: lago-basic-suite-master-host-0 2019-03-25 14:42:54.353+0000: 22186: warning : qemuDomainObjTaint:7521 : Domain id=1 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is tainted: host-cpu 2019-03-25 14:43:51.078+0000: 22188: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected 2019-03-25 14:45:12.597+0000: 22186: warning : qemuDomainObjTaint:7521 : Domain id=2 name='vm0' uuid=ed91d5d5-d342-4537-9807-a1fe4423ddcd is tainted: hook-script 2019-03-25 14:45:12.692+0000: 22186: error : virProcessRunInMountNamespace:1159 : internal error: child reported: unable to set security context 'system_u:object_r:virt_content_t:s0' on '/rhev/data-center/mnt/blockSD/cf0b3c9b-b447-48a8-a74a-ca8ffa775965/images/4319f48b-2d60-4295-886b-ecaf9f13d5d8/b1da85a2-3668-4ba2-be71-11e38e6ad772': No such file or directory 2019-03-25 14:45:33.130+0000: 22188: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected 2019-03-25 14:46:35.836+0000: 22183: error : qemuMonitorIO:718 : internal error: End of file from qemu monitor 2019-03-25 14:46:49.492+0000: 22184: warning : qemuDomainObjTaint:7521 : Domain id=3 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is tainted: host-cpu 2019-03-25 14:48:51.084+0000: 22187: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected (END) On Mon, Mar 25, 2019 at 4:04 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Hi, just seen: 2019-03-25 10:41:36,062-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-hostUpdatesChecker-Thread-2) [] EVENT_ID: HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates on host lago-basic-suite-master-host-1 with message 'Failed to run check-update of host 'lago-basic-suite-master-host-1'. Error: fatal: [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "msg": " http://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\n\n\n One of the configured repositories failed (Latest oVirt nightly),\n and yum doesn't have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work \"fix\" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=alocalsync ...\n\n 4. Disable the repository permanently, so yum won't use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable alocalsync\n or\n subscription-manager repos --disable=alocalsync\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=alocalsync.skip_if_unavailable=true\n\nfailure: repodata/repomd.xml from alocalsync: [Errno 256] No more mirrors to try.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nCannot upload enabled repos report, is this client registered?\n", "rc": 1, "results": []}
while looking at https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/3867/a...
for https://gerrit.ovirt.org/98862
Can you please have a look?
--
SANDRO BONAZZOLA
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://red.ht/sig>
Il giorno lun 25 mar 2019 alle ore 17:28 Dafna Ron <dron@redhat.com> ha scritto:
from what I can see, the host is unreachable at the time this is running. The actual job failed due to a reported issue (which is the hotplug_cpu) in libvirt:
I agree, the failure doesn't depend on this host being unreachable, but still, it should have been reachable, right?
2019-03-25 14:28:38.324+0000: 8010: info : libvirt version: 4.5.0, package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>, 2019-01-29-17:31:22, x86-01.bsys.centos.org) 2019-03-25 14:28:38.324+0000: 8010: info : hostname: lago-basic-suite-master-host-0 2019-03-25 14:28:38.324+0000: 8010: error : virNetSocketReadWire:1806 : End of file while reading data: Input/output error 2019-03-25 14:42:54.353+0000: 22186: info : libvirt version: 4.5.0, package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>, 2019-01-29-17:31:22, x86-01.bsys.centos.org) 2019-03-25 14:42:54.353+0000: 22186: info : hostname: lago-basic-suite-master-host-0 2019-03-25 14:42:54.353+0000: 22186: warning : qemuDomainObjTaint:7521 : Domain id=1 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is tainted: host-cpu 2019-03-25 14:43:51.078+0000: 22188: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected 2019-03-25 14:45:12.597+0000: 22186: warning : qemuDomainObjTaint:7521 : Domain id=2 name='vm0' uuid=ed91d5d5-d342-4537-9807-a1fe4423ddcd is tainted: hook-script 2019-03-25 14:45:12.692+0000: 22186: error : virProcessRunInMountNamespace:1159 : internal error: child reported: unable to set security context 'system_u:object_r:virt_content_t:s0' on '/rhev/data-center/mnt/blockSD/cf0b3c9b-b447-48a8-a74a-ca8ffa775965/images/4319f48b-2d60-4295-886b-ecaf9f13d5d8/b1da85a2-3668-4ba2-be71-11e38e6ad772': No such file or directory 2019-03-25 14:45:33.130+0000: 22188: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected 2019-03-25 14:46:35.836+0000: 22183: error : qemuMonitorIO:718 : internal error: End of file from qemu monitor 2019-03-25 14:46:49.492+0000: 22184: warning : qemuDomainObjTaint:7521 : Domain id=3 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is tainted: host-cpu 2019-03-25 14:48:51.084+0000: 22187: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected (END)
On Mon, Mar 25, 2019 at 4:04 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Hi, just seen: 2019-03-25 10:41:36,062-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-hostUpdatesChecker-Thread-2) [] EVENT_ID: HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates on host lago-basic-suite-master-host-1 with message 'Failed to run check-update of host 'lago-basic-suite-master-host-1'. Error: fatal: [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "msg": " http://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\n\n\n One of the configured repositories failed (Latest oVirt nightly),\n and yum doesn't have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work \"fix\" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=alocalsync ...\n\n 4. Disable the repository permanently, so yum won't use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable alocalsync\n or\n subscription-manager repos --disable=alocalsync\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=alocalsync.skip_if_unavailable=true\n\nfailure: repodata/repomd.xml from alocalsync: [Errno 256] No more mirrors to try.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nCannot upload enabled repos report, is this client registered?\n", "rc": 1, "results": []}
while looking at https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/3867/a...
for https://gerrit.ovirt.org/98862
Can you please have a look?
--
SANDRO BONAZZOLA
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://red.ht/sig>
-- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://red.ht/sig>
yes. I am not sure where its coming from but it seems its trying to run some upgrade flow (which is odd in basic). However, its not related to the actual failure. we will follow up to see if it repeat once the hotplug cpu issue is resolved. On Mon, Mar 25, 2019 at 5:04 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Il giorno lun 25 mar 2019 alle ore 17:28 Dafna Ron <dron@redhat.com> ha scritto:
from what I can see, the host is unreachable at the time this is running. The actual job failed due to a reported issue (which is the hotplug_cpu) in libvirt:
I agree, the failure doesn't depend on this host being unreachable, but still, it should have been reachable, right?
2019-03-25 14:28:38.324+0000: 8010: info : libvirt version: 4.5.0, package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>, 2019-01-29-17:31:22, x86-01.bsys.centos.org) 2019-03-25 14:28:38.324+0000: 8010: info : hostname: lago-basic-suite-master-host-0 2019-03-25 14:28:38.324+0000: 8010: error : virNetSocketReadWire:1806 : End of file while reading data: Input/output error 2019-03-25 14:42:54.353+0000: 22186: info : libvirt version: 4.5.0, package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>, 2019-01-29-17:31:22, x86-01.bsys.centos.org) 2019-03-25 14:42:54.353+0000: 22186: info : hostname: lago-basic-suite-master-host-0 2019-03-25 14:42:54.353+0000: 22186: warning : qemuDomainObjTaint:7521 : Domain id=1 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is tainted: host-cpu 2019-03-25 14:43:51.078+0000: 22188: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected 2019-03-25 14:45:12.597+0000: 22186: warning : qemuDomainObjTaint:7521 : Domain id=2 name='vm0' uuid=ed91d5d5-d342-4537-9807-a1fe4423ddcd is tainted: hook-script 2019-03-25 14:45:12.692+0000: 22186: error : virProcessRunInMountNamespace:1159 : internal error: child reported: unable to set security context 'system_u:object_r:virt_content_t:s0' on '/rhev/data-center/mnt/blockSD/cf0b3c9b-b447-48a8-a74a-ca8ffa775965/images/4319f48b-2d60-4295-886b-ecaf9f13d5d8/b1da85a2-3668-4ba2-be71-11e38e6ad772': No such file or directory 2019-03-25 14:45:33.130+0000: 22188: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected 2019-03-25 14:46:35.836+0000: 22183: error : qemuMonitorIO:718 : internal error: End of file from qemu monitor 2019-03-25 14:46:49.492+0000: 22184: warning : qemuDomainObjTaint:7521 : Domain id=3 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is tainted: host-cpu 2019-03-25 14:48:51.084+0000: 22187: error : qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest agent is not connected (END)
On Mon, Mar 25, 2019 at 4:04 PM Sandro Bonazzola <sbonazzo@redhat.com> wrote:
Hi, just seen: 2019-03-25 10:41:36,062-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-hostUpdatesChecker-Thread-2) [] EVENT_ID: HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates on host lago-basic-suite-master-host-1 with message 'Failed to run check-update of host 'lago-basic-suite-master-host-1'. Error: fatal: [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "msg": " http://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\n\n\n One of the configured repositories failed (Latest oVirt nightly),\n and yum doesn't have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work \"fix\" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=alocalsync ...\n\n 4. Disable the repository permanently, so yum won't use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable alocalsync\n or\n subscription-manager repos --disable=alocalsync\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=alocalsync.skip_if_unavailable=true\n\nfailure: repodata/repomd.xml from alocalsync: [Errno 256] No more mirrors to try.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nCannot upload enabled repos report, is this client registered?\n", "rc": 1, "results": []}
while looking at https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/3867/a...
for https://gerrit.ovirt.org/98862
Can you please have a look?
--
SANDRO BONAZZOLA
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://red.ht/sig>
--
SANDRO BONAZZOLA
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://red.ht/sig>
participants (2)
-
Dafna Ron -
Sandro Bonazzola