Hi, just seen: 2019-03-25 10:41:36,062-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-hostUpdatesChecker-Thread-2) [] EVENT_ID: HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates on host lago-basic-suite-master-host-1 with message 'Failed to run check-update of host 'lago-basic-suite-master-host-1'. Error: fatal: [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "msg": " http://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\nhttp:// 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying other mirror.\n\n\n One of the configured repositories failed (Latest oVirt nightly),\n and yum doesn't have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work \"fix\" this:\n\n 1. Contact the upstream for the repository and get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for the repository, to point to a working\n upstream. This is most often useful if you are using a newer\n distribution release than is supported by the repository (and the\n packages for the previous distribution release still work).\n\n 3. Run the command with the repository temporarily disabled\n yum --disablerepo=alocalsync ...\n\n 4. Disable the repository permanently, so yum won't use it by default. Yum\n will then just ignore the repository until you permanently enable it\n again or use --enablerepo for temporary usage:\n\n yum-config-manager --disable alocalsync\n or\n subscription-manager repos --disable=alocalsync\n\n 5. Configure the failing repository to be skipped, if it is unavailable.\n Note that yum will try to contact the repo. when it runs most commands,\n so will have to try and fail each time (and thus. yum will be be much\n slower). If it is a very temporary problem though, this is often a nice\n compromise:\n\n yum-config-manager --save --setopt=alocalsync.skip_if_unavailable=true\n\nfailure: repodata/repomd.xml from alocalsync: [Errno 256] No more mirrors to try.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection refused\"\nCannot upload enabled repos report, is this client registered?\n", "rc": 1, "results": []} while looking at https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/386... for https://gerrit.ovirt.org/98862 Can you please have a look? -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

Il giorno lun 25 mar 2019 alle ore 17:28 Dafna Ron <dron(a)redhat.com&gt; ha scritto: > from what I can see, the host is unreachable at the time this is running. > The actual job failed due to a reported issue (which is the hotplug_cpu) > in libvirt: > I agree, the failure doesn't depend on this host being unreachable, but still, it should have been reachable, right? > > 2019-03-25 14:28:38.324+0000: 8010: info : libvirt version: 4.5.0, > package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>;, > 2019-01-29-17:31:22, x86-01.bsys.centos.org) > 2019-03-25 14:28:38.324+0000: 8010: info : hostname: > lago-basic-suite-master-host-0 > 2019-03-25 14:28:38.324+0000: 8010: error : virNetSocketReadWire:1806 : > End of file while reading data: Input/output error > 2019-03-25 14:42:54.353+0000: 22186: info : libvirt version: 4.5.0, > package: 10.el7_6.4 (CentOS BuildSystem <http://bugs.centos.org>;, > 2019-01-29-17:31:22, x86-01.bsys.centos.org) > 2019-03-25 14:42:54.353+0000: 22186: info : hostname: > lago-basic-suite-master-host-0 > 2019-03-25 14:42:54.353+0000: 22186: warning : qemuDomainObjTaint:7521 : > Domain id=1 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is > tainted: host-cpu > 2019-03-25 14:43:51.078+0000: 22188: error : > qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest > agent is not connected > 2019-03-25 14:45:12.597+0000: 22186: warning : qemuDomainObjTaint:7521 : > Domain id=2 name='vm0' uuid=ed91d5d5-d342-4537-9807-a1fe4423ddcd is > tainted: hook-script > 2019-03-25 14:45:12.692+0000: 22186: error : > virProcessRunInMountNamespace:1159 : internal error: child reported: unable > to set security context 'system_u:object_r:virt_content_t:s0' on > '/rhev/data-center/mnt/blockSD/cf0b3c9b-b447-48a8-a74a-ca8ffa775965/images/4319f48b-2d60-4295-886b-ecaf9f13d5d8/b1da85a2-3668-4ba2-be71-11e38e6ad772': > No such file or directory > 2019-03-25 14:45:33.130+0000: 22188: error : > qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest > agent is not connected > 2019-03-25 14:46:35.836+0000: 22183: error : qemuMonitorIO:718 : internal > error: End of file from qemu monitor > 2019-03-25 14:46:49.492+0000: 22184: warning : qemuDomainObjTaint:7521 : > Domain id=3 name='vm2' uuid=c2f4dcf7-3adf-48f9-93aa-96a8742c0674 is > tainted: host-cpu > 2019-03-25 14:48:51.084+0000: 22187: error : > qemuDomainAgentAvailable:9133 : Guest agent is not responding: QEMU guest > agent is not connected > (END) > > > > On Mon, Mar 25, 2019 at 4:04 PM Sandro Bonazzola <sbonazzo(a)redhat.com&gt; > wrote: > >> Hi, >> just seen: >> 2019-03-25 10:41:36,062-04 ERROR >> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] >> (EE-ManagedThreadFactory-hostUpdatesChecker-Thread-2) [] EVENT_ID: >> HOST_AVAILABLE_UPDATES_FAILED(839), Failed to check for available updates >> on host lago-basic-suite-master-host-1 with message 'Failed to run >> check-update of host 'lago-basic-suite-master-host-1'. Error: fatal: >> [lago-basic-suite-master-host-1]: FAILED! => {"changed": false, "msg": " >> http://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] >> curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nTrying other mirror.\nhttp:// >> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - >> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying >> other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nTrying other mirror.\nhttp:// >> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - >> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying >> other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nTrying other mirror.\nhttp:// >> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - >> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying >> other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nTrying other mirror.\nhttp:// >> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - >> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying >> other mirror.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nTrying other mirror.\nhttp:// >> 192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - >> \"Failed connect to 192.168.201.1:8585; Connection refused\"\nTrying >> other mirror.\n\n\n One of the configured repositories failed (Latest oVirt >> nightly),\n and yum doesn't have enough cached data to continue. At this >> point the only\n safe thing yum can do is fail. There are a few ways to >> work \"fix\" this:\n\n 1. Contact the upstream for the repository and >> get them to fix the problem.\n\n 2. Reconfigure the baseurl/etc. for >> the repository, to point to a working\n upstream. This is most often >> useful if you are using a newer\n distribution release than is >> supported by the repository (and the\n packages for the previous >> distribution release still work).\n\n 3. Run the command with the >> repository temporarily disabled\n yum --disablerepo=alocalsync >> ...\n\n 4. Disable the repository permanently, so yum won't use it by >> default. Yum\n will then just ignore the repository until you >> permanently enable it\n again or use --enablerepo for temporary >> usage:\n\n yum-config-manager --disable alocalsync\n >> or\n subscription-manager repos --disable=alocalsync\n\n 5. >> Configure the failing repository to be skipped, if it is unavailable.\n >> Note that yum will try to contact the repo. when it runs most >> commands,\n so will have to try and fail each time (and thus. yum >> will be be much\n slower). If it is a very temporary problem though, >> this is often a nice\n compromise:\n\n yum-config-manager >> --save --setopt=alocalsync.skip_if_unavailable=true\n\nfailure: >> repodata/repomd.xml from alocalsync: [Errno 256] No more mirrors to >> try.\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: [Errno >> 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nhttp://192.168.201.1:8585/default/el7/repodata/repomd.xml: >> [Errno 14] curl#7 - \"Failed connect to 192.168.201.1:8585; Connection >> refused\"\nCannot upload enabled repos report, is this client >> registered?\n", "rc": 1, "results": []} >> >> while looking at >> https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/386... >> >> for https://gerrit.ovirt.org/98862 >> >> Can you please have a look? >> >> -- >> >> SANDRO BONAZZOLA >> >> MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV >> >> Red Hat EMEA <https://www.redhat.com/> >> >> sbonazzo(a)redhat.com >> <https://red.ht/sig> >> > -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>