I've updated this patch, it is running. to remove /dev/shm https://gerrit.ovirt.org/#/c/106627/ On Wed, Mar 11, 2020 at 8:30 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: > On Wed, Mar 11, 2020 at 3:43 AM <jenkins(a)jenkins.phx.ovirt.org&gt; wrote: > > > > Project: > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-role-remote-sui... > > Build: > https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-role-remote-sui... > > 03:30:11 + yum install '--disablerepo=*' > > --enablerepo=ovirt-4.3-snapshot-static-el7,sac-gluster-ansible-el7,glusterfs-6-el7,centos-updates-el7,centos-base-el7,centos-extras-el7,epel-el7,centos-ovirt-4.3-el7,centos-ovirt-common-el7,centos-qemu-ev-testing-el7,centos-opstools-testing-el7,centos-sclo-rh-release-el7 > -y yum-utils > 03:30:11 Error: Error writing to file > /dev/shm/yum/x86_64/7/epel-el7/gen/primary_db.sqlite: [Errno 28] No > space left on device > > What's the status of this? > > Thanks, > -- > Didi > > -- GALIT ROSENTHAL SOFTWARE ENGINEER Red Hat <https://www.redhat.com/> galit(a)redhat.com T: 972-9-7692230 <https://red.ht/sig>

On Wed, Mar 11, 2020 at 10:41 AM Galit Rosenthal <grosenth(a)redhat.com&gt; wrote: > > Hi Didi, > > Check the error of the suite he-basic-role-remote=suite-4.3 > https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_s... > > > > > DEPRECATION WARNING]: The 'ovirt_host_facts' module has been renamed to > > > 'ovirt_host_info', and the renamed one no longer returns ansible_facts. This > > > feature will be removed in version 2.13. Deprecation warnings can be disabled > > > by setting deprecation_warnings=False in ansible.cfg. That's just a warning, can be ignored for now. We do have an open bug to fix it (perhaps only in 4.4, we'll see). > > > > fatal: [lago-he-basic-role-remote-suite-4-3-host-0]: FAILED! => {"ansible_facts": {"ovirt_hosts": [{"address": "lago-he-basic-role-remote-suite-4-3-host-0.lago.local", "affinity_labels": [], "auto_numa_status": "unknown", "certificate": {"organization": "lago.local", "subject": "O=lago.local,CN=lago-he-basic-role-remote-suite-4-3-host-0.lago.local"}, "cluster": {"href": "/ovirt-engine/api/clusters/03967a8e-6368-11ea-a651-5452c0a8c863", "id": "03967a8e-6368-11ea-a651-5452c0a8c863"}, "comment": "", "cpu": {"speed": 0.0, "topology": {}}, "device_passthrough": {"enabled": false}, "devices": [], "external_network_provider_configurations": [], "external_status": "ok", "hardware_information": {"supported_rng_sources": []}, "hooks": [], "href": "/ovirt-engine/api/hosts/8d06089a-d562-458f-8720-0ba41ba082ef", "id": "8d06089a-d562-458f-8720-0ba41ba082ef", "katello_errata": [], "kdump_status": "unknown", "ksm": {"enabled": false}, "max_scheduling_memory": 0, "memory": 0, "name": "lago-he-basic-role-remote-suite-4-3-host-0", "network_attachments": [], "nics": [], "numa_nodes": [], "numa_supported": false, "os": {"custom_kernel_cmdline": ""}, "permissions": [], "port": 54321, "power_management": {"automatic_pm_enabled": true, "enabled": false, "kdump_detection": true, "pm_proxies": []}, "protocol": "stomp", "se_linux": {}, "spm": {"priority": 5, "status": "none"}, "ssh": {"fingerprint": "SHA256:tiWL/3QtoZ//DFHnGT+B3b80EBBcbpftGhcEgYpB7+8", "port": 22}, "statistics": [], "status": "install_failed", "storage_connection_extensions": [], "summary": {"total": 0}, "tags": [], "transparent_huge_pages": {"enabled": false}, "type": "rhel", "unmanaged_networks": [], "update_available": false, "vgpu_placement": "consolidated"}]}, "attempts": 120, "changed": false} This failed: https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/830... TASK [ovirt.hosted_engine_setup : Wait for the host to be up] ****************** FAILED - RETRYING: Wait for the host to be up (120 retries left). ... FAILED - RETRYING: Wait for the host to be up (1 retries left). Why the host didn't come up? https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/830... 2020-03-11 03:22:22,561-0400 DEBUG otopi.plugins.otopi.packagers.yumpackager yumpackager.verbose:76 Yum queue package dmidecode for install Determining fastest mirrors 2020-03-11 03:22:22,586-0400 ERROR otopi.plugins.otopi.packagers.yumpackager yumpackager.error:85 Yum Cannot queue package dmidecode: failure: repodata/repomd.xml from alocalsync: [Errno 256] No more mirrors to try. http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" http://192.168.200.1:8585/default/el7/repodata/repomd.xml: [Errno 14] curl#7 - "Failed connect to 192.168.200.1:8585; Connection refused" 2020-03-11 03:22:22,586-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/tmp/ovirt-P0rqhLG1My/pythonlib/otopi/context.py", line 132, in _executeMethod method['method']() File "/tmp/ovirt-P0rqhLG1My/otopi-plugins/ovirt-host-deploy/vdsm/vdsmid.py", line 84, in _packages self.packager.install(('dmidecode',)) File "/tmp/ovirt-P0rqhLG1My/otopi-plugins/otopi/packagers/yumpackager.py", line 293, in install ignoreErrors=ignoreErrors File "/tmp/ovirt-P0rqhLG1My/pythonlib/otopi/miniyum.py", line 852, in install **kwargs File "/tmp/ovirt-P0rqhLG1My/pythonlib/otopi/miniyum.py", line 495, in _queue provides = self._queryProvides(packages=(package,)) File "/tmp/ovirt-P0rqhLG1My/pythonlib/otopi/miniyum.py", line 433, in _queryProvides for po in self._yb.searchPackageProvides(args=packages): File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 3470, in searchPackageProvides where = self.returnPackagesByDep(arg) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 4296, in returnPackagesByDep return self.pkgSack.searchProvides(depstring) File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 1075, in <lambda> pkgSack = property(fget=lambda self: self._getSacks(), File "/usr/lib/python2.7/site-packages/yum/__init__.py", line 778, in _getSacks self.repos.populateSack(which=repos) File "/usr/lib/python2.7/site-packages/yum/repos.py", line 386, in populateSack sack.populate(repo, mdtype, callback, cacheonly) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 227, in populate if self._check_db_version(repo, mydbtype): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 319, in _check_db_version return repo._check_db_version(mdtype) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1515, in _check_db_version repoXML = self.repoXML File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1701, in <lambda> repoXML = property(fget=lambda self: self._getRepoXML(), File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1697, in _getRepoXML self._loadRepoXML(text=self.ui_id) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1688, in _loadRepoXML return self._groupLoadRepoXML(text, self._mdpolicy2mdtypes()) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1662, in _groupLoadRepoXML if self._commonLoadRepoXML(text): File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1482, in _commonLoadRepoXML result = self._getFileRepoXML(local, text) File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1259, in _getFileRepoXML size=102400) # setting max size as 100K File "/usr/lib/python2.7/site-packages/yum/yumRepo.py", line 1042, in _getFile raise e NoMoreMirrorsRepoError: failure: repodata/repomd.xml from alocalsync: [Errno 256] No more mirrors to try. host-deploy tried to install (with otopi's yum module) the package 'dmidecode', but failed to connect to 192.168.200.1:8585 . No idea why, or whether that's related to your patch. So it failed, and the host never came up from the engine's POV (which is what hosted-engine deploy checks).

A side note: These suites, he-basic-role-remote*, do not log ansible nicely as in the "normal" (ovirt-hosted-engine-setup) ones, so we rely only on ansible's output in script/stdout_stderr.log . I agree the output is not clear and needs some work. Feel free to open a bug on this and we'll try to handle. > > > > > > > On Wed, Mar 11, 2020 at 8:34 AM Galit Rosenthal <grosenth(a)redhat.com&gt; wrote: >> >> I've updated this patch, it is running. >> to remove /dev/shm >> https://gerrit.ovirt.org/#/c/106627/ >> >> >> On Wed, Mar 11, 2020 at 8:30 AM Yedidyah Bar David <didi(a)redhat.com&gt; wrote: >>> >>> On Wed, Mar 11, 2020 at 3:43 AM <jenkins(a)jenkins.phx.ovirt.org&gt; wrote: >>> > >>> > Project: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-role-remote-sui... >>> > Build: https://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-role-remote-sui... >>> >>> 03:30:11 + yum install '--disablerepo=*' >>> --enablerepo=ovirt-4.3-snapshot-static-el7,sac-gluster-ansible-el7,glusterfs-6-el7,centos-updates-el7,centos-base-el7,centos-extras-el7,epel-el7,centos-ovirt-4.3-el7,centos-ovirt-common-el7,centos-qemu-ev-testing-el7,centos-opstools-testing-el7,centos-sclo-rh-release-el7 >>> -y yum-utils >>> 03:30:11 Error: Error writing to file >>> /dev/shm/yum/x86_64/7/epel-el7/gen/primary_db.sqlite: [Errno 28] No >>> space left on device >>> >>> What's the status of this? >>> >>> Thanks, >>> -- >>> Didi >>> >> >> >> -- >> >> GALIT ROSENTHAL >> >> SOFTWARE ENGINEER >> >> Red Hat >> >> galit(a)redhat.com T: 972-9-7692230 > > > > -- > > GALIT ROSENTHAL > > SOFTWARE ENGINEER > > Red Hat > > galit(a)redhat.com T: 972-9-7692230 -- Didi