- Infra - oVirt List Archives

[JIRA] (OVIRT-2586) Memory leak in Jenkins SSE Gateway plugin
by Evgheni Dereveanchin (oVirt JIRA) 25 Mar '19

25 Mar '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2586?page=com.atlassian.jira.… ] Evgheni Dereveanchin updated OVIRT-2586: ---------------------------------------- Issue Type: Bug (was: Outage) > Memory leak in Jenkins SSE Gateway plugin > ----------------------------------------- > > Key: OVIRT-2586 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2586 > Project: oVirt - virtualization made easy > Issue Type: Bug > Reporter: sbonazzo > Assignee: Evgheni Dereveanchin > Attachments: usedMemory_year.png > > > Hi, > jenkins is terribly slow and becoming worse every day. > I tried to gain some speed by adding 4 cores to the VM through engine-phx. > It's a bit better but the real issue doesn't seem related to CPU power. > Can anybody investigate? > -- > SANDRO BONAZZOLA > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100099)

1 0

[JIRA] (OVIRT-2586) Memory leak in Jenkins SSE Gateway plugin
by Evgheni Dereveanchin (oVirt JIRA) 25 Mar '19

25 Mar '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2586?page=com.atlassian.jira.… ] Evgheni Dereveanchin updated OVIRT-2586: ---------------------------------------- Summary: Memory leak in Jenkins SSE Gateway plugin (was: Jenkins terribly slow and unresponsive) > Memory leak in Jenkins SSE Gateway plugin > ----------------------------------------- > > Key: OVIRT-2586 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2586 > Project: oVirt - virtualization made easy > Issue Type: Outage > Reporter: sbonazzo > Assignee: Evgheni Dereveanchin > Attachments: usedMemory_year.png > > > Hi, > jenkins is terribly slow and becoming worse every day. > I tried to gain some speed by adding 4 cores to the VM through engine-phx. > It's a bit better but the real issue doesn't seem related to CPU power. > Can anybody investigate? > -- > SANDRO BONAZZOLA > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100099)

1 0

[JIRA] (OVIRT-2586) Jenkins terribly slow and unresponsive
by Evgheni Dereveanchin (oVirt JIRA) 25 Mar '19

25 Mar '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2586?page=com.atlassian.jira.… ] Evgheni Dereveanchin updated OVIRT-2586: ---------------------------------------- Priority: Medium (was: Highest) > Jenkins terribly slow and unresponsive > -------------------------------------- > > Key: OVIRT-2586 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2586 > Project: oVirt - virtualization made easy > Issue Type: Outage > Reporter: sbonazzo > Assignee: Evgheni Dereveanchin > Attachments: usedMemory_year.png > > > Hi, > jenkins is terribly slow and becoming worse every day. > I tried to gain some speed by adding 4 cores to the VM through engine-phx. > It's a bit better but the real issue doesn't seem related to CPU power. > Can anybody investigate? > -- > SANDRO BONAZZOLA > MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV > Red Hat EMEA <https://www.redhat.com/> > sbonazzo(a)redhat.com > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100099)

1 0

[JIRA] (OVIRT-2707) please remove me from reviewers on ovirt-engine
by Eyal Edri (oVirt JIRA) 25 Mar '19

25 Mar '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2707?page=com.atlassian.jira.… ] Eyal Edri updated OVIRT-2707: ----------------------------- Resolution: Fixed Status: Done (was: To Do) > please remove me from reviewers on ovirt-engine > ----------------------------------------------- > > Key: OVIRT-2707 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2707 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please remove me everywhere from the default reviewers on ovirt-engine > https://gerrit.ovirt.org/#/x/reviewers/p/ovirt-engine > Best wishes, > Greg > -- > GREG SHEREMETA > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > Red Hat NA > <https://www.redhat.com/> > gshereme(a)redhat.com IRC: gshereme > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100099)

1 0

[JIRA] (OVIRT-2707) please remove me from reviewers on ovirt-engine
by Eyal Edri (oVirt JIRA) 25 Mar '19

25 Mar '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2707?page=com.atlassian.jira.… ] Eyal Edri commented on OVIRT-2707: ---------------------------------- I removed you from all the path's from the Gerrit UI. On Mon, Mar 25, 2019 at 2:37 PM Greg Sheremeta (oVirt JIRA) < -- Eyal edri MANAGER RHV/CNV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) > please remove me from reviewers on ovirt-engine > ----------------------------------------------- > > Key: OVIRT-2707 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2707 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please remove me everywhere from the default reviewers on ovirt-engine > https://gerrit.ovirt.org/#/x/reviewers/p/ovirt-engine > Best wishes, > Greg > -- > GREG SHEREMETA > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > Red Hat NA > <https://www.redhat.com/> > gshereme(a)redhat.com IRC: gshereme > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100099)

1 0

[JIRA] (OVIRT-2707) please remove me from reviewers on ovirt-engine
by Greg Sheremeta (oVirt JIRA) 25 Mar '19

25 Mar '19

Greg Sheremeta created OVIRT-2707: ------------------------------------- Summary: please remove me from reviewers on ovirt-engine Key: OVIRT-2707 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2707 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Greg Sheremeta Assignee: infra Hi, Please remove me everywhere from the default reviewers on ovirt-engine https://gerrit.ovirt.org/#/x/reviewers/p/ovirt-engine Best wishes, Greg -- GREG SHEREMETA SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX Red Hat NA <https://www.redhat.com/> gshereme(a)redhat.com IRC: gshereme <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100099)

2 1

[oVirt Jenkins] ovirt-system-tests_compat-4.3-suite-master - Build # 17 - Failure!
by jenkins＠jenkins.phx.ovirt.org 25 Mar '19

25 Mar '19

Project: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/17/ Build Number: 17 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #17 [Milan Zamazal] Add hc-basic-suite-4.3 ----------------- Failed Tests: ----------------- 2 tests failed. FAILED: 002_bootstrap.add_master_storage_domain_43 Error Message: Could not find hosts that are up in DC test-dc -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:eac93cd6-f7d5-4709-b8d4-6abf4ede9ba8:Get ssh client for lago-compat-4-3-suite-master-engine: lago.ssh: DEBUG: end task:eac93cd6-f7d5-4709-b8d4-6abf4ede9ba8:Get ssh client for lago-compat-4-3-suite-master-engine: lago.ssh: DEBUG: Running c6f4be5c on lago-compat-4-3-suite-master-engine: cat /root/multipath.txt lago.ssh: DEBUG: Command c6f4be5c on lago-compat-4-3-suite-master-engine returned with 0 lago.ssh: DEBUG: Command c6f4be5c on lago-compat-4-3-suite-master-engine output: 360014052ab13391ff944443894c2b0f7 360014052caa297d60e9440592a1f3d04 36001405bc22d1395bd6476299a975c10 36001405edd5271bd714406584bfb9d26 36001405f0546ac01e2c444ab58bb99d3 --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 417, in add_master_storage_domain add_iscsi_storage_domain(prefix) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 561, in add_iscsi_storage_domain host=_random_host_from_dc(api, DC_NAME), File "/home/jenkins/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 122, in _random_host_from_dc return _hosts_in_dc(api, dc_name, True) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/002_bootstrap.py", line 119, in _hosts_in_dc raise RuntimeError('Could not find hosts that are up in DC %s' % dc_name) 'Could not find hosts that are up in DC test-dc\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:eac93cd6-f7d5-4709-b8d4-6abf4ede9ba8:Get ssh client for lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end task:eac93cd6-f7d5-4709-b8d4-6abf4ede9ba8:Get ssh client for lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: Running c6f4be5c on lago-compat-4-3-suite-master-engine: cat /root/multipath.txt\nlago.ssh: DEBUG: Command c6f4be5c on lago-compat-4-3-suite-master-engine returned with 0\nlago.ssh: DEBUG: Command c6f4be5c on lago-compat-4-3-suite-master-engine output:\n 360014052ab13391ff944443894c2b0f7\n360014052caa297d60e9440592a1f3d04\n36001405bc22d1395bd6476299a975c10\n36001405edd5271bd714406584bfb9d26\n36001405f0546ac01e2c444ab58bb99d3\n\n--------------------- >> end captured logging << ---------------------' FAILED: 002_bootstrap.add_master_storage_domain_43 Error Message: Failed to copy files/directory from /var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-1/_var_log Unable to get valid context for root -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:eac93cd6-f7d5-4709-b8d4-6abf4ede9ba8:Get ssh client for lago-compat-4-3-suite-master-engine: lago.ssh: DEBUG: end task:eac93cd6-f7d5-4709-b8d4-6abf4ede9ba8:Get ssh client for lago-compat-4-3-suite-master-engine: lago.ssh: DEBUG: Running c6f4be5c on lago-compat-4-3-suite-master-engine: cat /root/multipath.txt lago.ssh: DEBUG: Command c6f4be5c on lago-compat-4-3-suite-master-engine returned with 0 lago.ssh: DEBUG: Command c6f4be5c on lago-compat-4-3-suite-master-engine output: 360014052ab13391ff944443894c2b0f7 360014052caa297d60e9440592a1f3d04 36001405bc22d1395bd6476299a975c10 36001405edd5271bd714406584bfb9d26 36001405f0546ac01e2c444ab58bb99d3 lago.prefix: INFO: * Collect artifacts: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.prefix: INFO: ?[0m?[0m lago.plugins.vm: DEBUG: Extracting scp://lago-compat-4-3-suite-master-host-0:/var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-0/_var_log lago.plugins.vm: INFO: ?[0m?[0m lago.plugins.vm: DEBUG: Extracting scp://lago-compat-4-3-suite-master-host-1:/var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-1/_var_log lago.ssh: DEBUG: start task:8f203123-42fa-4a8f-9c1b-7d4dfbd394aa:Get ssh client for lago-compat-4-3-suite-master-host-0: lago.plugins.vm: DEBUG: Extracting scp://lago-compat-4-3-suite-master-engine:/tmp/otopi* to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-engine/_tmp_otopi* lago.plugins.vm: INFO: ?[0m?[0m lago.plugins.vm: INFO: ?[0m?[0m lago.ssh: DEBUG: start task:9f190a0b-bcd0-46c2-93ea-713e1a42be0e:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: start task:6a194b08-2d10-4d75-9af6-f813f6113ea6:Get ssh client for lago-compat-4-3-suite-master-engine: lago.ssh: DEBUG: end task:6a194b08-2d10-4d75-9af6-f813f6113ea6:Get ssh client for lago-compat-4-3-suite-master-engine: lago.ssh: DEBUG: end task:8f203123-42fa-4a8f-9c1b-7d4dfbd394aa:Get ssh client for lago-compat-4-3-suite-master-host-0: lago.ssh: DEBUG: end task:9f190a0b-bcd0-46c2-93ea-713e1a42be0e:Get ssh client for lago-compat-4-3-suite-master-host-1: root: ERROR: ~ [Thread-561] Copy from lago-compat-4-3-suite-master-host-1:/var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-1/_var_log: ?[31mERROR?[0m (in 0:00:00) lago.plugins.vm: DEBUG: File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from remote_path, local_path, err.args[0] root: ERROR: - [Thread-561] lago-compat-4-3-suite-master-host-1: ?[31mERROR?[0m (in 0:00:00) lago.prefix: DEBUG: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp propagate_fail=not ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from remote_path, local_path, err.args[0] lago.utils: DEBUG: Error while running thread Thread-561 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp propagate_fail=not ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from remote_path, local_path, err.args[0] LagoCopyFilesFromVMError: Failed to copy files/directory from /var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-1/_var_log Unable to get valid context for root root: ERROR: ~ [Thread-562] Copy from lago-compat-4-3-suite-master-engine:/tmp/otopi* to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-engine/_tmp_otopi*: ?[31mERROR?[0m (in 0:00:00) lago.plugins.vm: DEBUG: File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 442, in copy_from raise ExtractPathNoPathError(err.args[0]) root: ERROR: - [Thread-562] lago-compat-4-3-suite-master-engine: ?[31mERROR?[0m (in 0:00:00) lago.prefix: DEBUG: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp propagate_fail=not ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 442, in copy_from raise ExtractPathNoPathError(err.args[0]) lago.utils: DEBUG: Error while running thread Thread-562 Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp propagate_fail=not ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 442, in copy_from raise ExtractPathNoPathError(err.args[0]) ExtractPathNoPathError: Failed to extract files: ?scp: /tmp/otopi*: No such file or directory lago.plugins.vm: INFO: ?[32mSuccess?[0m (in 0:00:01) lago.plugins.vm: DEBUG: Extracting scp://lago-compat-4-3-suite-master-host-0:/etc/resolv.conf to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-0/_etc_resolv.conf lago.plugins.vm: INFO: ?[0m?[0m lago.ssh: DEBUG: start task:4f64c62d-26aa-4e98-bd49-e0b46d4a10c3:Get ssh client for lago-compat-4-3-suite-master-host-0: lago.ssh: DEBUG: end task:4f64c62d-26aa-4e98-bd49-e0b46d4a10c3:Get ssh client for lago-compat-4-3-suite-master-host-0: lago.plugins.vm: INFO: ?[32mSuccess?[0m (in 0:00:00) lago.prefix: INFO: ?[31mERROR?[0m (in 0:00:01) root: ERROR: * Collect artifacts: ?[31mERROR?[0m (in 0:00:01) lago.prefix: DEBUG: File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 636, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1530, in collect_artifacts self.virt_env.get_vms().values(), File "/usr/lib/python2.7/site-packages/lago/utils.py", line 103, in invoke_in_parallel return vt.join_all() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp propagate_fail=not ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from remote_path, local_path, err.args[0] --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib/python2.7/site-packages/nose/case.py", line 133, in run self.runTest(result) File "/usr/lib/python2.7/site-packages/nose/case.py", line 151, in runTest test(result) File "/usr/lib64/python2.7/unittest/case.py", line 433, in __call__ return self.run(*args, **kwds) File "/usr/lib64/python2.7/unittest/case.py", line 393, in run result.addError(self, sys.exc_info()) File "/usr/lib/python2.7/site-packages/nose/proxy.py", line 131, in addError plugins.addError(self.test, err) File "/usr/lib/python2.7/site-packages/nose/plugins/manager.py", line 99, in __call__ return self.call(*arg, **kw) File "/usr/lib/python2.7/site-packages/nose/plugins/manager.py", line 167, in simple result = meth(*arg, **kw) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 163, in addError self._addFault(test, err) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 174, in _addFault self._prefix.paths.test_logs(test_name), False File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 636, in wrapper return func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1530, in collect_artifacts self.virt_env.get_vms().values(), File "/usr/lib/python2.7/site-packages/lago/utils.py", line 103, in invoke_in_parallel return vt.join_all() File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue queue.put({'return': func()}) File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts vm.collect_artifacts(path, ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts ignore_nopath=ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths return self.provider.extract_paths(paths, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths ignore_nopath=ignore_nopath, File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper return func(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp propagate_fail=not ignore_nopath File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from remote_path, local_path, err.args[0] 'Failed to copy files/directory from /var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-1/_var_log\nUnable to get valid context for root\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:eac93cd6-f7d5-4709-b8d4-6abf4ede9ba8:Get ssh client for lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end task:eac93cd6-f7d5-4709-b8d4-6abf4ede9ba8:Get ssh client for lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: Running c6f4be5c on lago-compat-4-3-suite-master-engine: cat /root/multipath.txt\nlago.ssh: DEBUG: Command c6f4be5c on lago-compat-4-3-suite-master-engine returned with 0\nlago.ssh: DEBUG: Command c6f4be5c on lago-compat-4-3-suite-master-engine output:\n 360014052ab13391ff944443894c2b0f7\n360014052caa297d60e9440592a1f3d04\n36001405bc22d1395bd6476299a975c10\n36001405edd5271bd714406584bfb9d26\n36001405f0546ac01e2c444ab58bb99d3\n\nlago.prefix: INFO: * Collect artifacts: \x1b[0m\x1b[0m\nlago.prefix: INFO: \x1b[0m\x1b[0m\nlago.prefix: INFO: \x1b[0m\x1b[0m\nlago.prefix: INFO: \x1b[0m\x1b[0m\nlago.plugins.vm: DEBUG: Extracting scp://lago-compat-4-3-suite-master-host-0:/var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-0/_var_log\nlago.plugins.vm: INFO: \x1b[0m\x1b[0m\nlago.plugins.vm: DEBUG: Extracting scp://lago-compat-4-3-suite-master-host-1:/var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-1/_var_log\nlago.ssh: DEBUG: start task:8f203123-42fa-4a8f-9c1b-7d4dfbd394aa:Get ssh client for lago-compat-4-3-suite-master-host-0:\nlago.plugins.vm: DEBUG: Extracting scp://lago-compat-4-3-suite-master-engine:/tmp/otopi* to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-engine/_tmp_otopi*\nlago.plugins.vm: INFO: \x1b[0m\x1b[0m\nlago.plugins.vm: INFO: \x1b[0m\x1b[0m\nlago.ssh: DEBUG: start task:9f190a0b-bcd0-46c2-93ea-713e1a42be0e:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: start task:6a194b08-2d10-4d75-9af6-f813f6113ea6:Get ssh client for lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end task:6a194b08-2d10-4d75-9af6-f813f6113ea6:Get ssh client for lago-compat-4-3-suite-master-engine:\nlago.ssh: DEBUG: end task:8f203123-42fa-4a8f-9c1b-7d4dfbd394aa:Get ssh client for lago-compat-4-3-suite-master-host-0:\nlago.ssh: DEBUG: end task:9f190a0b-bcd0-46c2-93ea-713e1a42be0e:Get ssh client for lago-compat-4-3-suite-master-host-1:\nroot: ERROR: ~ [Thread-561] Copy from lago-compat-4-3-suite-master-host-1:/var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-1/_var_log: \x1b[31mERROR\x1b[0m (in 0:00:00)\nlago.plugins.vm: DEBUG: File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from\n remote_path, local_path, err.args[0]\n\nroot: ERROR: - [Thread-561] lago-compat-4-3-suite-master-host-1: \x1b[31mERROR\x1b[0m (in 0:00:00)\nlago.prefix: DEBUG: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts\n vm.collect_artifacts(path, ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts\n ignore_nopath=ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths\n return self.provider.extract_paths(paths, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths\n ignore_nopath=ignore_nopath,\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths\n self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper\n return func(self, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp\n propagate_fail=not ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from\n remote_path, local_path, err.args[0]\n\nlago.utils: DEBUG: Error while running thread Thread-561\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue\n queue.put({\'return\': func()})\n File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts\n vm.collect_artifacts(path, ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts\n ignore_nopath=ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths\n return self.provider.extract_paths(paths, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths\n ignore_nopath=ignore_nopath,\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths\n self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper\n return func(self, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp\n propagate_fail=not ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from\n remote_path, local_path, err.args[0]\nLagoCopyFilesFromVMError: Failed to copy files/directory from /var/log to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-1/_var_log\nUnable to get valid context for root\nroot: ERROR: ~ [Thread-562] Copy from lago-compat-4-3-suite-master-engine:/tmp/otopi* to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-engine/_tmp_otopi*: \x1b[31mERROR\x1b[0m (in 0:00:00)\nlago.plugins.vm: DEBUG: File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 442, in copy_from\n raise ExtractPathNoPathError(err.args[0])\n\nroot: ERROR: - [Thread-562] lago-compat-4-3-suite-master-engine: \x1b[31mERROR\x1b[0m (in 0:00:00)\nlago.prefix: DEBUG: File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts\n vm.collect_artifacts(path, ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts\n ignore_nopath=ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths\n return self.provider.extract_paths(paths, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths\n ignore_nopath=ignore_nopath,\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths\n self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper\n return func(self, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp\n propagate_fail=not ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 442, in copy_from\n raise ExtractPathNoPathError(err.args[0])\n\nlago.utils: DEBUG: Error while running thread Thread-562\nTraceback (most recent call last):\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue\n queue.put({\'return\': func()})\n File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts\n vm.collect_artifacts(path, ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts\n ignore_nopath=ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths\n return self.provider.extract_paths(paths, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths\n ignore_nopath=ignore_nopath,\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths\n self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper\n return func(self, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp\n propagate_fail=not ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 442, in copy_from\n raise ExtractPathNoPathError(err.args[0])\nExtractPathNoPathError: Failed to extract files: \x01scp: /tmp/otopi*: No such file or directory\nlago.plugins.vm: INFO: \x1b[32mSuccess\x1b[0m (in 0:00:01)\nlago.plugins.vm: DEBUG: Extracting scp://lago-compat-4-3-suite-master-host-0:/etc/resolv.conf to /dev/shm/ost/deployment-compat-4.3-suite-master/default/test_logs/002_bootstrap.add_master_storage_domain_43-20190323005122/lago-compat-4-3-suite-master-host-0/_etc_resolv.conf\nlago.plugins.vm: INFO: \x1b[0m\x1b[0m\nlago.ssh: DEBUG: start task:4f64c62d-26aa-4e98-bd49-e0b46d4a10c3:Get ssh client for lago-compat-4-3-suite-master-host-0:\nlago.ssh: DEBUG: end task:4f64c62d-26aa-4e98-bd49-e0b46d4a10c3:Get ssh client for lago-compat-4-3-suite-master-host-0:\nlago.plugins.vm: INFO: \x1b[32mSuccess\x1b[0m (in 0:00:00)\nlago.prefix: INFO: \x1b[31mERROR\x1b[0m (in 0:00:01)\nroot: ERROR: * Collect artifacts: \x1b[31mERROR\x1b[0m (in 0:00:01)\nlago.prefix: DEBUG: File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 636, in wrapper\n return func(*args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1530, in collect_artifacts\n self.virt_env.get_vms().values(),\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 103, in invoke_in_parallel\n return vt.join_all()\n File "/usr/lib/python2.7/site-packages/lago/utils.py", line 57, in _ret_via_queue\n queue.put({\'return\': func()})\n File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1526, in _collect_artifacts\n vm.collect_artifacts(path, ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 659, in collect_artifacts\n ignore_nopath=ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 379, in extract_paths\n return self.provider.extract_paths(paths, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 398, in extract_paths\n ignore_nopath=ignore_nopath,\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 249, in extract_paths\n self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 99, in wrapper\n return func(self, *args, **kwargs)\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 269, in _extract_paths_scp\n propagate_fail=not ignore_nopath\n File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 445, in copy_from\n remote_path, local_path, err.args[0]\n\n--------------------- >> end captured logging << ---------------------'

1 3

[JIRA] (OVIRT-2706) how do we cache docker images in OST?
by Greg Sheremeta (oVirt JIRA) 25 Mar '19

25 Mar '19

[ https://ovirt-jira.atlassian.net/browse/OVIRT-2706?page=com.atlassian.jira.… ] Greg Sheremeta commented on OVIRT-2706: --------------------------------------- HUB_CONTAINER_IMAGE = 'selenium/hub:3.9.1-actinium' FIREFOX_CONTAINER_IMAGE = 'selenium/node-firefox-debug:3.9.1-actinium' CHROME_CONTAINER_IMAGE = 'selenium/node-chrome-debug:3.9.1-actinium' > how do we cache docker images in OST? > ------------------------------------- > > Key: OVIRT-2706 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-2706 > Project: oVirt - virtualization made easy > Issue Type: Improvement > Components: docker_cleanup.py > Reporter: Greg Sheremeta > Assignee: infra > > Currently we download the selenium grid containers on every run of 008 > basic ui sanity, and the containers are huge. Can we cache them somewhere > so OST doesn't need to download them every time? Please advise how and what > we need to do to enable this. Thanks! > Best wishes, > Greg > -- > GREG SHEREMETA > SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX > Red Hat NA > <https://www.redhat.com/> > gshereme(a)redhat.com IRC: gshereme > <https://red.ht/sig> -- This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100099)

1 0

[CQ]: d117169 (v2v-conversion-host) failed "ovirt-4.3" system tests
by oVirt Jenkins 25 Mar '19

25 Mar '19

Change d117169 (v2v-conversion-host) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://github.com/oVirt/v2v-conversion-host/commit/d117169276558f31d545682… For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/360/

1 0

[CQ]: 98824,4 (ovirt-engine) failed "ovirt-4.3" system tests
by oVirt Jenkins 25 Mar '19

25 Mar '19

Change 98824,4 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-4.3" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/98824/4 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/355/

1 0