Change 78316,3 (ovirt-engine) failed system tests
Change 78316,3 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/78316/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/833/
Looks like this is a false-positive we need to fix. We seem to have some sporadic failures in Lago log-collection that have to do with fluentd: 07:46:01 [basic-suit] Error while running thread 07:46:01 [basic-suit] Traceback (most recent call last): 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue 07:46:01 [basic-suit] queue.put({'return': func()}) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1435, in _collect_artifacts 07:46:01 [basic-suit] vm.collect_artifacts(path, ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 598, in collect_artifacts 07:46:01 [basic-suit] ignore_nopath=ignore_nopath 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 369, in extract_paths 07:46:01 [basic-suit] return self.provider.extract_paths(paths, *args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 299, in extract_paths 07:46:01 [basic-suit] ignore_nopath=ignore_nopath, 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 240, in extract_paths 07:46:01 [basic-suit] self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp 07:46:01 [basic-suit] raise ExtractPathNoPathError(err.message) 07:46:01 [basic-suit] ExtractPathNoPathError: scp: /var/log/fluentd/fluentdout.20170621.b552738936150792f: No such file or directory 07:46:01 [basic-suit] C0644 8190068 fluentdout.20170621.q5527383034d3a5a1 07:46:01 [basic-suit] @ Collect artifacts: ERROR (in 0:00:04) 07:46:01 [basic-suit] Error occured, aborting 07:46:01 [basic-suit] Traceback (most recent call last): 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 941, in main 07:46:01 [basic-suit] cli_plugins[args.verb].do_run(args) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run 07:46:01 [basic-suit] self._do_run(**vars(args)) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper 07:46:01 [basic-suit] return func(*args, prefix=prefix, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 738, in do_collect 07:46:01 [basic-suit] prefix.collect_artifacts(output, ignore_nopath=not no_skip) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 635, in wrapper 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1439, in collect_artifacts 07:46:01 [basic-suit] self.virt_env.get_vms().values(), 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 100, in invoke_in_parallel 07:46:01 [basic-suit] vt.join_all() 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue 07:46:01 [basic-suit] queue.put({'return': func()}) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1435, in _collect_artifacts 07:46:01 [basic-suit] vm.collect_artifacts(path, ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 598, in collect_artifacts 07:46:01 [basic-suit] ignore_nopath=ignore_nopath 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 369, in extract_paths 07:46:01 [basic-suit] return self.provider.extract_paths(paths, *args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 299, in extract_paths 07:46:01 [basic-suit] ignore_nopath=ignore_nopath, 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 240, in extract_paths 07:46:01 [basic-suit] self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp 07:46:01 [basic-suit] raise ExtractPathNoPathError(err.message) 07:46:01 [basic-suit] ExtractPathNoPathError: scp: /var/log/fluentd/fluentdout.20170621.b552738936150792f: No such file or directory 07:46:01 [basic-suit] C0644 8190068 fluentdout.20170621.q5527383034d3a5a1 07:46:01 [basic-suit] + res=1 07:46:01 [basic-suit] + exit 1 ---------- Forwarded message ---------- From: oVirt Jenkins <jenkins@ovirt.org> Date: 21 June 2017 at 10:46 Subject: Change 78316,3 (ovirt-engine) failed system tests To: infra@ovirt.org Change 78316,3 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/78316/3 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/833/ _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
Gal/Nadav, Can you have a look? if needed, let's open a ticket on it on JIRA. On Wed, Jun 21, 2017 at 11:19 AM, Barak Korren <bkorren@redhat.com> wrote:
Looks like this is a false-positive we need to fix.
We seem to have some sporadic failures in Lago log-collection that have to do with fluentd:
07:46:01 [basic-suit] Error while running thread 07:46:01 [basic-suit] Traceback (most recent call last): 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue 07:46:01 [basic-suit] queue.put({'return': func()}) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1435, in _collect_artifacts 07:46:01 [basic-suit] vm.collect_artifacts(path, ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 598, in collect_artifacts 07:46:01 [basic-suit] ignore_nopath=ignore_nopath 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 369, in extract_paths 07:46:01 [basic-suit] return self.provider.extract_paths(paths, *args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 299, in extract_paths 07:46:01 [basic-suit] ignore_nopath=ignore_nopath, 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 240, in extract_paths 07:46:01 [basic-suit] self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp 07:46:01 [basic-suit] raise ExtractPathNoPathError(err.message) 07:46:01 [basic-suit] ExtractPathNoPathError: scp: /var/log/fluentd/fluentdout.20170621.b552738936150792f: No such file or directory 07:46:01 [basic-suit] C0644 8190068 fluentdout.20170621.q5527383034d3a5a1 07:46:01 [basic-suit] @ Collect artifacts: ERROR (in 0:00:04) 07:46:01 [basic-suit] Error occured, aborting 07:46:01 [basic-suit] Traceback (most recent call last): 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 941, in main 07:46:01 [basic-suit] cli_plugins[args.verb].do_run(args) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run 07:46:01 [basic-suit] self._do_run(**vars(args)) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper 07:46:01 [basic-suit] return func(*args, prefix=prefix, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 738, in do_collect 07:46:01 [basic-suit] prefix.collect_artifacts(output, ignore_nopath=not no_skip) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 635, in wrapper 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1439, in collect_artifacts 07:46:01 [basic-suit] self.virt_env.get_vms().values(), 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 100, in invoke_in_parallel 07:46:01 [basic-suit] vt.join_all() 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue 07:46:01 [basic-suit] queue.put({'return': func()}) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1435, in _collect_artifacts 07:46:01 [basic-suit] vm.collect_artifacts(path, ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 598, in collect_artifacts 07:46:01 [basic-suit] ignore_nopath=ignore_nopath 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 369, in extract_paths 07:46:01 [basic-suit] return self.provider.extract_paths(paths, *args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 299, in extract_paths 07:46:01 [basic-suit] ignore_nopath=ignore_nopath, 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 240, in extract_paths 07:46:01 [basic-suit] self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp 07:46:01 [basic-suit] raise ExtractPathNoPathError(err.message) 07:46:01 [basic-suit] ExtractPathNoPathError: scp: /var/log/fluentd/fluentdout.20170621.b552738936150792f: No such file or directory 07:46:01 [basic-suit] C0644 8190068 fluentdout.20170621.q5527383034d3a5a1 07:46:01 [basic-suit] + res=1 07:46:01 [basic-suit] + exit 1
---------- Forwarded message ---------- From: oVirt Jenkins <jenkins@ovirt.org> Date: 21 June 2017 at 10:46 Subject: Change 78316,3 (ovirt-engine) failed system tests To: infra@ovirt.org
Change 78316,3 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed.
This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed.
For further details about the change see: https://gerrit.ovirt.org/#/c/78316/3
For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/833/ _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Eyal edri ASSOCIATE MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
Sure, will check and open a ticket accordingly. On Wed, Jun 21, 2017 at 2:04 PM, Eyal Edri <eedri@redhat.com> wrote:
Gal/Nadav, Can you have a look? if needed, let's open a ticket on it on JIRA.
On Wed, Jun 21, 2017 at 11:19 AM, Barak Korren <bkorren@redhat.com> wrote:
Looks like this is a false-positive we need to fix.
We seem to have some sporadic failures in Lago log-collection that have to do with fluentd:
07:46:01 [basic-suit] Error while running thread 07:46:01 [basic-suit] Traceback (most recent call last): 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue 07:46:01 [basic-suit] queue.put({'return': func()}) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1435, in _collect_artifacts 07:46:01 [basic-suit] vm.collect_artifacts(path, ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 598, in collect_artifacts 07:46:01 [basic-suit] ignore_nopath=ignore_nopath 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 369, in extract_paths 07:46:01 [basic-suit] return self.provider.extract_paths(paths, *args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 299, in extract_paths 07:46:01 [basic-suit] ignore_nopath=ignore_nopath, 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 240, in extract_paths 07:46:01 [basic-suit] self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp 07:46:01 [basic-suit] raise ExtractPathNoPathError(err.message) 07:46:01 [basic-suit] ExtractPathNoPathError: scp: /var/log/fluentd/fluentdout.20170621.b552738936150792f: No such file or directory 07:46:01 [basic-suit] C0644 8190068 fluentdout.20170621.q5527383034d3a5a1 07:46:01 [basic-suit] @ Collect artifacts: ERROR (in 0:00:04) 07:46:01 [basic-suit] Error occured, aborting 07:46:01 [basic-suit] Traceback (most recent call last): 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 941, in main 07:46:01 [basic-suit] cli_plugins[args.verb].do_run(args) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run 07:46:01 [basic-suit] self._do_run(**vars(args)) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper 07:46:01 [basic-suit] return func(*args, prefix=prefix, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 738, in do_collect 07:46:01 [basic-suit] prefix.collect_artifacts(output, ignore_nopath=not no_skip) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 635, in wrapper 07:46:01 [basic-suit] return func(*args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1439, in collect_artifacts 07:46:01 [basic-suit] self.virt_env.get_vms().values(), 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 100, in invoke_in_parallel 07:46:01 [basic-suit] vt.join_all() 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue 07:46:01 [basic-suit] queue.put({'return': func()}) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1435, in _collect_artifacts 07:46:01 [basic-suit] vm.collect_artifacts(path, ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 598, in collect_artifacts 07:46:01 [basic-suit] ignore_nopath=ignore_nopath 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 369, in extract_paths 07:46:01 [basic-suit] return self.provider.extract_paths(paths, *args, **kwargs) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 299, in extract_paths 07:46:01 [basic-suit] ignore_nopath=ignore_nopath, 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 240, in extract_paths 07:46:01 [basic-suit] self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath) 07:46:01 [basic-suit] File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp 07:46:01 [basic-suit] raise ExtractPathNoPathError(err.message) 07:46:01 [basic-suit] ExtractPathNoPathError: scp: /var/log/fluentd/fluentdout.20170621.b552738936150792f: No such file or directory 07:46:01 [basic-suit] C0644 8190068 fluentdout.20170621.q5527383034d3a5a1 07:46:01 [basic-suit] + res=1 07:46:01 [basic-suit] + exit 1
---------- Forwarded message ---------- From: oVirt Jenkins <jenkins@ovirt.org> Date: 21 June 2017 at 10:46 Subject: Change 78316,3 (ovirt-engine) failed system tests To: infra@ovirt.org
Change 78316,3 (ovirt-engine) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed.
This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed.
For further details about the change see: https://gerrit.ovirt.org/#/c/78316/3
For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/833/ _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
--
Eyal edri
ASSOCIATE MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ)
participants (4)
-
Barak Korren -
Eyal Edri -
Nadav Goldin -
oVirt Jenkins