<div dir="ltr">After updating the python sdk:<div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div>myoung at dev-vm in ~/repos/github/ovirt-system-tests on (no branch)▲</div><div>$ rpm -q python-ovirt-engine-sdk4</div><div>python-ovirt-engine-sdk4-4.1.6-2.20170712git1b99f36.el7.centos.x86_64</div></div></blockquote><div><div><br></div><div>I get more but different errors[1].</div><div><br></div><div>Then the lago log[2]</div><div>All engine Logs are even further[3]</div><div><br></div><div>[1]</div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div> # add_cluster: Success (in 0:00:03)</div><div> # add_hosts:</div><div> * Collect artifacts:</div><div> - [Thread-5] lago-basic-suite-4-1-host1: ERROR (in 0:00:19)</div><div> - [Thread-4] lago-basic-suite-4-1-engine: ERROR (in 0:00:20)</div><div>Error while running thread</div><div>Traceback (most recent call last):</div><div> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue</div><div> queue.put({'return': func()})</div><div> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1478, in _collect_artifacts</div><div> vm.collect_artifacts(path, ignore_nopath)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 624, in collect_artifacts</div><div> ignore_nopath=ignore_nopath</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 381, in extract_paths</div><div> return self.provider.extract_paths(paths, *args, **kwargs)</div><div> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 342, in extract_paths</div><div> ignore_nopath=ignore_nopath,</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 247, in extract_paths</div><div> self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp</div><div> propagate_fail=False</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 425, in copy_from</div><div> local_path=local_path,</div><div> File "/usr/lib/python2.7/site-packages/scp.py", line 125, in get</div><div> self._recv_all()</div><div> File "/usr/lib/python2.7/site-packages/scp.py", line 250, in _recv_all</div><div> msg = self.channel.recv(1024)</div><div> File "/usr/lib/python2.7/site-packages/paramiko/channel.py", line 615, in recv</div><div> raise socket.timeout()</div><div>timeout</div><div>Error while running thread</div><div>Traceback (most recent call last):</div><div> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue</div><div> queue.put({'return': func()})</div><div> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1478, in _collect_artifacts</div><div> vm.collect_artifacts(path, ignore_nopath)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 624, in collect_artifacts</div><div> ignore_nopath=ignore_nopath</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 381, in extract_paths</div><div> return self.provider.extract_paths(paths, *args, **kwargs)</div><div> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 342, in extract_paths</div><div> ignore_nopath=ignore_nopath,</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 247, in extract_paths</div><div> self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp</div><div> propagate_fail=False</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 425, in copy_from</div><div> local_path=local_path,</div><div> File "/usr/lib/python2.7/site-packages/scp.py", line 125, in get</div><div> self._recv_all()</div><div> File "/usr/lib/python2.7/site-packages/scp.py", line 250, in _recv_all</div><div> msg = self.channel.recv(1024)</div><div> File "/usr/lib/python2.7/site-packages/paramiko/channel.py", line 615, in recv</div><div> raise socket.timeout()</div><div>timeout</div></div><div><br></div></blockquote></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div>Error while running thread</div><div>Traceback (most recent call last):</div><div> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue</div><div> queue.put({'return': func()})</div><div> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1478, in _collect_artifacts</div><div> vm.collect_artifacts(path, ignore_nopath)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 624, in collect_artifacts</div><div> ignore_nopath=ignore_nopath</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 381, in extract_paths</div><div> return self.provider.extract_paths(paths, *args, **kwargs)</div><div> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 342, in extract_paths</div><div> ignore_nopath=ignore_nopath,</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 247, in extract_paths</div><div> self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp</div><div> propagate_fail=False</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 425, in copy_from</div><div> local_path=local_path,</div><div> File "/usr/lib/python2.7/site-packages/scp.py", line 125, in get</div><div> self._recv_all()</div><div> File "/usr/lib/python2.7/site-packages/scp.py", line 250, in _recv_all</div><div> msg = self.channel.recv(1024)</div><div> File "/usr/lib/python2.7/site-packages/paramiko/channel.py", line 615, in recv</div><div> raise socket.timeout()</div><div>timeout</div><div>Error while running thread</div><div>Traceback (most recent call last):</div><div> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 58, in _ret_via_queue</div><div> queue.put({'return': func()})</div><div> File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1478, in _collect_artifacts</div><div> vm.collect_artifacts(path, ignore_nopath)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 624, in collect_artifacts</div><div> ignore_nopath=ignore_nopath</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 381, in extract_paths</div><div> return self.provider.extract_paths(paths, *args, **kwargs)</div><div> File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/vm.py", line 342, in extract_paths</div><div> ignore_nopath=ignore_nopath,</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 247, in extract_paths</div><div> self._extract_paths_scp(paths=paths, ignore_nopath=ignore_nopath)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 266, in _extract_paths_scp</div><div> propagate_fail=False</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/vm.py", line 425, in copy_from</div><div> local_path=local_path,</div><div> File "/usr/lib/python2.7/site-packages/scp.py", line 125, in get</div><div> self._recv_all()</div><div> File "/usr/lib/python2.7/site-packages/scp.py", line 250, in _recv_all</div><div> msg = self.channel.recv(1024)</div><div> File "/usr/lib/python2.7/site-packages/paramiko/channel.py", line 615, in recv</div><div> raise socket.timeout()</div><div>timeout</div><div> * Collect artifacts: ERROR (in 0:00:33)</div></div><div><br></div></blockquote></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div> # add_hosts: ERROR (in 0:15:52)</div><div> # Collect artifacts:</div><div> * [Thread-19] lago-basic-suite-4-1-engine:</div><div> * [Thread-20] lago-basic-suite-4-1-host1:</div><div> * [Thread-21] lago-basic-suite-4-1-host0:</div><div> * [Thread-21] lago-basic-suite-4-1-host0: Success (in 0:00:03)</div><div> * [Thread-20] lago-basic-suite-4-1-host1: Success (in 0:00:05)</div><div> * [Thread-19] lago-basic-suite-4-1-engine: Success (in 0:00:16)</div><div> # Collect artifacts: Success (in 0:00:16)</div><div> # Results located at /home/myoung/repos/github/ovirt-system-tests/deployment-basic-suite-4.1/default/002_bootstrap.py.junit.xml</div><div>@ Run test: 002_bootstrap.py: ERROR (in 0:16:57)</div><div>Error occured, aborting</div><div>Traceback (most recent call last):</div><div> File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 360, in do_run</div><div> self.cli_plugins[args.ovirtverb].do_run(args)</div><div> File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run</div><div> self._do_run(**vars(args))</div><div> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper</div><div> return func(*args, **kwargs)</div><div> File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper</div><div> return func(*args, prefix=prefix, **kwargs)</div><div> File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 99, in do_ovirt_runtest</div><div> raise RuntimeError('Some tests failed')</div><div>RuntimeError: Some tests failed</div><div>+ res=1</div><div>+ cd -</div><div>/home/myoung/repos/github/ovirt-system-tests</div><div>+ return 1</div><div>+ failed=true</div><div>+ env_collect /home/myoung/repos/github/ovirt-system-tests/test_logs/basic-suite-4.1/post-002_bootstrap.py</div><div>+ local tests_out_dir=/home/myoung/repos/github/ovirt-system-tests/test_logs/basic-suite-4.1/post-002_bootstrap.py</div><div>+ echo '#########################'</div><div>#########################</div><div>+ [[ -e /home/myoung/repos/github/ovirt-system-tests/test_logs/basic-suite-4.1 ]]</div><div>+ cd /home/myoung/repos/github/ovirt-system-tests/deployment-basic-suite-4.1/current</div><div>+ lago collect --output /home/myoung/repos/github/ovirt-system-tests/test_logs/basic-suite-4.1/post-002_bootstrap.py</div><div>@ Collect artifacts:</div><div> # [Thread-1] lago-basic-suite-4-1-engine:</div><div> # [Thread-2] lago-basic-suite-4-1-host1:</div><div> # [Thread-3] lago-basic-suite-4-1-host0:</div><div> # [Thread-3] lago-basic-suite-4-1-host0: Success (in 0:00:02)</div><div> # [Thread-2] lago-basic-suite-4-1-host1: Success (in 0:00:03)</div><div> # [Thread-1] lago-basic-suite-4-1-engine: Success (in 0:00:06)</div><div>@ Collect artifacts: Success (in 0:00:07)</div><div>+ cp -a logs /home/myoung/repos/github/ovirt-system-tests/test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago_logs">post-002_bootstrap.py/lago_logs</a></div><div>+ cd -</div><div>/home/myoung/repos/github/ovirt-system-tests</div><div>+ true</div><div>+ echo '@@@@ ERROR: Failed running /home/myoung/repos/github/ovirt-system-tests/basic-suite-4.1/test-scenarios/002_bootstrap.py'</div><div>@@@@ ERROR: Failed running /home/myoung/repos/github/ovirt-system-tests/basic-suite-4.1/test-scenarios/002_bootstrap.py</div><div>+ return 1</div></div><div><br></div></blockquote>[2]</div></div></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div>$ cat './test_logs/basic-suite-4.1/post-002_bootstrap.py.2017-08-14_21:55:00/lago_logs/lago.log' | fpaste</div><div>Uploading (1202.8KiB)...<br></div><div><a href="https://da.gd/6aw2">https://da.gd/6aw2</a> -> <a href="https://paste.fedoraproject.org/paste/956xJBRY-m1D9T7giQktHQ/">https://paste.fedoraproject.org/paste/956xJBRY-m1D9T7giQktHQ/</a><br></div></div><div><br></div></blockquote><div><div><div>[3]</div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>$ for i in $(find test_logs/basic-suite-4.1/post-002_bootstrap.py -name *.log | grep ovirt-engine); do echo $i; cat $i | fpaste; echo ----; done</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170814212716-ql8ekq.log">post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170814212716-ql8ekq.log</a></div><div>Uploading (5544.7KiB)...<br></div><div><a href="https://da.gd/QLkSD">https://da.gd/QLkSD</a> -> <a href="https://paste.fedoraproject.org/paste/tQEEAfL3TQk7nYhrdyQIVQ/">https://paste.fedoraproject.org/paste/tQEEAfL3TQk7nYhrdyQIVQ/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/boot.log">post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/boot.log</a></div><div>Uploading (1.5KiB)...</div><div><a href="https://da.gd/eX4c6">https://da.gd/eX4c6</a> -> <a href="https://paste.fedoraproject.org/paste/7YFWv6uVzARJ8q0iCAhUNQ/">https://paste.fedoraproject.org/paste/7YFWv6uVzARJ8q0iCAhUNQ/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/server.log">post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/server.log</a></div><div>Uploading (33.9KiB)...</div><div><a href="https://da.gd/n1JTd">https://da.gd/n1JTd</a> -> <a href="https://paste.fedoraproject.org/paste/~cSehNxN77eGxprxliE3dQ/">https://paste.fedoraproject.org/paste/~cSehNxN77eGxprxliE3dQ/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/engine.log">post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/engine.log</a></div><div>Uploading (2697.0KiB)...<br></div><div><a href="https://da.gd/pRDI">https://da.gd/pRDI</a> -> <a href="https://paste.fedoraproject.org/paste/-jJF5TRug~pcQ7syEK~qnA/">https://paste.fedoraproject.org/paste/-jJF5TRug~pcQ7syEK~qnA/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine-dwh/ovirt-engine-dwhd.log">post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine-dwh/ovirt-engine-dwhd.log</a></div><div>Uploading (2.4KiB)...</div><div><a href="https://da.gd/7mkNW">https://da.gd/7mkNW</a> -> <a href="https://paste.fedoraproject.org/paste/PjqtJIsVK~1UzHd1PtjUJA/">https://paste.fedoraproject.org/paste/PjqtJIsVK~1UzHd1PtjUJA/</a></div><div>----</div></blockquote></div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 14, 2017 at 2:02 PM, Marc Young <span dir="ltr"><<a href="mailto:3vilpenguin@gmail.com" target="_blank">3vilpenguin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="font-size:12.8px">rpm -q python-ovirt-engine-sdk4</div></blockquote><div> </div><div>Mine is outdated: </div><div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>$ rpm -q python-ovirt-engine-sdk4</div><div>python-ovirt-engine-sdk4-4.1.<wbr>5-2.el7.centos.x86_64</div><div><br></div></blockquote>Ill update that and run again.<br><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div> </div></blockquote><div>Logs (better to send too many than not enough sometimes). Ovirt engine logs are there.</div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><span class=""><div>myoung at dev-vm in ~/repos/github/ovirt-system-<wbr>tests on (no branch)▲</div></span><div>$ for i in $(find test_logs/basic-suite-4.1/<wbr>post-002_bootstrap.py -name *.log); do echo $i; cat $i | fpaste; echo ----; done</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/tuned/tuned.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>tuned/tuned.log</a></div><div>Uploading (3.5KiB)...</div><div><a href="https://da.gd/LiG0" target="_blank">https://da.gd/LiG0</a> -> <a href="https://paste.fedoraproject.org/paste/wed7pYARTmHHTHcf0DvVAQ/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>wed7pYARTmHHTHcf0DvVAQ/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/audit/audit.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>audit/audit.log</a></div><div>Uploading (401.2KiB)...</div><div><a href="https://da.gd/tgSw" target="_blank">https://da.gd/tgSw</a> -> <a href="https://paste.fedoraproject.org/paste/WEqegOFiLpiNGCv0hvY0ww/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>WEqegOFiLpiNGCv0hvY0ww/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/yum.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/yum.<wbr>log</a></div><div>Uploading (24.3KiB)...</div><div><a href="https://da.gd/BzoMg" target="_blank">https://da.gd/BzoMg</a> -> <a href="https://paste.fedoraproject.org/paste/qla7z~hRsym8kkCr6NlXFg/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/qla7z~<wbr>hRsym8kkCr6NlXFg/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/setup-ds.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>setup-ds.log</a></div><div>Uploading (37.4KiB)...</div><div><a href="https://da.gd/1HHKA" target="_blank">https://da.gd/1HHKA</a> -> <a href="https://paste.fedoraproject.org/paste/4nRRDoC8lrv5CTxbS6TCxQ/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>4nRRDoC8lrv5CTxbS6TCxQ/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/setup/ovirt-engine-setup-20170814074645-vzih31.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>ovirt-engine/setup/ovirt-<wbr>engine-setup-20170814074645-<wbr>vzih31.log</a></div><div>Uploading (5544.4KiB)...<br></div><div><a href="https://da.gd/xYpYl" target="_blank">https://da.gd/xYpYl</a> -> <a href="https://paste.fedoraproject.org/paste/goz7RQVvusLXjiiheLD5xg/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>goz7RQVvusLXjiiheLD5xg/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/boot.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>ovirt-engine/boot.log</a></div><div>Uploading (1.3KiB)...</div><div><a href="https://da.gd/4Y6pE" target="_blank">https://da.gd/4Y6pE</a> -> <a href="https://paste.fedoraproject.org/paste/OCqWe6tjGkrNOrTkqex3mg/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>OCqWe6tjGkrNOrTkqex3mg/</a></div><div>----</div></blockquote></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/server.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>ovirt-engine/server.log</a></div><div>Uploading (34.1KiB)...</div><div><a href="https://da.gd/PZm7d" target="_blank">https://da.gd/PZm7d</a> -> <a href="https://paste.fedoraproject.org/paste/HpZOi8LTrgcWhKqfPeowPA/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>HpZOi8LTrgcWhKqfPeowPA/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine/engine.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>ovirt-engine/engine.log</a></div><div>Uploading (2724.0KiB)...<br></div><div><a href="https://da.gd/pIOO" target="_blank">https://da.gd/pIOO</a> -> <a href="https://paste.fedoraproject.org/paste/ufbtnfZOfSmun8sbOCs1xg/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>ufbtnfZOfSmun8sbOCs1xg/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/ovirt-engine-dwh/ovirt-engine-dwhd.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>ovirt-engine-dwh/ovirt-engine-<wbr>dwhd.log</a></div><div>Uploading (2.4KiB)...</div><div><a href="https://da.gd/Xf5R" target="_blank">https://da.gd/Xf5R</a> -> <a href="https://paste.fedoraproject.org/paste/MO-dlfNQXi19Tla3T~QXsA/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/MO-dlfNQXi19Tla3T~<wbr>QXsA/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-engine/_var_log/snmptrapd.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-engine/_var_log/<wbr>snmptrapd.log</a></div><div>Uploading (0.1KiB)...</div><div><a href="https://da.gd/cF48" target="_blank">https://da.gd/cF48</a> -> <a href="https://paste.fedoraproject.org/paste/NYqECCL1QVihcBQV~VWDsQ/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/NYqECCL1QVihcBQV~<wbr>VWDsQ/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-host1/_var_log/tuned/tuned.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-host1/_var_log/<wbr>tuned/tuned.log</a></div><div>Uploading (1.5KiB)...</div><div><a href="https://da.gd/G5oC" target="_blank">https://da.gd/G5oC</a> -> <a href="https://paste.fedoraproject.org/paste/JgvhOLoA7cEO2Q3ESjP5Aw/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>JgvhOLoA7cEO2Q3ESjP5Aw/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-host1/_var_log/audit/audit.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-host1/_var_log/<wbr>audit/audit.log</a></div><div>Uploading (183.1KiB)...</div><div><a href="https://da.gd/9bjYn" target="_blank">https://da.gd/9bjYn</a> -> <a href="https://paste.fedoraproject.org/paste/Ado1joqnZxkPL1us6UeOwQ/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>Ado1joqnZxkPL1us6UeOwQ/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-host1/_var_log/yum.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-host1/_var_log/yum.<wbr>log</a></div><div>Uploading (18.3KiB)...</div><div><a href="https://da.gd/i5ki" target="_blank">https://da.gd/i5ki</a> -> <a href="https://paste.fedoraproject.org/paste/9sxQziGLRqw1tLHKfBAi6Q/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>9sxQziGLRqw1tLHKfBAi6Q/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-host0/_var_log/tuned/tuned.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-host0/_var_log/<wbr>tuned/tuned.log</a></div><div>Uploading (1.5KiB)...</div><div><a href="https://da.gd/5Iiu4" target="_blank">https://da.gd/5Iiu4</a> -> <a href="https://paste.fedoraproject.org/paste/RaqsqKQdfgcTP1o2Epo~Pg/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/RaqsqKQdfgcTP1o2Epo~<wbr>Pg/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-host0/_var_log/audit/audit.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-host0/_var_log/<wbr>audit/audit.log</a></div><div>Uploading (179.9KiB)...</div><div><a href="https://da.gd/MyZ7" target="_blank">https://da.gd/MyZ7</a> -> <a href="https://paste.fedoraproject.org/paste/x7T7ZkIS3g5JE2dj028AVg/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>x7T7ZkIS3g5JE2dj028AVg/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago-basic-suite-4-1-host0/_var_log/yum.log" target="_blank">post<wbr>-002_bootstrap.py/lago-basic-<wbr>suite-4-1-host0/_var_log/yum.<wbr>log</a></div><div>Uploading (17.4KiB)...</div><div><a href="https://da.gd/Gmux" target="_blank">https://da.gd/Gmux</a> -> <a href="https://paste.fedoraproject.org/paste/kORmR4ShCv1EVGtTWSu2Zw/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/<wbr>kORmR4ShCv1EVGtTWSu2Zw/</a></div><div>----</div><div>test_logs/basic-suite-4.1/<a href="http://post-002_bootstrap.py/lago_logs/lago.log" target="_blank">post<wbr>-002_bootstrap.py/lago_logs/<wbr>lago.log</a></div><div>Uploading (1198.1KiB)...<br></div><div><a href="https://da.gd/vzVCA" target="_blank">https://da.gd/vzVCA</a> -> <a href="https://paste.fedoraproject.org/paste/TuQiMEro23I~OnFcM7wG-g/" target="_blank">https://paste.fedoraproject.<wbr>org/paste/TuQiMEro23I~<wbr>OnFcM7wG-g/</a></div><div>----</div></blockquote></div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 14, 2017 at 1:11 PM, Eyal Edri <span dir="ltr"><<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Mon, Aug 14, 2017 at 7:18 PM, Marc Young <span dir="ltr"><<a href="mailto:3vilpenguin@gmail.com" target="_blank">3vilpenguin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><span class="m_6863539624525807638m_5013702708423523820gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span style="font-size:12.8px">You should have all logs needed @/home/myoung/repos/github/</span><span style="font-size:12.8px">ovi<wbr>rt-system-tests/test_logs/</span><span style="font-size:12.8px">basi<wbr>c-suite-4.1/</span><a href="http://post-002_bootstrap.py/lago_logs" style="font-size:12.8px" target="_blank">post-002_bootstrap<wbr>.py/lago_logs</a></blockquote><div><br></div></span><div>The one I pasted into fpaste was the only one in that directory, if there are more logs Im just unsure where they live. </div><div>Heres all logs for basic suite 4.1 that I know how to find:</div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div>myoung at dev-vm in ~/repos/github/ovirt-system-te<wbr>sts on (no branch)▲</div></div></blockquote></div></div></blockquote><div><br></div></span><div>The engine and hosts logs are usually under the test_logs, which should be in the same subdir as lago_logs directory.</div><div>Also, can you verify you have the latest python v4 SDK installled?</div><div><br></div><div>rpm -q python-ovirt-engine-sdk4</div><div><br></div><div>If you don't have it installed, you can get it directly from the build job ( which acts also as yum repo ):</div><div><br></div><div>el7 build - <a href="http://jenkins.ovirt.org/job/python-ovirt-engine-sdk4_4.1_build-artifacts-el7-x86_64/lastSuccessfulBuild/artifact/exported-artifacts/" target="_blank">http://jenkins.ovirt.org/job/p<wbr>ython-ovirt-engine-sdk4_4.1_bu<wbr>ild-artifacts-el7-x86_64/lastS<wbr>uccessfulBuild/artifact/export<wbr>ed-artifacts/</a><br></div><div><br></div><div>fc26 build is available only for master [1] but should also work.</div><div><br></div><div>There are also fc24/25 builds, just follow the same pattern to find them on Jenkins.</div><div><br></div><div><br></div><div>[1] <a href="http://jenkins.ovirt.org/job/python-ovirt-engine-sdk4_master_build-artifacts-fc26-x86_64/lastSuccessfulBuild/artifact/exported-artifacts/" target="_blank">http://jenkins.ovirt.org/j<wbr>ob/python-ovirt-engine-sdk4_ma<wbr>ster_build-artifacts-fc26-x86_<wbr>64/lastSuccessfulBuild/artifac<wbr>t/exported-artifacts/</a></div><div><div class="m_6863539624525807638h5"><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div>$ for i in $(find . -name 'lago_logs' | grep basic-suite-4.1); do cat $i/*.log | fpaste; done</div><div>Uploading (1355.0KiB)...<br></div><div><a href="https://da.gd/OIlR" target="_blank">https://da.gd/OIlR</a> -> <a href="https://paste.fedoraproject.org/paste/9Xn2RuDgF2pY2I0CouBbGg/" target="_blank">https://paste.fedoraproject.or<wbr>g/paste/9Xn2RuDgF2pY2I0CouBbGg<wbr>/</a></div><div>Uploading (1154.0KiB)...<br></div><div><a href="https://da.gd/0Cti" target="_blank">https://da.gd/0Cti</a> -> <a href="https://paste.fedoraproject.org/paste/uJqvAiDSblnmN8OA6i5tXA/" target="_blank">https://paste.fedoraproject.or<wbr>g/paste/uJqvAiDSblnmN8OA6i5tXA<wbr>/</a></div><div>Uploading (1377.8KiB)...<br></div><div><a href="https://da.gd/GpDIf" target="_blank">https://da.gd/GpDIf</a> -> <a href="https://paste.fedoraproject.org/paste/~Cn6IJgbvateJfFulDNFCg/" target="_blank">https://paste.fedoraproject.or<wbr>g/paste/~Cn6IJgbvateJfFulDNFCg<wbr>/</a></div><div>Uploading (1176.7KiB)...<br></div><div><a href="https://da.gd/skhBy" target="_blank">https://da.gd/skhBy</a> -> <a href="https://paste.fedoraproject.org/paste/HIpEVm4TRPpH5nJCOyUfWA/" target="_blank">https://paste.fedoraproject.or<wbr>g/paste/HIpEVm4TRPpH5nJCOyUfWA<wbr>/</a></div><div>Uploading (1397.1KiB)...<br></div><div><a href="https://da.gd/fEtF" target="_blank">https://da.gd/fEtF</a> -> <a href="https://paste.fedoraproject.org/paste/cuycP~u9D~gJKNPsQ5NH2g/" target="_blank">https://paste.fedoraproject.or<wbr>g/paste/cuycP~u9D~gJKNPsQ5NH2g<wbr>/</a></div><div>Uploading (1198.1KiB)...<br></div><div><a href="https://da.gd/1zNm" target="_blank">https://da.gd/1zNm</a> -> <a href="https://paste.fedoraproject.org/paste/5~yEDwCSr~tdxHT~XUaLPg/" target="_blank">https://paste.fedoraproject.or<wbr>g/paste/5~yEDwCSr~tdxHT~XUaLPg<wbr>/</a></div></div><div><br></div></blockquote></div></div><div class="m_6863539624525807638m_5013702708423523820gmail-HOEnZb"><div class="m_6863539624525807638m_5013702708423523820gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 14, 2017 at 10:14 AM, Yaniv Kaul <span dir="ltr"><<a href="mailto:ykaul@redhat.com" target="_blank">ykaul@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Mon, Aug 14, 2017 at 5:10 PM, Marc Young <span dir="ltr"><<a href="mailto:3vilpenguin@gmail.com" target="_blank">3vilpenguin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Full log is just as uninteresting: <a href="https://paste.fedoraproject.org/paste/ZaZMcT5sg9SAGQ7OKC3-EA" target="_blank">https://paste.f<wbr>edoraproject.org/paste/ZaZMcT5<wbr>sg9SAGQ7OKC3-EA</a></div></blockquote><div><br></div></span><div>I was actually hoping for the engine log.</div><div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764h5"><div>Y.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152HOEnZb"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 14, 2017 at 8:03 AM, Yaniv Kaul <span dir="ltr"><<a href="mailto:ykaul@redhat.com" target="_blank">ykaul@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Mon, Aug 14, 2017 at 3:31 PM, Marc Young <span dir="ltr"><<a href="mailto:3vilpenguin@gmail.com" target="_blank">3vilpenguin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks for clarifying, that makes sense in hindsight since its testing things in their entirety. This is more of a rabbit hole for me as Im really just trying to learn Lago by using the OST project (since I'll be duplicating a lot of the setup).<div><br></div><div>stdout logs from a run: <a href="https://pastebin.com/KBDaCCYp" target="_blank">https://pastebin.com/KBDa<wbr>CCYp</a></div></div></blockquote><div><br></div></span><div>Can you send engine log so we'll understand why the host installation failed?</div><div>You should have all logs needed @/home/myoung/repos/github/ovi<wbr>rt-system-tests/test_logs/basi<wbr>c-suite-4.1/<a href="http://post-002_bootstrap.py/lago_logs" target="_blank">post-002_bootstrap<wbr>.py/lago_logs</a></div><span class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971HOEnZb"><font color="#888888"><div><br></div><div>Y.</div></font></span><div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971h5"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-HOEnZb"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 14, 2017 at 6:44 AM, Eyal Edri <span dir="ltr"><<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Mon, Aug 14, 2017 at 2:31 PM, Marc Young <span dir="ltr"><<a href="mailto:3vilpenguin@gmail.com" target="_blank">3vilpenguin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Ill try to get some detailed log files later, but fwiw I'm not running the hc suites (afaik, Im still getting inundated with the system tests/lago). The link I used for Jenkins was just to try to pull the latest 'passing' hash for ovirt-system-tests</div></blockquote><div><br></div></span><div>I see, its a bit more complicated than that I'm afraid, the HASH you see is probably on for OST code itself, while the suites can fail on any oVirt project ( ovirt-engine/vdsm/host-deploy/<wbr>etc.... ), so its more than just a single hash, but rather a list of RPMs and their versions, together with hash of OST ( for the tests themselves ) and also OS updates. </div><span><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>The errors have all been from master branch on the basic suite for 4.1 via:</div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>$ ./run_suite.sh basic-suite-4.1/<br></div></blockquote></div></div></blockquote><div><br></div></span><div>OK, then we need to investigate it and understand why add host fails on 4.1, please provide logs when possible.</div><div><br></div><div>BTW, you can also try running the manual job [1], but you'll need to provide it with custom yum repo URL with your built artifacts to test your code.</div><div>There is a section on it on the OST readthedocs page.</div><div><br></div><div>[1] <a href="http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_manual/" target="_blank">http://jenkins.ovirt.org/v<wbr>iew/oVirt%20system%20tests/job<wbr>/ovirt-system-tests_manual/</a></div><div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109h5"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div></div></blockquote></div></div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-HOEnZb"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 14, 2017 at 12:29 AM, Eyal Edri <span dir="ltr"><<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">I see you run the 'hc' suite, which means hyperconverged. This suite is running an hosted engine on gluster storage, It's a more complex suite than the basic one, and prone to more errors, however it should still work, but if you don't require it specifically, I would recommend running the basic suite, which should be easier to debug and also quicker to run.<div dir="auto"><br></div><div dir="auto">I'm also adding hc maintainer so she can check, can you share the link to the Jenkins job you're running? Or the log files if you're running locally.</div></div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353HOEnZb"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353h5"><div class="gmail_extra"><br><div class="gmail_quote">On Aug 14, 2017 06:00, "Marc Young" <<a href="mailto:3vilpenguin@gmail.com" target="_blank">3vilpenguin@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Actually I spoke too soon, still fails:<div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div>+ lago ovirt runtest /home/myoung/repos/github/ovir<wbr>t-system-tests/vagrant/test-sc<wbr>enarios/002_bootstrap.py</div><div>@ Run test: 002_bootstrap.py:</div><div>nose.config: INFO: Ignoring files matching ['^\\.', '^_', '^setup\\.py$']</div><div> # print_api_ver:</div><div> # print_api_ver: Success (in 0:00:00)</div><div> # add_dc:</div><div> # add_dc: Success (in 0:00:43)</div><div> # add_cluster:</div><div> # add_cluster: Success (in 0:00:03)</div><div> # add_hosts:</div><div>dd</div><div> * Collect artifacts:</div><div> * Collect artifacts: Success (in 0:01:14)</div><div> # add_hosts: Success (in 0:16:36)</div><div> # Results located at /home/myoung/repos/github/ovir<wbr>t-system-tests/deployment-vagr<wbr>ant/default/002_bootstrap.py.j<wbr>unit.xml</div><div>@ Run test: 002_bootstrap.py: Success (in 0:17:26)</div><div>Error occured, aborting</div><div>Traceback (most recent call last):</div><div> File "/usr/lib/python2.7/site-packa<wbr>ges/ovirtlago/cmd.py", line 360, in do_run</div><div> self.cli_plugins[args.ovirtver<wbr>b].do_run(args)</div><div> File "/usr/lib/python2.7/site-packa<wbr>ges/lago/plugins/cli.py", line 184, in do_run</div><div> self._do_run(**vars(args))</div><div> File "/usr/lib/python2.7/site-packa<wbr>ges/lago/utils.py", line 501, in wrapper</div><div> return func(*args, **kwargs)</div><div> File "/usr/lib/python2.7/site-packa<wbr>ges/lago/utils.py", line 512, in wrapper</div><div> return func(*args, prefix=prefix, **kwargs)</div><div> File "/usr/lib/python2.7/site-packa<wbr>ges/ovirtlago/cmd.py", line 99, in do_ovirt_runtest</div><div> raise RuntimeError('Some tests failed')</div><div>RuntimeError: Some tests failed</div></div><div><br></div><div><br></div></blockquote><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Aug 13, 2017 at 9:47 PM, Marc Young <span dir="ltr"><<a href="mailto:3vilpenguin@gmail.com" target="_blank">3vilpenguin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Edit: reply-all<span><div><br></div><div><span style="font-size:12.8px">It does, every time.</span><div style="font-size:12.8px">I got it to pass by using the last passing revision from Jenkins (hash <span style="color:rgb(51,51,51);font-size:15px;white-space:pre-wrap">98ae6d0b452d098f2703a197<wbr>deb082a091bba837 ), noted from </span><font color="#333333"><span style="font-size:15px;white-space:pre-wrap"><a href="http://jenkins.ovirt.org/job/system-tests_hc-basic-suite-master/15/consoleFull" target="_blank">http://jenkins.ovirt.org/job/s<wbr>ystem-tests_hc-basic-suite-mas<wbr>ter/15/consoleFull</a> </span></font></div></div><div style="font-size:12.8px"><font color="#333333"><span style="font-size:15px;white-space:pre-wrap"><br></span></font></div></span><div><span style="font-size:12.8px">Not sure if it's a true race condition, that build in Jenkins has failed consistently since #15</span></div></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On Sun, Aug 13, 2017 at 2:03 AM, Eyal Edri <span dir="ltr"><<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>></span> wrote:<br></span><div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973h5"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span>On Fri, Aug 11, 2017 at 9:34 PM, Marc Young <span dir="ltr"><<a href="mailto:3vilpenguin@gmail.com" target="_blank">3vilpenguin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">What's the fix for this for those of us using ovirt-system-tests?<div><br><div>Im trying to adapt some of the code for testing a third party tool, but master is still failing on 002_bootstrap</div></div></div></blockquote><div><br></div></span><div>It fails consistently? </div><div>AFAIK this is a race condition that happens maybe once a week, can you share your logs? is it also failing on add_secondary_storage? </div><div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050h5"><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012HOEnZb"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 1, 2017 at 9:20 AM, Benny Zlotnik <span dir="ltr"><<a href="mailto:bzlotnik@redhat.com" target="_blank">bzlotnik@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">I'm not sure it's related since the LSM test runs much later, in 004</div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012m_-3910692678509222295HOEnZb"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012m_-3910692678509222295h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Aug 1, 2017 at 3:33 PM, Eyal Edri <span dir="ltr"><<a href="mailto:eedri@redhat.com" target="_blank">eedri@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Adding Allon & Benny.<div><br></div><div>Is it possible to verify this is related to the LSM issue we've been handling in [1]? </div><div>If this is the case, we agreed to disable the test next time it fails, as the current workarounds with sleep isn't enough.</div><div><br></div><div>Can you confirm this is the case, and so we'll have to disable this test until one of the RFEs described in [1] is merged?</div><div><br></div><div><br></div><div>[1] <a href="https://gerrit.ovirt.org/#/c/78613/" target="_blank">https://gerrit.ovirt.org/#<wbr>/c/78613/</a></div></div><div class="gmail_extra"><div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012m_-3910692678509222295m_-4594783272345111135h5"><br><div class="gmail_quote">On Tue, Aug 1, 2017 at 2:45 PM, Barak Korren <span dir="ltr"><<a href="mailto:bkorren@redhat.com" target="_blank">bkorren@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>On 1 August 2017 at 14:39, Nir Soffer <<a href="mailto:nsoffer@redhat.com" target="_blank">nsoffer@redhat.com</a>> wrote:<br>
><br>
><br>
> On Tue, Aug 1, 2017 at 2:34 PM Barak Korren <<a href="mailto:bkorren@redhat.com" target="_blank">bkorren@redhat.com</a>> wrote:<br>
>><br>
>> Test failed: [ 002_bootstrap.add_secondary_st<wbr>orage_domains ]<br>
>><br>
>> Link to suspected patches:<br>
>> <a href="https://gerrit.ovirt.org/#/c/79974" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/7<wbr>9974</a><br>
><br>
><br>
> This patch adds missing log when resizing an online disk - why do you think<br>
> it is related to the failure?<br>
<br>
</span>Because it is the only patch participating in the test.<br>
(The test was equivalent to running the manual job with just this patch)<br>
<br>
Then again this may also one of the usual SD testing race conditions.<br>
Since the code in the patch seemed to be related to storage I didn't<br>
want to just assume that.<br>
<div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012m_-3910692678509222295m_-4594783272345111135m_4959142784202561159HOEnZb"><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012m_-3910692678509222295m_-4594783272345111135m_4959142784202561159h5"><br>
<br>
--<br>
Barak Korren<br>
RHV DevOps team , RHCE, RHCi<br>
Red Hat EMEA<br>
<a href="http://redhat.com" rel="noreferrer" target="_blank">redhat.com</a> | TRIED. TESTED. TRUSTED. | <a href="http://redhat.com/trusted" rel="noreferrer" target="_blank">redhat.com/trusted</a><br>
______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org" target="_blank">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/devel</a><br>
<br>
<br>
</div></div></blockquote></div><br><br clear="all"><div><br></div></div></div>-- <br><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012m_-3910692678509222295m_-4594783272345111135m_4959142784202561159gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><p style="font-family:overpass,sans-serif;margin:0px;padding:0px;font-size:14px;text-transform:uppercase;font-weight:bold"><font color="#cc0000">Eyal edri</font></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><br></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">ASSOCIATE MANAGER</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">RHV DevOps</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><br></p><p style="font-family:overpass,sans-serif;margin:0px;font-size:10px;color:rgb(153,153,153)"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat EMEA</a></p><table border="0" style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium"><tbody><tr><td width="100px"><a href="https://red.ht/sig" style="color:rgb(17,85,204)" target="_blank"><img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" width="90" height="auto"></a></td><td style="font-size:10px"><a href="https://redhat.com/trusted" style="color:rgb(204,0,0);font-weight:bold" target="_blank">TRIED. TESTED. TRUSTED.</a></td></tr></tbody></table></div><div>phone: <a href="tel:+972%209-769-2018" value="+97297692018" target="_blank">+972-9-7692018</a><br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div></div></div>
</div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org" target="_blank">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/devel</a><br></blockquote></div><br></div>
</div></div></blockquote></div></div></div><div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050h5"><br><br clear="all"><div><br></div>-- <br><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail-m_-3951477646076186353m_8511336199060387257m_-2911890537313925973m_4451550972513183050m_5979519333773425012gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><p style="font-family:overpass,sans-serif;margin:0px;padding:0px;font-size:14px;text-transform:uppercase;font-weight:bold"><font color="#cc0000">Eyal edri</font></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><br></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">ASSOCIATE MANAGER</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">RHV DevOps</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><br></p><p style="font-family:overpass,sans-serif;margin:0px;font-size:10px;color:rgb(153,153,153)"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat EMEA</a></p><table border="0" style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium"><tbody><tr><td width="100px"><a href="https://red.ht/sig" style="color:rgb(17,85,204)" target="_blank"><img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" width="90" height="auto"></a></td><td style="font-size:10px"><a href="https://redhat.com/trusted" style="color:rgb(204,0,0);font-weight:bold" target="_blank">TRIED. TESTED. TRUSTED.</a></td></tr></tbody></table></div><div>phone: <a href="tel:+972%209-769-2018" value="+97297692018" target="_blank">+972-9-7692018</a><br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div></div></div>
</div></div></div></div>
</blockquote></div></div></div><br></div>
</blockquote></div><br></div>
</blockquote></div></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div></div></div><div><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109h5"><br><br clear="all"><div><br></div>-- <br><div class="m_6863539624525807638m_5013702708423523820gmail-m_-1003698394600299764m_-1810784449302661152m_6262165642496514971m_3016493884151474243gmail-m_-1576273945199757109m_4865404260528103245gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><p style="font-family:overpass,sans-serif;margin:0px;padding:0px;font-size:14px;text-transform:uppercase;font-weight:bold"><font color="#cc0000">Eyal edri</font></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><br></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">ASSOCIATE MANAGER</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">RHV DevOps</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><br></p><p style="font-family:overpass,sans-serif;margin:0px;font-size:10px;color:rgb(153,153,153)"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat EMEA</a></p><table border="0" style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium"><tbody><tr><td width="100px"><a href="https://red.ht/sig" style="color:rgb(17,85,204)" target="_blank"><img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" width="90" height="auto"></a></td><td style="font-size:10px"><a href="https://redhat.com/trusted" style="color:rgb(204,0,0);font-weight:bold" target="_blank">TRIED. TESTED. TRUSTED.</a></td></tr></tbody></table></div><div>phone: <a href="tel:+972%209-769-2018" value="+97297692018" target="_blank">+972-9-7692018</a><br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div></div></div>
</div></div></div></div>
</blockquote></div><br></div>
</div></div><br>______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org" target="_blank">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman<wbr>/listinfo/devel</a><br></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><br></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div></div></div><div><div class="m_6863539624525807638h5"><br><br clear="all"><div><br></div>-- <br><div class="m_6863539624525807638m_5013702708423523820gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><p style="font-family:overpass,sans-serif;margin:0px;padding:0px;font-size:14px;text-transform:uppercase;font-weight:bold"><font color="#cc0000">Eyal edri</font></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-weight:bold;margin:0px;padding:0px;font-size:14px;text-transform:uppercase"><br></p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">ASSOCIATE MANAGER</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">RHV DevOps</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase">EMEA VIRTUALIZATION R&D</p><p style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:10px;margin:0px 0px 4px;text-transform:uppercase"><br></p><p style="font-family:overpass,sans-serif;margin:0px;font-size:10px;color:rgb(153,153,153)"><a href="https://www.redhat.com/" style="color:rgb(0,136,206);margin:0px" target="_blank">Red Hat EMEA</a></p><table border="0" style="color:rgb(0,0,0);font-family:overpass,sans-serif;font-size:medium"><tbody><tr><td width="100px"><a href="https://red.ht/sig" style="color:rgb(17,85,204)" target="_blank"><img src="https://www.redhat.com/profiles/rh/themes/redhatdotcom/img/logo-red-hat-black.png" width="90" height="auto"></a></td><td style="font-size:10px"><a href="https://redhat.com/trusted" style="color:rgb(204,0,0);font-weight:bold" target="_blank">TRIED. TESTED. TRUSTED.</a></td></tr></tbody></table></div><div>phone: <a href="tel:+972%209-769-2018" value="+97297692018" target="_blank">+972-9-7692018</a><br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div></div></div>
</div></div></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>