Project: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_compat-4.3-suite-master/144/ Build Number: 144 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #144 [jenkins CI] Auto create ovirt-4.3-release ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 004_basic_sanity.vdsm_recovery_43 Error Message: False != True after 180 seconds -------------------- >> begin captured logging << -------------------- lago.ssh: DEBUG: start task:0012d3c1-f5ce-4587-977d-9debfa6e2e19:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: end task:0012d3c1-f5ce-4587-977d-9debfa6e2e19:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: Running 92312e1e on lago-compat-4-3-suite-master-host-1: /usr/bin/systemctl status --lines=0 vdsmd lago.ssh: DEBUG: Command 92312e1e on lago-compat-4-3-suite-master-host-1 returned with 0 lago.ssh: DEBUG: Command 92312e1e on lago-compat-4-3-suite-master-host-1 output: ● vdsmd.service - Virtual Desktop Server Manager Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2019-07-26 21:11:18 EDT; 22min ago Process: 18081 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop (code=exited, status=0/SUCCESS) Process: 18330 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=0/SUCCESS) Main PID: 19523 (vdsmd) Tasks: 90 CGroup: /system.slice/vdsmd.service ├─ 8264 /usr/libexec/ioprocess --read-pipe-fd 58 --write-pipe-fd 57 --max-threads 10 --max-queued-requests 10 ├─ 8271 /usr/libexec/ioprocess --read-pipe-fd 67 --write-pipe-fd 66 --max-threads 10 --max-queued-requests 10 ├─ 8278 /usr/libexec/ioprocess --read-pipe-fd 81 --write-pipe-fd 77 --max-threads 10 --max-queued-requests 10 ├─19523 /usr/bin/python2 /usr/share/vdsm/vdsmd ├─29855 /usr/libexec/ioprocess --read-pipe-fd 66 --write-pipe-fd 65 --max-threads 10 --max-queued-requests 10 ├─29988 /usr/libexec/ioprocess --read-pipe-fd 81 --write-pipe-fd 80 --max-threads 10 --max-queued-requests 10 └─30062 /usr/libexec/ioprocess --read-pipe-fd 91 --write-pipe-fd 90 --max-threads 10 --max-queued-requests 10 lago.ssh: DEBUG: start task:29fa5efa-996a-46e6-976f-407f0aa618c6:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: end task:29fa5efa-996a-46e6-976f-407f0aa618c6:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: Running 92b411da on lago-compat-4-3-suite-master-host-1: /usr/bin/systemctl stop vdsmd lago.ssh: DEBUG: Command 92b411da on lago-compat-4-3-suite-master-host-1 returned with 0 lago.ssh: DEBUG: start task:41d3ddf0-cf9b-47d4-90d8-b4fa88b1164b:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: end task:41d3ddf0-cf9b-47d4-90d8-b4fa88b1164b:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: Running c5f04cda on lago-compat-4-3-suite-master-host-1: /usr/bin/systemctl status --lines=0 vdsmd lago.ssh: DEBUG: Command c5f04cda on lago-compat-4-3-suite-master-host-1 returned with 3 lago.ssh: DEBUG: Command c5f04cda on lago-compat-4-3-suite-master-host-1 output: ● vdsmd.service - Virtual Desktop Server Manager Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled) Active: activating (start-pre) since Fri 2019-07-26 21:35:10 EDT; 351ms ago Process: 8370 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop (code=exited, status=0/SUCCESS) Process: 19523 ExecStart=/usr/share/vdsm/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null /usr/share/vdsm/vdsmd (code=exited, status=0/SUCCESS) Main PID: 19523 (code=exited, status=0/SUCCESS); : 8486 (vdsmd_init_comm) Tasks: 2 CGroup: /system.slice/vdsmd.service └─control ├─8486 /bin/sh /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start └─8498 /usr/bin/python2 /usr/libexec/vdsm/wait_for_ipv4s lago.ssh: DEBUG: start task:e0527d05-8b28-4768-8fdb-6db7420510ea:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: end task:e0527d05-8b28-4768-8fdb-6db7420510ea:Get ssh client for lago-compat-4-3-suite-master-host-1: lago.ssh: DEBUG: Running c66e6bce on lago-compat-4-3-suite-master-host-1: /usr/bin/systemctl start vdsmd lago.ssh: DEBUG: Command c66e6bce on lago-compat-4-3-suite-master-host-1 returned with 0 --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_compat-4.3-suite-master/ovirt-system-tests/compat-4.3-suite-master/test-scenarios/004_basic_sanity.py", line 753, in vdsm_recovery lambda: File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 282, in assert_true_within_short assert_equals_within_short(func, True, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 266, in assert_equals_within_short func, value, SHORT_TIMEOUT, allowed_exceptions=allowed_exceptions File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 252, in assert_equals_within '%s != %s after %s seconds' % (res, value, timeout) 'False != True after 180 seconds\n-------------------- >> begin captured logging << --------------------\nlago.ssh: DEBUG: start task:0012d3c1-f5ce-4587-977d-9debfa6e2e19:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: end task:0012d3c1-f5ce-4587-977d-9debfa6e2e19:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: Running 92312e1e on lago-compat-4-3-suite-master-host-1: /usr/bin/systemctl status --lines=0 vdsmd\nlago.ssh: DEBUG: Command 92312e1e on lago-compat-4-3-suite-master-host-1 returned with 0\nlago.ssh: DEBUG: Command 92312e1e on lago-compat-4-3-suite-master-host-1 output:\n \xe2\x97\x8f vdsmd.service - Virtual Desktop Server Manager\n Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled)\n Active: active (running) since Fri 2019-07-26 21:11:18 EDT; 22min ago\n Process: 18081 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop (code=exited, status=0/SUCCESS)\n Process: 18330 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=0/SUCCESS)\n Main PID: 19523 (vdsmd)\n Tasks: 90\n CGroup: /system.slice/vdsmd.service\n \xe2\x94\x9c\xe2\x94\x80 8264 /usr/libexec/ioprocess --read-pipe-fd 58 --write-pipe-fd 57 --max-threads 10 --max-queued-requests 10\n \xe2\x94\x9c\xe2\x94\x80 8271 /usr/libexec/ioprocess --read-pipe-fd 67 --write-pipe-fd 66 --max-threads 10 --max-queued-requests 10\n \xe2\x94\x9c\xe2\x94\x80 8278 /usr/libexec/ioprocess --read-pipe-fd 81 --write-pipe-fd 77 --max-threads 10 --max-queued-requests 10\n \xe2\x94\x9c\xe2\x94\x8019523 /usr/bin/python2 /usr/share/vdsm/vdsmd\n \xe2\x94\x9c\xe2\x94\x8029855 /usr/libexec/ioprocess --read-pipe-fd 66 --write-pipe-fd 65 --max-threads 10 --max-queued-requests 10\n \xe2\x94\x9c\xe2\x94\x8029988 /usr/libexec/ioprocess --read-pipe-fd 81 --write-pipe-fd 80 --max-threads 10 --max-queued-requests 10\n \xe2\x94\x94\xe2\x94\x8030062 /usr/libexec/ioprocess --read-pipe-fd 91 --write-pipe-fd 90 --max-threads 10 --max-queued-requests 10\n\nlago.ssh: DEBUG: start task:29fa5efa-996a-46e6-976f-407f0aa618c6:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: end task:29fa5efa-996a-46e6-976f-407f0aa618c6:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: Running 92b411da on lago-compat-4-3-suite-master-host-1: /usr/bin/systemctl stop vdsmd\nlago.ssh: DEBUG: Command 92b411da on lago-compat-4-3-suite-master-host-1 returned with 0\nlago.ssh: DEBUG: start task:41d3ddf0-cf9b-47d4-90d8-b4fa88b1164b:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: end task:41d3ddf0-cf9b-47d4-90d8-b4fa88b1164b:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: Running c5f04cda on lago-compat-4-3-suite-master-host-1: /usr/bin/systemctl status --lines=0 vdsmd\nlago.ssh: DEBUG: Command c5f04cda on lago-compat-4-3-suite-master-host-1 returned with 3\nlago.ssh: DEBUG: Command c5f04cda on lago-compat-4-3-suite-master-host-1 output:\n \xe2\x97\x8f vdsmd.service - Virtual Desktop Server Manager\n Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled)\n Active: activating (start-pre) since Fri 2019-07-26 21:35:10 EDT; 351ms ago\n Process: 8370 ExecStopPost=/usr/libexec/vdsm/vdsmd_init_common.sh --post-stop (code=exited, status=0/SUCCESS)\n Process: 19523 ExecStart=/usr/share/vdsm/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null /usr/share/vdsm/vdsmd (code=exited, status=0/SUCCESS)\n Main PID: 19523 (code=exited, status=0/SUCCESS); : 8486 (vdsmd_init_comm)\n Tasks: 2\n CGroup: /system.slice/vdsmd.service\n \xe2\x94\x94\xe2\x94\x80control\n \xe2\x94\x9c\xe2\x94\x808486 /bin/sh /usr/libexec/vdsm/vdsmd_init_common.sh --pre-start\n \xe2\x94\x94\xe2\x94\x808498 /usr/bin/python2 /usr/libexec/vdsm/wait_for_ipv4s\n\nlago.ssh: DEBUG: start task:e0527d05-8b28-4768-8fdb-6db7420510ea:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: end task:e0527d05-8b28-4768-8fdb-6db7420510ea:Get ssh client for lago-compat-4-3-suite-master-host-1:\nlago.ssh: DEBUG: Running c66e6bce on lago-compat-4-3-suite-master-host-1: /usr/bin/systemctl start vdsmd\nlago.ssh: DEBUG: Command c66e6bce on lago-compat-4-3-suite-master-host-1 returned with 0\n--------------------- >> end captured logging << ---------------------'