[ OST Failure Report ] [ oVirt master ] [ 14-11-2017 ] [ 002_bootstrap.add_hosts ]
by Dafna Ron
This is a multi-part message in MIME format.
--------------5185435FD38913B821CED967
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We had a failure in upgrade suite for 002_bootstrap.add_hosts. I am not
seeing any error that can suggest on an issue in engine.
I can see in the host messages host that we have stopped writing to the
log for 15 minutes and it may suggest that there is something that is
keeping the host from starting which causes us to fail the test on timeout.
However, i can use some help in determining the cause for this failure
and weather its connected to the bootstrap_add_host test in upgrade.
**
*Link to suspected patches: As I said,**I do not think its related, but
this is the patch that was reported. *
*
https://gerrit.ovirt.org/#/c/83854/
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/*
*
*
*Link to all logs:*
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/artifact/*
*
*
*(Relevant) error snippet from the log: *
*
<error>
**
_*Test error: *_**
**
*
Error Message
False != True after 900 seconds
Stacktrace
Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/002_bootstrap.py", line 187, in add_hosts
testlib.assert_true_within(_host_is_up_4, timeout=15*60)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within
'%s != %s after %s seconds' % (res, value, timeout)
AssertionError: False != True after 900 seconds
**
**
**
**_lago log: _*
*
*2017-11-13
15:31:23,212::log_utils.py::__enter__::600::lago.prefix::INFO::ESC[0mESC[0m
2017-11-13
15:31:23,213::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:0468ed2f-b174-4d94-bc66-2b6e08087a86:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:23,213::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:91372413-f7fd-4b72-85b9-9f5216ca7ae9:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:23,213::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:23,213::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:26,220::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-engine: [Errno
None] Unable to connect to port 22 on 192.168.200.3
2017-11-13
15:31:26,221::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2
2017-11-13
15:31:27,222::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:91372413-f7fd-4b72-85b9-9f5216ca7ae9:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:27,222::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:0468ed2f-b174-4d94-bc66-2b6e08087a86:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:27,222::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-engine:
Timed out (in 4 s) trying to ssh to
lago-upgrade-from-release-suite-master-engine
2017-11-13 15:31:27,222::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-host0:
Timed out (in 4 s) trying to ssh to
lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:28,224::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:fc88f641-b012-4636-a471-9ccaaf361a53:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:28,224::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:afb01a46-6338-407f-b7c9-9d5c6b91404d:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:28,224::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:28,225::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:28,226::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2
2017-11-13
15:31:29,228::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:afb01a46-6338-407f-b7c9-9d5c6b91404d:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:29,228::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-host0:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:29,229::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-engine: [Errno
None] Unable to connect to port 22 on 192.168.200.3
2017-11-13
15:31:30,229::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:fc88f641-b012-4636-a471-9ccaaf361a53:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:30,230::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:9df5ddca-bbc4-485a-9f3a-b0b9a5bb5990:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:30,230::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-engine:
Timed out (in 2 s) trying to ssh to
lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:30,230::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:30,231::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2
2017-11-13
15:31:31,231::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:53fe67da-a632-49fe-b697-e8a2c4cb7d23:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:31,232::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:31,232::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-engine: [Errno
None] Unable to connect to port 22 on 192.168.200.3
2017-11-13
15:31:31,232::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:9df5ddca-bbc4-485a-9f3a-b0b9a5bb5990:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:31,233::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-host0:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:32,234::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:53fe67da-a632-49fe-b697-e8a2c4cb7d23:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13 15:31:32,234::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-engine:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:32,234::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:1969bf5b-4e66-43d7-91c6-a28949e98fe8:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:32,234::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:32,235::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2
2017-11-13
15:31:33,235::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:5fd0e3c3-d83c-46f9-9c88-bc739aa9c430:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13
15:31:33,235::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:33,236::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:1969bf5b-4e66-43d7-91c6-a28949e98fe8:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:33,236::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket error
connecting to lago-upgrade-from-release-suite-master-engine: [Errno
None] Unable to connect to port 22 on 192.168.200.3
2017-11-13 15:31:33,237::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-host0:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:34,238::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:5fd0e3c3-d83c-46f9-9c88-bc739aa9c430:Get ssh client for
lago-upgrade-from-release-suite-master-engine:
2017-11-13 15:31:34,238::ssh.py::wait_for_ssh::129::lago.ssh::DEBUG::Got
exception while sshing to lago-upgrade-from-release-suite-master-engine:
Timed out (in 1 s) trying to ssh to
lago-upgrade-from-release-suite-master-engine
2017-11-13
15:31:34,238::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:22550d6e-c397-43dc-9691-c8d9debfe416:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:34,239::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got 1
tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:34,407::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:22550d6e-c397-43dc-9691-c8d9debfe416:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:34,639::ssh.py::ssh::58::lago.ssh::DEBUG::Running
bb49adfa on lago-upgrade-from-release-suite-master-host0: true
2017-11-13 15:31:34,665::ssh.py::ssh::81::lago.ssh::DEBUG::Command
bb49adfa on lago-upgrade-from-release-suite-master-host0 returned with 0
2017-11-13
15:31:34,665::ssh.py::wait_for_ssh::153::lago.ssh::DEBUG::Wait succeeded
for ssh to lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:34,665::log_utils.py::__exit__::611::lago.prefix::INFO::ESC[32mSuccessESC[0m
(in 0:00:11)
2017-11-13
15:31:34,666::log_utils.py::__enter__::600::lago.prefix::INFO::ESC[0mESC[0m
2017-11-13
15:31:34,666::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:4e2a7421-28f7-4679-955f-7261fe377939:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13
15:31:34,667::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still got
100 tries for lago-upgrade-from-release-suite-master-host0
2017-11-13
15:31:34,805::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:4e2a7421-28f7-4679-955f-7261fe377939:Get ssh client for
lago-upgrade-from-release-suite-master-host0:
2017-11-13 15:31:34,906::ssh.py::ssh::58::lago.ssh::DEBUG::Running
bb724bf2 on lago-upgrade-from-release-suite-master-host0: bash -s < "set -xe
DIST=$(uname -r | sed -r 's/^.*\.([^\.]+)\.[^\.]+$/\1/')
*
*
*
**_from host: _*
*
ions. Proceeding anyway.
Nov 13 10:31:34 lago-upgrade-from-release-suite-master-host0 systemd:
Configuration file /usr/lib/systemd/system/ebtables.service is marked
executable. Please remove executable permission bits. Proceeding anyway.
Nov 13 10:31:34 lago-upgrade-from-release-suite-master-host0 systemd:
Configuration file /usr/lib/systemd/system/wpa_supplicant.service is
marked executable. Please remove executable permission bits. Proceeding
anyway.
Nov 13 10:31:55 lago-upgrade-from-release-suite-master-host0 yum[1294]:
Updated: iptables-1.4.21-18.2.el7_4.x86_64
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 3.
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0 systemd:
Removed slice user-0.slice.
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0 systemd:
Stopping user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Created slice user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Starting user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Started Session 4 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Starting Session 4 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: New session 4 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 4.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Removed slice user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Stopping user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Created slice user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Starting user-0.slice.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Started Session 5 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0 systemd:
Starting Session 5 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: New session 5 of user root.
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 5.
Nov 13 10:44:01 lago-upgrade-from-release-suite-master-host0 systemd:
Removed slice user-0.slice.
Nov 13 10:44:01 lago-upgrade-from-release-suite-master-host0 systemd:
Stopping user-0.slice.
Nov 13 10:46:27 lago-upgrade-from-release-suite-master-host0 systemd:
Starting Cleanup of Temporary Directories...
Nov 13 10:46:27 lago-upgrade-from-release-suite-master-host0 systemd:
Started Cleanup of Temporary Directories.
Nov 13 10:54:36 lago-upgrade-from-release-suite-master-host0
dhclient[906]: DHCPREQUEST on eth0 to 192.168.200.1 port 67 (xid=0x564629cb)
Nov 13 10:54:36 lago-upgrade-from-release-suite-master-host0
dhclient[906]: DHCPACK from 192.168.200.1 (xid=0x564629cb)**
*
*
*</error>*
**
--------------5185435FD38913B821CED967
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We had a failure in upgrade suite for 002_bootstrap.add_hosts. I
am not seeing any error that can suggest on an issue in engine. <br>
</p>
<p>I can see in the host messages host that we have stopped writing
to the log for 15 minutes and it may suggest that there is
something that is keeping the host from starting which causes us
to fail the test on timeout. <br>
</p>
<p>However, i can use some help in determining the cause for this
failure and weather its connected to the bootstrap_add_host test
in upgrade.<br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to suspected patches: As I said,<b> </b>I do not think its related, but this is the patch that was reported.
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/83854/">https://gerrit.ovirt.org/#/c/83854/</a></span></p>
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:</span></p>
<a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/</a></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs:</span></p>
<a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/artifact/</a></b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><u><font size="+1"><b>Test error: </b></font></u><b>
</b></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><b>
</b></span></p>
</b></p>
<h3 style="box-sizing: border-box; color: rgb(51, 51, 51);
font-family: Helvetica, Arial, sans-serif; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
letter-spacing: normal; orphans: 2; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration-style: initial; text-decoration-color: initial;">Error
Message</h3>
<pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">False != True after 900 seconds</pre>
<h3 style="box-sizing: border-box; color: rgb(51, 51, 51);
font-family: Helvetica, Arial, sans-serif; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
letter-spacing: normal; orphans: 2; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
text-decoration-style: initial; text-decoration-color: initial;">Stacktrace</h3>
<pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/upgrade-from-release-suite-master/test-scenarios-after-upgrade/002_bootstrap.py", line 187, in add_hosts
testlib.assert_true_within(_host_is_up_4, timeout=15*60)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, in assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 237, in assert_equals_within
'%s != %s after %s seconds' % (res, value, timeout)
AssertionError: False != True after 900 seconds</pre>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></span></p>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><b><u><font
size="+1">lago log: </font></u></b><br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57">2017-11-13
15:31:23,212::log_utils.py::__enter__::600::lago.prefix::<a class="moz-txt-link-freetext" href="INFO::ESC">INFO::ESC</a>[0mESC[0m<br>
2017-11-13
15:31:23,213::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:0468ed2f-b174-4d94-bc66-2b6e08087a86:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:23,213::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:91372413-f7fd-4b72-85b9-9f5216ca7ae9:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:23,213::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:23,213::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:26,220::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-engine: [Errno None]
Unable to connect to port 22 on 192.168.200.3<br>
2017-11-13
15:31:26,221::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2<br>
2017-11-13
15:31:27,222::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:91372413-f7fd-4b72-85b9-9f5216ca7ae9:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:27,222::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:0468ed2f-b174-4d94-bc66-2b6e08087a86:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:27,222::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-engine: Timed out (in 4
s) trying to ssh to
lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:27,222::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-host0: Timed out (in 4 s)
trying to ssh to lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:28,224::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:fc88f641-b012-4636-a471-9ccaaf361a53:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:28,224::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:afb01a46-6338-407f-b7c9-9d5c6b91404d:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:28,224::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:28,225::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:28,226::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2<br>
2017-11-13
15:31:29,228::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:afb01a46-6338-407f-b7c9-9d5c6b91404d:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:29,228::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-host0: Timed out (in 1 s)
trying to ssh to lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:29,229::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-engine: [Errno None]
Unable to connect to port 22 on 192.168.200.3<br>
2017-11-13
15:31:30,229::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:fc88f641-b012-4636-a471-9ccaaf361a53:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:30,230::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:9df5ddca-bbc4-485a-9f3a-b0b9a5bb5990:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:30,230::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-engine: Timed out (in 2
s) trying to ssh to
lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:30,230::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:30,231::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2<br>
2017-11-13
15:31:31,231::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:53fe67da-a632-49fe-b697-e8a2c4cb7d23:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:31,232::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:31,232::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-engine: [Errno None]
Unable to connect to port 22 on 192.168.200.3<br>
2017-11-13
15:31:31,232::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:9df5ddca-bbc4-485a-9f3a-b0b9a5bb5990:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:31,233::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-host0: Timed out (in 1 s)
trying to ssh to lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:32,234::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:53fe67da-a632-49fe-b697-e8a2c4cb7d23:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:32,234::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-engine: Timed out (in 1
s) trying to ssh to
lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:32,234::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:1969bf5b-4e66-43d7-91c6-a28949e98fe8:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:32,234::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:32,235::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-host0: [Errno None]
Unable to connect to port 22 on 192.168.200.2<br>
2017-11-13
15:31:33,235::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:5fd0e3c3-d83c-46f9-9c88-bc739aa9c430:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:33,235::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:33,236::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:1969bf5b-4e66-43d7-91c6-a28949e98fe8:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:33,236::ssh.py::get_ssh_client::354::lago.ssh::DEBUG::Socket
error connecting to
lago-upgrade-from-release-suite-master-engine: [Errno None]
Unable to connect to port 22 on 192.168.200.3<br>
2017-11-13
15:31:33,237::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-host0: Timed out (in 1 s)
trying to ssh to lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:34,238::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:5fd0e3c3-d83c-46f9-9c88-bc739aa9c430:Get ssh client for
lago-upgrade-from-release-suite-master-engine:<br>
2017-11-13
15:31:34,238::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::129::lago.ssh::DEBUG::Got">ssh::129::lago.ssh::DEBUG::Got</a>
exception while sshing to
lago-upgrade-from-release-suite-master-engine: Timed out (in 1
s) trying to ssh to
lago-upgrade-from-release-suite-master-engine<br>
2017-11-13
15:31:34,238::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:22550d6e-c397-43dc-9691-c8d9debfe416:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:34,239::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 1 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:34,407::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:22550d6e-c397-43dc-9691-c8d9debfe416:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:34,639::ssh.py::<a class="moz-txt-link-freetext" href="ssh::58::lago.ssh::DEBUG::Running">ssh::58::lago.ssh::DEBUG::Running</a> bb49adfa
on lago-upgrade-from-release-suite-master-host0: true<br>
2017-11-13
15:31:34,665::ssh.py::<a class="moz-txt-link-freetext" href="ssh::81::lago.ssh::DEBUG::Command">ssh::81::lago.ssh::DEBUG::Command</a> bb49adfa
on lago-upgrade-from-release-suite-master-host0 returned with 0<br>
2017-11-13
15:31:34,665::ssh.py::wait_for_<a class="moz-txt-link-freetext" href="ssh::153::lago.ssh::DEBUG::Wait">ssh::153::lago.ssh::DEBUG::Wait</a>
succeeded for ssh to
lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:34,665::log_utils.py::__exit__::611::lago.prefix::<a class="moz-txt-link-freetext" href="INFO::ESC">INFO::ESC</a>[32mSuccessESC[0m
(in 0:00:11)<br>
2017-11-13
15:31:34,666::log_utils.py::__enter__::600::lago.prefix::<a class="moz-txt-link-freetext" href="INFO::ESC">INFO::ESC</a>[0mESC[0m<br>
2017-11-13
15:31:34,666::log_utils.py::__enter__::600::lago.ssh::DEBUG::start
task:4e2a7421-28f7-4679-955f-7261fe377939:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:34,667::ssh.py::get_ssh_client::339::lago.ssh::DEBUG::Still
got 100 tries for lago-upgrade-from-release-suite-master-host0<br>
2017-11-13
15:31:34,805::log_utils.py::__exit__::611::lago.ssh::DEBUG::end
task:4e2a7421-28f7-4679-955f-7261fe377939:Get ssh client for
lago-upgrade-from-release-suite-master-host0:<br>
2017-11-13
15:31:34,906::ssh.py::<a class="moz-txt-link-freetext" href="ssh::58::lago.ssh::DEBUG::Running">ssh::58::lago.ssh::DEBUG::Running</a> bb724bf2
on lago-upgrade-from-release-suite-master-host0: bash -s <
"set -xe<br>
DIST=$(uname -r | sed -r 's/^.*\.([^\.]+)\.[^\.]+$/\1/')<br>
<br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><br>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><b><u><font
size="+1">from host: </font></u></b><br>
</b></p>
<p><span style="font-weight:normal;">ions. Proceeding anyway.<br>
Nov 13 10:31:34 lago-upgrade-from-release-suite-master-host0
systemd: Configuration file
/usr/lib/systemd/system/ebtables.service is marked executable.
Please remove executable permission bits. Proceeding anyway.<br>
Nov 13 10:31:34 lago-upgrade-from-release-suite-master-host0
systemd: Configuration file
/usr/lib/systemd/system/wpa_supplicant.service is marked
executable. Please remove executable permission bits. Proceeding
anyway.<br>
Nov 13 10:31:55 lago-upgrade-from-release-suite-master-host0
yum[1294]: Updated: iptables-1.4.21-18.2.el7_4.x86_64<br>
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 3.<br>
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0
systemd: Removed slice user-0.slice.<br>
Nov 13 10:31:56 lago-upgrade-from-release-suite-master-host0
systemd: Stopping user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Created slice user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Starting user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Started Session 4 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Starting Session 4 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: New session 4 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 4.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Removed slice user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Stopping user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Created slice user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Starting user-0.slice.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Started Session 5 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd: Starting Session 5 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: New session 5 of user root.<br>
Nov 13 10:44:00 lago-upgrade-from-release-suite-master-host0
systemd-logind: Removed session 5.<br>
Nov 13 10:44:01 lago-upgrade-from-release-suite-master-host0
systemd: Removed slice user-0.slice.<br>
Nov 13 10:44:01 lago-upgrade-from-release-suite-master-host0
systemd: Stopping user-0.slice.<br>
Nov 13 10:46:27 lago-upgrade-from-release-suite-master-host0
systemd: Starting Cleanup of Temporary Directories...<br>
Nov 13 10:46:27 lago-upgrade-from-release-suite-master-host0
systemd: Started Cleanup of Temporary Directories.<br>
Nov 13 10:54:36 lago-upgrade-from-release-suite-master-host0
dhclient[906]: DHCPREQUEST on eth0 to 192.168.200.1 port 67
(xid=0x564629cb)<br>
Nov 13 10:54:36 lago-upgrade-from-release-suite-master-host0
dhclient[906]: DHCPACK from 192.168.200.1 (xid=0x564629cb)</span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-ba60-843c-cfdd-7366571bcb57"><b><br>
</b><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p>
</b><br class="Apple-interchange-newline">
</p>
</body>
</html>
--------------5185435FD38913B821CED967--
7 years
[JIRA] (OVIRT-1763) Increase entropy for hosts
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1510756591-26566-475
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1763?page=com.atlassian.jir... ]
Barak Korren edited comment on OVIRT-1763 at 11/15/17 2:36 PM:
---------------------------------------------------------------
[~ederevea] OST uses a simple check to determine if to run in RAM or not:
{code}
# if above RAM_THRESHOLD KBs are available in /dev/shm, run there
RAM_THRESHOLD=15000000
# ...
get_run_path() {
local avail_shm
avail_shm=$(df --output=avail /dev/shm | sed 1d)
[[ "$avail_shm" -ge "$RAM_THRESHOLD" ]] && \
mkdir -p "/dev/shm/ost" && \
echo "/dev/shm/ost/deployment-$SUITE" || \
echo "$PWD/deployment-$SUITE"
}
{code}
This seems to be 15G so we need to figure out why wasn't this available when these lines ran (These lines are amount the very first things that run when an OST suit starts up, so its unlikely to be cause by anything the suit is doing)
Code quoted above is part of '{{automation/suit.sh}}' in OST:
https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=automat...
was (Author: bkorren(a)redhat.com):
[~ederevea] OST uses a simple check to determine if to run in RAM or not:
{code}
# if above RAM_THRESHOLD KBs are available in /dev/shm, run there
RAM_THRESHOLD=15000000
# ...
get_run_path() {
local avail_shm
avail_shm=$(df --output=avail /dev/shm | sed 1d)
[[ "$avail_shm" -ge "$RAM_THRESHOLD" ]] && \
mkdir -p "/dev/shm/ost" && \
echo "/dev/shm/ost/deployment-$SUITE" || \
echo "$PWD/deployment-$SUITE"
}
{code}
This seems to be 15G so we need to figure out why wasn't this available when these lines ran (These lines are amount the very first things that run when an OST suit starts up, so its unlikely to be cause by anything the suit is doing)
> Increase entropy for hosts
> --------------------------
>
> Key: OVIRT-1763
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1763
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Reporter: Dafna Ron
> Assignee: infra
>
> we had a failure in ost that was really hard to debug: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/
> There are no failures in the logs and the test itself was terminated by a timeout.
> It took the vms a long time to download packages and install and didi seems to think that this is due to limited entropy on the physical host.
> we need to review this issue and increase the entropy on the hosts.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100071)
------------=_1510756591-26566-475
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1763?page=com.atlassian.jir... ]</pre>
<h3>Barak Korren edited comment on OVIRT-1763 at 11/15/17 2:36 PM:</h3>
<p>[~ederevea] OST uses a simple check to determine if to run in RAM or not:</p>
<p>{code} # if above RAM_THRESHOLD KBs are available in /dev/shm, run there RAM_THRESHOLD=15000000</p>
<p># …</p>
<p>get_run_path() {</p>
<pre>local avail_shm
avail_shm=$(df --output=avail /dev/shm | sed 1d)
[[ "$avail_shm" -ge "$RAM_THRESHOLD" ]] && \
mkdir -p "/dev/shm/ost" && \
echo "/dev/shm/ost/deployment-$SUITE" || \
echo "$PWD/deployment-$SUITE"</pre>
<p>} {code}</p>
<p>This seems to be 15G so we need to figure out why wasn't this available when these lines ran (These lines are amount the very first things that run when an OST suit starts up, so its unlikely to be cause by anything the suit is doing)</p>
<p>Code quoted above is part of ‘{{automation/suit.sh}}’ in OST: <a href="https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=automat...">https://gerrit.ovirt.org/gitweb?p=ovirt-system-tests.git;a=blob;f=automat...</a></p>
<p>was (Author: bkorren(a)redhat.com): [~ederevea] OST uses a simple check to determine if to run in RAM or not:</p>
<p>{code} # if above RAM_THRESHOLD KBs are available in /dev/shm, run there RAM_THRESHOLD=15000000</p>
<p># …</p>
<p>get_run_path() {</p>
<pre>local avail_shm
avail_shm=$(df --output=avail /dev/shm | sed 1d)
[[ "$avail_shm" -ge "$RAM_THRESHOLD" ]] && \
mkdir -p "/dev/shm/ost" && \
echo "/dev/shm/ost/deployment-$SUITE" || \
echo "$PWD/deployment-$SUITE"</pre>
<p>} {code}</p>
<p>This seems to be 15G so we need to figure out why wasn't this available when these lines ran (These lines are amount the very first things that run when an OST suit starts up, so its unlikely to be cause by anything the suit is doing)</p>
<blockquote><h3>Increase entropy for hosts</h3>
<pre> Key: OVIRT-1763
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1763
Project: oVirt - virtualization made easy
Issue Type: Bug
Reporter: Dafna Ron
Assignee: infra</pre>
<p>we had a failure in ost that was really hard to debug: <a href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/</a> There are no failures in the logs and the test itself was terminated by a timeout. It took the vms a long time to download packages and install and didi seems to think that this is due to limited entropy on the physical host. we need to review this issue and increase the entropy on the hosts.</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100071)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYL..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1510756591-26566-475--
7 years
[JIRA] (OVIRT-1763) Increase entropy for hosts
by Barak Korren (oVirt JIRA)
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1763?page=com.atlassian.jir... ]
Barak Korren commented on OVIRT-1763:
-------------------------------------
[~ederevea] OST uses a simple check to determine if to run in RAM or not:
{code}
# if above RAM_THRESHOLD KBs are available in /dev/shm, run there
RAM_THRESHOLD=15000000
# ...
get_run_path() {
local avail_shm
avail_shm=$(df --output=avail /dev/shm | sed 1d)
[[ "$avail_shm" -ge "$RAM_THRESHOLD" ]] && \
mkdir -p "/dev/shm/ost" && \
echo "/dev/shm/ost/deployment-$SUITE" || \
echo "$PWD/deployment-$SUITE"
}
{code}
This seems to be 15G so we need to figure out why wasn't this available when these lines ran (These lines are amount the very first things that run when an OST suit starts up, so its unlikely to be cause by anything the suit is doing)
> Increase entropy for hosts
> --------------------------
>
> Key: OVIRT-1763
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1763
> Project: oVirt - virtualization made easy
> Issue Type: Bug
> Reporter: Dafna Ron
> Assignee: infra
>
> we had a failure in ost that was really hard to debug: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3795/
> There are no failures in the logs and the test itself was terminated by a timeout.
> It took the vms a long time to download packages and install and didi seems to think that this is due to limited entropy on the physical host.
> we need to review this issue and increase the entropy on the hosts.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100071)
7 years
[JIRA] (OVIRT-1767) Look into replacing gitweb for oVirt Gerrit
by Barak Korren (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1510754998-15183-222
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1767?page=com.atlassian.jir... ]
Barak Korren updated OVIRT-1767:
--------------------------------
Description:
One of the faults of our Gerrit system is that the source browser look like it belongs to a different century.
There seems to be a Gerrit plugin which allows using Gitiles rather then gitweb in Gerrit:
https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...
Gitiles itself seems to be a more modern Git UI then gitweb, at the very least is supports showing markdown files, which is a staple of modern source browsers:
https://gerrit.googlesource.com/gitiles/
Another plugin to consider is the one for using GitBlit:
https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...
Gitblit is another Java based Gir bowser:
http://gitblit.com/features.html
was:
There seems to be a Gerrit plugin which allows using Gitiles rather then gitweb in Gerrit:
https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...
Gitiles itself seems to be a more modern Git UI then gitweb, at the very least is supports showing markdown files, which is a staple of modern source browsers:
https://gerrit.googlesource.com/gitiles/
> Look into replacing gitweb for oVirt Gerrit
> -------------------------------------------
>
> Key: OVIRT-1767
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1767
> Project: oVirt - virtualization made easy
> Issue Type: Improvement
> Components: Gerrit/git
> Reporter: Barak Korren
> Assignee: infra
> Labels: gerrit
>
> One of the faults of our Gerrit system is that the source browser look like it belongs to a different century.
> There seems to be a Gerrit plugin which allows using Gitiles rather then gitweb in Gerrit:
> https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...
> Gitiles itself seems to be a more modern Git UI then gitweb, at the very least is supports showing markdown files, which is a staple of modern source browsers:
> https://gerrit.googlesource.com/gitiles/
> Another plugin to consider is the one for using GitBlit:
> https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...
> Gitblit is another Java based Gir bowser:
> http://gitblit.com/features.html
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100071)
------------=_1510754998-15183-222
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1767?page=com.atlassian.jir... ]</pre>
<h3>Barak Korren updated OVIRT-1767:</h3>
<pre>Description:</pre>
<p>One of the faults of our Gerrit system is that the source browser look like it belongs to a different century.</p>
<p>There seems to be a Gerrit plugin which allows using Gitiles rather then gitweb in Gerrit: <a href="https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...">https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...</a></p>
<p>Gitiles itself seems to be a more modern Git UI then gitweb, at the very least is supports showing markdown files, which is a staple of modern source browsers: <a href="https://gerrit.googlesource.com/gitiles/">https://gerrit.googlesource.com/gitiles/</a></p>
<p>Another plugin to consider is the one for using GitBlit: <a href="https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...">https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...</a></p>
<p>Gitblit is another Java based Gir bowser: <a href="http://gitblit.com/features.html">http://gitblit.com/features.html</a></p>
<pre>was:</pre>
<p>There seems to be a Gerrit plugin which allows using Gitiles rather then gitweb in Gerrit: <a href="https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...">https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...</a></p>
<p>Gitiles itself seems to be a more modern Git UI then gitweb, at the very least is supports showing markdown files, which is a staple of modern source browsers: <a href="https://gerrit.googlesource.com/gitiles/">https://gerrit.googlesource.com/gitiles/</a></p>
<blockquote><h3>Look into replacing gitweb for oVirt Gerrit</h3>
<pre> Key: OVIRT-1767
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1767
Project: oVirt - virtualization made easy
Issue Type: Improvement
Components: Gerrit/git
Reporter: Barak Korren
Assignee: infra
Labels: gerrit</pre>
<p>One of the faults of our Gerrit system is that the source browser look like it belongs to a different century. There seems to be a Gerrit plugin which allows using Gitiles rather then gitweb in Gerrit: <a href="https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...">https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...</a> Gitiles itself seems to be a more modern Git UI then gitweb, at the very least is supports showing markdown files, which is a staple of modern source browsers: <a href="https://gerrit.googlesource.com/gitiles/">https://gerrit.googlesource.com/gitiles/</a> Another plugin to consider is the one for using GitBlit: <a href="https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...">https://gerrit-review.googlesource.com/Documentation/config-plugins.html#...</a> Gitblit is another Java based Gir bowser: <a href="http://gitblit.com/features.html">http://gitblit.com/features.html</a></p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100071)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYL..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1510754998-15183-222--
7 years
