[oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.1 - Build # 37 - Failure!
by jenkins＠jenkins.phx.ovirt.org
------=_Part_627_898823252.1505878358235
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/37/
Build Number: 37
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #37
[Sahina Bose] fixes for the 4.1 and master reposyncfiles
[Dan Kenigsberg] enable fc26 build for mom
[Dan Kenigsberg] enable fc26 build for ovirt-vmconsole.
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 002_bootstrap.list_glance_images
Error Message:
Fault reason is "Operation Failed". Fault detail is "[Failed to communicate with the external provider, see log for additional details.]". HTTP response code is 400.
Stack Trace:
Traceback (most recent call last):
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.1/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 697, in list_glance_images
list_glance_images_4(api)
File "/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.1/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 748, in list_glance_images_4
if not check_glance_connectivity_4(api):
File "/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.1/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 846, in check_glance_connectivity_4
glance.test_connectivity()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 14026, in test_connectivity
return self._internal_action(action, 'testconnectivity', None, headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 290, in _internal_action
return future.wait() if wait else future
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53, in wait
return self._code(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 287, in callback
self._check_fault(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 123, in _check_fault
self._raise_error(response, body)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 109, in _raise_error
raise error
Error: Fault reason is "Operation Failed". Fault detail is "[Failed to communicate with the external provider, see log for additional details.]". HTTP response code is 400.
------=_Part_627_898823252.1505878358235--
7 years, 2 months
[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.1 - Build # 34 - Failure!
by jenkins＠jenkins.phx.ovirt.org
------=_Part_625_1969086870.1505877115773
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.1=
/=20
Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.1/3=
4/
Build Number: 34
Build Status: Failure
Triggered By: Started by timer
-------------------------------------
Changes Since Last Success:
-------------------------------------
Changes for Build #34
[Sahina Bose] fixes for the 4.1 and master reposyncfiles
[Dan Kenigsberg] enable fc26 build for mom
[Dan Kenigsberg] enable fc26 build for ovirt-vmconsole.
-----------------
Failed Tests:
-----------------
1 tests failed.
FAILED: 002_bootstrap.add_hosts
Error Message:
status: 503
reason: Service Unavailable
detail:=20
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Unavailable</title>
</head><body>
<h1>Service Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
</body></html>
-------------------- >> begin captured logging << --------------------
ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u=
p at 0x46ace60>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i=
n assert_equals_within
res =3D func()
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.1/ovirt=
-system-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 144=
, in _host_is_up
cur_state =3D api.hosts.get(host.name()).status.state
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py=
", line 18338, in get
headers=3D{"All-Content":all_content}
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",=
line 46, in get
return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls=
=3Dcls)
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",=
line 122, in request
persistent_auth=3Dself.__persistent_auth
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection=
spool.py", line 79, in do_request
persistent_auth)
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection=
spool.py", line 162, in __do_request
raise errors.RequestError(response_code, response_reason, response_body=
)
RequestError:=20
status: 503
reason: Service Unavailable
detail:=20
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>503 Service Unavailable</title>
</head><body>
<h1>Service Unavailable</h1>
<p>The server is temporarily unable to service your
request due to maintenance downtime or capacity
problems. Please try again later.</p>
</body></html>
--------------------- >> end captured logging << ---------------------
Stack Trace:
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTes=
t
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, i=
n wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in=
wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.1/ovirt=
-system-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 163=
, in add_hosts
testlib.assert_true_within(_host_is_up, timeout=3D15 * 60)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, i=
n assert_true_within
assert_equals_within(func, True, timeout, allowed_exceptions)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i=
n assert_equals_within
res =3D func()
File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.1/ovirt=
-system-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 144=
, in _host_is_up
cur_state =3D api.hosts.get(host.name()).status.state
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py=
", line 18338, in get
headers=3D{"All-Content":all_content}
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",=
line 46, in get
return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls=
=3Dcls)
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",=
line 122, in request
persistent_auth=3Dself.__persistent_auth
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection=
spool.py", line 79, in do_request
persistent_auth)
File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection=
spool.py", line 162, in __do_request
raise errors.RequestError(response_code, response_reason, response_body=
)
'\r\nstatus: 503\r\nreason: Service Unavailable\r\ndetail: \n<!DOCTYPE HTML=
PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>503 Service Unav=
ailable</title>\n</head><body>\n<h1>Service Unavailable</h1>\n<p>The server=
is temporarily unable to service your\nrequest due to maintenance downtime=
or capacity\nproblems. Please try again later.</p>\n</body></html>\n\n----=
---------------- >> begin captured logging << --------------------\novirtla=
go.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x=
46ace60>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/si=
te-packages/ovirtlago/testlib.py", line 219, in assert_equals_within\n r=
es =3D func()\n File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-=
suite-4.1/ovirt-system-tests/hc-basic-suite-4.1/test-scenarios/002_bootstra=
p.py", line 144, in _host_is_up\n cur_state =3D api.hosts.get(host.name(=
)).status.state\n File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastr=
ucture/brokers.py", line 18338, in get\n headers=3D{"All-Content":all_co=
ntent}\n File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/pr=
oxy.py", line 46, in get\n return self.request(method=3D\'GET\', url=3Du=
rl, headers=3Dheaders, cls=3Dcls)\n File "/usr/lib/python2.7/site-packages=
/ovirtsdk/infrastructure/proxy.py", line 122, in request\n persistent_au=
th=3Dself.__persistent_auth\n File "/usr/lib/python2.7/site-packages/ovirt=
sdk/infrastructure/connectionspool.py", line 79, in do_request\n persist=
ent_auth)\n File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure=
/connectionspool.py", line 162, in __do_request\n raise errors.RequestEr=
ror(response_code, response_reason, response_body)\nRequestError: \r\nstatu=
s: 503\r\nreason: Service Unavailable\r\ndetail: \n<!DOCTYPE HTML PUBLIC "-=
//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>503 Service Unavailable</t=
itle>\n</head><body>\n<h1>Service Unavailable</h1>\n<p>The server is tempor=
arily unable to service your\nrequest due to maintenance downtime or capaci=
ty\nproblems. Please try again later.</p>\n</body></html>\n\n--------------=
------- >> end captured logging << ---------------------'
------=_Part_625_1969086870.1505877115773--
7 years, 2 months
[JIRA] (OVIRT-1659) add ipv6 accept_ra to sysctl on jenkins slaves
by Evgheni Dereveanchin (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1505907708-16017-2467
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Evgheni Dereveanchin created OVIRT-1659:
-------------------------------------------
Summary: add ipv6 accept_ra to sysctl on jenkins slaves
Key: OVIRT-1659
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1659
Project: oVirt - virtualization made easy
Issue Type: Task
Reporter: Evgheni Dereveanchin
Assignee: infra
Priority: High
To work around new libvirt behavior, net.ipv6.conf.all.accept_ra=2 sysctl parameter needs to be set on slaves, specifically the bare metals.
More info here: https://github.com/lago-project/lago/pull/654/files
--
This message was sent by Atlassian {0}
(v1001.0.0-SNAPSHOT#100060)
------------=_1505907708-16017-2467
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<h3>Evgheni Dereveanchin created OVIRT-1659:</h3>
<pre> Summary: add ipv6 accept_ra to sysctl on jenkins slaves
Key: OVIRT-1659
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1659
Project: oVirt - virtualization made easy
Issue Type: Task
Reporter: Evgheni Dereveanchin
Assignee: infra
Priority: High</pre>
<p>To work around new libvirt behavior, net.ipv6.conf.all.accept_ra=2 sysctl parameter needs to be set on slaves, specifically the bare metals.</p>
<p>More info here: <a href="https://github.com/lago-project/lago/pull/654/files">https://github.com/lago-project/lago/pull/654/files</a></p>
<p>— This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYL..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1505907708-16017-2467--
7 years, 2 months
[JIRA] (OVIRT-1658) Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)
by Evgheni Dereveanchin (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1505906876-10461-2701
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1658?page=com.atlassian.jir... ]
Evgheni Dereveanchin commented on OVIRT-1658:
---------------------------------------------
Seen this as well. Logged a CentOS bug and will run the localedef command manually across our systems to avoid this breaking stuff.
> Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)
> ----------------------------------------------------------------------
>
> Key: OVIRT-1658
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1658
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: Barak Korren
> Assignee: Evgheni Dereveanchin
>
> Seems there is an issue with locale files in recent CentOS versions,
> and we're hitting it in slaves that have been updated recently.
> The symptoms are that Jenkins disconnects from the slave and then
> refuses to reconnect to it. The agent log in Jenkins shows:
> [09/19/17 15:42:56] [SSH] Connection closed.
> [09/19/17 15:58:48] [SSH] Opening SSH connection to
> vm0002.workers-phx.ovirt.org:22.
> [09/19/17 15:58:48] [SSH] WARNING: SSH Host Keys are not being
> verified. Man-in-the-middle attacks may be possible against this
> connection.
> [09/19/17 15:58:48] [SSH] Authentication successful.
> SSH connection reports a garbage before a command execution.
> Check your .bashrc, .profile, and so on to make sure it is quiet.
> The received junk text is as follows:
> /etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot
> change locale (en_US.utf8): No such file or directory
> /etc/profile.d/lang.sh: line 20: warning: setlocale: LC_COLLATE:
> cannot change locale (en_US.utf8): No such file or directory
> /etc/profile.d/lang.sh: line 23: warning: setlocale: LC_MESSAGES:
> cannot change locale (en_US.utf8): No such file or directory
> /etc/profile.d/lang.sh: line 26: warning: setlocale: LC_NUMERIC:
> cannot change locale (en_US.utf8): No such file or directory
> /etc/profile.d/lang.sh: line 29: warning: setlocale: LC_TIME: cannot
> change locale (en_US.utf8): No such file or directory
> null
> [09/19/17 15:58:48] Launch failed - cleaning up connection
> [09/19/17 15:58:48] [SSH] Connection closed.
> The same locale error messages can also be reproduced on the slave by
> running an interactive login from the console or 'su -'. When running
> 'locale -a' you can also see the en_US.UTF-8 locale is somehow
> missing.
> Looking around for this I found the following:
> https://github.com/CentOS/sig-cloud-instance-images/issues/71
> I tried downgrading glibc back to the version we had before, but that
> did not seem to resolve the issue. Eventually I managed to resolve it
> by running 'localedef -i en_US -f UTF-8 en_US.UTF-8' on the slave.
> I've seen this happen on 'vm0002.workers-phx.ovirt.org' which is
> attached to the staging Jenkins, but I've no reason to believe this
> won't start impacting production slaves.
> We need to research this further and find out if we need to do
> something to prevent this issue from surfacing on production slaves.
> --
> Barak Korren
> RHV DevOps team , RHCE, RHCi
> Red Hat EMEA
> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
--
This message was sent by Atlassian {0}
(v1001.0.0-SNAPSHOT#100060)
------------=_1505906876-10461-2701
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1658?page=com.atlassian.jir... ]</pre>
<h3>Evgheni Dereveanchin commented on OVIRT-1658:</h3>
<p>Seen this as well. Logged a CentOS bug and will run the localedef command manually across our systems to avoid this breaking stuff.</p>
<blockquote><h3>Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)</h3>
<pre> Key: OVIRT-1658
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1658
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: Barak Korren
Assignee: Evgheni Dereveanchin</pre>
<p>Seems there is an issue with locale files in recent CentOS versions, and we're hitting it in slaves that have been updated recently. The symptoms are that Jenkins disconnects from the slave and then refuses to reconnect to it. The agent log in Jenkins shows: [09/19/17 15:42:56] [SSH] Connection closed. [09/19/17 15:58:48] [SSH] Opening SSH connection to vm0002.workers-phx.ovirt.org:22. [09/19/17 15:58:48] [SSH] WARNING: SSH Host Keys are not being verified. Man-in-the-middle attacks may be possible against this connection. [09/19/17 15:58:48] [SSH] Authentication successful. SSH connection reports a garbage before a command execution. Check your .bashrc, .profile, and so on to make sure it is quiet. The received junk text is as follows: /etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 20: warning: setlocale: LC_COLLATE: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 23: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 26: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 29: warning: setlocale: LC_TIME: cannot change locale (en_US.utf8): No such file or directory null [09/19/17 15:58:48] Launch failed – cleaning up connection [09/19/17 15:58:48] [SSH] Connection closed. The same locale error messages can also be reproduced on the slave by running an interactive login from the console or ‘su -’. When running ‘locale -a’ you can also see the en_US.UTF-8 locale is somehow missing. Looking around for this I found the following: <a href="https://github.com/CentOS/sig-cloud-instance-images/issues/71">https://github.com/CentOS/sig-cloud-instance-images/issues/71</a> I tried downgrading glibc back to the version we had before, but that did not seem to resolve the issue. Eventually I managed to resolve it by running ‘localedef -i en_US -f UTF-8 en_US.UTF-8’ on the slave. I've seen this happen on ‘vm0002.workers-phx.ovirt.org’ which is attached to the staging Jenkins, but I've no reason to believe this won't start impacting production slaves. We need to research this further and find out if we need to do something to prevent this issue from surfacing on production slaves. — Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted</p></blockquote>
<p>— This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYL..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1505906876-10461-2701--
7 years, 2 months
Subject: [ OST Failure Report ] [ master ] [ 20-09-2017] [ No Test Failed - missing packages in fc25 ]
by Dafna Ron
This is a multi-part message in MIME format.
--------------60D60522EFE450E2C842A076
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
CQ reported a failure in one of the builds and looking at the console I
found that we had a failure before the OST started due to:
ovirt-hosted-engine-ha_master_build-artifacts-fc25-x86_64 (67) failed building
The build failed of ovirt-hosted-engine-ha_master_build-artifacts-fc25-x86_64 failed due to missing packages.
*
Link to suspected patches: patch reported as failed is
https://gerrit.ovirt.org/#/c/81792/
**
Link to Job:
*cq job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2732/console*
*failed build job:
http://jenkins.ovirt.org/job/ovirt-hosted-engine-ha_master_build-artifact...
Link to all logs:
*CQ job logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2732/artifa...
*failed build log:
http://jenkins.ovirt.org/job/ovirt-hosted-engine-ha_master_build-artifact...
**
*(Relevant) error snippet from the log: *
*<error>*
**07:46:21* Getting requirements for ovirt-hosted-engine-ha.spec
*07:46:21* --> Already installed : python-2.7.13-2.fc25.x86_64
*07:46:21* --> python2-mock-2.0.0-2.fc25.noarch *07:46:22* -->
python2-nose-1.3.7-11.fc25.noarch *07:46:22* --> Already installed :
python-devel-2.7.13-2.fc25.x86_64 *07:46:22* --> Already installed :
systemd-231-17.fc25.x86_64 *07:46:22* -->
vdsm-4.20.3-47.git4801123.fc25.x86_64 *07:46:22* -->
vdsm-client-4.20.3-47.git4801123.fc25.noarch *07:46:22* -->
vdsm-python-4.20.3-47.git4801123.fc25.noarch *07:46:31* Error: Package:
vdsm-4.20.3-47.git4801123.fc25.x86_64 (ovirt-master-snapshot) *07:46:31*
Requires: openvswitch >= 2.7.0 *07:46:31* Available:
openvswitch-2.5.0-4.fc25.x86_64 (fedora-base-fc25) *07:46:31*
openvswitch = 2.5.0-4.fc25 *07:46:31* You could try using --skip-broken
to work around the problem *07:46:31* You could try running: rpm -Va
--nofiles --nodigest *07:46:31* Took 18 seconds*
</error>
*Thanks,
Dafna
--------------60D60522EFE450E2C842A076
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: base64
PGh0bWw+CiAgPGhlYWQ+CgogICAgPG1ldGEgaHR0cC1lcXVpdj0iY29udGVudC10eXBlIiBj
b250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPgogIDwvaGVhZD4KICA8Ym9keSB0
ZXh0PSIjMDAwMDAwIiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIDxwPkhpLCA8YnI+CiAgICA8
L3A+CiAgICA8cD5DUSByZXBvcnRlZCBhIGZhaWx1cmUgaW4gb25lIG9mIHRoZSBidWlsZHMg
YW5kIGxvb2tpbmcgYXQgdGhlCiAgICAgIGNvbnNvbGUgSSBmb3VuZCB0aGF0IHdlIGhhZCBh
IGZhaWx1cmUgYmVmb3JlIHRoZSBPU1Qgc3RhcnRlZCBkdWUKICAgICAgdG86IDxicj4KICAg
IDwvcD4KICAgIDxwcmUgY2xhc3M9ImNvbnNvbGUtb3V0cHV0IiBzdHlsZT0iYm94LXNpemlu
ZzogYm9yZGVyLWJveDsgd2hpdGUtc3BhY2U6IHByZS13cmFwOyB3b3JkLXdyYXA6IGJyZWFr
LXdvcmQ7IG1hcmdpbjogMHB4OyBjb2xvcjogcmdiKDUxLCA1MSwgNTEpOyBmb250LXNpemU6
IDE0cHg7IGZvbnQtc3R5bGU6IG5vcm1hbDsgZm9udC12YXJpYW50LWxpZ2F0dXJlczogbm9y
bWFsOyBmb250LXZhcmlhbnQtY2Fwczogbm9ybWFsOyBmb250LXdlaWdodDogbm9ybWFsOyBs
ZXR0ZXItc3BhY2luZzogbm9ybWFsOyBvcnBoYW5zOiAyOyB0ZXh0LWFsaWduOiBzdGFydDsg
dGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdpZG93czogMjsgd29y
ZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1k
ZWNvcmF0aW9uLXN0eWxlOiBpbml0aWFsOyB0ZXh0LWRlY29yYXRpb24tY29sb3I6IGluaXRp
YWw7Ij5vdmlydC1ob3N0ZWQtZW5naW5lLWhhX21hc3Rlcl9idWlsZC1hcnRpZmFjdHMtZmMy
NS14ODZfNjQgKDY3KSBmYWlsZWQgYnVpbGRpbmcKClRoZSBidWlsZCBmYWlsZWQgb2Ygb3Zp
cnQtaG9zdGVkLWVuZ2luZS1oYV9tYXN0ZXJfYnVpbGQtYXJ0aWZhY3RzLWZjMjUteDg2XzY0
IGZhaWxlZCBkdWUgdG8gbWlzc2luZyBwYWNrYWdlcy4gCgo8YiBzdHlsZT0iZm9udC13ZWln
aHQ6bm9ybWFsOyIgaWQ9ImRvY3MtaW50ZXJuYWwtZ3VpZC01ODU5YjdhMS05ZWI0LTgyYjct
NDgwNS05Njc3MTljNjIxYTkiPjxwIGRpcj0ibHRyIiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4z
ODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBwdDsiPjxzcGFuIHN0eWxlPSJmb250
LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQt
Y29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2Zv
bnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246
YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXA7Ij5MaW5rIHRvIHN1c3BlY3RlZCBwYXRj
aGVzOiAgcGF0Y2ggcmVwb3J0ZWQgYXMgZmFpbGVkIGlzIDxhIGNsYXNzPSJtb3otdHh0LWxp
bmstZnJlZXRleHQiIGhyZWY9Imh0dHBzOi8vZ2Vycml0Lm92aXJ0Lm9yZy8jL2MvODE3OTIv
Ij5odHRwczovL2dlcnJpdC5vdmlydC5vcmcvIy9jLzgxNzkyLzwvYT48L3NwYW4+PC9wPjwv
Yj48YiBzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsOyIgaWQ9ImRvY3MtaW50ZXJuYWwtZ3Vp
ZC01ODU5YjdhMS05ZWI0LTgyYjctNDgwNS05Njc3MTljNjIxYTkiPgo8cCBkaXI9Imx0ciIg
c3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTow
cHQ7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJpYWw7Y29s
b3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2VpZ2h0OjQw
MDtmb250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVjb3JhdGlv
bjpub25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13cmFwOyI+
TGluayB0byBKb2I6IAo8L3NwYW4+PC9wPjxwIGRpcj0ibHRyIiBzdHlsZT0ibGluZS1oZWln
aHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBwdDsiPjxzcGFuIHN0eWxl
PSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tn
cm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9y
bWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwt
YWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXA7Ij48Yj5jcSBqb2I6IDxhIGNs
YXNzPSJtb3otdHh0LWxpbmstZnJlZXRleHQiIGhyZWY9Imh0dHA6Ly9qZW5raW5zLm92aXJ0
Lm9yZy9qb2Ivb3ZpcnQtbWFzdGVyX2NoYW5nZS1xdWV1ZS10ZXN0ZXIvMjczMi9jb25zb2xl
Ij5odHRwOi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LW1hc3Rlcl9jaGFuZ2UtcXVl
dWUtdGVzdGVyLzI3MzIvY29uc29sZTwvYT48L2I+PC9zcGFuPjwvcD48cCBkaXI9Imx0ciIg
c3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTow
cHQ7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJpYWw7Y29s
b3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2VpZ2h0OjQw
MDtmb250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVjb3JhdGlv
bjpub25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13cmFwOyI+
PGI+ZmFpbGVkIGJ1aWxkIGpvYjogPGEgY2xhc3M9Im1vei10eHQtbGluay1mcmVldGV4dCIg
aHJlZj0iaHR0cDovL2plbmtpbnMub3ZpcnQub3JnL2pvYi9vdmlydC1ob3N0ZWQtZW5naW5l
LWhhX21hc3Rlcl9idWlsZC1hcnRpZmFjdHMtZmMyNS14ODZfNjQvNjcvIj5odHRwOi8vamVu
a2lucy5vdmlydC5vcmcvam9iL292aXJ0LWhvc3RlZC1lbmdpbmUtaGFfbWFzdGVyX2J1aWxk
LWFydGlmYWN0cy1mYzI1LXg4Nl82NC82Ny88L2E+PC9iPgo8L3NwYW4+PC9wPgo8cCBkaXI9
Imx0ciIgc3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJv
dHRvbTowcHQ7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJp
YWw7Y29sb3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2Vp
Z2h0OjQwMDtmb250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVj
b3JhdGlvbjpub25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13
cmFwOyI+TGluayB0byBhbGwgbG9nczogCjwvc3Bhbj48L3A+PHAgZGlyPSJsdHIiIHN0eWxl
PSJsaW5lLWhlaWdodDoxLjM4O21hcmdpbi10b3A6MHB0O21hcmdpbi1ib3R0b206MHB0OyI+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFtaWx5OkFyaWFsO2NvbG9yOiMw
MDAwMDA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtmb250LXdlaWdodDo0MDA7Zm9u
dC1zdHlsZTpub3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0ZXh0LWRlY29yYXRpb246bm9u
ZTt2ZXJ0aWNhbC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFjZTpwcmUtd3JhcDsiPjxiPkNR
IGpvYiBsb2dzOiA8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVmPSJodHRw
Oi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LW1hc3Rlcl9jaGFuZ2UtcXVldWUtdGVz
dGVyLzI3MzIvYXJ0aWZhY3QvZXhwb3J0ZWQtYXJ0aWZhY3RzLyI+aHR0cDovL2plbmtpbnMu
b3ZpcnQub3JnL2pvYi9vdmlydC1tYXN0ZXJfY2hhbmdlLXF1ZXVlLXRlc3Rlci8yNzMyL2Fy
dGlmYWN0L2V4cG9ydGVkLWFydGlmYWN0cy88L2E+PC9iPgo8L3NwYW4+PC9wPjxwIGRpcj0i
bHRyIiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90
dG9tOjBwdDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlh
bDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWln
aHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNv
cmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdy
YXA7Ij48Yj5mYWlsZWQgYnVpbGQgbG9nOiA8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0
ZXh0IiBocmVmPSJodHRwOi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LWhvc3RlZC1l
bmdpbmUtaGFfbWFzdGVyX2J1aWxkLWFydGlmYWN0cy1mYzI1LXg4Nl82NC82Ny9hcnRpZmFj
dC8iPmh0dHA6Ly9qZW5raW5zLm92aXJ0Lm9yZy9qb2Ivb3ZpcnQtaG9zdGVkLWVuZ2luZS1o
YV9tYXN0ZXJfYnVpbGQtYXJ0aWZhY3RzLWZjMjUteDg2XzY0LzY3L2FydGlmYWN0LzwvYT48
L2I+Cjwvc3Bhbj48L3A+PHAgZGlyPSJsdHIiIHN0eWxlPSJsaW5lLWhlaWdodDoxLjM4O21h
cmdpbi10b3A6MHB0O21hcmdpbi1ib3R0b206MHB0OyI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6
ZToxMXB0O2ZvbnQtZmFtaWx5OkFyaWFsO2NvbG9yOiMwMDAwMDA7YmFja2dyb3VuZC1jb2xv
cjp0cmFuc3BhcmVudDtmb250LXdlaWdodDo0MDA7Zm9udC1zdHlsZTpub3JtYWw7Zm9udC12
YXJpYW50Om5vcm1hbDt0ZXh0LWRlY29yYXRpb246bm9uZTt2ZXJ0aWNhbC1hbGlnbjpiYXNl
bGluZTt3aGl0ZS1zcGFjZTpwcmUtd3JhcDsiPgo8L3NwYW4+PC9wPjxwIGRpcj0ibHRyIiBz
dHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBw
dDsiPgo8YiBzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsOyIgaWQ9ImRvY3MtaW50ZXJuYWwt
Z3VpZC01ODU5YjdhMS05ZWI3LWFmOWYtNTZjNC0wY2M5M2YwZmNiMWIiPjxwIGRpcj0ibHRy
IiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9t
OjBwdDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtj
b2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6
NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0
aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXA7
Ij4oUmVsZXZhbnQpIGVycm9yIHNuaXBwZXQgZnJvbSB0aGUgbG9nOiA8L3NwYW4+PC9wPjxw
IGRpcj0ibHRyIiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJn
aW4tYm90dG9tOjBwdDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWls
eTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9u
dC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4
dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6
cHJlLXdyYXA7Ij4mbHQ7ZXJyb3ImZ3Q7PC9zcGFuPjwvcD48L2I+CjwvcD4KPHNwYW4gY2xh
c3M9InRpbWVzdGFtcCIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHls
ZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0
eWxlPSJib3gtc2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJv
cmRlci1ib3g7Ij4wNzo0NjoyMTwvYj4gPC9zcGFuPkdldHRpbmcgcmVxdWlyZW1lbnRzIGZv
ciBvdmlydC1ob3N0ZWQtZW5naW5lLWhhLnNwZWMKPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIg
c3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzog
Ym9yZGVyLWJveDsiPjA3OjQ2OjIxPC9iPiA8L3NwYW4+IC0tJmd0OyBBbHJlYWR5IGluc3Rh
bGxlZCA6IHB5dGhvbi0yLjcuMTMtMi5mYzI1Lng4Nl82NAo8c3BhbiBjbGFzcz0idGltZXN0
YW1wIiBzdHlsZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjxiIHN0eWxlPSJib3gtc2l6
aW5nOiBib3JkZXItYm94OyI+MDc6NDY6MjE8L2I+IDwvc3Bhbj4gLS0mZ3Q7IHB5dGhvbjIt
bW9jay0yLjAuMC0yLmZjMjUubm9hcmNoCjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0eWxl
PSJib3gtc2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRl
ci1ib3g7Ij4wNzo0NjoyMjwvYj4gPC9zcGFuPiAtLSZndDsgcHl0aG9uMi1ub3NlLTEuMy43
LTExLmZjMjUubm9hcmNoCjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0eWxlPSJib3gtc2l6
aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij4w
Nzo0NjoyMjwvYj4gPC9zcGFuPiAtLSZndDsgQWxyZWFkeSBpbnN0YWxsZWQgOiBweXRob24t
ZGV2ZWwtMi43LjEzLTIuZmMyNS54ODZfNjQKPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIgc3R5
bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzogYm9y
ZGVyLWJveDsiPjA3OjQ2OjIyPC9iPiA8L3NwYW4+IC0tJmd0OyBBbHJlYWR5IGluc3RhbGxl
ZCA6IHN5c3RlbWQtMjMxLTE3LmZjMjUueDg2XzY0CjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAi
IHN0eWxlPSJib3gtc2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6
IGJvcmRlci1ib3g7Ij4wNzo0NjoyMjwvYj4gPC9zcGFuPiAtLSZndDsgdmRzbS00LjIwLjMt
NDcuZ2l0NDgwMTEyMy5mYzI1Lng4Nl82NAo8c3BhbiBjbGFzcz0idGltZXN0YW1wIiBzdHls
ZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjxiIHN0eWxlPSJib3gtc2l6aW5nOiBib3Jk
ZXItYm94OyI+MDc6NDY6MjI8L2I+IDwvc3Bhbj4gLS0mZ3Q7IHZkc20tY2xpZW50LTQuMjAu
My00Ny5naXQ0ODAxMTIzLmZjMjUubm9hcmNoCjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0
eWxlPSJib3gtc2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJv
cmRlci1ib3g7Ij4wNzo0NjoyMjwvYj4gPC9zcGFuPiAtLSZndDsgdmRzbS1weXRob24tNC4y
MC4zLTQ3LmdpdDQ4MDExMjMuZmMyNS5ub2FyY2gKPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIg
c3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzog
Ym9yZGVyLWJveDsiPjA3OjQ2OjMxPC9iPiA8L3NwYW4+RXJyb3I6IFBhY2thZ2U6IHZkc20t
NC4yMC4zLTQ3LmdpdDQ4MDExMjMuZmMyNS54ODZfNjQgKG92aXJ0LW1hc3Rlci1zbmFwc2hv
dCkKPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1i
b3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjA3OjQ2OjMxPC9iPiA8
L3NwYW4+ICAgICAgICAgICBSZXF1aXJlczogb3BlbnZzd2l0Y2ggJmd0Oz0gMi43LjAKPHNw
YW4gY2xhc3M9InRpbWVzdGFtcCIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48
YiBzdHlsZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjA3OjQ2OjMxPC9iPiA8L3NwYW4+
ICAgICAgICAgICBBdmFpbGFibGU6IG9wZW52c3dpdGNoLTIuNS4wLTQuZmMyNS54ODZfNjQg
KGZlZG9yYS1iYXNlLWZjMjUpCjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0eWxlPSJib3gt
c2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7
Ij4wNzo0NjozMTwvYj4gPC9zcGFuPiAgICAgICAgICAgICAgIG9wZW52c3dpdGNoID0gMi41
LjAtNC5mYzI1CjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0eWxlPSJib3gtc2l6aW5nOiBi
b3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij4wNzo0Njoz
MTwvYj4gPC9zcGFuPiBZb3UgY291bGQgdHJ5IHVzaW5nIC0tc2tpcC1icm9rZW4gdG8gd29y
ayBhcm91bmQgdGhlIHByb2JsZW0KPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIgc3R5bGU9ImJv
eC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzogYm9yZGVyLWJv
eDsiPjA3OjQ2OjMxPC9iPiA8L3NwYW4+IFlvdSBjb3VsZCB0cnkgcnVubmluZzogcnBtIC1W
YSAtLW5vZmlsZXMgLS1ub2RpZ2VzdAo8c3BhbiBjbGFzcz0idGltZXN0YW1wIiBzdHlsZT0i
Ym94LXNpemluZzogYm9yZGVyLWJveDsiPjxiIHN0eWxlPSJib3gtc2l6aW5nOiBib3JkZXIt
Ym94OyI+MDc6NDY6MzE8L2I+IDwvc3Bhbj5Ub29rIDE4IHNlY29uZHM8L2I+PC9zcGFuPgoK
CjxwIGRpcj0ibHRyIiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDtt
YXJnaW4tYm90dG9tOjBwdDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZh
bWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7
Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7
dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3Bh
Y2U6cHJlLXdyYXA7Ij4mbHQ7L2Vycm9yJmd0Ozwvc3Bhbj48L3A+CjwvYj5UaGFua3MsIApE
YWZuYQrCoDwvcHJlPgogIDwvYm9keT4KPC9odG1sPgo=
--------------60D60522EFE450E2C842A076--
7 years, 2 months
[JIRA] (OVIRT-1658) Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)
by Evgheni Dereveanchin (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1505900748-14133-1694
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1658?page=com.atlassian.jir... ]
Evgheni Dereveanchin reassigned OVIRT-1658:
-------------------------------------------
Assignee: Evgheni Dereveanchin (was: infra)
> Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)
> ----------------------------------------------------------------------
>
> Key: OVIRT-1658
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1658
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: Barak Korren
> Assignee: Evgheni Dereveanchin
>
> Seems there is an issue with locale files in recent CentOS versions,
> and we're hitting it in slaves that have been updated recently.
> The symptoms are that Jenkins disconnects from the slave and then
> refuses to reconnect to it. The agent log in Jenkins shows:
> [09/19/17 15:42:56] [SSH] Connection closed.
> [09/19/17 15:58:48] [SSH] Opening SSH connection to
> vm0002.workers-phx.ovirt.org:22.
> [09/19/17 15:58:48] [SSH] WARNING: SSH Host Keys are not being
> verified. Man-in-the-middle attacks may be possible against this
> connection.
> [09/19/17 15:58:48] [SSH] Authentication successful.
> SSH connection reports a garbage before a command execution.
> Check your .bashrc, .profile, and so on to make sure it is quiet.
> The received junk text is as follows:
> /etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot
> change locale (en_US.utf8): No such file or directory
> /etc/profile.d/lang.sh: line 20: warning: setlocale: LC_COLLATE:
> cannot change locale (en_US.utf8): No such file or directory
> /etc/profile.d/lang.sh: line 23: warning: setlocale: LC_MESSAGES:
> cannot change locale (en_US.utf8): No such file or directory
> /etc/profile.d/lang.sh: line 26: warning: setlocale: LC_NUMERIC:
> cannot change locale (en_US.utf8): No such file or directory
> /etc/profile.d/lang.sh: line 29: warning: setlocale: LC_TIME: cannot
> change locale (en_US.utf8): No such file or directory
> null
> [09/19/17 15:58:48] Launch failed - cleaning up connection
> [09/19/17 15:58:48] [SSH] Connection closed.
> The same locale error messages can also be reproduced on the slave by
> running an interactive login from the console or 'su -'. When running
> 'locale -a' you can also see the en_US.UTF-8 locale is somehow
> missing.
> Looking around for this I found the following:
> https://github.com/CentOS/sig-cloud-instance-images/issues/71
> I tried downgrading glibc back to the version we had before, but that
> did not seem to resolve the issue. Eventually I managed to resolve it
> by running 'localedef -i en_US -f UTF-8 en_US.UTF-8' on the slave.
> I've seen this happen on 'vm0002.workers-phx.ovirt.org' which is
> attached to the staging Jenkins, but I've no reason to believe this
> won't start impacting production slaves.
> We need to research this further and find out if we need to do
> something to prevent this issue from surfacing on production slaves.
> --
> Barak Korren
> RHV DevOps team , RHCE, RHCi
> Red Hat EMEA
> redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
--
This message was sent by Atlassian {0}
(v1001.0.0-SNAPSHOT#100060)
------------=_1505900748-14133-1694
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1658?page=com.atlassian.jir... ]</pre>
<h3>Evgheni Dereveanchin reassigned OVIRT-1658:</h3>
<pre>Assignee: Evgheni Dereveanchin (was: infra)</pre>
<blockquote><h3>Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)</h3>
<pre> Key: OVIRT-1658
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1658
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: Barak Korren
Assignee: Evgheni Dereveanchin</pre>
<p>Seems there is an issue with locale files in recent CentOS versions, and we're hitting it in slaves that have been updated recently. The symptoms are that Jenkins disconnects from the slave and then refuses to reconnect to it. The agent log in Jenkins shows: [09/19/17 15:42:56] [SSH] Connection closed. [09/19/17 15:58:48] [SSH] Opening SSH connection to vm0002.workers-phx.ovirt.org:22. [09/19/17 15:58:48] [SSH] WARNING: SSH Host Keys are not being verified. Man-in-the-middle attacks may be possible against this connection. [09/19/17 15:58:48] [SSH] Authentication successful. SSH connection reports a garbage before a command execution. Check your .bashrc, .profile, and so on to make sure it is quiet. The received junk text is as follows: /etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 20: warning: setlocale: LC_COLLATE: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 23: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 26: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 29: warning: setlocale: LC_TIME: cannot change locale (en_US.utf8): No such file or directory null [09/19/17 15:58:48] Launch failed – cleaning up connection [09/19/17 15:58:48] [SSH] Connection closed. The same locale error messages can also be reproduced on the slave by running an interactive login from the console or ‘su -’. When running ‘locale -a’ you can also see the en_US.UTF-8 locale is somehow missing. Looking around for this I found the following: <a href="https://github.com/CentOS/sig-cloud-instance-images/issues/71">https://github.com/CentOS/sig-cloud-instance-images/issues/71</a> I tried downgrading glibc back to the version we had before, but that did not seem to resolve the issue. Eventually I managed to resolve it by running ‘localedef -i en_US -f UTF-8 en_US.UTF-8’ on the slave. I've seen this happen on ‘vm0002.workers-phx.ovirt.org’ which is attached to the staging Jenkins, but I've no reason to believe this won't start impacting production slaves. We need to research this further and find out if we need to do something to prevent this issue from surfacing on production slaves. — Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted</p></blockquote>
<p>— This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYL..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1505900748-14133-1694--
7 years, 2 months
