September 2017 - Infra - oVirt List Archives

[oVirt Jenkins] ovirt-system-tests_he-basic-suite-4.1 - Build # 37 - Failure!
by jenkins＠jenkins.phx.ovirt.org 21 Sep '17

21 Sep '17

------=_Part_627_898823252.1505878358235 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Project: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/ Build: http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/37/ Build Number: 37 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #37 [Sahina Bose] fixes for the 4.1 and master reposyncfiles [Dan Kenigsberg] enable fc26 build for mom [Dan Kenigsberg] enable fc26 build for ovirt-vmconsole. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.list_glance_images Error Message: Fault reason is "Operation Failed". Fault detail is "[Failed to communicate with the external provider, see log for additional details.]". HTTP response code is 400. Stack Trace: Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.1/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 697, in list_glance_images list_glance_images_4(api) File "/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.1/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 748, in list_glance_images_4 if not check_glance_connectivity_4(api): File "/home/jenkins/workspace/ovirt-system-tests_he-basic-suite-4.1/ovirt-system-tests/he-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 846, in check_glance_connectivity_4 glance.test_connectivity() File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 14026, in test_connectivity return self._internal_action(action, 'testconnectivity', None, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 290, in _internal_action return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 287, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 123, in _check_fault self._raise_error(response, body) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 109, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Failed to communicate with the external provider, see log for additional details.]". HTTP response code is 400. ------=_Part_627_898823252.1505878358235--

1 1

[oVirt Jenkins] ovirt-system-tests_hc-basic-suite-4.1 - Build # 34 - Failure!
by jenkins＠jenkins.phx.ovirt.org 21 Sep '17

21 Sep '17

------=_Part_625_1969086870.1505877115773 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Project: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.1= /=20 Build: http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.1/3= 4/ Build Number: 34 Build Status: Failure Triggered By: Started by timer ------------------------------------- Changes Since Last Success: ------------------------------------- Changes for Build #34 [Sahina Bose] fixes for the 4.1 and master reposyncfiles [Dan Kenigsberg] enable fc26 build for mom [Dan Kenigsberg] enable fc26 build for ovirt-vmconsole. ----------------- Failed Tests: ----------------- 1 tests failed. FAILED: 002_bootstrap.add_hosts Error Message: status: 503 reason: Service Unavailable detail:=20 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>503 Service Unavailable</title> </head><body> <h1>Service Unavailable</h1> <p>The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.</p> </body></html> -------------------- >> begin captured logging << -------------------- ovirtlago.testlib: ERROR: * Unhandled exception in <function _host_is_u= p at 0x46ace60> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.1/ovirt= -system-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 144= , in _host_is_up cur_state =3D api.hosts.get(host.name()).status.state File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py= ", line 18338, in get headers=3D{"All-Content":all_content} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 46, in get return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls= =3Dcls) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 122, in request persistent_auth=3Dself.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body= ) RequestError:=20 status: 503 reason: Service Unavailable detail:=20 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>503 Service Unavailable</title> </head><body> <h1>Service Unavailable</h1> <p>The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later.</p> </body></html> --------------------- >> end captured logging << --------------------- Stack Trace: File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTes= t self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, i= n wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in= wrapper return func(get_test_prefix(), *args, **kwargs) File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.1/ovirt= -system-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 163= , in add_hosts testlib.assert_true_within(_host_is_up, timeout=3D15 * 60) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 263, i= n assert_true_within assert_equals_within(func, True, timeout, allowed_exceptions) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 219, i= n assert_equals_within res =3D func() File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-suite-4.1/ovirt= -system-tests/hc-basic-suite-4.1/test-scenarios/002_bootstrap.py", line 144= , in _host_is_up cur_state =3D api.hosts.get(host.name()).status.state File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/brokers.py= ", line 18338, in get headers=3D{"All-Content":all_content} File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 46, in get return self.request(method=3D'GET', url=3Durl, headers=3Dheaders, cls= =3Dcls) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/proxy.py",= line 122, in request persistent_auth=3Dself.__persistent_auth File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 79, in do_request persistent_auth) File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/connection= spool.py", line 162, in __do_request raise errors.RequestError(response_code, response_reason, response_body= ) '\r\nstatus: 503\r\nreason: Service Unavailable\r\ndetail: \n<!DOCTYPE HTML= PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>503 Service Unav= ailable</title>\n</head><body>\n<h1>Service Unavailable</h1>\n<p>The server= is temporarily unable to service your\nrequest due to maintenance downtime= or capacity\nproblems. Please try again later.</p>\n</body></html>\n\n----= ---------------- >> begin captured logging << --------------------\novirtla= go.testlib: ERROR: * Unhandled exception in <function _host_is_up at 0x= 46ace60>\nTraceback (most recent call last):\n File "/usr/lib/python2.7/si= te-packages/ovirtlago/testlib.py", line 219, in assert_equals_within\n r= es =3D func()\n File "/home/jenkins/workspace/ovirt-system-tests_hc-basic-= suite-4.1/ovirt-system-tests/hc-basic-suite-4.1/test-scenarios/002_bootstra= p.py", line 144, in _host_is_up\n cur_state =3D api.hosts.get(host.name(= )).status.state\n File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastr= ucture/brokers.py", line 18338, in get\n headers=3D{"All-Content":all_co= ntent}\n File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure/pr= oxy.py", line 46, in get\n return self.request(method=3D\'GET\', url=3Du= rl, headers=3Dheaders, cls=3Dcls)\n File "/usr/lib/python2.7/site-packages= /ovirtsdk/infrastructure/proxy.py", line 122, in request\n persistent_au= th=3Dself.__persistent_auth\n File "/usr/lib/python2.7/site-packages/ovirt= sdk/infrastructure/connectionspool.py", line 79, in do_request\n persist= ent_auth)\n File "/usr/lib/python2.7/site-packages/ovirtsdk/infrastructure= /connectionspool.py", line 162, in __do_request\n raise errors.RequestEr= ror(response_code, response_reason, response_body)\nRequestError: \r\nstatu= s: 503\r\nreason: Service Unavailable\r\ndetail: \n<!DOCTYPE HTML PUBLIC "-= //IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>503 Service Unavailable</t= itle>\n</head><body>\n<h1>Service Unavailable</h1>\n<p>The server is tempor= arily unable to service your\nrequest due to maintenance downtime or capaci= ty\nproblems. Please try again later.</p>\n</body></html>\n\n--------------= ------- >> end captured logging << ---------------------' ------=_Part_625_1969086870.1505877115773--

1 1

oVirt infra daily report - unstable production jobs - 450
by jenkins＠jenkins.phx.ovirt.org 20 Sep '17

20 Sep '17

------=_Part_645_820653108.1505948406105 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Good morning! Attached is the HTML page with the jenkins status report. You can see it also here: - http://jenkins.ovirt.org/job/system_jenkins-report/450//artifact/exported-a… Cheers, Jenkins ------=_Part_645_820653108.1505948406105 Content-Type: text/html; charset=us-ascii; name=upstream_report.html Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=upstream_report.html Content-ID: <upstream_report.html> <!DOCTYPE html><head><style type="text/css"> table.gridtable { border-collapse: collapse; table-layout:fixed; width:1600px; font-family: monospace; font-size:13px; } .head { font-size:20px; font-family: arial; } .sub { font-size:18px; background-color:#e5e5e5; font-family: arial; } pre { font-family: monospace; display: inline; white-space: pre-wrap; white-space: -moz-pre-wrap !important; white-space: -pre-wrap; white-space: -o-pre-wrap; word-wrap: break-word; } </style> </head> <body> <table class="gridtable" border=2> <tr><th colspan=2 class=head> RHEVM CI Jenkins Daily Report - 20/09/2017 </th></tr><tr><th colspan=2 class=sub> <font color="blue"><a href="http://jenkins.ovirt.org/">00 Unstable Critical</a></font> </th></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.1/">ovirt-system-tests_hc-basic-suite-4.1</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-4.1/">ovirt-system-tests_he-basic-suite-4.1</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> <tr><td> <a href="http://jenkins.ovirt.org/job/system-sync_mirrors-fedora-updates-fc25-x86_64/">system-sync_mirrors-fedora-updates-fc25-x86_64</a> </td><td> This job is automatically updated by jenkins job builder, any manual change will be lost in the next update. If you want to make permanent changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;hb=…"> jenkins</a> repo.  </td></tr> ------=_Part_645_820653108.1505948406105--

1 0

[JIRA] (OVIRT-1659) add ipv6 accept_ra to sysctl on jenkins slaves
by Evgheni Dereveanchin (oVirt JIRA) 20 Sep '17

20 Sep '17

This is a multi-part message in MIME format... ------------=_1505919776-7137-3131 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1659?page=com.atlassian.jira.… ] Evgheni Dereveanchin reassigned OVIRT-1659: ------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > add ipv6 accept_ra to sysctl on jenkins slaves > ---------------------------------------------- > > Key: OVIRT-1659 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1659 > Project: oVirt - virtualization made easy > Issue Type: Task > Reporter: Evgheni Dereveanchin > Assignee: Evgheni Dereveanchin > Priority: High > > To work around new libvirt behavior, net.ipv6.conf.all.accept_ra=2 sysctl parameter needs to be set on slaves, specifically the bare metals. > More info here: https://github.com/lago-project/lago/pull/654/files -- This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060) ------------=_1505919776-7137-3131 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1659?page=com.atlassian.jira.… ]</pre> <h3>Evgheni Dereveanchin reassigned OVIRT-1659:</h3> <pre>Assignee: Evgheni Dereveanchin (was: infra)</pre> <blockquote><h3>add ipv6 accept_ra to sysctl on jenkins slaves</h3> <pre> Key: OVIRT-1659 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1659 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: Evgheni Dereveanchin Priority: High</pre> <p>To work around new libvirt behavior, net.ipv6.conf.all.accept_ra=2 sysctl parameter needs to be set on slaves, specifically the bare metals. More info here: <a href="https://github.com/lago-project/lago/pull/654/files">https://github.com/lago-project/lago/pull/654/files</a></p></blockquote> <p>— This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYLyz…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1505919776-7137-3131--

1 0

[CQ]: 82035, 1 (ovirt-engine-sdk-ruby) failed "ovirt-4.1" system tests
by oVirt Jenkins 20 Sep '17

20 Sep '17

Change 82035,1 (ovirt-engine-sdk-ruby) is probably the reason behind recent system test failures in the "ovirt-4.1" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/82035/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-4.1_change-queue-tester/1034/

1 0

[JIRA] (OVIRT-1659) add ipv6 accept_ra to sysctl on jenkins slaves
by Evgheni Dereveanchin (oVirt JIRA) 20 Sep '17

20 Sep '17

This is a multi-part message in MIME format... ------------=_1505907708-16017-2467 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Evgheni Dereveanchin created OVIRT-1659: ------------------------------------------- Summary: add ipv6 accept_ra to sysctl on jenkins slaves Key: OVIRT-1659 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1659 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: infra Priority: High To work around new libvirt behavior, net.ipv6.conf.all.accept_ra=2 sysctl parameter needs to be set on slaves, specifically the bare metals. More info here: https://github.com/lago-project/lago/pull/654/files -- This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060) ------------=_1505907708-16017-2467 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <h3>Evgheni Dereveanchin created OVIRT-1659:</h3> <pre> Summary: add ipv6 accept_ra to sysctl on jenkins slaves Key: OVIRT-1659 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1659 Project: oVirt - virtualization made easy Issue Type: Task Reporter: Evgheni Dereveanchin Assignee: infra Priority: High</pre> <p>To work around new libvirt behavior, net.ipv6.conf.all.accept_ra=2 sysctl parameter needs to be set on slaves, specifically the bare metals.</p> <p>More info here: <a href="https://github.com/lago-project/lago/pull/654/files">https://github.com/lago-project/lago/pull/654/files</a></p> <p>— This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYLyz…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1505907708-16017-2467--

1 0

[JIRA] (OVIRT-1658) Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)
by Evgheni Dereveanchin (oVirt JIRA) 20 Sep '17

20 Sep '17

This is a multi-part message in MIME format... ------------=_1505906876-10461-2701 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1658?page=com.atlassian.jira.… ] Evgheni Dereveanchin commented on OVIRT-1658: --------------------------------------------- Seen this as well. Logged a CentOS bug and will run the localedef command manually across our systems to avoid this breaking stuff. > Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale) > ---------------------------------------------------------------------- > > Key: OVIRT-1658 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1658 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Barak Korren > Assignee: Evgheni Dereveanchin > > Seems there is an issue with locale files in recent CentOS versions, > and we're hitting it in slaves that have been updated recently. > The symptoms are that Jenkins disconnects from the slave and then > refuses to reconnect to it. The agent log in Jenkins shows: > [09/19/17 15:42:56] [SSH] Connection closed. > [09/19/17 15:58:48] [SSH] Opening SSH connection to > vm0002.workers-phx.ovirt.org:22. > [09/19/17 15:58:48] [SSH] WARNING: SSH Host Keys are not being > verified. Man-in-the-middle attacks may be possible against this > connection. > [09/19/17 15:58:48] [SSH] Authentication successful. > SSH connection reports a garbage before a command execution. > Check your .bashrc, .profile, and so on to make sure it is quiet. > The received junk text is as follows: > /etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot > change locale (en_US.utf8): No such file or directory > /etc/profile.d/lang.sh: line 20: warning: setlocale: LC_COLLATE: > cannot change locale (en_US.utf8): No such file or directory > /etc/profile.d/lang.sh: line 23: warning: setlocale: LC_MESSAGES: > cannot change locale (en_US.utf8): No such file or directory > /etc/profile.d/lang.sh: line 26: warning: setlocale: LC_NUMERIC: > cannot change locale (en_US.utf8): No such file or directory > /etc/profile.d/lang.sh: line 29: warning: setlocale: LC_TIME: cannot > change locale (en_US.utf8): No such file or directory > null > [09/19/17 15:58:48] Launch failed - cleaning up connection > [09/19/17 15:58:48] [SSH] Connection closed. > The same locale error messages can also be reproduced on the slave by > running an interactive login from the console or 'su -'. When running > 'locale -a' you can also see the en_US.UTF-8 locale is somehow > missing. > Looking around for this I found the following: > https://github.com/CentOS/sig-cloud-instance-images/issues/71 > I tried downgrading glibc back to the version we had before, but that > did not seem to resolve the issue. Eventually I managed to resolve it > by running 'localedef -i en_US -f UTF-8 en_US.UTF-8' on the slave. > I've seen this happen on 'vm0002.workers-phx.ovirt.org' which is > attached to the staging Jenkins, but I've no reason to believe this > won't start impacting production slaves. > We need to research this further and find out if we need to do > something to prevent this issue from surfacing on production slaves. > -- > Barak Korren > RHV DevOps team , RHCE, RHCi > Red Hat EMEA > redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted -- This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060) ------------=_1505906876-10461-2701 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1658?page=com.atlassian.jira.… ]</pre> <h3>Evgheni Dereveanchin commented on OVIRT-1658:</h3> <p>Seen this as well. Logged a CentOS bug and will run the localedef command manually across our systems to avoid this breaking stuff.</p> <blockquote><h3>Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)</h3> <pre> Key: OVIRT-1658 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1658 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Barak Korren Assignee: Evgheni Dereveanchin</pre> <p>Seems there is an issue with locale files in recent CentOS versions, and we're hitting it in slaves that have been updated recently. The symptoms are that Jenkins disconnects from the slave and then refuses to reconnect to it. The agent log in Jenkins shows: [09/19/17 15:42:56] [SSH] Connection closed. [09/19/17 15:58:48] [SSH] Opening SSH connection to vm0002.workers-phx.ovirt.org:22. [09/19/17 15:58:48] [SSH] WARNING: SSH Host Keys are not being verified. Man-in-the-middle attacks may be possible against this connection. [09/19/17 15:58:48] [SSH] Authentication successful. SSH connection reports a garbage before a command execution. Check your .bashrc, .profile, and so on to make sure it is quiet. The received junk text is as follows: /etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 20: warning: setlocale: LC_COLLATE: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 23: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 26: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 29: warning: setlocale: LC_TIME: cannot change locale (en_US.utf8): No such file or directory null [09/19/17 15:58:48] Launch failed – cleaning up connection [09/19/17 15:58:48] [SSH] Connection closed. The same locale error messages can also be reproduced on the slave by running an interactive login from the console or ‘su -’. When running ‘locale -a’ you can also see the en_US.UTF-8 locale is somehow missing. Looking around for this I found the following: <a href="https://github.com/CentOS/sig-cloud-instance-images/issues/71">https://github.com/CentOS/sig-cloud-instance-images/issues/71</a> I tried downgrading glibc back to the version we had before, but that did not seem to resolve the issue. Eventually I managed to resolve it by running ‘localedef -i en_US -f UTF-8 en_US.UTF-8’ on the slave. I've seen this happen on ‘vm0002.workers-phx.ovirt.org’ which is attached to the staging Jenkins, but I've no reason to believe this won't start impacting production slaves. We need to research this further and find out if we need to do something to prevent this issue from surfacing on production slaves. — Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted</p></blockquote> <p>— This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYLyz…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1505906876-10461-2701--

1 0

[CQ]: 82022, 1 (ovirt-engine-sdk-ruby) failed "ovirt-master" system tests
by oVirt Jenkins 20 Sep '17

20 Sep '17

Change 82022,1 (ovirt-engine-sdk-ruby) is probably the reason behind recent system test failures in the "ovirt-master" change queue and needs to be fixed. This change had been removed from the testing queue. Artifacts build from this change will not be released until it is fixed. For further details about the change see: https://gerrit.ovirt.org/#/c/82022/1 For failed test results see: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2734/

1 0

Subject: [ OST Failure Report ] [ master ] [ 20-09-2017] [ No Test Failed - missing packages in fc25 ]
by Dafna Ron 20 Sep '17

20 Sep '17

This is a multi-part message in MIME format. --------------60D60522EFE450E2C842A076 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi, CQ reported a failure in one of the builds and looking at the console I found that we had a failure before the OST started due to: ovirt-hosted-engine-ha_master_build-artifacts-fc25-x86_64 (67) failed building The build failed of ovirt-hosted-engine-ha_master_build-artifacts-fc25-x86_64 failed due to missing packages. * Link to suspected patches: patch reported as failed is https://gerrit.ovirt.org/#/c/81792/ ** Link to Job: *cq job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2732/console* *failed build job: http://jenkins.ovirt.org/job/ovirt-hosted-engine-ha_master_build-artifacts-… Link to all logs: *CQ job logs: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2732/artifact… *failed build log: http://jenkins.ovirt.org/job/ovirt-hosted-engine-ha_master_build-artifacts-… ** *(Relevant) error snippet from the log: * *<error>* **07:46:21* Getting requirements for ovirt-hosted-engine-ha.spec *07:46:21* --> Already installed : python-2.7.13-2.fc25.x86_64 *07:46:21* --> python2-mock-2.0.0-2.fc25.noarch *07:46:22* --> python2-nose-1.3.7-11.fc25.noarch *07:46:22* --> Already installed : python-devel-2.7.13-2.fc25.x86_64 *07:46:22* --> Already installed : systemd-231-17.fc25.x86_64 *07:46:22* --> vdsm-4.20.3-47.git4801123.fc25.x86_64 *07:46:22* --> vdsm-client-4.20.3-47.git4801123.fc25.noarch *07:46:22* --> vdsm-python-4.20.3-47.git4801123.fc25.noarch *07:46:31* Error: Package: vdsm-4.20.3-47.git4801123.fc25.x86_64 (ovirt-master-snapshot) *07:46:31* Requires: openvswitch >= 2.7.0 *07:46:31* Available: openvswitch-2.5.0-4.fc25.x86_64 (fedora-base-fc25) *07:46:31* openvswitch = 2.5.0-4.fc25 *07:46:31* You could try using --skip-broken to work around the problem *07:46:31* You could try running: rpm -Va --nofiles --nodigest *07:46:31* Took 18 seconds* </error> *Thanks, Dafna --------------60D60522EFE450E2C842A076 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGh0bWw+CiAgPGhlYWQ+CgogICAgPG1ldGEgaHR0cC1lcXVpdj0iY29udGVudC10eXBlIiBj b250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPgogIDwvaGVhZD4KICA8Ym9keSB0 ZXh0PSIjMDAwMDAwIiBiZ2NvbG9yPSIjRkZGRkZGIj4KICAgIDxwPkhpLCA8YnI+CiAgICA8 L3A+CiAgICA8cD5DUSByZXBvcnRlZCBhIGZhaWx1cmUgaW4gb25lIG9mIHRoZSBidWlsZHMg YW5kIGxvb2tpbmcgYXQgdGhlCiAgICAgIGNvbnNvbGUgSSBmb3VuZCB0aGF0IHdlIGhhZCBh IGZhaWx1cmUgYmVmb3JlIHRoZSBPU1Qgc3RhcnRlZCBkdWUKICAgICAgdG86IDxicj4KICAg IDwvcD4KICAgIDxwcmUgY2xhc3M9ImNvbnNvbGUtb3V0cHV0IiBzdHlsZT0iYm94LXNpemlu ZzogYm9yZGVyLWJveDsgd2hpdGUtc3BhY2U6IHByZS13cmFwOyB3b3JkLXdyYXA6IGJyZWFr LXdvcmQ7IG1hcmdpbjogMHB4OyBjb2xvcjogcmdiKDUxLCA1MSwgNTEpOyBmb250LXNpemU6 IDE0cHg7IGZvbnQtc3R5bGU6IG5vcm1hbDsgZm9udC12YXJpYW50LWxpZ2F0dXJlczogbm9y bWFsOyBmb250LXZhcmlhbnQtY2Fwczogbm9ybWFsOyBmb250LXdlaWdodDogbm9ybWFsOyBs ZXR0ZXItc3BhY2luZzogbm9ybWFsOyBvcnBoYW5zOiAyOyB0ZXh0LWFsaWduOiBzdGFydDsg dGV4dC1pbmRlbnQ6IDBweDsgdGV4dC10cmFuc2Zvcm06IG5vbmU7IHdpZG93czogMjsgd29y ZC1zcGFjaW5nOiAwcHg7IC13ZWJraXQtdGV4dC1zdHJva2Utd2lkdGg6IDBweDsgdGV4dC1k ZWNvcmF0aW9uLXN0eWxlOiBpbml0aWFsOyB0ZXh0LWRlY29yYXRpb24tY29sb3I6IGluaXRp YWw7Ij5vdmlydC1ob3N0ZWQtZW5naW5lLWhhX21hc3Rlcl9idWlsZC1hcnRpZmFjdHMtZmMy NS14ODZfNjQgKDY3KSBmYWlsZWQgYnVpbGRpbmcKClRoZSBidWlsZCBmYWlsZWQgb2Ygb3Zp cnQtaG9zdGVkLWVuZ2luZS1oYV9tYXN0ZXJfYnVpbGQtYXJ0aWZhY3RzLWZjMjUteDg2XzY0 IGZhaWxlZCBkdWUgdG8gbWlzc2luZyBwYWNrYWdlcy4gCgo8YiBzdHlsZT0iZm9udC13ZWln aHQ6bm9ybWFsOyIgaWQ9ImRvY3MtaW50ZXJuYWwtZ3VpZC01ODU5YjdhMS05ZWI0LTgyYjct NDgwNS05Njc3MTljNjIxYTkiPjxwIGRpcj0ibHRyIiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4z ODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBwdDsiPjxzcGFuIHN0eWxlPSJmb250 LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQt Y29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2Zv bnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246 YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXA7Ij5MaW5rIHRvIHN1c3BlY3RlZCBwYXRj aGVzOiAgcGF0Y2ggcmVwb3J0ZWQgYXMgZmFpbGVkIGlzIDxhIGNsYXNzPSJtb3otdHh0LWxp bmstZnJlZXRleHQiIGhyZWY9Imh0dHBzOi8vZ2Vycml0Lm92aXJ0Lm9yZy8jL2MvODE3OTIv Ij5odHRwczovL2dlcnJpdC5vdmlydC5vcmcvIy9jLzgxNzkyLzwvYT48L3NwYW4+PC9wPjwv Yj48YiBzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsOyIgaWQ9ImRvY3MtaW50ZXJuYWwtZ3Vp ZC01ODU5YjdhMS05ZWI0LTgyYjctNDgwNS05Njc3MTljNjIxYTkiPgo8cCBkaXI9Imx0ciIg c3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTow cHQ7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJpYWw7Y29s b3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2VpZ2h0OjQw MDtmb250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVjb3JhdGlv bjpub25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13cmFwOyI+ TGluayB0byBKb2I6IAo8L3NwYW4+PC9wPjxwIGRpcj0ibHRyIiBzdHlsZT0ibGluZS1oZWln aHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBwdDsiPjxzcGFuIHN0eWxl PSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tn cm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9y bWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwt YWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXA7Ij48Yj5jcSBqb2I6IDxhIGNs YXNzPSJtb3otdHh0LWxpbmstZnJlZXRleHQiIGhyZWY9Imh0dHA6Ly9qZW5raW5zLm92aXJ0 Lm9yZy9qb2Ivb3ZpcnQtbWFzdGVyX2NoYW5nZS1xdWV1ZS10ZXN0ZXIvMjczMi9jb25zb2xl Ij5odHRwOi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LW1hc3Rlcl9jaGFuZ2UtcXVl dWUtdGVzdGVyLzI3MzIvY29uc29sZTwvYT48L2I+PC9zcGFuPjwvcD48cCBkaXI9Imx0ciIg c3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTow cHQ7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJpYWw7Y29s b3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2VpZ2h0OjQw MDtmb250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVjb3JhdGlv bjpub25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13cmFwOyI+ PGI+ZmFpbGVkIGJ1aWxkIGpvYjogPGEgY2xhc3M9Im1vei10eHQtbGluay1mcmVldGV4dCIg aHJlZj0iaHR0cDovL2plbmtpbnMub3ZpcnQub3JnL2pvYi9vdmlydC1ob3N0ZWQtZW5naW5l LWhhX21hc3Rlcl9idWlsZC1hcnRpZmFjdHMtZmMyNS14ODZfNjQvNjcvIj5odHRwOi8vamVu a2lucy5vdmlydC5vcmcvam9iL292aXJ0LWhvc3RlZC1lbmdpbmUtaGFfbWFzdGVyX2J1aWxk LWFydGlmYWN0cy1mYzI1LXg4Nl82NC82Ny88L2E+PC9iPgo8L3NwYW4+PC9wPgo8cCBkaXI9 Imx0ciIgc3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJv dHRvbTowcHQ7Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJp YWw7Y29sb3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2Vp Z2h0OjQwMDtmb250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVj b3JhdGlvbjpub25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13 cmFwOyI+TGluayB0byBhbGwgbG9nczogCjwvc3Bhbj48L3A+PHAgZGlyPSJsdHIiIHN0eWxl PSJsaW5lLWhlaWdodDoxLjM4O21hcmdpbi10b3A6MHB0O21hcmdpbi1ib3R0b206MHB0OyI+ PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFtaWx5OkFyaWFsO2NvbG9yOiMw MDAwMDA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtmb250LXdlaWdodDo0MDA7Zm9u dC1zdHlsZTpub3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0ZXh0LWRlY29yYXRpb246bm9u ZTt2ZXJ0aWNhbC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFjZTpwcmUtd3JhcDsiPjxiPkNR IGpvYiBsb2dzOiA8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVmPSJodHRw Oi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LW1hc3Rlcl9jaGFuZ2UtcXVldWUtdGVz dGVyLzI3MzIvYXJ0aWZhY3QvZXhwb3J0ZWQtYXJ0aWZhY3RzLyI+aHR0cDovL2plbmtpbnMu b3ZpcnQub3JnL2pvYi9vdmlydC1tYXN0ZXJfY2hhbmdlLXF1ZXVlLXRlc3Rlci8yNzMyL2Fy dGlmYWN0L2V4cG9ydGVkLWFydGlmYWN0cy88L2E+PC9iPgo8L3NwYW4+PC9wPjxwIGRpcj0i bHRyIiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90 dG9tOjBwdDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlh bDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWln aHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNv cmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdy YXA7Ij48Yj5mYWlsZWQgYnVpbGQgbG9nOiA8YSBjbGFzcz0ibW96LXR4dC1saW5rLWZyZWV0 ZXh0IiBocmVmPSJodHRwOi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LWhvc3RlZC1l bmdpbmUtaGFfbWFzdGVyX2J1aWxkLWFydGlmYWN0cy1mYzI1LXg4Nl82NC82Ny9hcnRpZmFj dC8iPmh0dHA6Ly9qZW5raW5zLm92aXJ0Lm9yZy9qb2Ivb3ZpcnQtaG9zdGVkLWVuZ2luZS1o YV9tYXN0ZXJfYnVpbGQtYXJ0aWZhY3RzLWZjMjUteDg2XzY0LzY3L2FydGlmYWN0LzwvYT48 L2I+Cjwvc3Bhbj48L3A+PHAgZGlyPSJsdHIiIHN0eWxlPSJsaW5lLWhlaWdodDoxLjM4O21h cmdpbi10b3A6MHB0O21hcmdpbi1ib3R0b206MHB0OyI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6 ZToxMXB0O2ZvbnQtZmFtaWx5OkFyaWFsO2NvbG9yOiMwMDAwMDA7YmFja2dyb3VuZC1jb2xv cjp0cmFuc3BhcmVudDtmb250LXdlaWdodDo0MDA7Zm9udC1zdHlsZTpub3JtYWw7Zm9udC12 YXJpYW50Om5vcm1hbDt0ZXh0LWRlY29yYXRpb246bm9uZTt2ZXJ0aWNhbC1hbGlnbjpiYXNl bGluZTt3aGl0ZS1zcGFjZTpwcmUtd3JhcDsiPgo8L3NwYW4+PC9wPjxwIGRpcj0ibHRyIiBz dHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBw dDsiPgo8YiBzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsOyIgaWQ9ImRvY3MtaW50ZXJuYWwt Z3VpZC01ODU5YjdhMS05ZWI3LWFmOWYtNTZjNC0wY2M5M2YwZmNiMWIiPjxwIGRpcj0ibHRy IiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9t OjBwdDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtj b2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6 NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0 aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXA7 Ij4oUmVsZXZhbnQpIGVycm9yIHNuaXBwZXQgZnJvbSB0aGUgbG9nOiA8L3NwYW4+PC9wPjxw IGRpcj0ibHRyIiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJn aW4tYm90dG9tOjBwdDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWls eTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9u dC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4 dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6 cHJlLXdyYXA7Ij4mbHQ7ZXJyb3ImZ3Q7PC9zcGFuPjwvcD48L2I+CjwvcD4KPHNwYW4gY2xh c3M9InRpbWVzdGFtcCIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHls ZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0 eWxlPSJib3gtc2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJv cmRlci1ib3g7Ij4wNzo0NjoyMTwvYj4gPC9zcGFuPkdldHRpbmcgcmVxdWlyZW1lbnRzIGZv ciBvdmlydC1ob3N0ZWQtZW5naW5lLWhhLnNwZWMKPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIg c3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzog Ym9yZGVyLWJveDsiPjA3OjQ2OjIxPC9iPiA8L3NwYW4+IC0tJmd0OyBBbHJlYWR5IGluc3Rh bGxlZCA6IHB5dGhvbi0yLjcuMTMtMi5mYzI1Lng4Nl82NAo8c3BhbiBjbGFzcz0idGltZXN0 YW1wIiBzdHlsZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjxiIHN0eWxlPSJib3gtc2l6 aW5nOiBib3JkZXItYm94OyI+MDc6NDY6MjE8L2I+IDwvc3Bhbj4gLS0mZ3Q7IHB5dGhvbjIt bW9jay0yLjAuMC0yLmZjMjUubm9hcmNoCjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0eWxl PSJib3gtc2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRl ci1ib3g7Ij4wNzo0NjoyMjwvYj4gPC9zcGFuPiAtLSZndDsgcHl0aG9uMi1ub3NlLTEuMy43 LTExLmZjMjUubm9hcmNoCjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0eWxlPSJib3gtc2l6 aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij4w Nzo0NjoyMjwvYj4gPC9zcGFuPiAtLSZndDsgQWxyZWFkeSBpbnN0YWxsZWQgOiBweXRob24t ZGV2ZWwtMi43LjEzLTIuZmMyNS54ODZfNjQKPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIgc3R5 bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzogYm9y ZGVyLWJveDsiPjA3OjQ2OjIyPC9iPiA8L3NwYW4+IC0tJmd0OyBBbHJlYWR5IGluc3RhbGxl ZCA6IHN5c3RlbWQtMjMxLTE3LmZjMjUueDg2XzY0CjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAi IHN0eWxlPSJib3gtc2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6 IGJvcmRlci1ib3g7Ij4wNzo0NjoyMjwvYj4gPC9zcGFuPiAtLSZndDsgdmRzbS00LjIwLjMt NDcuZ2l0NDgwMTEyMy5mYzI1Lng4Nl82NAo8c3BhbiBjbGFzcz0idGltZXN0YW1wIiBzdHls ZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjxiIHN0eWxlPSJib3gtc2l6aW5nOiBib3Jk ZXItYm94OyI+MDc6NDY6MjI8L2I+IDwvc3Bhbj4gLS0mZ3Q7IHZkc20tY2xpZW50LTQuMjAu My00Ny5naXQ0ODAxMTIzLmZjMjUubm9hcmNoCjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0 eWxlPSJib3gtc2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJv cmRlci1ib3g7Ij4wNzo0NjoyMjwvYj4gPC9zcGFuPiAtLSZndDsgdmRzbS1weXRob24tNC4y MC4zLTQ3LmdpdDQ4MDExMjMuZmMyNS5ub2FyY2gKPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIg c3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzog Ym9yZGVyLWJveDsiPjA3OjQ2OjMxPC9iPiA8L3NwYW4+RXJyb3I6IFBhY2thZ2U6IHZkc20t NC4yMC4zLTQ3LmdpdDQ4MDExMjMuZmMyNS54ODZfNjQgKG92aXJ0LW1hc3Rlci1zbmFwc2hv dCkKPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1i b3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjA3OjQ2OjMxPC9iPiA8 L3NwYW4+ICAgICAgICAgICBSZXF1aXJlczogb3BlbnZzd2l0Y2ggJmd0Oz0gMi43LjAKPHNw YW4gY2xhc3M9InRpbWVzdGFtcCIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij48 YiBzdHlsZT0iYm94LXNpemluZzogYm9yZGVyLWJveDsiPjA3OjQ2OjMxPC9iPiA8L3NwYW4+ ICAgICAgICAgICBBdmFpbGFibGU6IG9wZW52c3dpdGNoLTIuNS4wLTQuZmMyNS54ODZfNjQg KGZlZG9yYS1iYXNlLWZjMjUpCjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0eWxlPSJib3gt c2l6aW5nOiBib3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7 Ij4wNzo0NjozMTwvYj4gPC9zcGFuPiAgICAgICAgICAgICAgIG9wZW52c3dpdGNoID0gMi41 LjAtNC5mYzI1CjxzcGFuIGNsYXNzPSJ0aW1lc3RhbXAiIHN0eWxlPSJib3gtc2l6aW5nOiBi b3JkZXItYm94OyI+PGIgc3R5bGU9ImJveC1zaXppbmc6IGJvcmRlci1ib3g7Ij4wNzo0Njoz MTwvYj4gPC9zcGFuPiBZb3UgY291bGQgdHJ5IHVzaW5nIC0tc2tpcC1icm9rZW4gdG8gd29y ayBhcm91bmQgdGhlIHByb2JsZW0KPHNwYW4gY2xhc3M9InRpbWVzdGFtcCIgc3R5bGU9ImJv eC1zaXppbmc6IGJvcmRlci1ib3g7Ij48YiBzdHlsZT0iYm94LXNpemluZzogYm9yZGVyLWJv eDsiPjA3OjQ2OjMxPC9iPiA8L3NwYW4+IFlvdSBjb3VsZCB0cnkgcnVubmluZzogcnBtIC1W YSAtLW5vZmlsZXMgLS1ub2RpZ2VzdAo8c3BhbiBjbGFzcz0idGltZXN0YW1wIiBzdHlsZT0i Ym94LXNpemluZzogYm9yZGVyLWJveDsiPjxiIHN0eWxlPSJib3gtc2l6aW5nOiBib3JkZXIt Ym94OyI+MDc6NDY6MzE8L2I+IDwvc3Bhbj5Ub29rIDE4IHNlY29uZHM8L2I+PC9zcGFuPgoK CjxwIGRpcj0ibHRyIiBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDtt YXJnaW4tYm90dG9tOjBwdDsiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZh bWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7 Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7 dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3Bh Y2U6cHJlLXdyYXA7Ij4mbHQ7L2Vycm9yJmd0Ozwvc3Bhbj48L3A+CjwvYj5UaGFua3MsIApE YWZuYQrCoDwvcHJlPgogIDwvYm9keT4KPC9odG1sPgo= --------------60D60522EFE450E2C842A076--

2 2

[JIRA] (OVIRT-1658) Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)
by Evgheni Dereveanchin (oVirt JIRA) 20 Sep '17

20 Sep '17

This is a multi-part message in MIME format... ------------=_1505900748-14133-1694 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit [ https://ovirt-jira.atlassian.net/browse/OVIRT-1658?page=com.atlassian.jira.… ] Evgheni Dereveanchin reassigned OVIRT-1658: ------------------------------------------- Assignee: Evgheni Dereveanchin (was: infra) > Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale) > ---------------------------------------------------------------------- > > Key: OVIRT-1658 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1658 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Barak Korren > Assignee: Evgheni Dereveanchin > > Seems there is an issue with locale files in recent CentOS versions, > and we're hitting it in slaves that have been updated recently. > The symptoms are that Jenkins disconnects from the slave and then > refuses to reconnect to it. The agent log in Jenkins shows: > [09/19/17 15:42:56] [SSH] Connection closed. > [09/19/17 15:58:48] [SSH] Opening SSH connection to > vm0002.workers-phx.ovirt.org:22. > [09/19/17 15:58:48] [SSH] WARNING: SSH Host Keys are not being > verified. Man-in-the-middle attacks may be possible against this > connection. > [09/19/17 15:58:48] [SSH] Authentication successful. > SSH connection reports a garbage before a command execution. > Check your .bashrc, .profile, and so on to make sure it is quiet. > The received junk text is as follows: > /etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot > change locale (en_US.utf8): No such file or directory > /etc/profile.d/lang.sh: line 20: warning: setlocale: LC_COLLATE: > cannot change locale (en_US.utf8): No such file or directory > /etc/profile.d/lang.sh: line 23: warning: setlocale: LC_MESSAGES: > cannot change locale (en_US.utf8): No such file or directory > /etc/profile.d/lang.sh: line 26: warning: setlocale: LC_NUMERIC: > cannot change locale (en_US.utf8): No such file or directory > /etc/profile.d/lang.sh: line 29: warning: setlocale: LC_TIME: cannot > change locale (en_US.utf8): No such file or directory > null > [09/19/17 15:58:48] Launch failed - cleaning up connection > [09/19/17 15:58:48] [SSH] Connection closed. > The same locale error messages can also be reproduced on the slave by > running an interactive login from the console or 'su -'. When running > 'locale -a' you can also see the en_US.UTF-8 locale is somehow > missing. > Looking around for this I found the following: > https://github.com/CentOS/sig-cloud-instance-images/issues/71 > I tried downgrading glibc back to the version we had before, but that > did not seem to resolve the issue. Eventually I managed to resolve it > by running 'localedef -i en_US -f UTF-8 en_US.UTF-8' on the slave. > I've seen this happen on 'vm0002.workers-phx.ovirt.org' which is > attached to the staging Jenkins, but I've no reason to believe this > won't start impacting production slaves. > We need to research this further and find out if we need to do > something to prevent this issue from surfacing on production slaves. > -- > Barak Korren > RHV DevOps team , RHCE, RHCi > Red Hat EMEA > redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted -- This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060) ------------=_1505900748-14133-1694 Content-Type: text/html; charset="UTF-8" Content-Disposition: inline Content-Transfer-Encoding: 7bit <html><body> <pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1658?page=com.atlassian.jira.… ]</pre> <h3>Evgheni Dereveanchin reassigned OVIRT-1658:</h3> <pre>Assignee: Evgheni Dereveanchin (was: infra)</pre> <blockquote><h3>Potential issue coming to CentOS7 slaves (glibc, missing UTF-8 locale)</h3> <pre> Key: OVIRT-1658 URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1658 Project: oVirt - virtualization made easy Issue Type: By-EMAIL Reporter: Barak Korren Assignee: Evgheni Dereveanchin</pre> <p>Seems there is an issue with locale files in recent CentOS versions, and we're hitting it in slaves that have been updated recently. The symptoms are that Jenkins disconnects from the slave and then refuses to reconnect to it. The agent log in Jenkins shows: [09/19/17 15:42:56] [SSH] Connection closed. [09/19/17 15:58:48] [SSH] Opening SSH connection to vm0002.workers-phx.ovirt.org:22. [09/19/17 15:58:48] [SSH] WARNING: SSH Host Keys are not being verified. Man-in-the-middle attacks may be possible against this connection. [09/19/17 15:58:48] [SSH] Authentication successful. SSH connection reports a garbage before a command execution. Check your .bashrc, .profile, and so on to make sure it is quiet. The received junk text is as follows: /etc/profile.d/lang.sh: line 19: warning: setlocale: LC_CTYPE: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 20: warning: setlocale: LC_COLLATE: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 23: warning: setlocale: LC_MESSAGES: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 26: warning: setlocale: LC_NUMERIC: cannot change locale (en_US.utf8): No such file or directory /etc/profile.d/lang.sh: line 29: warning: setlocale: LC_TIME: cannot change locale (en_US.utf8): No such file or directory null [09/19/17 15:58:48] Launch failed – cleaning up connection [09/19/17 15:58:48] [SSH] Connection closed. The same locale error messages can also be reproduced on the slave by running an interactive login from the console or ‘su -’. When running ‘locale -a’ you can also see the en_US.UTF-8 locale is somehow missing. Looking around for this I found the following: <a href="https://github.com/CentOS/sig-cloud-instance-images/issues/71">https://github.com/CentOS/sig-cloud-instance-images/issues/71</a> I tried downgrading glibc back to the version we had before, but that did not seem to resolve the issue. Eventually I managed to resolve it by running ‘localedef -i en_US -f UTF-8 en_US.UTF-8’ on the slave. I've seen this happen on ‘vm0002.workers-phx.ovirt.org’ which is attached to the staging Jenkins, but I've no reason to believe this won't start impacting production slaves. We need to research this further and find out if we need to do something to prevent this issue from surfacing on production slaves. — Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted</p></blockquote> <p>— This message was sent by Atlassian {0} (v1001.0.0-SNAPSHOT#100060)</p> <img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYLyz…" alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/> </body></html> ------------=_1505900748-14133-1694--

1 0