[ OST Failure Report ] [ oVirt master ] [ 23-11-2017 ] [ 001_initialize_engine.test_initialize_engine ]
This is a multi-part message in MIME format. --------------F9CBFA00796816161239008C Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Hi, We have a failing on test 001_initialize_engine.test_initialize_engine. This is failing with error Failed to start service 'ovirt-imageio-proxy ** *Link and headline ofto suspected patches: * *** build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/ Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055* * * *Link to all logs:* * * http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/ ** *http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/... * (Relevant) error snippet from the log: <error> *from lago log: Failed to start service 'ovirt-imageio-proxy messages logs: Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. ** *</error>* ** --------------F9CBFA00796816161239008C Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body text="#000000" bgcolor="#FFFFFF"> <p>Hi, <br> </p> <p>We have a failing on test 001_initialize_engine.test_initialize_engine. <br> </p> <p>This is failing with error Failed to start service 'ovirt-imageio-proxy</p> <p><br> </p> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e975-9a0d-3b01-8de053cdfef7"> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link and headline ofto suspected patches: </span></p> </b><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e975-9a0d-3b01-8de053cdfef7"> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">build: Make resulting RPMs architecture-specific - <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/84534/">https://gerrit.ovirt.org/#/c/84534/</a> </span></p> <br> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to Job:</span></p> <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055</a></b></p> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e975-9a0d-3b01-8de053cdfef7"><br> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link to all logs:</span></p> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> </span></p> </b></p> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/</a></span></p> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e975-9a0d-3b01-8de053cdfef7"> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/</a></span></p> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"> </span></p> <br> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant) error snippet from the log: </span></p> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p> <br> </b>from lago log: <br> </p> <pre style="box-sizing: border-box; white-space: pre-wrap; word-wrap: break-word; margin: 0px; color: rgb(51, 51, 51); font-size: 14px; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: 2; text-align: start; text-indent: 0px; text-transform: none; widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration-style: initial; text-decoration-color: initial;">Failed to start service 'ovirt-imageio-proxy messages logs: Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.</pre> <p><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-e975-9a0d-3b01-8de053cdfef7"> <p dir="ltr" style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"></error></span></p> </b><br class="Apple-interchange-newline"> </p> </body> </html> --------------F9CBFA00796816161239008C--
Daniel/Nir? On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
*Link and headline ofto suspected patches: *
* build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/ <https://gerrit.ovirt.org/#/c/84534/> Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055>*
*Link to all logs:*
http://jenkins.ovirt.org/job/ovirt-master_change-queue- tester/4055/artifact/
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/... <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/>*
* (Relevant) error snippet from the log: <error> *from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
*</error>*
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
This is a multi-part message in MIME format. --------------D11D052340527DBBD99C6D52 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit there were two different patches reported as failing cq today with the ovirt-imageio-proxy service failing to start. Here is the latest failure: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4130/artifact On 11/23/2017 03:39 PM, Allon Mureinik wrote:
Daniel/Nir?
On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com <mailto:dron@redhat.com>> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
**
*Link and headline ofto suspected patches: *
***
build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/ <https://gerrit.ovirt.org/#/c/84534/>
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055>*
* *
*Link to all logs:*
*
*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/>
**
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/... <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/>*
*
(Relevant) error snippet from the log:
<error>
*from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
**
*</error>*
**
_______________________________________________ Infra mailing list Infra@ovirt.org <mailto:Infra@ovirt.org> http://lists.ovirt.org/mailman/listinfo/infra <http://lists.ovirt.org/mailman/listinfo/infra>
--------------D11D052340527DBBD99C6D52 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiIGNv bnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD11dGYtOCI+CiAgPC9oZWFkPgogIDxib2R5IHRl eHQ9IiMwMDAwMDAiIGJnY29sb3I9IiNGRkZGRkYiPgogICAgPGRpdiBjbGFzcz0ibW96LWNp dGUtcHJlZml4Ij50aGVyZSB3ZXJlIHR3byBkaWZmZXJlbnQgcGF0Y2hlcwogICAgICByZXBv cnRlZCBhcyBmYWlsaW5nIGNxIHRvZGF5IHdpdGggdGhlIG92aXJ0LWltYWdlaW8tcHJveHkg c2VydmljZQogICAgICBmYWlsaW5nIHRvIHN0YXJ0LiA8YnI+CiAgICAgIDxicj4KICAgICAg SGVyZSBpcyB0aGUgbGF0ZXN0IGZhaWx1cmU6CjxhIGNsYXNzPSJtb3otdHh0LWxpbmstZnJl ZXRleHQiIGhyZWY9Imh0dHA6Ly9qZW5raW5zLm92aXJ0Lm9yZy9qb2Ivb3ZpcnQtbWFzdGVy X2NoYW5nZS1xdWV1ZS10ZXN0ZXIvNDEzMC9hcnRpZmFjdCI+aHR0cDovL2plbmtpbnMub3Zp cnQub3JnL2pvYi9vdmlydC1tYXN0ZXJfY2hhbmdlLXF1ZXVlLXRlc3Rlci80MTMwL2FydGlm YWN0PC9hPjxicj4KICAgICAgPGJyPgogICAgICA8YnI+CiAgICAgIDxicj4KICAgICAgT24g MTEvMjMvMjAxNyAwMzozOSBQTSwgQWxsb24gTXVyZWluaWsgd3JvdGU6PGJyPgogICAgPC9k aXY+CiAgICA8YmxvY2txdW90ZSB0eXBlPSJjaXRlIgpjaXRlPSJtaWQ6Q0FEZ1hmM3ljSFdf MHNDYmFOU0o2eVZlVEJTdVYtZFhoRTRXLTdlV083LXF1VFk0eS1nQG1haWwuZ21haWwuY29t Ij4KICAgICAgPGRpdiBkaXI9Imx0ciI+CiAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxfZGVm YXVsdCIKICAgICAgICAgIHN0eWxlPSJmb250LWZhbWlseTptb25vc3BhY2UsbW9ub3NwYWNl Ij5EYW5pZWwvTmlyPzwvZGl2PgogICAgICA8L2Rpdj4KICAgICAgPGRpdiBjbGFzcz0iZ21h aWxfZXh0cmEiPjxicj4KICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+T24gVGh1 LCBOb3YgMjMsIDIwMTcgYXQgNToyOSBQTSwgRGFmbmEKICAgICAgICAgIFJvbiA8c3BhbiBk aXI9Imx0ciI+Jmx0OzxhIGhyZWY9Im1haWx0bzpkcm9uQHJlZGhhdC5jb20iCiAgICAgICAg ICAgICAgdGFyZ2V0PSJfYmxhbmsiIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+ZHJvbkByZWRo YXQuY29tPC9hPiZndDs8L3NwYW4+CiAgICAgICAgICB3cm90ZTo8YnI+CiAgICAgICAgICA8 YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAKICAg ICAgICAgICAgLjhleDtib3JkZXItbGVmdDoxcHggI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6 MWV4Ij4KICAgICAgICAgICAgPGRpdiB0ZXh0PSIjMDAwMDAwIiBiZ2NvbG9yPSIjRkZGRkZG Ij4KICAgICAgICAgICAgICA8cD5IaSwgPGJyPgogICAgICAgICAgICAgIDwvcD4KICAgICAg ICAgICAgICA8cD5XZSBoYXZlIGEgZmFpbGluZyBvbiB0ZXN0IDAwMV9pbml0aWFsaXplX2Vu Z2luZS50ZXN0Xzx3YnI+aW5pdGlhbGl6ZV9lbmdpbmUuCiAgICAgICAgICAgICAgICA8YnI+ CiAgICAgICAgICAgICAgPC9wPgogICAgICAgICAgICAgIDxwPlRoaXMgaXMgZmFpbGluZyB3 aXRoIGVycm9yIEZhaWxlZCB0byBzdGFydCBzZXJ2aWNlCiAgICAgICAgICAgICAgICAnb3Zp cnQtaW1hZ2Vpby1wcm94eTwvcD4KICAgICAgICAgICAgICA8cD48YnI+CiAgICAgICAgICAg ICAgPC9wPgogICAgICAgICAgICAgIDxwPjxiIHN0eWxlPSJmb250LXdlaWdodDpub3JtYWwi CmlkPSJtXy0xNjUzMzkwMDAwMDU2NDEyMjMxZG9jcy1pbnRlcm5hbC1ndWlkLTU4NTliN2Ex LWU5NzUtOWEwZC0zYjAxLThkZTA1M2NkZmVmNyI+CiAgICAgICAgICAgICAgICA8L2I+PC9w PgogICAgICAgICAgICAgIDxwIGRpcj0ibHRyIgogICAgICAgICAgICAgICAgc3R5bGU9Imxp bmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTowcHQiPjxiCiAg ICAgICAgICAgICAgICAgIHN0eWxlPSJmb250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0xNjUz MzkwMDAwMDU2NDEyMjMxZG9jcy1pbnRlcm5hbC1ndWlkLTU4NTliN2ExLWU5NzUtOWEwZC0z YjAxLThkZTA1M2NkZmVmNyI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFt aWx5OkFyaWFsO2NvbG9yOiMwMDAwMDA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtm b250LXdlaWdodDo0MDA7Zm9udC1zdHlsZTpub3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0 ZXh0LWRlY29yYXRpb246bm9uZTt2ZXJ0aWNhbC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFj ZTpwcmUtd3JhcCI+TGluayBhbmQgaGVhZGxpbmUgb2Z0byBzdXNwZWN0ZWQgcGF0Y2hlczog Cjwvc3Bhbj48L2I+PC9wPgogICAgICAgICAgICAgIDxiIHN0eWxlPSJmb250LXdlaWdodDpu b3JtYWwiCmlkPSJtXy0xNjUzMzkwMDAwMDU2NDEyMjMxZG9jcy1pbnRlcm5hbC1ndWlkLTU4 NTliN2ExLWU5NzUtOWEwZC0zYjAxLThkZTA1M2NkZmVmNyI+CiAgICAgICAgICAgICAgPC9i PjxiIHN0eWxlPSJmb250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0xNjUzMzkwMDAwMDU2NDEy MjMxZG9jcy1pbnRlcm5hbC1ndWlkLTU4NTliN2ExLWU5NzUtOWEwZC0zYjAxLThkZTA1M2Nk ZmVmNyI+CiAgICAgICAgICAgICAgICA8cCBkaXI9Imx0ciIKICAgICAgICAgICAgICAgICAg c3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTow cHQiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xv cjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAw O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9u Om5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXAiPmJ1 aWxkOiBNYWtlIHJlc3VsdGluZyBSUE1zIGFyY2hpdGVjdHVyZS1zcGVjaWZpYyAtIDxhIGNs YXNzPSJtXy0xNjUzMzkwMDAwMDU2NDEyMjMxbW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVm PSJodHRwczovL2dlcnJpdC5vdmlydC5vcmcvIy9jLzg0NTM0LyIgdGFyZ2V0PSJfYmxhbmsi IG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+aHR0cHM6Ly9nZXJyaXQub3ZpcnQub3JnLyMvYy88 d2JyPjg0NTM0LzwvYT4KPC9zcGFuPjwvcD4KICAgICAgICAgICAgICAgIDxicj4KICAgICAg ICAgICAgICAgIDxwIGRpcj0ibHRyIgogICAgICAgICAgICAgICAgICBzdHlsZT0ibGluZS1o ZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBwdCI+PHNwYW4gc3R5 bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFtaWx5OkFyaWFsO2NvbG9yOiMwMDAwMDA7YmFj a2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtmb250LXdlaWdodDo0MDA7Zm9udC1zdHlsZTpu b3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0ZXh0LWRlY29yYXRpb246bm9uZTt2ZXJ0aWNh bC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFjZTpwcmUtd3JhcCI+TGluayB0byBKb2I6PC9z cGFuPjwvcD4KICAgICAgICAgICAgICAgIDxhIGNsYXNzPSJtXy0xNjUzMzkwMDAwMDU2NDEy MjMxbW96LXR4dC1saW5rLWZyZWV0ZXh0IgpocmVmPSJodHRwOi8vamVua2lucy5vdmlydC5v cmcvam9iL292aXJ0LW1hc3Rlcl9jaGFuZ2UtcXVldWUtdGVzdGVyLzQwNTUiCiAgICAgICAg ICAgICAgICAgIHRhcmdldD0iX2JsYW5rIiBtb3otZG8tbm90LXNlbmQ9InRydWUiPmh0dHA6 Ly9qZW5raW5zLm92aXJ0Lm9yZy9qb2IvPHdicj5vdmlydC1tYXN0ZXJfY2hhbmdlLXF1ZXVl LTx3YnI+dGVzdGVyLzQwNTU8L2E+PC9iPgogICAgICAgICAgICAgIDxwPjxiIHN0eWxlPSJm b250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0xNjUzMzkwMDAwMDU2NDEyMjMxZG9jcy1pbnRl cm5hbC1ndWlkLTU4NTliN2ExLWU5NzUtOWEwZC0zYjAxLThkZTA1M2NkZmVmNyI+PGJyPgog ICAgICAgICAgICAgICAgPC9iPjwvcD4KICAgICAgICAgICAgICA8cCBkaXI9Imx0ciIKICAg ICAgICAgICAgICAgIHN0eWxlPSJsaW5lLWhlaWdodDoxLjM4O21hcmdpbi10b3A6MHB0O21h cmdpbi1ib3R0b206MHB0Ij48YgogICAgICAgICAgICAgICAgICBzdHlsZT0iZm9udC13ZWln aHQ6bm9ybWFsIgppZD0ibV8tMTY1MzM5MDAwMDA1NjQxMjIzMWRvY3MtaW50ZXJuYWwtZ3Vp ZC01ODU5YjdhMS1lOTc1LTlhMGQtM2IwMS04ZGUwNTNjZGZlZjciPjxzcGFuIHN0eWxlPSJm b250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tncm91 bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFs O2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwtYWxp Z246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXAiPkxpbmsgdG8gYWxsIGxvZ3M6PC9z cGFuPjwvYj48L3A+CiAgICAgICAgICAgICAgPGIgc3R5bGU9ImZvbnQtd2VpZ2h0Om5vcm1h bCIKaWQ9Im1fLTE2NTMzOTAwMDAwNTY0MTIyMzFkb2NzLWludGVybmFsLWd1aWQtNTg1OWI3 YTEtZTk3NS05YTBkLTNiMDEtOGRlMDUzY2RmZWY3Ij4KICAgICAgICAgICAgICAgIDxwIGRp cj0ibHRyIgogICAgICAgICAgICAgICAgICBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJn aW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBwdCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox MXB0O2ZvbnQtZmFtaWx5OkFyaWFsO2NvbG9yOiMwMDAwMDA7YmFja2dyb3VuZC1jb2xvcjp0 cmFuc3BhcmVudDtmb250LXdlaWdodDo0MDA7Zm9udC1zdHlsZTpub3JtYWw7Zm9udC12YXJp YW50Om5vcm1hbDt0ZXh0LWRlY29yYXRpb246bm9uZTt2ZXJ0aWNhbC1hbGlnbjpiYXNlbGlu ZTt3aGl0ZS1zcGFjZTpwcmUtd3JhcCI+Cjwvc3Bhbj48L3A+CiAgICAgICAgICAgICAgPC9i PgogICAgICAgICAgICAgIDxwIGRpcj0ibHRyIgogICAgICAgICAgICAgICAgc3R5bGU9Imxp bmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTowcHQiPjxzcGFu IHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAw O2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5 bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVy dGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXAiPjxhIGNsYXNzPSJt Xy0xNjUzMzkwMDAwMDU2NDEyMjMxbW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVmPSJodHRw Oi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LW1hc3Rlcl9jaGFuZ2UtcXVldWUtdGVz dGVyLzQwNTUvYXJ0aWZhY3QvIiB0YXJnZXQ9Il9ibGFuayIgbW96LWRvLW5vdC1zZW5kPSJ0 cnVlIj5odHRwOi8vamVua2lucy5vdmlydC5vcmcvam9iLzx3YnI+b3ZpcnQtbWFzdGVyX2No YW5nZS1xdWV1ZS08d2JyPnRlc3Rlci80MDU1L2FydGlmYWN0LzwvYT48L3NwYW4+PC9wPgog ICAgICAgICAgICAgIDxwPjxiIHN0eWxlPSJmb250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0x NjUzMzkwMDAwMDU2NDEyMjMxZG9jcy1pbnRlcm5hbC1ndWlkLTU4NTliN2ExLWU5NzUtOWEw ZC0zYjAxLThkZTA1M2NkZmVmNyI+CiAgICAgICAgICAgICAgICA8L2I+PC9wPgogICAgICAg ICAgICAgIDxwIGRpcj0ibHRyIgogICAgICAgICAgICAgICAgc3R5bGU9ImxpbmUtaGVpZ2h0 OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTowcHQiPjxiCiAgICAgICAgICAg ICAgICAgIHN0eWxlPSJmb250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0xNjUzMzkwMDAwMDU2 NDEyMjMxZG9jcy1pbnRlcm5hbC1ndWlkLTU4NTliN2ExLWU5NzUtOWEwZC0zYjAxLThkZTA1 M2NkZmVmNyI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFtaWx5OkFyaWFs O2NvbG9yOiMwMDAwMDA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtmb250LXdlaWdo dDo0MDA7Zm9udC1zdHlsZTpub3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0ZXh0LWRlY29y YXRpb246bm9uZTt2ZXJ0aWNhbC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFjZTpwcmUtd3Jh cCI+PGEgY2xhc3M9Im1fLTE2NTMzOTAwMDAwNTY0MTIyMzFtb3otdHh0LWxpbmstZnJlZXRl eHQiIGhyZWY9Imh0dHA6Ly9qZW5raW5zLm92aXJ0Lm9yZy9qb2Ivb3ZpcnQtbWFzdGVyX2No YW5nZS1xdWV1ZS10ZXN0ZXIvNDA1NS9hcnRpZmFjdC9leHBvcnRlZC1hcnRpZmFjdHMvdXBn cmFkZS1mcm9tLXJlbGVhc2Utc3VpdC1tYXN0ZXItZWw3L3Rlc3RfbG9ncy91cGdyYWRlLWZy b20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXIvcG9zdC0wMDFfaW5pdGlhbGl6ZV9lbmdpbmUucHkv bGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lL192YXJfbG9n L21lc3NhZ2VzLyp2aWV3Ki8iIHRhcmdldD0iX2JsYW5rIiBtb3otZG8tbm90LXNlbmQ9InRy dWUiPmh0dHA6Ly9qZW5raW5zLm92aXJ0Lm9yZy9qb2IvPHdicj5vdmlydC1tYXN0ZXJfY2hh bmdlLXF1ZXVlLTx3YnI+dGVzdGVyLzQwNTUvYXJ0aWZhY3QvZXhwb3J0ZWQtPHdicj5hcnRp ZmFjdHMvdXBncmFkZS1mcm9tLTx3YnI+cmVsZWFzZS1zdWl0LW1hc3Rlci1lbDcvdGVzdF88 d2JyPmxvZ3MvdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXIvcG9zdC0w MDFfPHdicj5pbml0aWFsaXplX2VuZ2luZS5weS9sYWdvLTx3YnI+dXBncmFkZS1mcm9tLXJl bGVhc2Utc3VpdGUtPHdicj5tYXN0ZXItZW5naW5lL192YXJfbG9nLzx3YnI+bWVzc2FnZXMv KnZpZXcqLzwvYT48L3NwYW4+PC9iPjwvcD4KICAgICAgICAgICAgICA8YiBzdHlsZT0iZm9u dC13ZWlnaHQ6bm9ybWFsIgppZD0ibV8tMTY1MzM5MDAwMDA1NjQxMjIzMWRvY3MtaW50ZXJu YWwtZ3VpZC01ODU5YjdhMS1lOTc1LTlhMGQtM2IwMS04ZGUwNTNjZGZlZjciPgogICAgICAg ICAgICAgICAgPHAgZGlyPSJsdHIiCiAgICAgICAgICAgICAgICAgIHN0eWxlPSJsaW5lLWhl aWdodDoxLjM4O21hcmdpbi10b3A6MHB0O21hcmdpbi1ib3R0b206MHB0Ij48c3BhbiBzdHls ZT0iZm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJpYWw7Y29sb3I6IzAwMDAwMDtiYWNr Z3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2VpZ2h0OjQwMDtmb250LXN0eWxlOm5v cm1hbDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVjb3JhdGlvbjpub25lO3ZlcnRpY2Fs LWFsaWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13cmFwIj4KPC9zcGFuPjwvcD4KICAg ICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAgIDxwIGRpcj0ibHRyIgogICAgICAg ICAgICAgICAgICBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJn aW4tYm90dG9tOjBwdCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFtaWx5 OkFyaWFsO2NvbG9yOiMwMDAwMDA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtmb250 LXdlaWdodDo0MDA7Zm9udC1zdHlsZTpub3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0ZXh0 LWRlY29yYXRpb246bm9uZTt2ZXJ0aWNhbC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFjZTpw cmUtd3JhcCI+KFJlbGV2YW50KSBlcnJvciBzbmlwcGV0IGZyb20gdGhlIGxvZzogPC9zcGFu PjwvcD4KICAgICAgICAgICAgICAgIDxwIGRpcj0ibHRyIgogICAgICAgICAgICAgICAgICBz dHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBw dCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFtaWx5OkFyaWFsO2NvbG9y OiMwMDAwMDA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtmb250LXdlaWdodDo0MDA7 Zm9udC1zdHlsZTpub3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0ZXh0LWRlY29yYXRpb246 bm9uZTt2ZXJ0aWNhbC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFjZTpwcmUtd3JhcCI+Jmx0 O2Vycm9yJmd0Ozwvc3Bhbj48L3A+CiAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAg ICAgPC9iPmZyb20gbGFnbyBsb2c6IDxicj4KICAgICAgICAgICAgICA8cHJlIHN0eWxlPSJi b3gtc2l6aW5nOmJvcmRlci1ib3g7d2hpdGUtc3BhY2U6cHJlLXdyYXA7d29yZC13cmFwOmJy ZWFrLXdvcmQ7bWFyZ2luOjBweDtjb2xvcjpyZ2IoNTEsNTEsNTEpO2ZvbnQtc2l6ZToxNHB4 O2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudC1saWdhdHVyZXM6bm9ybWFsO2ZvbnQt dmFyaWFudC1jYXBzOm5vcm1hbDtmb250LXdlaWdodDpub3JtYWw7bGV0dGVyLXNwYWNpbmc6 bm9ybWFsO3RleHQtYWxpZ246c3RhcnQ7dGV4dC1pbmRlbnQ6MHB4O3RleHQtdHJhbnNmb3Jt Om5vbmU7d29yZC1zcGFjaW5nOjBweDt0ZXh0LWRlY29yYXRpb24tc3R5bGU6aW5pdGlhbDt0 ZXh0LWRlY29yYXRpb24tY29sb3I6aW5pdGlhbCI+RmFpbGVkIHRvIHN0YXJ0IHNlcnZpY2Ug J292aXJ0LWltYWdlaW8tcHJveHkKCm1lc3NhZ2VzIGxvZ3M6IAoKTm92IDIzIDA3OjMwOjQ3 IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIHN5 c3RlbWQ6IFN0YXJ0aW5nIFNlc3Npb24gOCBvZiB1c2VyIHJvb3QuCk5vdiAyMyAwNzozMDo0 NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBv dmlydC1pbWFnZWlvLXByb3h5OiBUcmFjZWJhY2sgKG1vc3QgcmVjZW50IGNhbGwgbGFzdCk6 Ck5vdiAyMyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUt bWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2Jpbi9vdmly dC1pbWFnZWlvLXByb3h5Ijx3YnI+LCBsaW5lIDg1LCBpbiAmbHQ7bW9kdWxlJmd0OwpOb3Yg MjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rl ci1lbmdpbmUgb3ZpcnQtaW1hZ2Vpby1wcm94eTogc3RhdHVzID0gaW1hZ2VfcHJveHkubWFp bihhcmdzLCBjb25maWcpCk5vdiAyMyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxl YXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxl ICIvdXNyL2xpYi9weXRob24yLjcvc2l0ZS08d2JyPnBhY2thZ2VzL292aXJ0X2ltYWdlaW9f cHJveHkvPHdicj5pbWFnZV9wcm94eS5weSIsIGxpbmUgMjEsIGluIG1haW4KTm92IDIzIDA3 OjMwOjQ3IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5n aW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IGltYWdlX3NlcnZlci5zdGFydChjb25maWcpCk5v diAyMyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFz dGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xpYi9weXRob24y Ljcvc2l0ZS08d2JyPnBhY2thZ2VzL292aXJ0X2ltYWdlaW9fcHJveHkvPHdicj5zZXJ2ZXIu cHkiLCBsaW5lIDQ1LCBpbiBzdGFydApOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZy b20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQtaW1hZ2Vpby1wcm94 eTogV1NHSVJlcXVlc3RIYW5kbGVyKQpOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZy b20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQtaW1hZ2Vpby1wcm94 eTogRmlsZSAiL3Vzci9saWI2NC9weXRob24yLjcvPHdicj5Tb2NrZXRTZXJ2ZXIucHkiLCBs aW5lIDQxOSwgaW4gX19pbml0X18KTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBncmFkZS1mcm9t LXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6 IHNlbGYuc2VydmVyX2JpbmQoKQpOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20t cmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQtaW1hZ2Vpby1wcm94eTog RmlsZSAiL3Vzci9saWI2NC9weXRob24yLjcvd3NnaXJlZi88d2JyPnNpbXBsZV9zZXJ2ZXIu cHkiLCBsaW5lIDQ4LCBpbiBzZXJ2ZXJfYmluZApOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdy YWRlLWZyb20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQtaW1hZ2Vp by1wcm94eTogSFRUUFNlcnZlci5zZXJ2ZXJfYmluZChzZWxmKQpOb3YgMjMgMDc6MzA6NDcg bGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgb3Zp cnQtaW1hZ2Vpby1wcm94eTogRmlsZSAiL3Vzci9saWI2NC9weXRob24yLjcvPHdicj5CYXNl SFRUUFNlcnZlci5weSIsIGxpbmUgMTA4LCBpbiBzZXJ2ZXJfYmluZApOb3YgMjMgMDc6MzA6 NDcgbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUg b3ZpcnQtaW1hZ2Vpby1wcm94eTogU29ja2V0U2VydmVyLlRDUFNlcnZlci5zZXJ2ZXJfPHdi cj5iaW5kKHNlbGYpCk5vdiAyMyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNl LTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIv dXNyL2xpYjY0L3B5dGhvbjIuNy88d2JyPlNvY2tldFNlcnZlci5weSIsIGxpbmUgNDMwLCBp biBzZXJ2ZXJfYmluZApOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20tcmVsZWFz ZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQtaW1hZ2Vpby1wcm94eTogc2VsZi5z b2NrZXQuYmluZChzZWxmLnNlcnZlcl88d2JyPmFkZHJlc3MpCk5vdiAyMyAwNzozMDo0NyBs YWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmly dC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xpYjY0L3B5dGhvbjIuNy9zb2NrZXQuPHdi cj5weSIsIGxpbmUgMjI0LCBpbiBtZXRoCk5vdiAyMyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUt ZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXBy b3h5OiByZXR1cm4gZ2V0YXR0cihzZWxmLl9zb2NrLG5hbWUpKCo8d2JyPmFyZ3MpCk5vdiAy MyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVy LWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBzb2NrZXQuZXJyb3I6IFtFcnJubyA5OF0g QWRkcmVzcyBhbHJlYWR5IGluIHVzZQpOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZy b20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDogb3ZpcnQtaW1h Z2Vpby1wcm94eS5zZXJ2aWNlOiBtYWluIHByb2Nlc3MgZXhpdGVkLCBjb2RlPWV4aXRlZCwg c3RhdHVzPTEvRkFJTFVSRQpOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20tcmVs ZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDogRmFpbGVkIHRvIHN0YXJ0 IG9WaXJ0IEltYWdlSU8gUHJveHkuCk5vdiAyMyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJv bS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBzeXN0ZW1kOiBVbml0IG92aXJ0 LWltYWdlaW8tcHJveHkuc2VydmljZSBlbnRlcmVkIGZhaWxlZCBzdGF0ZS4KTm92IDIzIDA3 OjMwOjQ3IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5n aW5lIHN5c3RlbWQ6IG92aXJ0LWltYWdlaW8tcHJveHkuc2VydmljZSBmYWlsZWQuCk5vdiAy MyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVy LWVuZ2luZSBzeXN0ZW1kOiBvdmlydC1pbWFnZWlvLXByb3h5LnNlcnZpY2UgaG9sZG9mZiB0 aW1lIG92ZXIsIHNjaGVkdWxpbmcgcmVzdGFydC4KTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBn cmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIHN5c3RlbWQ6IFN0 YXJ0aW5nIG9WaXJ0IEltYWdlSU8gUHJveHkuLi4KTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBn cmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdl aW8tcHJveHk6IFRyYWNlYmFjayAobW9zdCByZWNlbnQgY2FsbCBsYXN0KToKTm92IDIzIDA3 OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5n aW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IEZpbGUgIi91c3IvYmluL292aXJ0LWltYWdlaW8t cHJveHkiPHdicj4sIGxpbmUgODUsIGluICZsdDttb2R1bGUmZ3Q7Ck5vdiAyMyAwNzozMDo0 OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBv dmlydC1pbWFnZWlvLXByb3h5OiBzdGF0dXMgPSBpbWFnZV9wcm94eS5tYWluKGFyZ3MsIGNv bmZpZykKTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5z dWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IEZpbGUgIi91c3IvbGli L3B5dGhvbjIuNy9zaXRlLTx3YnI+cGFja2FnZXMvb3ZpcnRfaW1hZ2Vpb19wcm94eS88d2Jy PmltYWdlX3Byb3h5LnB5IiwgbGluZSAyMSwgaW4gbWFpbgpOb3YgMjMgMDc6MzA6NDggbGFn by11cGdyYWRlLWZyb20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQt aW1hZ2Vpby1wcm94eTogaW1hZ2Vfc2VydmVyLnN0YXJ0KGNvbmZpZykKTm92IDIzIDA3OjMw OjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5l IG92aXJ0LWltYWdlaW8tcHJveHk6IEZpbGUgIi91c3IvbGliL3B5dGhvbjIuNy9zaXRlLTx3 YnI+cGFja2FnZXMvb3ZpcnRfaW1hZ2Vpb19wcm94eS88d2JyPnNlcnZlci5weSIsIGxpbmUg NDUsIGluIHN0YXJ0Ck5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNl LTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBXU0dJUmVx dWVzdEhhbmRsZXIpCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNl LTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIv dXNyL2xpYjY0L3B5dGhvbjIuNy88d2JyPlNvY2tldFNlcnZlci5weSIsIGxpbmUgNDE5LCBp biBfX2luaXRfXwpOb3YgMjMgMDc6MzA6NDggbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS08 d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQtaW1hZ2Vpby1wcm94eTogc2VsZi5zZXJ2 ZXJfYmluZCgpCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3 YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNy L2xpYjY0L3B5dGhvbjIuNy93c2dpcmVmLzx3YnI+c2ltcGxlX3NlcnZlci5weSIsIGxpbmUg NDgsIGluIHNlcnZlcl9iaW5kCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1y ZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBI VFRQU2VydmVyLnNlcnZlcl9iaW5kKHNlbGYpCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3Jh ZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlv LXByb3h5OiBGaWxlICIvdXNyL2xpYjY0L3B5dGhvbjIuNy88d2JyPkJhc2VIVFRQU2VydmVy LnB5IiwgbGluZSAxMDgsIGluIHNlcnZlcl9iaW5kCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVw Z3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFn ZWlvLXByb3h5OiBTb2NrZXRTZXJ2ZXIuVENQU2VydmVyLnNlcnZlcl88d2JyPmJpbmQoc2Vs ZikKTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0 ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IEZpbGUgIi91c3IvbGliNjQv cHl0aG9uMi43Lzx3YnI+U29ja2V0U2VydmVyLnB5IiwgbGluZSA0MzAsIGluIHNlcnZlcl9i aW5kCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3Vp dGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBzZWxmLnNvY2tldC5iaW5k KHNlbGYuc2VydmVyXzx3YnI+YWRkcmVzcykKTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFk ZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8t cHJveHk6IEZpbGUgIi91c3IvbGliNjQvcHl0aG9uMi43L3NvY2tldC48d2JyPnB5IiwgbGlu ZSAyMjQsIGluIG1ldGgKTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVh c2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IHJldHVy biBnZXRhdHRyKHNlbGYuX3NvY2ssbmFtZSkoKjx3YnI+YXJncykKTm92IDIzIDA3OjMwOjQ4 IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIG92 aXJ0LWltYWdlaW8tcHJveHk6IHNvY2tldC5lcnJvcjogW0Vycm5vIDk4XSBBZGRyZXNzIGFs cmVhZHkgaW4gdXNlCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNl LTx3YnI+c3VpdGUtbWFzdGVyLWVuZ2luZSBzeXN0ZW1kOiBvdmlydC1pbWFnZWlvLXByb3h5 LnNlcnZpY2U6IG1haW4gcHJvY2VzcyBleGl0ZWQsIGNvZGU9ZXhpdGVkLCBzdGF0dXM9MS9G QUlMVVJFCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+ c3VpdGUtbWFzdGVyLWVuZ2luZSBzeXN0ZW1kOiBGYWlsZWQgdG8gc3RhcnQgb1ZpcnQgSW1h Z2VJTyBQcm94eS4KTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Ut PHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIHN5c3RlbWQ6IFVuaXQgb3ZpcnQtaW1hZ2Vpby1w cm94eS5zZXJ2aWNlIGVudGVyZWQgZmFpbGVkIHN0YXRlLgpOb3YgMjMgMDc6MzA6NDggbGFn by11cGdyYWRlLWZyb20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVt ZDogb3ZpcnQtaW1hZ2Vpby1wcm94eS5zZXJ2aWNlIGZhaWxlZC4KTm92IDIzIDA3OjMwOjQ4 IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5zdWl0ZS1tYXN0ZXItZW5naW5lIHN5 c3RlbWQ6IG92aXJ0LWltYWdlaW8tcHJveHkuc2VydmljZSBob2xkb2ZmIHRpbWUgb3Zlciwg c2NoZWR1bGluZyByZXN0YXJ0LgpOb3YgMjMgMDc6MzA6NDggbGFnby11cGdyYWRlLWZyb20t cmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDogc3RhcnQgcmVxdWVz dCByZXBlYXRlZCB0b28gcXVpY2tseSBmb3Igb3ZpcnQtaW1hZ2Vpby1wcm94eS5zZXJ2aWNl Ck5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLTx3YnI+c3VpdGUt bWFzdGVyLWVuZ2luZSBzeXN0ZW1kOiBGYWlsZWQgdG8gc3RhcnQgb1ZpcnQgSW1hZ2VJTyBQ cm94eS4KTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2UtPHdicj5z dWl0ZS1tYXN0ZXItZW5naW5lIHN5c3RlbWQ6IFVuaXQgb3ZpcnQtaW1hZ2Vpby1wcm94eS5z ZXJ2aWNlIGVudGVyZWQgZmFpbGVkIHN0YXRlLgpOb3YgMjMgMDc6MzA6NDggbGFnby11cGdy YWRlLWZyb20tcmVsZWFzZS08d2JyPnN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDogb3Zp cnQtaW1hZ2Vpby1wcm94eS5zZXJ2aWNlIGZhaWxlZC48L3ByZT4KICAgICAgICAgICAgICA8 cD48YiBzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsIgppZD0ibV8tMTY1MzM5MDAwMDA1NjQx MjIzMWRvY3MtaW50ZXJuYWwtZ3VpZC01ODU5YjdhMS1lOTc1LTlhMGQtM2IwMS04ZGUwNTNj ZGZlZjciPgogICAgICAgICAgICAgICAgPC9iPjwvcD4KICAgICAgICAgICAgICA8cCBkaXI9 Imx0ciIKICAgICAgICAgICAgICAgIHN0eWxlPSJsaW5lLWhlaWdodDoxLjM4O21hcmdpbi10 b3A6MHB0O21hcmdpbi1ib3R0b206MHB0Ij48YgogICAgICAgICAgICAgICAgICBzdHlsZT0i Zm9udC13ZWlnaHQ6bm9ybWFsIgppZD0ibV8tMTY1MzM5MDAwMDA1NjQxMjIzMWRvY3MtaW50 ZXJuYWwtZ3VpZC01ODU5YjdhMS1lOTc1LTlhMGQtM2IwMS04ZGUwNTNjZGZlZjciPjxzcGFu IHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAw O2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5 bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVy dGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXAiPiZsdDsvZXJyb3Im Z3Q7PC9zcGFuPjwvYj48L3A+CiAgICAgICAgICAgICAgPGIgc3R5bGU9ImZvbnQtd2VpZ2h0 Om5vcm1hbCIKaWQ9Im1fLTE2NTMzOTAwMDAwNTY0MTIyMzFkb2NzLWludGVybmFsLWd1aWQt NTg1OWI3YTEtZTk3NS05YTBkLTNiMDEtOGRlMDUzY2RmZWY3Ij4KICAgICAgICAgICAgICA8 L2I+PGJyCiAgICAgICAgICAgICAgICBjbGFzcz0ibV8tMTY1MzM5MDAwMDA1NjQxMjIzMUFw cGxlLWludGVyY2hhbmdlLW5ld2xpbmUiPgogICAgICAgICAgICA8L2Rpdj4KICAgICAgICAg ICAgPGJyPgogICAgICAgICAgICBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX188d2Jy Pl9fX19fX19fX19fX19fX19fPGJyPgogICAgICAgICAgICBJbmZyYSBtYWlsaW5nIGxpc3Q8 YnI+CiAgICAgICAgICAgIDxhIGhyZWY9Im1haWx0bzpJbmZyYUBvdmlydC5vcmciIG1vei1k by1ub3Qtc2VuZD0idHJ1ZSI+SW5mcmFAb3ZpcnQub3JnPC9hPjxicj4KICAgICAgICAgICAg PGEgaHJlZj0iaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2luZnJh IgogICAgICAgICAgICAgIHJlbD0ibm9yZWZlcnJlciIgdGFyZ2V0PSJfYmxhbmsiIG1vei1k by1ub3Qtc2VuZD0idHJ1ZSI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy88d2JyPm1haWxtYW4v bGlzdGluZm8vaW5mcmE8L2E+PGJyPgogICAgICAgICAgICA8YnI+CiAgICAgICAgICA8L2Js b2NrcXVvdGU+CiAgICAgICAgPC9kaXY+CiAgICAgICAgPGJyPgogICAgICA8L2Rpdj4KICAg IDwvYmxvY2txdW90ZT4KICAgIDxwPjxicj4KICAgIDwvcD4KICA8L2JvZHk+CjwvaHRtbD4K --------------D11D052340527DBBD99C6D52--
We still see this issue on the upgrade suite from latest release to master [1]. I don't see any evidence in "/var/log/messages" [2] that "ovirt-imageio-proxy" was started twice. [1] http://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/ovirt-mas... [2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-... On Fri, Nov 24, 2017 at 8:16 PM, Dafna Ron <dron@redhat.com> wrote:
there were two different patches reported as failing cq today with the ovirt-imageio-proxy service failing to start.
Here is the latest failure: http://jenkins.ovirt.org/job/ ovirt-master_change-queue-tester/4130/artifact
On 11/23/2017 03:39 PM, Allon Mureinik wrote:
Daniel/Nir?
On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
*Link and headline ofto suspected patches: *
* build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/ <https://gerrit.ovirt.org/#/c/84534/> Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055>*
*Link to all logs:*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste r/4055/artifact/
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/... <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/>*
* (Relevant) error snippet from the log: <error> *from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
*</error>*
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- *GAL bEN HAIM* RHV DEVOPS
On Sun, Nov 26, 2017 at 4:53 PM, Gal Ben Haim <gbenhaim@redhat.com> wrote:
We still see this issue on the upgrade suite from latest release to master [1]. I don't see any evidence in "/var/log/messages" [2] that "ovirt-imageio-proxy" was started twice.
Since it's not a registered port and a high port, could it be used by something else (what are the odds though ? Is it consistent? Y.
[1] http://jenkins.ovirt.org/blue/rest/organizations/ jenkins/pipelines/ovirt-master_change-queue-tester/ runs/4153/nodes/123/steps/241/log/?start=0
[2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ ovirt-master_change-queue-tester/4153/artifact/exported- artifacts/upgrade-from-release-suit-master-el7/test_ logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago- upgrade-from-release-suite-master-engine/_var_log/messages/*view*/
On Fri, Nov 24, 2017 at 8:16 PM, Dafna Ron <dron@redhat.com> wrote:
there were two different patches reported as failing cq today with the ovirt-imageio-proxy service failing to start.
Here is the latest failure: http://jenkins.ovirt.org/job/o virt-master_change-queue-tester/4130/artifact
On 11/23/2017 03:39 PM, Allon Mureinik wrote:
Daniel/Nir?
On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
*Link and headline ofto suspected patches: *
* build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/ <https://gerrit.ovirt.org/#/c/84534/> Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055>*
*Link to all logs:*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste r/4055/artifact/
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/... <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/>*
* (Relevant) error snippet from the log: <error> *from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
*</error>*
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- *GAL bEN HAIM* RHV DEVOPS
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
The failure is not consistent. On Sun, Nov 26, 2017 at 5:33 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Sun, Nov 26, 2017 at 4:53 PM, Gal Ben Haim <gbenhaim@redhat.com> wrote:
We still see this issue on the upgrade suite from latest release to master [1]. I don't see any evidence in "/var/log/messages" [2] that "ovirt-imageio-proxy" was started twice.
Since it's not a registered port and a high port, could it be used by something else (what are the odds though ? Is it consistent? Y.
[1] http://jenkins.ovirt.org/blue/rest/organizations/jenkins /pipelines/ovirt-master_change-queue-tester/runs/4153/ nodes/123/steps/241/log/?start=0
[2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovir t-master_change-queue-tester/4153/artifact/exported-artifac ts/upgrade-from-release-suit-master-el7/test_logs/upgrade- from-release-suite-master/post-001_initialize_engine.py/ lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/
On Fri, Nov 24, 2017 at 8:16 PM, Dafna Ron <dron@redhat.com> wrote:
there were two different patches reported as failing cq today with the ovirt-imageio-proxy service failing to start.
Here is the latest failure: http://jenkins.ovirt.org/job/o virt-master_change-queue-tester/4130/artifact
On 11/23/2017 03:39 PM, Allon Mureinik wrote:
Daniel/Nir?
On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
*Link and headline ofto suspected patches: *
* build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/ <https://gerrit.ovirt.org/#/c/84534/> Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055>*
*Link to all logs:*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste r/4055/artifact/
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/... <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/>*
* (Relevant) error snippet from the log: <error> *from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
*</error>*
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- *GAL bEN HAIM* RHV DEVOPS
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- *GAL bEN HAIM* RHV DEVOPS
I think we need to check and report which process is listening on a port when starting a server on that port fail. Didi, do you think we can integrate this in the deploy code, or this should be implemented in each server? Maybe when deployment fails, the deploy code can report all the listening sockets and the processes bound to these sockets? Nir On Sun, Nov 26, 2017 at 7:11 PM Gal Ben Haim <gbenhaim@redhat.com> wrote:
The failure is not consistent.
On Sun, Nov 26, 2017 at 5:33 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Sun, Nov 26, 2017 at 4:53 PM, Gal Ben Haim <gbenhaim@redhat.com> wrote:
We still see this issue on the upgrade suite from latest release to master [1]. I don't see any evidence in "/var/log/messages" [2] that "ovirt-imageio-proxy" was started twice.
Since it's not a registered port and a high port, could it be used by something else (what are the odds though ? Is it consistent? Y.
[1] http://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/ovirt-mas...
[2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-...
On Fri, Nov 24, 2017 at 8:16 PM, Dafna Ron <dron@redhat.com> wrote:
there were two different patches reported as failing cq today with the ovirt-imageio-proxy service failing to start.
Here is the latest failure: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4130/artifact
On 11/23/2017 03:39 PM, Allon Mureinik wrote:
Daniel/Nir?
On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
*Link and headline ofto suspected patches: *
* build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/ <https://gerrit.ovirt.org/#/c/84534/> Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055>*
*Link to all logs:*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/... <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/>*
* (Relevant) error snippet from the log: <error> *from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
*</error>*
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- *GAL bEN HAIM* RHV DEVOPS
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- *GAL bEN HAIM* RHV DEVOPS _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Sun, Nov 26, 2017 at 7:24 PM, Nir Soffer <nsoffer@redhat.com> wrote:
I think we need to check and report which process is listening on a port when starting a server on that port fail.
How do you know that a server was "started on that port", and that if failed specifically because it failed to bind? There is no standardized (Unix) way to mark that a service wants to listen on a specific port, or that it failed because a specific port was bound by some other process. There are various classical *inetd* daemons, and modern systemd.socket, that listen *instead* of some service. Then they can manage the port resources and perhaps do something intelligent about them.
Didi, do you think we can integrate this in the deploy code, or this should be implemented in each server?
It should be quite easy to patch otopi's services.state to run something if start fails, e.g. 'ss -anp' or whatever you want. It should even be not-too-hard to do this in a self-contained plugin, so can be part of otopi-debug-plugins. If we decide that something needs to be implemented by each server, perhaps "something" should be to be controlled by a systemd.socket unit. Didn't try, though, to see what this actually buys us.
Maybe when deployment fails, the deploy code can report all the listening sockets and the processes bound to these sockets?
Pushed now: https://gerrit.ovirt.org/84699 core: Name TRANSACTION_INIT https://gerrit.ovirt.org/84700 plugins: debug: Add debug_failure https://gerrit.ovirt.org/84701 automation: Test failure Will merge soon, if all goes well. Feel free to open BZ for other things discussed above, if relevant.
Nir
On Sun, Nov 26, 2017 at 7:11 PM Gal Ben Haim <gbenhaim@redhat.com> wrote:
The failure is not consistent.
On Sun, Nov 26, 2017 at 5:33 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Sun, Nov 26, 2017 at 4:53 PM, Gal Ben Haim <gbenhaim@redhat.com> wrote:
We still see this issue on the upgrade suite from latest release to master [1]. I don't see any evidence in "/var/log/messages" [2] that "ovirt-imageio-proxy" was started twice.
Since it's not a registered port and a high port, could it be used by something else (what are the odds though ? Is it consistent? Y.
[1] http://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/ovirt-mas...
[2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-...
On Fri, Nov 24, 2017 at 8:16 PM, Dafna Ron <dron@redhat.com> wrote:
there were two different patches reported as failing cq today with the ovirt-imageio-proxy service failing to start.
Here is the latest failure: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4130/artifact
On 11/23/2017 03:39 PM, Allon Mureinik wrote:
Daniel/Nir?
On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
Link and headline ofto suspected patches:
build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/...
(Relevant) error snippet from the log:
<error>
from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
</error>
_______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- GAL bEN HAIM RHV DEVOPS
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- GAL bEN HAIM RHV DEVOPS _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Didi
On Mon, Nov 27, 2017 at 10:38 AM, Yedidyah Bar David <didi@redhat.com> wrote:
On Sun, Nov 26, 2017 at 7:24 PM, Nir Soffer <nsoffer@redhat.com> wrote:
I think we need to check and report which process is listening on a port when starting a server on that port fail.
How do you know that a server was "started on that port", and that if failed specifically because it failed to bind?
There is no standardized (Unix) way to mark that a service wants to listen on a specific port, or that it failed because a specific port was bound by some other process.
There are various classical *inetd* daemons, and modern systemd.socket, that listen *instead* of some service. Then they can manage the port resources and perhaps do something intelligent about them.
Didi, do you think we can integrate this in the deploy code, or this should be implemented in each server?
It should be quite easy to patch otopi's services.state to run something if start fails, e.g. 'ss -anp' or whatever you want.
It should even be not-too-hard to do this in a self-contained plugin, so can be part of otopi-debug-plugins.
If we decide that something needs to be implemented by each server, perhaps "something" should be to be controlled by a systemd.socket unit. Didn't try, though, to see what this actually buys us.
Maybe when deployment fails, the deploy code can report all the listening sockets and the processes bound to these sockets?
Pushed now:
https://gerrit.ovirt.org/84699 core: Name TRANSACTION_INIT https://gerrit.ovirt.org/84700 plugins: debug: Add debug_failure https://gerrit.ovirt.org/84701 automation: Test failure
Will merge soon, if all goes well.
Merged them. Pushed to OST: https://gerrit.ovirt.org/84710 Dafna - thanks for opening the bug on ovirt-imageio, but I am not sure anyone can do much about it without more info, such as might be provided by above patches. When I suggested below to open BZ I meant on otopi or host-deploy to provide more debugging info, not for imageio - obviously no harm in opening it, and it's good to have it even if only for reference.
Feel free to open BZ for other things discussed above, if relevant.
Nir
On Sun, Nov 26, 2017 at 7:11 PM Gal Ben Haim <gbenhaim@redhat.com>
The failure is not consistent.
On Sun, Nov 26, 2017 at 5:33 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Sun, Nov 26, 2017 at 4:53 PM, Gal Ben Haim <gbenhaim@redhat.com> wrote:
We still see this issue on the upgrade suite from latest release to master [1]. I don't see any evidence in "/var/log/messages" [2] that "ovirt-imageio-proxy" was started twice.
Since it's not a registered port and a high port, could it be used by something else (what are the odds though ? Is it consistent? Y.
[1] http://jenkins.ovirt.org/blue/rest/organizations/jenkins/
[2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/
ovirt-master_change-queue-tester/4153/artifact/exported- artifacts/upgrade-from-release-suit-master-el7/test_ logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago- upgrade-from-release-suite-master-engine/_var_log/messages/*view*/
On Fri, Nov 24, 2017 at 8:16 PM, Dafna Ron <dron@redhat.com> wrote:
there were two different patches reported as failing cq today with
ovirt-imageio-proxy service failing to start.
Here is the latest failure: http://jenkins.ovirt.org/job/ovirt-master_change-queue- tester/4130/artifact
On 11/23/2017 03:39 PM, Allon Mureinik wrote:
Daniel/Nir?
On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com> wrote: > > Hi, > > We have a failing on test > 001_initialize_engine.test_initialize_engine. > > This is failing with error Failed to start service > 'ovirt-imageio-proxy > > > Link and headline ofto suspected patches: > > build: Make resulting RPMs architecture-specific - > https://gerrit.ovirt.org/#/c/84534/ > > > Link to Job: > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 > > > Link to all logs: > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue- tester/4055/artifact/ > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue- tester/4055/artifact/exported-artifacts/upgrade-from- release-suit-master-el7/test_logs/upgrade-from-release- suite-master/post-001_initialize_engine.py/lago- upgrade-from-release-suite-master-engine/_var_log/messages/*view*/ > > > (Relevant) error snippet from the log: > > <error> > > > from lago log: > > Failed to start service 'ovirt-imageio-proxy > > messages logs: > > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: > Starting Session 8 of user root. > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: Traceback (most recent call last): > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in > <module> > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: status = image_proxy.main(args, config) > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File > "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py",
> 21, in main > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: image_server.start(config) > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File > "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py",
> in start > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: WSGIRequestHandler) > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py",
> in __init__ > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: self.server_bind() > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/ simple_server.py", > line 48, in server_bind > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: HTTPServer.server_bind(self) > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py",
> 108, in server_bind > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py",
> in server_bind > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: self.socket.bind(self.server_address) > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in > meth > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: return getattr(self._sock,name)(*args) > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: > ovirt-imageio-proxy.service: main process exited, code=exited, > status=1/FAILURE > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: > Failed to start oVirt ImageIO Proxy. > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: > Unit ovirt-imageio-proxy.service entered failed state. > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: > ovirt-imageio-proxy.service failed. > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: > ovirt-imageio-proxy.service holdoff time over, scheduling restart. > Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: > Starting oVirt ImageIO Proxy... > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: Traceback (most recent call last): > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in > <module> > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: status = image_proxy.main(args, config) > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File > "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py",
> 21, in main > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: image_server.start(config) > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File > "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py",
> in start > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: WSGIRequestHandler) > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py",
> in __init__ > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: self.server_bind() > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/ simple_server.py", > line 48, in server_bind > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: HTTPServer.server_bind(self) > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py",
> 108, in server_bind > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py",
wrote: pipelines/ovirt-master_change-queue-tester/runs/4153/nodes/ 123/steps/241/log/?start=0 the line line 45, line 419, line line 430, line line 45, line 419, line line 430,
> in server_bind > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: self.socket.bind(self.server_address) > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in > meth > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: return getattr(self._sock,name)(*args) > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine > ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > ovirt-imageio-proxy.service: main process exited, code=exited, > status=1/FAILURE > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > Failed to start oVirt ImageIO Proxy. > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > Unit ovirt-imageio-proxy.service entered failed state. > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > ovirt-imageio-proxy.service failed. > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > ovirt-imageio-proxy.service holdoff time over, scheduling restart. > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > start request repeated too quickly for ovirt-imageio-proxy.service > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > Failed to start oVirt ImageIO Proxy. > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > Unit ovirt-imageio-proxy.service entered failed state. > Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: > ovirt-imageio-proxy.service failed. > > </error> > > > > _______________________________________________ > Infra mailing list > Infra@ovirt.org > http://lists.ovirt.org/mailman/listinfo/infra >
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- GAL bEN HAIM RHV DEVOPS
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- GAL bEN HAIM RHV DEVOPS _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Didi
-- Didi
Didi - we just saw a report of a similar failure on engine-4.1's OST. COuld you please backport these patches there too? On Mon, Nov 27, 2017 at 2:57 PM, Yedidyah Bar David <didi@redhat.com> wrote:
On Mon, Nov 27, 2017 at 10:38 AM, Yedidyah Bar David <didi@redhat.com> wrote:
I think we need to check and report which process is listening on a
On Sun, Nov 26, 2017 at 7:24 PM, Nir Soffer <nsoffer@redhat.com> wrote: port
when starting a server on that port fail.
How do you know that a server was "started on that port", and that if failed specifically because it failed to bind?
There is no standardized (Unix) way to mark that a service wants to listen on a specific port, or that it failed because a specific port was bound by some other process.
There are various classical *inetd* daemons, and modern systemd.socket, that listen *instead* of some service. Then they can manage the port resources and perhaps do something intelligent about them.
Didi, do you think we can integrate this in the deploy code, or this should be implemented in each server?
It should be quite easy to patch otopi's services.state to run something if start fails, e.g. 'ss -anp' or whatever you want.
It should even be not-too-hard to do this in a self-contained plugin, so can be part of otopi-debug-plugins.
If we decide that something needs to be implemented by each server, perhaps "something" should be to be controlled by a systemd.socket unit. Didn't try, though, to see what this actually buys us.
Maybe when deployment fails, the deploy code can report all the listening sockets and the processes bound to these sockets?
Pushed now:
https://gerrit.ovirt.org/84699 core: Name TRANSACTION_INIT https://gerrit.ovirt.org/84700 plugins: debug: Add debug_failure https://gerrit.ovirt.org/84701 automation: Test failure
Will merge soon, if all goes well.
Merged them.
Pushed to OST:
https://gerrit.ovirt.org/84710
Dafna - thanks for opening the bug on ovirt-imageio, but I am not sure anyone can do much about it without more info, such as might be provided by above patches. When I suggested below to open BZ I meant on otopi or host-deploy to provide more debugging info, not for imageio - obviously no harm in opening it, and it's good to have it even if only for reference.
Feel free to open BZ for other things discussed above, if relevant.
Nir
On Sun, Nov 26, 2017 at 7:11 PM Gal Ben Haim <gbenhaim@redhat.com>
The failure is not consistent.
On Sun, Nov 26, 2017 at 5:33 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Sun, Nov 26, 2017 at 4:53 PM, Gal Ben Haim <gbenhaim@redhat.com> wrote:
We still see this issue on the upgrade suite from latest release to master [1]. I don't see any evidence in "/var/log/messages" [2] that "ovirt-imageio-proxy" was started twice.
Since it's not a registered port and a high port, could it be used by something else (what are the odds though ? Is it consistent? Y.
[1] http://jenkins.ovirt.org/blue/rest/organizations/jenkins/pip
elines/ovirt-master_change-queue-tester/runs/4153/nodes/123/ steps/241/log/?start=0
[2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovir
t-master_change-queue-tester/4153/artifact/exported-artifac ts/upgrade-from-release-suit-master-el7/test_logs/upgrade- from-release-suite-master/post-001_initialize_engine.py/ lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/
On Fri, Nov 24, 2017 at 8:16 PM, Dafna Ron <dron@redhat.com> wrote: > > there were two different patches reported as failing cq today with
> ovirt-imageio-proxy service failing to start. > > Here is the latest failure: > http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste r/4130/artifact > > > > > On 11/23/2017 03:39 PM, Allon Mureinik wrote: > > Daniel/Nir? > > On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com> wrote: >> >> Hi, >> >> We have a failing on test >> 001_initialize_engine.test_initialize_engine. >> >> This is failing with error Failed to start service >> 'ovirt-imageio-proxy >> >> >> Link and headline ofto suspected patches: >> >> build: Make resulting RPMs architecture-specific - >> https://gerrit.ovirt.org/#/c/84534/ >> >> >> Link to Job: >> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 >> >> >> Link to all logs: >> >> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste r/4055/artifact/ >> >> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-teste r/4055/artifact/exported-artifacts/upgrade-from-release- suit-master-el7/test_logs/upgrade-from-release-suite- master/post-001_initialize_engine.py/lago-upgrade-from- release-suite-master-engine/_var_log/messages/*view*/ >> >> >> (Relevant) error snippet from the log: >> >> <error> >> >> >> from lago log: >> >> Failed to start service 'ovirt-imageio-proxy >> >> messages logs: >> >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: >> Starting Session 8 of user root. >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: Traceback (most recent call last): >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in >> <module> >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: status = image_proxy.main(args, config) >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File >> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py",
>> 21, in main >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: image_server.start(config) >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File >> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py",
>> in start >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: WSGIRequestHandler) >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py",
>> in __init__ >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: self.server_bind() >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/ simple_server.py", >> line 48, in server_bind >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: HTTPServer.server_bind(self) >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py",
>> 108, in server_bind >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py",
>> in server_bind >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: self.socket.bind(self.server_address) >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in >> meth >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: return getattr(self._sock,name)(*args) >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: >> ovirt-imageio-proxy.service: main process exited, code=exited, >> status=1/FAILURE >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: >> Failed to start oVirt ImageIO Proxy. >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: >> Unit ovirt-imageio-proxy.service entered failed state. >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: >> ovirt-imageio-proxy.service failed. >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: >> ovirt-imageio-proxy.service holdoff time over, scheduling restart. >> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: >> Starting oVirt ImageIO Proxy... >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: Traceback (most recent call last): >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in >> <module> >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: status = image_proxy.main(args, config) >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File >> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py",
>> 21, in main >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: image_server.start(config) >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File >> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py",
>> in start >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: WSGIRequestHandler) >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py",
>> in __init__ >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: self.server_bind() >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/ simple_server.py", >> line 48, in server_bind >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: HTTPServer.server_bind(self) >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py",
>> 108, in server_bind >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py",
wrote: the line line 45, line 419, line line 430, line line 45, line 419, line line 430,
>> in server_bind >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: self.socket.bind(self.server_address) >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in >> meth >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: return getattr(self._sock,name)(*args) >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine >> ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> ovirt-imageio-proxy.service: main process exited, code=exited, >> status=1/FAILURE >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> Failed to start oVirt ImageIO Proxy. >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> Unit ovirt-imageio-proxy.service entered failed state. >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> ovirt-imageio-proxy.service failed. >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> ovirt-imageio-proxy.service holdoff time over, scheduling restart. >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> start request repeated too quickly for ovirt-imageio-proxy.service >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> Failed to start oVirt ImageIO Proxy. >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> Unit ovirt-imageio-proxy.service entered failed state. >> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: >> ovirt-imageio-proxy.service failed. >> >> </error> >> >> >> >> _______________________________________________ >> Infra mailing list >> Infra@ovirt.org >> http://lists.ovirt.org/mailman/listinfo/infra >> > > > > _______________________________________________ > Devel mailing list > Devel@ovirt.org > http://lists.ovirt.org/mailman/listinfo/devel
-- GAL bEN HAIM RHV DEVOPS
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- GAL bEN HAIM RHV DEVOPS _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Didi
-- Didi
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
This is a multi-part message in MIME format. --------------510D2C3E2A3E3638E7FA4B2B Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit I opened a bug for this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1517764 On 11/26/2017 05:24 PM, Nir Soffer wrote:
I think we need to check and report which process is listening on a port when starting a server on that port fail.
Didi, do you think we can integrate this in the deploy code, or this should be implemented in each server?
Maybe when deployment fails, the deploy code can report all the listening sockets and the processes bound to these sockets?
Nir
On Sun, Nov 26, 2017 at 7:11 PM Gal Ben Haim <gbenhaim@redhat.com <mailto:gbenhaim@redhat.com>> wrote:
The failure is not consistent.
On Sun, Nov 26, 2017 at 5:33 PM, Yaniv Kaul <ykaul@redhat.com <mailto:ykaul@redhat.com>> wrote:
On Sun, Nov 26, 2017 at 4:53 PM, Gal Ben Haim <gbenhaim@redhat.com <mailto:gbenhaim@redhat.com>> wrote:
We still see this issue on the upgrade suite from latest release to master [1]. I don't see any evidence in "/var/log/messages" [2] that "ovirt-imageio-proxy" was started twice.
Since it's not a registered port and a high port, could it be used by something else (what are the odds though ? Is it consistent? Y.
[1] http://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/ovirt-mas...
[2] http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-...
On Fri, Nov 24, 2017 at 8:16 PM, Dafna Ron <dron@redhat.com <mailto:dron@redhat.com>> wrote:
there were two different patches reported as failing cq today with the ovirt-imageio-proxy service failing to start.
Here is the latest failure: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4130/artifact
On 11/23/2017 03:39 PM, Allon Mureinik wrote:
Daniel/Nir?
On Thu, Nov 23, 2017 at 5:29 PM, Dafna Ron <dron@redhat.com <mailto:dron@redhat.com>> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
**
*Link and headline ofto suspected patches: *
***
build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055*
* *
*Link to all logs:*
*
*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/
**
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/...
*
(Relevant) error snippet from the log:
<error>
*from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
**
*</error>*
**
_______________________________________________ Infra mailing list Infra@ovirt.org <mailto:Infra@ovirt.org> http://lists.ovirt.org/mailman/listinfo/infra
_______________________________________________ Devel mailing list Devel@ovirt.org <mailto:Devel@ovirt.org> http://lists.ovirt.org/mailman/listinfo/devel
-- *GAL bEN HAIM* RHV DEVOPS
_______________________________________________ Devel mailing list Devel@ovirt.org <mailto:Devel@ovirt.org> http://lists.ovirt.org/mailman/listinfo/devel
-- *GAL bEN HAIM* RHV DEVOPS _______________________________________________ Devel mailing list Devel@ovirt.org <mailto:Devel@ovirt.org> http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--------------510D2C3E2A3E3638E7FA4B2B Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: base64 PGh0bWw+CiAgPGhlYWQ+CiAgICA8bWV0YSBodHRwLWVxdWl2PSJDb250ZW50LVR5cGUiIGNv bnRlbnQ9InRleHQvaHRtbDsgY2hhcnNldD11dGYtOCI+CiAgPC9oZWFkPgogIDxib2R5IHRl eHQ9IiMwMDAwMDAiIGJnY29sb3I9IiNGRkZGRkYiPgogICAgPGRpdiBjbGFzcz0ibW96LWNp dGUtcHJlZml4Ij5JIG9wZW5lZCBhIGJ1ZyBmb3IgdGhpcyBpc3N1ZTogPGJyPgogICAgICA8 YnI+CiAgICAgIDxhIGNsYXNzPSJtb3otdHh0LWxpbmstZnJlZXRleHQiIGhyZWY9Imh0dHBz Oi8vYnVnemlsbGEucmVkaGF0LmNvbS9zaG93X2J1Zy5jZ2k/aWQ9MTUxNzc2NCI+aHR0cHM6 Ly9idWd6aWxsYS5yZWRoYXQuY29tL3Nob3dfYnVnLmNnaT9pZD0xNTE3NzY0PC9hPjxicj4K ICAgICAgPGJyPgogICAgICA8YnI+CiAgICAgIE9uIDExLzI2LzIwMTcgMDU6MjQgUE0sIE5p ciBTb2ZmZXIgd3JvdGU6PGJyPgogICAgPC9kaXY+CiAgICA8YmxvY2txdW90ZSB0eXBlPSJj aXRlIgpjaXRlPSJtaWQ6Q0FNUmJ5eXNVQjYyT251aSszTXRTRGNKUkFtMlNvSE5kX2pwMz1Z eTloV2dabkNBVHVnQG1haWwuZ21haWwuY29tIj4KICAgICAgPGRpdiBkaXI9Imx0ciI+SSB0 aGluayB3ZSBuZWVkIHRvIGNoZWNrIGFuZCByZXBvcnQgd2hpY2jCoCBwcm9jZXNzCiAgICAg ICAgaXMgbGlzdGVuaW5nIG9uIGEgcG9ydAogICAgICAgIDxkaXY+d2hlbiBzdGFydGluZyBh IHNlcnZlciBvbiB0aGF0IHBvcnQgZmFpbC4KICAgICAgICAgIDxkaXY+CiAgICAgICAgICAg IDxkaXY+PGJyPgogICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgPGRpdj5EaWRpLCBk byB5b3UgdGhpbmsgd2UgY2FuIGludGVncmF0ZSB0aGlzIGluIHRoZSBkZXBsb3kKICAgICAg ICAgICAgICBjb2RlLCBvciB0aGlzPC9kaXY+CiAgICAgICAgICAgIDxkaXY+c2hvdWxkIGJl IGltcGxlbWVudGVkIGluIGVhY2ggc2VydmVyPzwvZGl2PgogICAgICAgICAgICA8ZGl2Pjxi cj4KICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgIDxkaXY+TWF5YmUgd2hlbiBkZXBs b3ltZW50IGZhaWxzLCB0aGUgZGVwbG95IGNvZGUgY2FuIHJlcG9ydAogICAgICAgICAgICAg IGFsbCB0aGU8L2Rpdj4KICAgICAgICAgICAgPGRpdj5saXN0ZW5pbmcgc29ja2V0cyBhbmQg dGhlIHByb2Nlc3NlcyBib3VuZCB0byB0aGVzZQogICAgICAgICAgICAgIHNvY2tldHM/PC9k aXY+CiAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICA8L2Rpdj4KICAgICAgICAg ICAgPGRpdj5OaXI8L2Rpdj4KICAgICAgICAgICAgPGJyPgogICAgICAgICAgICA8ZGl2IGNs YXNzPSJnbWFpbF9xdW90ZSI+CiAgICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+T24gU3Vu LCBOb3YgMjYsIDIwMTcgYXQgNzoxMSBQTSBHYWwgQmVuCiAgICAgICAgICAgICAgICBIYWlt ICZsdDs8YSBocmVmPSJtYWlsdG86Z2JlbmhhaW1AcmVkaGF0LmNvbSIKICAgICAgICAgICAg ICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIj5nYmVuaGFpbUByZWRoYXQuY29tPC9hPiZn dDsKICAgICAgICAgICAgICAgIHdyb3RlOjxicj4KICAgICAgICAgICAgICA8L2Rpdj4KICAg ICAgICAgICAgICA8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiIHN0eWxlPSJtYXJn aW46MCAwIDAKICAgICAgICAgICAgICAgIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29s aWQ7cGFkZGluZy1sZWZ0OjFleCI+CiAgICAgICAgICAgICAgICA8ZGl2IGRpcj0ibHRyIj5U aGUgZmFpbHVyZSBpcyBub3QgY29uc2lzdGVudC48L2Rpdj4KICAgICAgICAgICAgICAgIDxk aXYgY2xhc3M9ImdtYWlsX2V4dHJhIj48YnI+CiAgICAgICAgICAgICAgICAgIDxkaXYgY2xh c3M9ImdtYWlsX3F1b3RlIj5PbiBTdW4sIE5vdiAyNiwgMjAxNyBhdCA1OjMzCiAgICAgICAg ICAgICAgICAgICAgUE0sIFlhbml2IEthdWwgPHNwYW4gZGlyPSJsdHIiPiZsdDs8YQogICAg ICAgICAgICAgICAgICAgICAgICBocmVmPSJtYWlsdG86eWthdWxAcmVkaGF0LmNvbSIgdGFy Z2V0PSJfYmxhbmsiCiAgICAgICAgICAgICAgICAgICAgICAgIG1vei1kby1ub3Qtc2VuZD0i dHJ1ZSI+eWthdWxAcmVkaGF0LmNvbTwvYT4mZ3Q7PC9zcGFuPgogICAgICAgICAgICAgICAg ICAgIHdyb3RlOjxicj4KICAgICAgICAgICAgICAgICAgICA8YmxvY2txdW90ZSBjbGFzcz0i Z21haWxfcXVvdGUiIHN0eWxlPSJtYXJnaW46MCAwIDAKICAgICAgICAgICAgICAgICAgICAg IC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0OjFleCI+CiAg ICAgICAgICAgICAgICAgICAgICA8ZGl2IGRpcj0ibHRyIj48YnI+CiAgICAgICAgICAgICAg ICAgICAgICAgIDxkaXYgY2xhc3M9ImdtYWlsX2V4dHJhIj48YnI+CiAgICAgICAgICAgICAg ICAgICAgICAgICAgPGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjxzcGFuPk9uIFN1biwgTm92 IDI2LAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAyMDE3IGF0IDQ6NTMgUE0sIEdh bCBCZW4gSGFpbSA8c3BhbgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGRpcj0i bHRyIj4mbHQ7PGEKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGhyZWY9Im1h aWx0bzpnYmVuaGFpbUByZWRoYXQuY29tIgogICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgdGFyZ2V0PSJfYmxhbmsiIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+Z2JlbmhhaW1A cmVkaGF0LmNvbTwvYT4mZ3Q7PC9zcGFuPgogICAgICAgICAgICAgICAgICAgICAgICAgICAg ICB3cm90ZTo8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxibG9ja3F1b3Rl IGNsYXNzPSJnbWFpbF9xdW90ZSIKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBz dHlsZT0ibWFyZ2luOjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4CiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgI2NjYyBzb2xpZDtwYWRkaW5nLWxlZnQ6MWV4Ij4KICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICA8ZGl2IGRpcj0ibHRyIj5XZSBzdGlsbCBzZWUg dGhpcyBpc3N1ZQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgb24gdGhlIHVw Z3JhZGUgc3VpdGUgZnJvbSBsYXRlc3QKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHJlbGVhc2UgdG8gbWFzdGVyIFsxXS4KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIDxkaXY+SSBkb24ndCBzZWUgYW55IGV2aWRlbmNlIGluCiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICIvdmFyL2xvZy9tZXNzYWdlcyIgWzJdIHRoYXQKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIm92aXJ0LWltYWdlaW8tcHJveHki IHdhcyBzdGFydGVkCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHR3aWNl LjwvZGl2PgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvZGl2PgogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8 L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvc3Bhbj4KICAgICAgICAgICAg ICAgICAgICAgICAgICAgIDxkaXY+U2luY2UgaXQncyBub3QgYSByZWdpc3RlcmVkIHBvcnQg YW5kIGEKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaGlnaCBwb3J0LCBjb3VsZCBp dCBiZSB1c2VkIGJ5IHNvbWV0aGluZwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICBl bHNlICh3aGF0IGFyZSB0aGUgb2RkcyB0aG91Z2ggPzwvZGl2PgogICAgICAgICAgICAgICAg ICAgICAgICAgICAgPGRpdj5JcyBpdCBjb25zaXN0ZW50PzwvZGl2PgogICAgICAgICAgICAg ICAgICAgICAgICAgICAgPHNwYW4gY2xhc3M9Im1fLTM0NzE4MDcyOTA5OTE3NDMzNDZIT0Vu WmIiPjxmb250CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29sb3I9IiM4ODg4 ODgiPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxkaXY+WS48L2Rpdj4KICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9mb250Pjwvc3Bhbj4KICAgICAgICAgICAg ICAgICAgICAgICAgICAgIDxkaXY+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxk aXYgY2xhc3M9Im1fLTM0NzE4MDcyOTA5OTE3NDMzNDZoNSI+CiAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgPGRpdj7CoDwvZGl2PgogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIDxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9xdW90ZSIKICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIHN0eWxlPSJtYXJnaW46MCAwIDAKICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2MKICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHNvbGlkO3BhZGRpbmctbGVmdDoxZXgiPgogICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+CiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxkaXY+PGJyPgogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgPGRpdj5bMV3CoDxhCmhyZWY9Imh0dHA6Ly9qZW5raW5zLm92aXJ0Lm9yZy9i bHVlL3Jlc3Qvb3JnYW5pemF0aW9ucy9qZW5raW5zL3BpcGVsaW5lcy9vdmlydC1tYXN0ZXJf Y2hhbmdlLXF1ZXVlLXRlc3Rlci9ydW5zLzQxNTMvbm9kZXMvMTIzL3N0ZXBzLzI0MS9sb2cv P3N0YXJ0PTAiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB0YXJn ZXQ9Il9ibGFuayIKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1v ei1kby1ub3Qtc2VuZD0idHJ1ZSI+aHR0cDovL2plbmtpbnMub3ZpcnQub3JnL2JsdWUvcmVz dC9vcmdhbml6YXRpb25zL2plbmtpbnMvcGlwZWxpbmVzL292aXJ0LW1hc3Rlcl9jaGFuZ2Ut cXVldWUtdGVzdGVyL3J1bnMvNDE1My9ub2Rlcy8xMjMvc3RlcHMvMjQxL2xvZy8/c3RhcnQ9 MDwvYT48L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGRpdj48 YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvZGl2PgogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8ZGl2PlsyXSA8YQpocmVmPSJodHRwOi8v amVua2lucy5vdmlydC5vcmcvdmlldy9DaGFuZ2UlMjBxdWV1ZSUyMGpvYnMvam9iL292aXJ0 LW1hc3Rlcl9jaGFuZ2UtcXVldWUtdGVzdGVyLzQxNTMvYXJ0aWZhY3QvZXhwb3J0ZWQtYXJ0 aWZhY3RzL3VwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXQtbWFzdGVyLWVsNy90ZXN0X2xvZ3Mv dXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyL3Bvc3QtMDAxX2luaXRpYWxpemVf ZW5naW5lLnB5L2xhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2lu ZS9fdmFyX2xvZy9tZXNzYWdlcy8qdmlldyovIgogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgdGFyZ2V0PSJfYmxhbmsiCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBtb3otZG8tbm90LXNlbmQ9InRydWUiPmh0dHA6Ly9qZW5raW5z Lm92aXJ0Lm9yZy92aWV3L0NoYW5nZSUyMHF1ZXVlJTIwam9icy9qb2Ivb3ZpcnQtbWFzdGVy X2NoYW5nZS1xdWV1ZS10ZXN0ZXIvNDE1My9hcnRpZmFjdC9leHBvcnRlZC1hcnRpZmFjdHMv dXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdC1tYXN0ZXItZWw3L3Rlc3RfbG9ncy91cGdyYWRl LWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXIvcG9zdC0wMDFfaW5pdGlhbGl6ZV9lbmdpbmUu cHkvbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lL192YXJf bG9nL21lc3NhZ2VzLyp2aWV3Ki88L2E+wqDCoDwvZGl2PgogICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICA8ZGl2IGNsYXNzPSJnbWFpbF9leHRyYSI+PGJyPgogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICA8ZGl2IGNsYXNzPSJnbWFpbF9xdW90ZSI+CiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgPGRpdj4KICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIDxkaXYKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgY2xhc3M9Im1fLTM0NzE4MDcyOTA5OTE3NDMzNDZtXy03NDI5Njc2MDQxMDk4 MTEwMTU3aDUiPk9uCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IEZyaSwgTm92IDI0LCAyMDE3IGF0IDg6MTYgUE0sCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgIERhZm5hIFJvbiA8c3BhbiBkaXI9Imx0ciI+Jmx0OzxhCmhy ZWY9Im1haWx0bzpkcm9uQHJlZGhhdC5jb20iIHRhcmdldD0iX2JsYW5rIiBtb3otZG8tbm90 LXNlbmQ9InRydWUiPmRyb25AcmVkaGF0LmNvbTwvYT4mZ3Q7PC9zcGFuPgogICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB3cm90ZTo8YnI+CiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICA8YmxvY2txdW90ZSBjbGFzcz0iZ21haWxfcXVvdGUiCiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHlsZT0ibWFyZ2luOjAgMCAwCiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAuOGV4O2JvcmRlci1sZWZ0OjFw eCAjY2NjCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzb2xpZDtw YWRkaW5nLWxlZnQ6MWV4Ij4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIDxkaXY+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxk aXYKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjbGFzcz0i bV8tMzQ3MTgwNzI5MDk5MTc0MzM0Nm1fLTc0Mjk2NzYwNDEwOTgxMTAxNTdoNSI+CiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGRpdiB0ZXh0PSIjMDAw MDAwIgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgYmdj b2xvcj0iI0ZGRkZGRiI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICA8ZGl2CmNsYXNzPSJtXy0zNDcxODA3MjkwOTkxNzQzMzQ2bV8tNzQyOTY3NjA0 MTA5ODExMDE1N21fMjEyNzI3NTc1NTAyNjY3NDEwbV8yNzAwMzEyNjA1OTM2MjM2MDIzbW96 LWNpdGUtcHJlZml4Ij50aGVyZQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICB3ZXJlIHR3byBkaWZmZXJlbnQKICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgcGF0Y2hlcyByZXBvcnRlZCBhcwogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBmYWlsaW5nIGNxIHRv ZGF5IHdpdGgKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgdGhlIG92aXJ0LWltYWdlaW8tcHJveHkKICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgc2VydmljZSBmYWlsaW5nIHRvCiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0YXJ0LiA8YnI+CiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgSGVyZSBpcyB0aGUgbGF0 ZXN0CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZh aWx1cmU6CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IDxhCmNsYXNzPSJtXy0zNDcxODA3MjkwOTkxNzQzMzQ2bV8tNzQyOTY3NjA0MTA5ODExMDE1 N21fMjEyNzI3NTc1NTAyNjY3NDEwbV8yNzAwMzEyNjA1OTM2MjM2MDIzbW96LXR4dC1saW5r LWZyZWV0ZXh0IgpocmVmPSJodHRwOi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LW1h c3Rlcl9jaGFuZ2UtcXVldWUtdGVzdGVyLzQxMzAvYXJ0aWZhY3QiCiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdGFyZ2V0PSJfYmxhbmsiCiAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbW96LWRv LW5vdC1zZW5kPSJ0cnVlIj5odHRwOi8vamVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LW1h c3Rlcl9jaGFuZ2UtcXVldWUtdGVzdGVyLzQxMzAvYXJ0aWZhY3Q8L2E+CiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxkaXY+CiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGRpdgpjbGFzcz0ibV8t MzQ3MTgwNzI5MDk5MTc0MzM0Nm1fLTc0Mjk2NzYwNDEwOTgxMTAxNTdtXzIxMjcyNzU3NTUw MjY2NzQxMGg1Ij48YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBPbiAxMS8yMy8yMDE3IDAzOjM5CiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBQTSwgQWxsb24gTXVy ZWluaWsKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIHdyb3RlOjxicj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIDxkaXY+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIDxkaXYKY2xhc3M9Im1fLTM0NzE4MDcyOTA5OTE3NDMzNDZtXy03NDI5Njc2 MDQxMDk4MTEwMTU3bV8yMTI3Mjc1NzU1MDI2Njc0MTBoNSI+CiAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGJsb2NrcXVvdGUKICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHR5cGU9ImNpdGUi PgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg PGRpdiBkaXI9Imx0ciI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIDxkaXYKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBzdHlsZT0iZm9udC1mYW1pbHk6bW9ub3NwYWNlLG1v bm9zcGFjZSI+RGFuaWVsL05pcj88L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGRpdgogICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBjbGFzcz0iZ21haWxfZXh0cmEi Pjxicj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgPGRpdgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIGNsYXNzPSJnbWFpbF9xdW90ZSI+T24KICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBUaHUsIE5vdiAyMywKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAy MDE3IGF0IDU6MjkgUE0sCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgRGFmbmEgUm9uIDxzcGFuCiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkaXI9Imx0ciI+Jmx0Ozxh CmhyZWY9Im1haWx0bzpkcm9uQHJlZGhhdC5jb20iIHRhcmdldD0iX2JsYW5rIiBtb3otZG8t bm90LXNlbmQ9InRydWUiPmRyb25AcmVkaGF0LmNvbTwvYT4mZ3Q7PC9zcGFuPgogICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHdyb3Rl Ojxicj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICA8YmxvY2txdW90ZQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgY2xhc3M9ImdtYWlsX3F1b3RlIgpzdHlsZT0ibWFy Z2luOjAgMCAwIC44ZXg7Ym9yZGVyLWxlZnQ6MXB4ICNjY2Mgc29saWQ7cGFkZGluZy1sZWZ0 OjFleCI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICA8ZGl2CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICB0ZXh0PSIjMDAwMDAwIgpiZ2NvbG9yPSIjRkZGRkZGIj4K ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIDxwPkhpLCA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICA8L3A+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8cD5XZSBoYXZlIGEKICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGZhaWxpbmcg b24KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIHRlc3QKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIDAwMV9pbml0aWFsaXplX2VuZ2luZS50ZXN0X2luaXRpYWxpemVf ZW5naW5lLgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgPC9wPgogICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPHA+VGhpcyBpcwogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgZmFpbGluZyB3 aXRoCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICBlcnJvciBGYWlsZWQKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHRvIHN0YXJ0CiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzZXJ2aWNlCiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnb3Zp cnQtaW1hZ2Vpby1wcm94eTwvcD4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIDxwPjxicj4KICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvcD4KICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxwPjxiCiAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsIgppZD0ibV8tMzQ3MTgwNzI5MDk5MTc0MzM0 Nm1fLTc0Mjk2NzYwNDEwOTgxMTAxNTdtXzIxMjcyNzU3NTUwMjY2NzQxMG1fMjcwMDMxMjYw NTkzNjIzNjAyM21fLTE2NTMzOTAwMDAwNTY0MTIyMzFkb2NzLWludGVybmFsLWd1aWQtNTg1 OWI3YTEtZTk3NS05YTBkLTNiMDEtOGRlMDUzY2RmZWY3Ij4KICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvYj48L3A+CiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8 cCBkaXI9Imx0ciIKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHN0eWxlPSJsaW5lLWhlaWdodDoxLjM4O21hcmdpbi10b3A6MHB0 O21hcmdpbi1ib3R0b206MHB0Ij48YgpzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsIgppZD0i bV8tMzQ3MTgwNzI5MDk5MTc0MzM0Nm1fLTc0Mjk2NzYwNDEwOTgxMTAxNTdtXzIxMjcyNzU3 NTUwMjY2NzQxMG1fMjcwMDMxMjYwNTkzNjIzNjAyM21fLTE2NTMzOTAwMDAwNTY0MTIyMzFk b2NzLWludGVybmFsLWd1aWQtNTg1OWI3YTEtZTk3NS05YTBkLTNiMDEtOGRlMDUzY2RmZWY3 Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJpYWw7Y29sb3I6 IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2VpZ2h0OjQwMDtm b250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVjb3JhdGlvbjpu b25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13cmFwIj5MaW5r IGFuZCBoZWFkbGluZSBvZnRvIHN1c3BlY3RlZCBwYXRjaGVzOiAKPC9zcGFuPjwvYj48L3A+ CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICA8YgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgc3R5bGU9ImZvbnQtd2VpZ2h0Om5vcm1hbCIKaWQ9Im1fLTM0NzE4MDcy OTA5OTE3NDMzNDZtXy03NDI5Njc2MDQxMDk4MTEwMTU3bV8yMTI3Mjc1NzU1MDI2Njc0MTBt XzI3MDAzMTI2MDU5MzYyMzYwMjNtXy0xNjUzMzkwMDAwMDU2NDEyMjMxZG9jcy1pbnRlcm5h bC1ndWlkLTU4NTliN2ExLWU5NzUtOWEwZC0zYjAxLThkZTA1M2NkZmVmNyI+CiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8L2I+ PGIKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgIHN0eWxlPSJmb250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0zNDcxODA3MjkwOTkx NzQzMzQ2bV8tNzQyOTY3NjA0MTA5ODExMDE1N21fMjEyNzI3NTc1NTAyNjY3NDEwbV8yNzAw MzEyNjA1OTM2MjM2MDIzbV8tMTY1MzM5MDAwMDA1NjQxMjIzMWRvY3MtaW50ZXJuYWwtZ3Vp ZC01ODU5YjdhMS1lOTc1LTlhMGQtM2IwMS04ZGUwNTNjZGZlZjciPgogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPHAgZGlyPSJs dHIiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICBzdHlsZT0ibGluZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4t Ym90dG9tOjBwdCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFtaWx5OkFy aWFsO2NvbG9yOiMwMDAwMDA7YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtmb250LXdl aWdodDo0MDA7Zm9udC1zdHlsZTpub3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0ZXh0LWRl Y29yYXRpb246bm9uZTt2ZXJ0aWNhbC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFjZTpwcmUt d3JhcCI+YnVpbGQ6IE1ha2UgcmVzdWx0aW5nIFJQTXMgYXJjaGl0ZWN0dXJlLXNwZWNpZmlj IC0gPGEgY2xhc3M9Im1fLTM0NzE4MDcyOTA5OTE3NDMzNDZtXy03NDI5Njc2MDQxMDk4MTEw MTU3bV8yMTI3Mjc1NzU1MDI2Njc0MTBtXzI3MDAzMTI2MDU5MzYyMzYwMjNtXy0xNjUzMzkw MDAwMDU2NDEyMjMxbW96LXR4dC1saW5rLWZyZWV0ZXh0IiBocmVmPSJodHRwczovL2dlcnJp dC5vdmlydC5vcmcvIy9jLzg0NTM0LyIgdGFyZ2V0PSJfYmxhbmsiIG1vei1kby1ub3Qtc2Vu ZD0idHJ1ZSI+aHR0cHM6Ly9nZXJyaXQub3ZpcnQub3JnLyMvYy84NDUzNC88L2E+Cjwvc3Bh bj48L3A+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICA8cCBkaXI9Imx0ciIKICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0eWxlPSJsaW5lLWhlaWdo dDoxLjM4O21hcmdpbi10b3A6MHB0O21hcmdpbi1ib3R0b206MHB0Ij48c3BhbiBzdHlsZT0i Zm9udC1zaXplOjExcHQ7Zm9udC1mYW1pbHk6QXJpYWw7Y29sb3I6IzAwMDAwMDtiYWNrZ3Jv dW5kLWNvbG9yOnRyYW5zcGFyZW50O2ZvbnQtd2VpZ2h0OjQwMDtmb250LXN0eWxlOm5vcm1h bDtmb250LXZhcmlhbnQ6bm9ybWFsO3RleHQtZGVjb3JhdGlvbjpub25lO3ZlcnRpY2FsLWFs aWduOmJhc2VsaW5lO3doaXRlLXNwYWNlOnByZS13cmFwIj5MaW5rIHRvIEpvYjo8L3NwYW4+ PC9wPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgPGEKY2xhc3M9Im1fLTM0NzE4MDcyOTA5OTE3NDMzNDZtXy03NDI5Njc2MDQx MDk4MTEwMTU3bV8yMTI3Mjc1NzU1MDI2Njc0MTBtXzI3MDAzMTI2MDU5MzYyMzYwMjNtXy0x NjUzMzkwMDAwMDU2NDEyMjMxbW96LXR4dC1saW5rLWZyZWV0ZXh0IgpocmVmPSJodHRwOi8v amVua2lucy5vdmlydC5vcmcvam9iL292aXJ0LW1hc3Rlcl9jaGFuZ2UtcXVldWUtdGVzdGVy LzQwNTUiCnRhcmdldD0iX2JsYW5rIiBtb3otZG8tbm90LXNlbmQ9InRydWUiPmh0dHA6Ly9q ZW5raW5zLm92aXJ0Lm9yZy9qb2Ivb3ZpcnQtbWFzdGVyX2NoYW5nZS1xdWV1ZS10ZXN0ZXIv NDA1NTwvYT48L2I+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICA8cD48YgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgc3R5bGU9ImZvbnQtd2VpZ2h0Om5vcm1hbCIK aWQ9Im1fLTM0NzE4MDcyOTA5OTE3NDMzNDZtXy03NDI5Njc2MDQxMDk4MTEwMTU3bV8yMTI3 Mjc1NzU1MDI2Njc0MTBtXzI3MDAzMTI2MDU5MzYyMzYwMjNtXy0xNjUzMzkwMDAwMDU2NDEy MjMxZG9jcy1pbnRlcm5hbC1ndWlkLTU4NTliN2ExLWU5NzUtOWEwZC0zYjAxLThkZTA1M2Nk ZmVmNyI+PGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgPC9iPjwvcD4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxwIGRpcj0ibHRyIgogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3R5bGU9Imxp bmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRvbTowcHQiPjxiCnN0 eWxlPSJmb250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0zNDcxODA3MjkwOTkxNzQzMzQ2bV8t NzQyOTY3NjA0MTA5ODExMDE1N21fMjEyNzI3NTc1NTAyNjY3NDEwbV8yNzAwMzEyNjA1OTM2 MjM2MDIzbV8tMTY1MzM5MDAwMDA1NjQxMjIzMWRvY3MtaW50ZXJuYWwtZ3VpZC01ODU5Yjdh MS1lOTc1LTlhMGQtM2IwMS04ZGUwNTNjZGZlZjciPjxzcGFuIHN0eWxlPSJmb250LXNpemU6 MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6 dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFy aWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxp bmU7d2hpdGUtc3BhY2U6cHJlLXdyYXAiPkxpbmsgdG8gYWxsIGxvZ3M6PC9zcGFuPjwvYj48 L3A+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICA8YgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgc3R5bGU9ImZvbnQtd2VpZ2h0Om5vcm1hbCIKaWQ9Im1fLTM0NzE4 MDcyOTA5OTE3NDMzNDZtXy03NDI5Njc2MDQxMDk4MTEwMTU3bV8yMTI3Mjc1NzU1MDI2Njc0 MTBtXzI3MDAzMTI2MDU5MzYyMzYwMjNtXy0xNjUzMzkwMDAwMDU2NDEyMjMxZG9jcy1pbnRl cm5hbC1ndWlkLTU4NTliN2ExLWU5NzUtOWEwZC0zYjAxLThkZTA1M2NkZmVmNyI+CiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8 cCBkaXI9Imx0ciIKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHN0eWxlPSJsaW5lLWhlaWdodDoxLjM4O21hcmdpbi10b3A6MHB0 O21hcmdpbi1ib3R0b206MHB0Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1m YW1pbHk6QXJpYWw7Y29sb3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50 O2ZvbnQtd2VpZ2h0OjQwMDtmb250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFs O3RleHQtZGVjb3JhdGlvbjpub25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNw YWNlOnByZS13cmFwIj4KPC9zcGFuPjwvcD4KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvYj4KICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxwIGRpcj0ibHRyIgog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgc3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRv bTowcHQiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtj b2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6 NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0 aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXAi PjxhIGNsYXNzPSJtXy0zNDcxODA3MjkwOTkxNzQzMzQ2bV8tNzQyOTY3NjA0MTA5ODExMDE1 N21fMjEyNzI3NTc1NTAyNjY3NDEwbV8yNzAwMzEyNjA1OTM2MjM2MDIzbV8tMTY1MzM5MDAw MDA1NjQxMjIzMW1vei10eHQtbGluay1mcmVldGV4dCIgaHJlZj0iaHR0cDovL2plbmtpbnMu b3ZpcnQub3JnL2pvYi9vdmlydC1tYXN0ZXJfY2hhbmdlLXF1ZXVlLXRlc3Rlci80MDU1L2Fy dGlmYWN0LyIgdGFyZ2V0PSJfYmxhbmsiIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+aHR0cDov L2plbmtpbnMub3ZpcnQub3JnL2pvYi9vdmlydC1tYXN0ZXJfY2hhbmdlLXF1ZXVlLXRlc3Rl ci80MDU1L2FydGlmYWN0LzwvYT48L3NwYW4+PC9wPgogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPHA+PGIKICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0eWxlPSJm b250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0zNDcxODA3MjkwOTkxNzQzMzQ2bV8tNzQyOTY3 NjA0MTA5ODExMDE1N21fMjEyNzI3NTc1NTAyNjY3NDEwbV8yNzAwMzEyNjA1OTM2MjM2MDIz bV8tMTY1MzM5MDAwMDA1NjQxMjIzMWRvY3MtaW50ZXJuYWwtZ3VpZC01ODU5YjdhMS1lOTc1 LTlhMGQtM2IwMS04ZGUwNTNjZGZlZjciPgogICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9iPjwvcD4KICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxwIGRpcj0ibHRy IgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgc3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJv dHRvbTowcHQiPjxiCnN0eWxlPSJmb250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0zNDcxODA3 MjkwOTkxNzQzMzQ2bV8tNzQyOTY3NjA0MTA5ODExMDE1N21fMjEyNzI3NTc1NTAyNjY3NDEw bV8yNzAwMzEyNjA1OTM2MjM2MDIzbV8tMTY1MzM5MDAwMDA1NjQxMjIzMWRvY3MtaW50ZXJu YWwtZ3VpZC01ODU5YjdhMS1lOTc1LTlhMGQtM2IwMS04ZGUwNTNjZGZlZjciPjxzcGFuIHN0 eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtjb2xvcjojMDAwMDAwO2Jh Y2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6NDAwO2ZvbnQtc3R5bGU6 bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0aW9uOm5vbmU7dmVydGlj YWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXAiPjxhIGNsYXNzPSJtXy0z NDcxODA3MjkwOTkxNzQzMzQ2bV8tNzQyOTY3NjA0MTA5ODExMDE1N21fMjEyNzI3NTc1NTAy NjY3NDEwbV8yNzAwMzEyNjA1OTM2MjM2MDIzbV8tMTY1MzM5MDAwMDA1NjQxMjIzMW1vei10 eHQtbGluay1mcmVldGV4dCIgaHJlZj0iaHR0cDovL2plbmtpbnMub3ZpcnQub3JnL2pvYi9v dmlydC1tYXN0ZXJfY2hhbmdlLXF1ZXVlLXRlc3Rlci80MDU1L2FydGlmYWN0L2V4cG9ydGVk LWFydGlmYWN0cy91cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0LW1hc3Rlci1lbDcvdGVzdF9s b2dzL3VwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci9wb3N0LTAwMV9pbml0aWFs aXplX2VuZ2luZS5weS9sYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1l bmdpbmUvX3Zhcl9sb2cvbWVzc2FnZXMvKnZpZXcqLyIgdGFyZ2V0PSJfYmxhbmsiIG1vei1k by1ub3Qtc2VuZD0idHJ1ZSI+aHR0cDovL2plbmtpbnMub3ZpcnQub3JnL2pvYi9vdmlydC1t YXN0ZXJfY2hhbmdlLXF1ZXVlLXRlc3Rlci80MDU1L2FydGlmYWN0L2V4cG9ydGVkLWFydGlm YWN0cy91cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0LW1hc3Rlci1lbDcvdGVzdF9sb2dzL3Vw Z3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci9wb3N0LTAwMV9pbml0aWFsaXplX2Vu Z2luZS5weS9sYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1lbmdpbmUv X3Zhcl9sb2cvbWVzc2FnZXMvKnZpZXcqLzwvYT48L3NwYW4+PC9iPjwvcD4KICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxiCiAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICBzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsIgppZD0ibV8tMzQ3MTgwNzI5MDk5MTc0MzM0 Nm1fLTc0Mjk2NzYwNDEwOTgxMTAxNTdtXzIxMjcyNzU3NTUwMjY2NzQxMG1fMjcwMDMxMjYw NTkzNjIzNjAyM21fLTE2NTMzOTAwMDAwNTY0MTIyMzFkb2NzLWludGVybmFsLWd1aWQtNTg1 OWI3YTEtZTk3NS05YTBkLTNiMDEtOGRlMDUzY2RmZWY3Ij4KICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxwIGRpcj0ibHRyIgog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgc3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJvdHRv bTowcHQiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlhbDtj b2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWlnaHQ6 NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNvcmF0 aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdyYXAi Pgo8L3NwYW4+PC9wPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgPHAgZGlyPSJsdHIiCiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHlsZT0ibGlu ZS1oZWlnaHQ6MS4zODttYXJnaW4tdG9wOjBwdDttYXJnaW4tYm90dG9tOjBwdCI+PHNwYW4g c3R5bGU9ImZvbnQtc2l6ZToxMXB0O2ZvbnQtZmFtaWx5OkFyaWFsO2NvbG9yOiMwMDAwMDA7 YmFja2dyb3VuZC1jb2xvcjp0cmFuc3BhcmVudDtmb250LXdlaWdodDo0MDA7Zm9udC1zdHls ZTpub3JtYWw7Zm9udC12YXJpYW50Om5vcm1hbDt0ZXh0LWRlY29yYXRpb246bm9uZTt2ZXJ0 aWNhbC1hbGlnbjpiYXNlbGluZTt3aGl0ZS1zcGFjZTpwcmUtd3JhcCI+KFJlbGV2YW50KSBl cnJvciBzbmlwcGV0IGZyb20gdGhlIGxvZzogPC9zcGFuPjwvcD4KICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxwIGRpcj0ibHRy IgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgc3R5bGU9ImxpbmUtaGVpZ2h0OjEuMzg7bWFyZ2luLXRvcDowcHQ7bWFyZ2luLWJv dHRvbTowcHQiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTFwdDtmb250LWZhbWlseTpBcmlh bDtjb2xvcjojMDAwMDAwO2JhY2tncm91bmQtY29sb3I6dHJhbnNwYXJlbnQ7Zm9udC13ZWln aHQ6NDAwO2ZvbnQtc3R5bGU6bm9ybWFsO2ZvbnQtdmFyaWFudDpub3JtYWw7dGV4dC1kZWNv cmF0aW9uOm5vbmU7dmVydGljYWwtYWxpZ246YmFzZWxpbmU7d2hpdGUtc3BhY2U6cHJlLXdy YXAiPiZsdDtlcnJvciZndDs8L3NwYW4+PC9wPgogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGJyPgogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9iPmZyb20gbGFn bwogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgbG9nOiA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICA8cHJlIHN0eWxlPSJib3gtc2l6aW5nOmJvcmRlci1ib3g7 d2hpdGUtc3BhY2U6cHJlLXdyYXA7d29yZC13cmFwOmJyZWFrLXdvcmQ7bWFyZ2luOjBweDtj b2xvcjpyZ2IoNTEsNTEsNTEpO2ZvbnQtc2l6ZToxNHB4O2ZvbnQtc3R5bGU6bm9ybWFsO2Zv bnQtdmFyaWFudC1saWdhdHVyZXM6bm9ybWFsO2ZvbnQtdmFyaWFudC1jYXBzOm5vcm1hbDtm b250LXdlaWdodDpub3JtYWw7bGV0dGVyLXNwYWNpbmc6bm9ybWFsO3RleHQtYWxpZ246c3Rh cnQ7dGV4dC1pbmRlbnQ6MHB4O3RleHQtdHJhbnNmb3JtOm5vbmU7d29yZC1zcGFjaW5nOjBw eDt0ZXh0LWRlY29yYXRpb24tc3R5bGU6aW5pdGlhbDt0ZXh0LWRlY29yYXRpb24tY29sb3I6 aW5pdGlhbCI+RmFpbGVkIHRvIHN0YXJ0IHNlcnZpY2UgJ292aXJ0LWltYWdlaW8tcHJveHkK Cm1lc3NhZ2VzIGxvZ3M6IAoKTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBncmFkZS1mcm9tLXJl bGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBzeXN0ZW1kOiBTdGFydGluZyBTZXNzaW9uIDgg b2YgdXNlciByb290LgpOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20tcmVsZWFz ZS1zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IFRyYWNlYmFjayAo bW9zdCByZWNlbnQgY2FsbCBsYXN0KToKTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBncmFkZS1m cm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBG aWxlICIvdXNyL2Jpbi9vdmlydC1pbWFnZWlvLXByb3h5IiwgbGluZSA4NSwgaW4gJmx0O21v ZHVsZSZndDsKTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3Vp dGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBzdGF0dXMgPSBpbWFnZV9w cm94eS5tYWluKGFyZ3MsIGNvbmZpZykKTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBncmFkZS1m cm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBG aWxlICIvdXNyL2xpYi9weXRob24yLjcvc2l0ZS1wYWNrYWdlcy9vdmlydF9pbWFnZWlvX3By b3h5L2ltYWdlX3Byb3h5LnB5IiwgbGluZSAyMSwgaW4gbWFpbgpOb3YgMjMgMDc6MzA6NDcg bGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWlt YWdlaW8tcHJveHk6IGltYWdlX3NlcnZlci5zdGFydChjb25maWcpCk5vdiAyMyAwNzozMDo0 NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQt aW1hZ2Vpby1wcm94eTogRmlsZSAiL3Vzci9saWIvcHl0aG9uMi43L3NpdGUtcGFja2FnZXMv b3ZpcnRfaW1hZ2Vpb19wcm94eS9zZXJ2ZXIucHkiLCBsaW5lIDQ1LCBpbiBzdGFydApOb3Yg MjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5n aW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IFdTR0lSZXF1ZXN0SGFuZGxlcikKTm92IDIzIDA3 OjMwOjQ3IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBv dmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xpYjY0L3B5dGhvbjIuNy9Tb2NrZXRT ZXJ2ZXIucHkiLCBsaW5lIDQxOSwgaW4gX19pbml0X18KTm92IDIzIDA3OjMwOjQ3IGxhZ28t dXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlv LXByb3h5OiBzZWxmLnNlcnZlcl9iaW5kKCkKTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBncmFk ZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5 OiBGaWxlICIvdXNyL2xpYjY0L3B5dGhvbjIuNy93c2dpcmVmL3NpbXBsZV9zZXJ2ZXIucHki LCBsaW5lIDQ4LCBpbiBzZXJ2ZXJfYmluZApOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRl LWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6 IEhUVFBTZXJ2ZXIuc2VydmVyX2JpbmQoc2VsZikKTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBn cmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXBy b3h5OiBGaWxlICIvdXNyL2xpYjY0L3B5dGhvbjIuNy9CYXNlSFRUUFNlcnZlci5weSIsIGxp bmUgMTA4LCBpbiBzZXJ2ZXJfYmluZApOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZy b20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IFNv Y2tldFNlcnZlci5UQ1BTZXJ2ZXIuc2VydmVyX2JpbmQoc2VsZikKTm92IDIzIDA3OjMwOjQ3 IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1p bWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xpYjY0L3B5dGhvbjIuNy9Tb2NrZXRTZXJ2ZXIu cHkiLCBsaW5lIDQzMCwgaW4gc2VydmVyX2JpbmQKTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBn cmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXBy b3h5OiBzZWxmLnNvY2tldC5iaW5kKHNlbGYuc2VydmVyX2FkZHJlc3MpCk5vdiAyMyAwNzoz MDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1lbmdpbmUgb3Zp cnQtaW1hZ2Vpby1wcm94eTogRmlsZSAiL3Vzci9saWI2NC9weXRob24yLjcvc29ja2V0LnB5 IiwgbGluZSAyMjQsIGluIG1ldGgKTm92IDIzIDA3OjMwOjQ3IGxhZ28tdXBncmFkZS1mcm9t LXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiByZXR1 cm4gZ2V0YXR0cihzZWxmLl9zb2NrLG5hbWUpKCphcmdzKQpOb3YgMjMgMDc6MzA6NDcgbGFn by11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdl aW8tcHJveHk6IHNvY2tldC5lcnJvcjogW0Vycm5vIDk4XSBBZGRyZXNzIGFscmVhZHkgaW4g dXNlCk5vdiAyMyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1h c3Rlci1lbmdpbmUgc3lzdGVtZDogb3ZpcnQtaW1hZ2Vpby1wcm94eS5zZXJ2aWNlOiBtYWlu IHByb2Nlc3MgZXhpdGVkLCBjb2RlPWV4aXRlZCwgc3RhdHVzPTEvRkFJTFVSRQpOb3YgMjMg MDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5l IHN5c3RlbWQ6IEZhaWxlZCB0byBzdGFydCBvVmlydCBJbWFnZUlPIFByb3h5LgpOb3YgMjMg MDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5l IHN5c3RlbWQ6IFVuaXQgb3ZpcnQtaW1hZ2Vpby1wcm94eS5zZXJ2aWNlIGVudGVyZWQgZmFp bGVkIHN0YXRlLgpOb3YgMjMgMDc6MzA6NDcgbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1z dWl0ZS1tYXN0ZXItZW5naW5lIHN5c3RlbWQ6IG92aXJ0LWltYWdlaW8tcHJveHkuc2Vydmlj ZSBmYWlsZWQuCk5vdiAyMyAwNzozMDo0NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1 aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDogb3ZpcnQtaW1hZ2Vpby1wcm94eS5zZXJ2aWNl IGhvbGRvZmYgdGltZSBvdmVyLCBzY2hlZHVsaW5nIHJlc3RhcnQuCk5vdiAyMyAwNzozMDo0 NyBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVt ZDogU3RhcnRpbmcgb1ZpcnQgSW1hZ2VJTyBQcm94eS4uLgpOb3YgMjMgMDc6MzA6NDggbGFn by11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdl aW8tcHJveHk6IFRyYWNlYmFjayAobW9zdCByZWNlbnQgY2FsbCBsYXN0KToKTm92IDIzIDA3 OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBv dmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2Jpbi9vdmlydC1pbWFnZWlvLXByb3h5 IiwgbGluZSA4NSwgaW4gJmx0O21vZHVsZSZndDsKTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBn cmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXBy b3h5OiBzdGF0dXMgPSBpbWFnZV9wcm94eS5tYWluKGFyZ3MsIGNvbmZpZykKTm92IDIzIDA3 OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBv dmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xpYi9weXRob24yLjcvc2l0ZS1wYWNr YWdlcy9vdmlydF9pbWFnZWlvX3Byb3h5L2ltYWdlX3Byb3h5LnB5IiwgbGluZSAyMSwgaW4g bWFpbgpOb3YgMjMgMDc6MzA6NDggbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1t YXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IGltYWdlX3NlcnZlci5zdGFydChj b25maWcpCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRl LW1hc3Rlci1lbmdpbmUgb3ZpcnQtaW1hZ2Vpby1wcm94eTogRmlsZSAiL3Vzci9saWIvcHl0 aG9uMi43L3NpdGUtcGFja2FnZXMvb3ZpcnRfaW1hZ2Vpb19wcm94eS9zZXJ2ZXIucHkiLCBs aW5lIDQ1LCBpbiBzdGFydApOb3YgMjMgMDc6MzA6NDggbGFnby11cGdyYWRlLWZyb20tcmVs ZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IFdTR0lSZXF1 ZXN0SGFuZGxlcikKTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Ut c3VpdGUtbWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xp YjY0L3B5dGhvbjIuNy9Tb2NrZXRTZXJ2ZXIucHkiLCBsaW5lIDQxOSwgaW4gX19pbml0X18K Tm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVy LWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBzZWxmLnNlcnZlcl9iaW5kKCkKTm92IDIz IDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2lu ZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xpYjY0L3B5dGhvbjIuNy93c2dp cmVmL3NpbXBsZV9zZXJ2ZXIucHkiLCBsaW5lIDQ4LCBpbiBzZXJ2ZXJfYmluZApOb3YgMjMg MDc6MzA6NDggbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5l IG92aXJ0LWltYWdlaW8tcHJveHk6IEhUVFBTZXJ2ZXIuc2VydmVyX2JpbmQoc2VsZikKTm92 IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVu Z2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xpYjY0L3B5dGhvbjIuNy9C YXNlSFRUUFNlcnZlci5weSIsIGxpbmUgMTA4LCBpbiBzZXJ2ZXJfYmluZApOb3YgMjMgMDc6 MzA6NDggbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lIG92 aXJ0LWltYWdlaW8tcHJveHk6IFNvY2tldFNlcnZlci5UQ1BTZXJ2ZXIuc2VydmVyX2JpbmQo c2VsZikKTm92IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUt bWFzdGVyLWVuZ2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBGaWxlICIvdXNyL2xpYjY0L3B5 dGhvbjIuNy9Tb2NrZXRTZXJ2ZXIucHkiLCBsaW5lIDQzMCwgaW4gc2VydmVyX2JpbmQKTm92 IDIzIDA3OjMwOjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVu Z2luZSBvdmlydC1pbWFnZWlvLXByb3h5OiBzZWxmLnNvY2tldC5iaW5kKHNlbGYuc2VydmVy X2FkZHJlc3MpCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1 aXRlLW1hc3Rlci1lbmdpbmUgb3ZpcnQtaW1hZ2Vpby1wcm94eTogRmlsZSAiL3Vzci9saWI2 NC9weXRob24yLjcvc29ja2V0LnB5IiwgbGluZSAyMjQsIGluIG1ldGgKTm92IDIzIDA3OjMw OjQ4IGxhZ28tdXBncmFkZS1mcm9tLXJlbGVhc2Utc3VpdGUtbWFzdGVyLWVuZ2luZSBvdmly dC1pbWFnZWlvLXByb3h5OiByZXR1cm4gZ2V0YXR0cihzZWxmLl9zb2NrLG5hbWUpKCphcmdz KQpOb3YgMjMgMDc6MzA6NDggbGFnby11cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0 ZXItZW5naW5lIG92aXJ0LWltYWdlaW8tcHJveHk6IHNvY2tldC5lcnJvcjogW0Vycm5vIDk4 XSBBZGRyZXNzIGFscmVhZHkgaW4gdXNlCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUt ZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDogb3ZpcnQtaW1hZ2Vp by1wcm94eS5zZXJ2aWNlOiBtYWluIHByb2Nlc3MgZXhpdGVkLCBjb2RlPWV4aXRlZCwgc3Rh dHVzPTEvRkFJTFVSRQpOb3YgMjMgMDc6MzA6NDggbGFnby11cGdyYWRlLWZyb20tcmVsZWFz ZS1zdWl0ZS1tYXN0ZXItZW5naW5lIHN5c3RlbWQ6IEZhaWxlZCB0byBzdGFydCBvVmlydCBJ bWFnZUlPIFByb3h5LgpOb3YgMjMgMDc6MzA6NDggbGFnby11cGdyYWRlLWZyb20tcmVsZWFz ZS1zdWl0ZS1tYXN0ZXItZW5naW5lIHN5c3RlbWQ6IFVuaXQgb3ZpcnQtaW1hZ2Vpby1wcm94 eS5zZXJ2aWNlIGVudGVyZWQgZmFpbGVkIHN0YXRlLgpOb3YgMjMgMDc6MzA6NDggbGFnby11 cGdyYWRlLWZyb20tcmVsZWFzZS1zdWl0ZS1tYXN0ZXItZW5naW5lIHN5c3RlbWQ6IG92aXJ0 LWltYWdlaW8tcHJveHkuc2VydmljZSBmYWlsZWQuCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVw Z3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDogb3ZpcnQt aW1hZ2Vpby1wcm94eS5zZXJ2aWNlIGhvbGRvZmYgdGltZSBvdmVyLCBzY2hlZHVsaW5nIHJl c3RhcnQuCk5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRl LW1hc3Rlci1lbmdpbmUgc3lzdGVtZDogc3RhcnQgcmVxdWVzdCByZXBlYXRlZCB0b28gcXVp Y2tseSBmb3Igb3ZpcnQtaW1hZ2Vpby1wcm94eS5zZXJ2aWNlCk5vdiAyMyAwNzozMDo0OCBs YWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDog RmFpbGVkIHRvIHN0YXJ0IG9WaXJ0IEltYWdlSU8gUHJveHkuCk5vdiAyMyAwNzozMDo0OCBs YWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rlci1lbmdpbmUgc3lzdGVtZDog VW5pdCBvdmlydC1pbWFnZWlvLXByb3h5LnNlcnZpY2UgZW50ZXJlZCBmYWlsZWQgc3RhdGUu Ck5vdiAyMyAwNzozMDo0OCBsYWdvLXVwZ3JhZGUtZnJvbS1yZWxlYXNlLXN1aXRlLW1hc3Rl ci1lbmdpbmUgc3lzdGVtZDogb3ZpcnQtaW1hZ2Vpby1wcm94eS5zZXJ2aWNlIGZhaWxlZC48 L3ByZT4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgIDxwPjxiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICBzdHlsZT0iZm9udC13ZWlnaHQ6bm9ybWFsIgppZD0ibV8t MzQ3MTgwNzI5MDk5MTc0MzM0Nm1fLTc0Mjk2NzYwNDEwOTgxMTAxNTdtXzIxMjcyNzU3NTUw MjY2NzQxMG1fMjcwMDMxMjYwNTkzNjIzNjAyM21fLTE2NTMzOTAwMDAwNTY0MTIyMzFkb2Nz LWludGVybmFsLWd1aWQtNTg1OWI3YTEtZTk3NS05YTBkLTNiMDEtOGRlMDUzY2RmZWY3Ij4K ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgIDwvYj48L3A+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICA8cCBkaXI9Imx0ciIKICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHN0eWxlPSJsaW5lLWhlaWdodDox LjM4O21hcmdpbi10b3A6MHB0O21hcmdpbi1ib3R0b206MHB0Ij48YgpzdHlsZT0iZm9udC13 ZWlnaHQ6bm9ybWFsIgppZD0ibV8tMzQ3MTgwNzI5MDk5MTc0MzM0Nm1fLTc0Mjk2NzYwNDEw OTgxMTAxNTdtXzIxMjcyNzU3NTUwMjY2NzQxMG1fMjcwMDMxMjYwNTkzNjIzNjAyM21fLTE2 NTMzOTAwMDAwNTY0MTIyMzFkb2NzLWludGVybmFsLWd1aWQtNTg1OWI3YTEtZTk3NS05YTBk LTNiMDEtOGRlMDUzY2RmZWY3Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExcHQ7Zm9udC1m YW1pbHk6QXJpYWw7Y29sb3I6IzAwMDAwMDtiYWNrZ3JvdW5kLWNvbG9yOnRyYW5zcGFyZW50 O2ZvbnQtd2VpZ2h0OjQwMDtmb250LXN0eWxlOm5vcm1hbDtmb250LXZhcmlhbnQ6bm9ybWFs O3RleHQtZGVjb3JhdGlvbjpub25lO3ZlcnRpY2FsLWFsaWduOmJhc2VsaW5lO3doaXRlLXNw YWNlOnByZS13cmFwIj4mbHQ7L2Vycm9yJmd0Ozwvc3Bhbj48L2I+PC9wPgogICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGIKICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IHN0eWxlPSJmb250LXdlaWdodDpub3JtYWwiCmlkPSJtXy0zNDcxODA3MjkwOTkxNzQzMzQ2 bV8tNzQyOTY3NjA0MTA5ODExMDE1N21fMjEyNzI3NTc1NTAyNjY3NDEwbV8yNzAwMzEyNjA1 OTM2MjM2MDIzbV8tMTY1MzM5MDAwMDA1NjQxMjIzMWRvY3MtaW50ZXJuYWwtZ3VpZC01ODU5 YjdhMS1lOTc1LTlhMGQtM2IwMS04ZGUwNTNjZGZlZjciPgogICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9iPjxicgpjbGFzcz0i bV8tMzQ3MTgwNzI5MDk5MTc0MzM0Nm1fLTc0Mjk2NzYwNDEwOTgxMTAxNTdtXzIxMjcyNzU3 NTUwMjY2NzQxMG1fMjcwMDMxMjYwNTkzNjIzNjAyM21fLTE2NTMzOTAwMDAwNTY0MTIyMzFB cHBsZS1pbnRlcmNoYW5nZS1uZXdsaW5lIj4KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGJyPgpfX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXzxicj4KICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIEluZnJh IG1haWxpbmcKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIGxpc3Q8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICA8YQogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgaHJlZj0ibWFpbHRvOkluZnJhQG92 aXJ0Lm9yZyIKdGFyZ2V0PSJfYmxhbmsiIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+SW5mcmFA b3ZpcnQub3JnPC9hPjxicj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgIDxhCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQu b3JnL21haWxtYW4vbGlzdGluZm8vaW5mcmEiCnJlbD0ibm9yZWZlcnJlciIgdGFyZ2V0PSJf YmxhbmsiIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9t YWlsbWFuL2xpc3RpbmZvL2luZnJhPC9hPjxicj4KICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxicj4KICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8L2Jsb2NrcXVvdGU+ CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IDwvZGl2PgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICA8L2Jsb2NrcXVvdGU+CiAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgPHA+PGJyPgogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvcD4KICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8 c3Bhbj5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXzxi cj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgRGV2ZWwgbWFp bGluZyBsaXN0PGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICA8YQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGhyZWY9 Im1haWx0bzpEZXZlbEBvdmlydC5vcmciCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgdGFyZ2V0PSJfYmxhbmsiCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0cnVlIj5EZXZlbEBvdmly dC5vcmc8L2E+PGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICA8YQogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGhyZWY9 Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby9kZXZlbCIKICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICByZWw9Im5vcmVmZXJyZXIi CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdGFyZ2V0PSJf YmxhbmsiCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbW96 LWRvLW5vdC1zZW5kPSJ0cnVlIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlz dGluZm8vZGV2ZWw8L2E+PGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgPC9zcGFuPjwvYmxvY2txdW90ZT4KICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxz cGFuCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY2xhc3M9Im1fLTM0 NzE4MDcyOTA5OTE3NDMzNDZtXy03NDI5Njc2MDQxMDk4MTEwMTU3SE9FblpiIj48Zm9udAog ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgY29sb3I9IiM4ODg4ODgi Pjxicj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxiciBjbGVh cj0iYWxsIj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxkaXY+ PGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9kaXY+CiAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAtLSA8YnI+CiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8ZGl2CmNsYXNzPSJtXy0zNDcxODA3 MjkwOTkxNzQzMzQ2bV8tNzQyOTY3NjA0MTA5ODExMDE1N21fMjEyNzI3NTc1NTAyNjY3NDEw Z21haWxfc2lnbmF0dXJlIgpkYXRhLXNtYXJ0bWFpbD0iZ21haWxfc2lnbmF0dXJlIj4KICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPGRpdiBkaXI9Imx0ciI+ PGZvbnQKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGNv bG9yPSIjMDAwMDAwIiBzaXplPSIyIgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgZmFjZT0ib3ZlcnBhc3MsCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICBzYW5zLXNlcmlmIj48c3BhbgogICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzdHlsZT0idGV4dC10cmFuc2Zv cm06dXBwZXJjYXNlIj48Yj5HQUwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBiRU4gSEFJTTwvYj48L3NwYW4+PC9mb250PgogICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDxkaXY+PHNwYW4Kc3R5bGU9ImNv bG9yOnJnYigwLDAsMCk7Zm9udC1mYW1pbHk6b3ZlcnBhc3Msc2Fucy1zZXJpZjt0ZXh0LXRy YW5zZm9ybTp1cHBlcmNhc2UiPjxmb250CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgc2l6ZT0iMiI+UkhWIERFVk9QUzwvZm9udD48L3NwYW4+ PGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvZGl2 PgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvZGl2PgogICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvZm9udD48L3NwYW4+PC9kaXY+CiAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICA8YnI+Cl9fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fPGJyPgogICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgRGV2ZWwgbWFpbGluZyBsaXN0PGJyPgogICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgPGEgaHJlZj0ibWFpbHRvOkRldmVsQG92aXJ0Lm9yZyIKICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgdGFyZ2V0PSJfYmxhbmsiCiAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+RGV2 ZWxAb3ZpcnQub3JnPC9hPjxicj4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IDxhCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIGhyZWY9Imh0dHA6Ly9s aXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby9kZXZlbCIKICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgcmVsPSJub3JlZmVycmVyIiB0YXJnZXQ9Il9ibGFuayIK ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgbW96LWRvLW5vdC1zZW5kPSJ0 cnVlIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vZGV2ZWw8L2E+ PGJyPgogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIDwvYmxvY2txdW90ZT4KICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAg ICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAg ICAgICAgICAgICAgICAgICAgICA8YnI+CiAgICAgICAgICAgICAgICAgICAgICAgIDwvZGl2 PgogICAgICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgICAgPC9i bG9ja3F1b3RlPgogICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAgICAg PGJyPgogICAgICAgICAgICAgICAgICA8YnIgY2xlYXI9ImFsbCI+CiAgICAgICAgICAgICAg ICAgIDxkaXY+PGJyPgogICAgICAgICAgICAgICAgICA8L2Rpdj4KICAgICAgICAgICAgICAg ICAgLS0gPGJyPgogICAgICAgICAgICAgICAgICA8ZGl2IGNsYXNzPSJtXy0zNDcxODA3Mjkw OTkxNzQzMzQ2Z21haWxfc2lnbmF0dXJlIgogICAgICAgICAgICAgICAgICAgIGRhdGEtc21h cnRtYWlsPSJnbWFpbF9zaWduYXR1cmUiPgogICAgICAgICAgICAgICAgICAgIDxkaXYgZGly PSJsdHIiPjxmb250IGNvbG9yPSIjMDAwMDAwIiBzaXplPSIyIgogICAgICAgICAgICAgICAg ICAgICAgICBmYWNlPSJvdmVycGFzcywgc2Fucy1zZXJpZiI+PHNwYW4KICAgICAgICAgICAg ICAgICAgICAgICAgICBzdHlsZT0idGV4dC10cmFuc2Zvcm06dXBwZXJjYXNlIj48Yj5HQUwg YkVOCiAgICAgICAgICAgICAgICAgICAgICAgICAgICBIQUlNPC9iPjwvc3Bhbj48L2ZvbnQ+ CiAgICAgICAgICAgICAgICAgICAgICA8ZGl2PjxzcGFuCnN0eWxlPSJjb2xvcjpyZ2IoMCww LDApO2ZvbnQtZmFtaWx5Om92ZXJwYXNzLHNhbnMtc2VyaWY7dGV4dC10cmFuc2Zvcm06dXBw ZXJjYXNlIj48Zm9udAogICAgICAgICAgICAgICAgICAgICAgICAgICAgc2l6ZT0iMiI+UkhW IERFVk9QUzwvZm9udD48L3NwYW4+PGJyPgogICAgICAgICAgICAgICAgICAgICAgPC9kaXY+ CiAgICAgICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICAgIDwvZGl2Pgog ICAgICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICAgICAgICBfX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXzxicj4KICAgICAgICAgICAgICAgIERl dmVsIG1haWxpbmcgbGlzdDxicj4KICAgICAgICAgICAgICAgIDxhIGhyZWY9Im1haWx0bzpE ZXZlbEBvdmlydC5vcmciIHRhcmdldD0iX2JsYW5rIgogICAgICAgICAgICAgICAgICBtb3ot ZG8tbm90LXNlbmQ9InRydWUiPkRldmVsQG92aXJ0Lm9yZzwvYT48YnI+CiAgICAgICAgICAg ICAgICA8YSBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8v ZGV2ZWwiCiAgICAgICAgICAgICAgICAgIHJlbD0ibm9yZWZlcnJlciIgdGFyZ2V0PSJfYmxh bmsiCiAgICAgICAgICAgICAgICAgIG1vei1kby1ub3Qtc2VuZD0idHJ1ZSI+aHR0cDovL2xp c3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2RldmVsPC9hPjwvYmxvY2txdW90ZT4K ICAgICAgICAgICAgPC9kaXY+CiAgICAgICAgICA8L2Rpdj4KICAgICAgICA8L2Rpdj4KICAg ICAgPC9kaXY+CiAgICAgIDxicj4KICAgICAgPGZpZWxkc2V0IGNsYXNzPSJtaW1lQXR0YWNo bWVudEhlYWRlciI+PC9maWVsZHNldD4KICAgICAgPGJyPgogICAgICA8cHJlIHdyYXA9IiI+ X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KRGV2ZWwg bWFpbGluZyBsaXN0CjxhIGNsYXNzPSJtb3otdHh0LWxpbmstYWJicmV2aWF0ZWQiIGhyZWY9 Im1haWx0bzpEZXZlbEBvdmlydC5vcmciPkRldmVsQG92aXJ0Lm9yZzwvYT4KPGEgY2xhc3M9 Im1vei10eHQtbGluay1mcmVldGV4dCIgaHJlZj0iaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9t YWlsbWFuL2xpc3RpbmZvL2RldmVsIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4v bGlzdGluZm8vZGV2ZWw8L2E+PC9wcmU+CiAgICA8L2Jsb2NrcXVvdGU+CiAgICA8cD48YnI+ CiAgICA8L3A+CiAgPC9ib2R5Pgo8L2h0bWw+Cg== --------------510D2C3E2A3E3638E7FA4B2B--
On Thu, Nov 23, 2017 at 5:30 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
Can your reproduce it? If fail with: Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use This means the proxy is already running, and someone is trying to start ti again. Systemd should not allow this, but we have seen this in the daemon few weeks ago. The owner of the code trying to start the proxy should check this.
*Link and headline ofto suspected patches: *
* build: Make resulting RPMs architecture-specific - https://gerrit.ovirt.org/#/c/84534/ <https://gerrit.ovirt.org/#/c/84534/> Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055>*
*Link to all logs:*
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/... <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/exported-artifacts/upgrade-from-release-suit-master-el7/test_logs/upgrade-from-release-suite-master/post-001_initialize_engine.py/lago-upgrade-from-release-suite-master-engine/_var_log/messages/*view*/>*
* (Relevant) error snippet from the log: <error> *from lago log:
Failed to start service 'ovirt-imageio-proxy
messages logs:
Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting Session 8 of user root. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:47 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: start request repeated too quickly for ovirt-imageio-proxy.service Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 23 07:30:48 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed.
*</error>*
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 23.11.2017 16:29, Dafna Ron wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
[...]
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/
The patch hasn't changed any code, so I *think* I'm innocent here. FWIW, I've noticed the following error in http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/consoleTe... ... basic-suit] RuntimeError: Failed to run reposync 3 times for repoid: centos-extras-el7, aborting. .... Not sure whether this indicates a real error condition. [...] -- Mit freundlichen Grüßen/Kind Regards Viktor Mihajlovski
There was an issue with the mirrors and I see the patch merged 7 hours ago so its possible it was a late alert. On 11/23/2017 03:58 PM, Viktor Mihajlovski wrote:
On 23.11.2017 16:29, Dafna Ron wrote:
Hi,
We have a failing on test 001_initialize_engine.test_initialize_engine.
This is failing with error Failed to start service 'ovirt-imageio-proxy
[...]
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/artifact/
The patch hasn't changed any code, so I *think* I'm innocent here.
FWIW, I've noticed the following error in http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4055/consoleTe... ... basic-suit] RuntimeError: Failed to run reposync 3 times for repoid: centos-extras-el7, aborting. .... Not sure whether this indicates a real error condition. [...]
participants (7)
-
Allon Mureinik -
Dafna Ron -
Gal Ben Haim -
Nir Soffer -
Viktor Mihajlovski -
Yaniv Kaul -
Yedidyah Bar David