On Wed, Nov 29, 2017 at 4:57 PM, Yedidyah Bar David <didi@redhat.com> wrote:
On Wed, Nov 29, 2017 at 3:56 PM, Dafna Ron <dron@redhat.com> wrote:
we had a failure on 002_bootstrap.verify_add_hosts but the error is on imageio
I looked at the host log that Nir added and I can only see that the address is in use which seems to be the same issue we have in initialize engine.
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4205/artifact/...
I cannot see anything in host-deploy. Didi, would we be able to see anything here?
Sorry, seems like my plugin is not enough. Will have a look. Now merged an updated plugin, should hopefully pass changequeue soon. Let's see what happens next time a service fails. Search engine-setup/host-deploy logs for 'tcp connections'.
Best regards,
On 11/30/2017 03:03 PM, Yedidyah Bar David wrote: thank you :) will update once we have another failure.
Thanks, Dafna
On 11/29/2017 11:03 AM, Yedidyah Bar David wrote:
On Wed, Nov 29, 2017 at 1:00 PM, Dafna Ron <dron@redhat.com> wrote:
this is the plugin info from steup log but I don't see anything more than we have seen except a timeout.
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/artifact/...
Didi, is there anywhere else I should look?
Sadly, as already replied, not yet. Hopefully next time...
On 11/29/2017 10:18 AM, Nir Soffer wrote:
Do we have more info from Didi's debug plugin now?
On Wed, Nov 29, 2017 at 12:07 PM Dafna Ron <dron@redhat.com> wrote:
Hi,
We have failed cq with ovirt-imageio failing to start on upgrade suite. I can still only see errors in the messages log.
I'm writing the reported patch but I don't think it has anything to do with this issue.
Link and headline of suspected patches:
restapi: Enable update to no default network provider of cluster - https://gerrit.ovirt.org/#/c/84814/
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/artifact/
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/testRepor...
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/artifact/...
(Relevant) error snippet from the log:
<error>
From messages log
Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: Started oVirt Engine. Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: Reloading. Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: Configuration file /usr/lib/systemd/system/ebtables.service is marked executable. Please remove executable permission bits. Proceeding anyway. Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt Engine Data Warehouse... Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: Started oVirt Engine Data Warehouse. Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: Reloading. Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: Configuration file /usr/lib/systemd/system/ebtables.service is marked executable. Please remove executable permission bits. Proceeding anyway. Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart. Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: Starting oVirt ImageIO Proxy... Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: Traceback (most recent call last): Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in <module> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: status = image_proxy.main(args, config) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line 21, in main Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: image_server.start(config) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, in start Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: WSGIRequestHandler) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__ Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.server_bind() Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", line 48, in server_bind Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: HTTPServer.server_bind(self) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: self.socket.bind(self.server_address) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in meth Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: return getattr(self._sock,name)(*args) Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service: main process exited, code=exited, status=1/FAILURE Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: Failed to start oVirt ImageIO Proxy. Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: Unit ovirt-imageio-proxy.service entered failed state. Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service failed. Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: ovirt-imageio-proxy.service holdoff time over, scheduling restart.
</error>
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Didi
-- Didi