Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ 20-11-2017 ] [ 001_initialize_engine.test_initialize_engine ]

On Wed, Nov 29, 2017 at 3:56 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: > > we had a failure on 002_bootstrap.verify_add_hosts but the error is on > imageio > > I looked at the host log that Nir added and I can only see that the > address is in use which seems to be the same issue we have in initialize > engine. > > > http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4205/artifa... > > I cannot see anything in host-deploy. > Didi, would we be able to see anything here? Sorry, seems like my plugin is not enough. Will have a look.

> > > Thanks, > Dafna > > > > On 11/29/2017 11:03 AM, Yedidyah Bar David wrote: > > On Wed, Nov 29, 2017 at 1:00 PM, Dafna Ron <dron(a)redhat.com&gt; wrote: >> >> this is the plugin info from steup log but I don't see anything more than >> we have seen except a timeout. >> >> https://pastebin.com/QVtNRNWV >> >> >> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/artifa... >> >> Didi, is there anywhere else I should look? > > > Sadly, as already replied, not yet. Hopefully next time... > >> >> >> >> On 11/29/2017 10:18 AM, Nir Soffer wrote: >> >> Do we have more info from Didi's debug plugin now? >> >> On Wed, Nov 29, 2017 at 12:07 PM Dafna Ron <dron(a)redhat.com&gt; wrote: >>> >>> Hi, >>> >>> We have failed cq with ovirt-imageio failing to start on upgrade suite. >>> I can still only see errors in the messages log. >>> >>> I'm writing the reported patch but I don't think it has anything to do >>> with this issue. >>> >>> Link and headline of suspected patches: >>> >>> restapi: Enable update to no default network provider of cluster - >>> https://gerrit.ovirt.org/#/c/84814/ >>> >>> Link to Job: >>> >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/ >>> >>> Link to all logs: >>> >>> >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/artifact/ >>> >>> >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/testRe... >>> >>> >>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4194/artifa... >>> >>> (Relevant) error snippet from the log: >>> >>> <error> >>> >>> From messages log >>> >>> >>> Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: >>> Started oVirt Engine. >>> Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: >>> Reloading. >>> Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: >>> Configuration file /usr/lib/systemd/system/ebtables.service is marked >>> executable. Please remove executable permission bits. Proceeding anyway. >>> Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: >>> Starting oVirt Engine Data Warehouse... >>> Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: >>> Started oVirt Engine Data Warehouse. >>> Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: >>> Reloading. >>> Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: >>> Configuration file /usr/lib/systemd/system/ebtables.service is marked >>> executable. Please remove executable permission bits. Proceeding anyway. >>> Nov 28 12:32:13 lago-upgrade-from-release-suite-master-engine systemd: >>> Starting oVirt ImageIO Proxy... >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: Traceback (most recent call last): >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in >>> <module> >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: status = image_proxy.main(args, config) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File >>> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line >>> 21, in main >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: image_server.start(config) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File >>> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, >>> in start >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: WSGIRequestHandler) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, >>> in __init__ >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: self.server_bind() >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", >>> line 48, in server_bind >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: HTTPServer.server_bind(self) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line >>> 108, in server_bind >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, >>> in server_bind >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: self.socket.bind(self.server_address) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in >>> meth >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: return getattr(self._sock,name)(*args) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> ovirt-imageio-proxy.service: main process exited, code=exited, >>> status=1/FAILURE >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> Failed to start oVirt ImageIO Proxy. >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> Unit ovirt-imageio-proxy.service entered failed state. >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> ovirt-imageio-proxy.service failed. >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> ovirt-imageio-proxy.service holdoff time over, scheduling restart. >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> Starting oVirt ImageIO Proxy... >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: Traceback (most recent call last): >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/bin/ovirt-imageio-proxy", line 85, in >>> <module> >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: status = image_proxy.main(args, config) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File >>> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/image_proxy.py", line >>> 21, in main >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: image_server.start(config) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File >>> "/usr/lib/python2.7/site-packages/ovirt_imageio_proxy/server.py", line 45, >>> in start >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: WSGIRequestHandler) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 419, >>> in __init__ >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: self.server_bind() >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/wsgiref/simple_server.py", >>> line 48, in server_bind >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: HTTPServer.server_bind(self) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/BaseHTTPServer.py", line >>> 108, in server_bind >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: SocketServer.TCPServer.server_bind(self) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/SocketServer.py", line 430, >>> in server_bind >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: self.socket.bind(self.server_address) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: File "/usr/lib64/python2.7/socket.py", line 224, in >>> meth >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: return getattr(self._sock,name)(*args) >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine >>> ovirt-imageio-proxy: socket.error: [Errno 98] Address already in use >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> ovirt-imageio-proxy.service: main process exited, code=exited, >>> status=1/FAILURE >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> Failed to start oVirt ImageIO Proxy. >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> Unit ovirt-imageio-proxy.service entered failed state. >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> ovirt-imageio-proxy.service failed. >>> Nov 28 12:32:14 lago-upgrade-from-release-suite-master-engine systemd: >>> ovirt-imageio-proxy.service holdoff time over, scheduling restart. >>> >>> </error> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> Devel mailing list >>> Devel(a)ovirt.org >>> http://lists.ovirt.org/mailman/listinfo/devel >> >> > > > > -- > Didi > > -- Didi