It seems that there was one job that timed out while performing a cleanup [1], and ever since every build that tries to use VMs fail.

I've cleaned the faulty host.

I've enqueued the specific change that was dropped out of the cq to be tested again.

[1] http://jenkins.ovirt.org/job/ovirt-system-tests_he-basic-suite-master/50/console

On Mon, Oct 9, 2017 at 1:50 PM, Daniel Belenky <dbelenky@redhat.com> wrote:

It seems like a cleanup issue from the previous job that ran on that specific node. I'll try to figure out what happened there.

On Mon, Oct 9, 2017 at 1:16 PM, Dafna Ron <dron@redhat.com> wrote:

Hi,

We have a CQ failure with the error:

error: [Errno 98] Address already in use

Link to suspected patches: This failure is pointing to: https://gerrit.ovirt.org/#/c/81704/

Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3055/

Link to all logs:

http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3055/consoleFull
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3055/artifact/

(Relevant) error snippet from the log:

<error>

08:36:38 [upgrade-from-prevrelease-suit] @ Deploy oVirt environment: ERROR (in 0:00:00)
08:36:38 [upgrade-from-prevrelease-suit] Error occured, aborting
08:36:38 [upgrade-from-prevrelease-suit] Traceback (most recent call last):
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 362, in do_run
08:36:38 [upgrade-from-prevrelease-suit]     self.cli_plugins[args.ovirtverb].do_run(args)
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run
08:36:38 [upgrade-from-prevrelease-suit]     self._do_run(**vars(args))
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in wrapper
08:36:38 [upgrade-from-prevrelease-suit]     return func(*args, **kwargs)
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in wrapper
08:36:38 [upgrade-from-prevrelease-suit]     return func(*args, prefix=prefix, **kwargs)
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 166, in do_deploy
08:36:38 [upgrade-from-prevrelease-suit]     prefix.deploy()
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 636, in wrapper
08:36:38 [upgrade-from-prevrelease-suit]     return func(*args, **kwargs)
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/ovirtlago/reposetup.py", line 111, in wrapper
08:36:38 [upgrade-from-prevrelease-suit]     with utils.repo_server_context(args[0]):
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
08:36:38 [upgrade-from-prevrelease-suit]     return self.gen.next()
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line 100, in repo_server_context
08:36:38 [upgrade-from-prevrelease-suit]     root_dir=prefix.paths.internal_repo(),
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line 76, in _create_http_server
08:36:38 [upgrade-from-prevrelease-suit]     generate_request_handler(root_dir),
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__
08:36:38 [upgrade-from-prevrelease-suit]     self.server_bind()
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind
08:36:38 [upgrade-from-prevrelease-suit]     SocketServer.TCPServer.server_bind(self)
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind
08:36:38 [upgrade-from-prevrelease-suit]     self.socket.bind(self.server_address)
08:36:38 [upgrade-from-prevrelease-suit]   File "/usr/lib64/python2.7/socket.py", line 224, in meth
08:36:38 [upgrade-from-prevrelease-suit]     return getattr(self._sock,name)(*args)
08:36:38 [upgrade-from-prevrelease-suit] error: [Errno 98] Address already in use
08:36:38 [upgrade-from-prevrelease-suit] + res=1
08:36:38 [upgrade-from-prevrelease-suit] + cd -

</error>

thank you.
Dafna



-------- Forwarded Message --------

Subject:	[CQ]: 81704, 6 (ovirt-hosted-engine-setup) failed "ovirt-master" system tests
Date:	Mon, 9 Oct 2017 10:05:39 +0000 (UTC)
From:	oVirt Jenkins <jenkins@ovirt.org>
To:	infra@ovirt.org

Change 81704,6 (ovirt-hosted-engine-setup) is probably the reason behind recent
system test failures in the "ovirt-master" change queue and needs to be fixed.

This change had been removed from the testing queue. Artifacts build from this
change will not be released until it is fixed.

For further details about the change see:
https://gerrit.ovirt.org/#/c/81704/6

For failed test results see:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3055/
_______________________________________________
Infra mailing list
Infra@ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra

_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

--

DANIEL BELENKY

RHV DEVOPS

EMEA VIRTUALIZATION R&D

--

DANIEL BELENKY

RHV DEVOPS

EMEA VIRTUALIZATION R&D