[JIRA] (OVIRT-1712) OST jobs fails on "address already in use"
by eyal edri (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1509355640-17188-136
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1712?page=com.atlassian.jir... ]
eyal edri commented on OVIRT-1712:
----------------------------------
This is happening for me locally now, even with cleaning the networks and running lago destroy, what is the recommended cleanup action needed to resolve this?
> OST jobs fails on "address already in use"
> ------------------------------------------
>
> Key: OVIRT-1712
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1712
> Project: oVirt - virtualization made easy
> Issue Type: By-EMAIL
> Reporter: eyal edri
> Assignee: infra
>
> Evgheni,
> Was there any change recently to Lago slaves?
> On Fri, Oct 20, 2017 at 11:05 AM, Piotr Kliczewski <
> piotr.kliczewski(a)gmail.com> wrote:
> > I attempted to run manual OST twice and both failed with below issue.
> > Can someone take a look?
> >
> > Thanks,
> > Piotr
> >
> > 2017-10-20 07:59:12,485::log_utils.py::__exit__::607::ovirtlago.prefix:
> > :DEBUG::
> > File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 636,
> > in wrapper
> > return func(*args, **kwargs)
> > File "/usr/lib/python2.7/site-packages/ovirtlago/reposetup.py", line
> > 111, in wrapper
> > with utils.repo_server_context(args[0]):
> > File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
> > return self.gen.next()
> > File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line
> > 100, in repo_server_context
> > root_dir=prefix.paths.internal_repo(),
> > File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line 76,
> > in _create_http_server
> > generate_request_handler(root_dir),
> > File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__
> > self.server_bind()
> > File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind
> > SocketServer.TCPServer.server_bind(self)
> > File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind
> > self.socket.bind(self.server_address)
> > File "/usr/lib64/python2.7/socket.py", line 224, in meth
> > return getattr(self._sock,name)(*args)
> >
> > 2017-10-20 07:59:12,485::cmd.py::do_run::365::root::ERROR::Error
> > occured, aborting
> > Traceback (most recent call last):
> > File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 362, in
> > do_run
> > self.cli_plugins[args.ovirtverb].do_run(args)
> > File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line
> > 184, in do_run
> > self._do_run(**vars(args))
> > File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in
> > wrapper
> > return func(*args, **kwargs)
> > File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in
> > wrapper
> > return func(*args, prefix=prefix, **kwargs)
> > File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 166,
> > in do_deploy
> > prefix.deploy()
> > File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 636,
> > in wrapper
> > return func(*args, **kwargs)
> > File "/usr/lib/python2.7/site-packages/ovirtlago/reposetup.py", line
> > 111, in wrapper
> > with utils.repo_server_context(args[0]):
> > File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
> > return self.gen.next()
> > File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line
> > 100, in repo_server_context
> > root_dir=prefix.paths.internal_repo(),
> > File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line 76,
> > in _create_http_server
> > generate_request_handler(root_dir),
> > File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__
> > self.server_bind()
> > File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind
> > SocketServer.TCPServer.server_bind(self)
> > File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind
> > self.socket.bind(self.server_address)
> > File "/usr/lib64/python2.7/socket.py", line 224, in meth
> > return getattr(self._sock,name)(*args)
> > error: [Errno 98] Address already in use
> > _______________________________________________
> > Infra mailing list
> > Infra(a)ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/infra
> >
> >
> >
> --
> Eyal edri
> MANAGER
> RHV DevOps
> EMEA VIRTUALIZATION R&D
> Red Hat EMEA <https://www.redhat.com/>
> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
> phone: +972-9-7692018
> irc: eedri (on #tlv #rhev-dev #rhev-integ)
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100070)
------------=_1509355640-17188-136
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1712?page=com.atlassian.jir... ]</pre>
<h3>eyal edri commented on OVIRT-1712:</h3>
<p>This is happening for me locally now, even with cleaning the networks and running lago destroy, what is the recommended cleanup action needed to resolve this?</p>
<blockquote><h3>OST jobs fails on “address already in use”</h3>
<pre> Key: OVIRT-1712
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1712
Project: oVirt - virtualization made easy
Issue Type: By-EMAIL
Reporter: eyal edri
Assignee: infra</pre>
<p>Evgheni, Was there any change recently to Lago slaves? On Fri, Oct 20, 2017 at 11:05 AM, Piotr Kliczewski < piotr.kliczewski(a)gmail.com> wrote:</p>
<blockquote><p>I attempted to run manual OST twice and both failed with below issue. Can someone take a look?</p>
<p>Thanks, Piotr</p>
<p>2017-10-20 07:59:12,485::log_utils.py::__exit__::607::ovirtlago.prefix: :DEBUG::</p>
<pre>File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 636,</pre>
<p>in wrapper</p>
<pre> return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/ovirtlago/reposetup.py", line</pre>
<p>111, in wrapper</p>
<pre> with utils.repo_server_context(args[0]):
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line</pre>
<p>100, in repo_server_context</p>
<pre> root_dir=prefix.paths.internal_repo(),
File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line 76,</pre>
<p>in _create_http_server</p>
<pre> generate_request_handler(root_dir),
File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__
self.server_bind()
File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind
self.socket.bind(self.server_address)
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)</pre>
<p>2017-10-20 07:59:12,485::cmd.py::do_run::365::root::ERROR::Error occured, aborting Traceback (most recent call last):</p>
<pre>File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 362, in</pre>
<p>do_run</p>
<pre> self.cli_plugins[args.ovirtverb].do_run(args)
File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line</pre>
<p>184, in do_run</p>
<pre> self._do_run(**vars(args))
File "/usr/lib/python2.7/site-packages/lago/utils.py", line 501, in</pre>
<p>wrapper</p>
<pre> return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/lago/utils.py", line 512, in</pre>
<p>wrapper</p>
<pre> return func(*args, prefix=prefix, **kwargs)
File "/usr/lib/python2.7/site-packages/ovirtlago/cmd.py", line 166,</pre>
<p>in do_deploy</p>
<pre> prefix.deploy()
File "/usr/lib/python2.7/site-packages/lago/log_utils.py", line 636,</pre>
<p>in wrapper</p>
<pre> return func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/ovirtlago/reposetup.py", line</pre>
<p>111, in wrapper</p>
<pre> with utils.repo_server_context(args[0]):
File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line</pre>
<p>100, in repo_server_context</p>
<pre> root_dir=prefix.paths.internal_repo(),
File "/usr/lib/python2.7/site-packages/ovirtlago/utils.py", line 76,</pre>
<p>in _create_http_server</p>
<pre> generate_request_handler(root_dir),
File "/usr/lib64/python2.7/SocketServer.py", line 419, in __init__
self.server_bind()
File "/usr/lib64/python2.7/BaseHTTPServer.py", line 108, in server_bind
SocketServer.TCPServer.server_bind(self)
File "/usr/lib64/python2.7/SocketServer.py", line 430, in server_bind
self.socket.bind(self.server_address)
File "/usr/lib64/python2.7/socket.py", line 224, in meth
return getattr(self._sock,name)(*args)</pre>
<p>error: [Errno 98] Address already in use <em>____________________________________________</em>_ Infra mailing list Infra(a)ovirt.org <a href="http://lists.ovirt.org/mailman/listinfo/infra">http://lists.ovirt.org/mailman/listinfo/infra</a></p></blockquote>
<p>— Eyal edri MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <<a href="https://www.redhat.com/">https://www.redhat.com/</a>> <<a href="https://red.ht/sig">https://red.ht/sig</a>> TRIED. TESTED. TRUSTED. <<a href="https://redhat.com/trusted">https://redhat.com/trusted</a>> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100070)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYL..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1509355640-17188-136--
7 years
[JIRA] (OVIRT-1719) More VMs for Foreman
by eyal edri (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1509355388-27719-154
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1719?page=com.atlassian.jir... ]
eyal edri reassigned OVIRT-1719:
--------------------------------
Assignee: Evgheni Dereveanchin (was: infra)
> More VMs for Foreman
> --------------------
>
> Key: OVIRT-1719
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1719
> Project: oVirt - virtualization made easy
> Issue Type: Task
> Components: oVirt Infra
> Reporter: Greg Sutcliffe
> Assignee: Evgheni Dereveanchin
>
> You may have seen the fun with Rackspace this week - Rackspace first tried to withdraw all the OSS discounts it currently funds, and then backed down to just not allowing new participants.
> This makes me wary of our continuing reliance on Rackspace, so I'm looking for more hardware in other places. [~eedri] has offered more VMs for Foreman, but I know you guys are short on public IPs.
> Thus, could we get 1 more VM? specs:
> Network: same network config as https://ovirt-jira.atlassian.net/browse/OVIRT-1484
> Compute: 4Gb / 4CPU / 100Gb storage
> Keys: http://emeraldreverie.org/downloads/sshkey.pub
> With this, we can then migrate the Jenkins master and run it alongside the puppetmaster. I won't request anything more until I see what other sponsors I can get to take the slaves off Rackspace :)
> Thanks!
> Greg
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100070)
------------=_1509355388-27719-154
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1719?page=com.atlassian.jir... ]</pre>
<h3>eyal edri reassigned OVIRT-1719:</h3>
<pre>Assignee: Evgheni Dereveanchin (was: infra)</pre>
<blockquote><h3>More VMs for Foreman</h3>
<pre> Key: OVIRT-1719
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1719
Project: oVirt - virtualization made easy
Issue Type: Task
Components: oVirt Infra
Reporter: Greg Sutcliffe
Assignee: Evgheni Dereveanchin</pre>
<p>You may have seen the fun with Rackspace this week – Rackspace first tried to withdraw all the OSS discounts it currently funds, and then backed down to just not allowing new participants. This makes me wary of our continuing reliance on Rackspace, so I'm looking for more hardware in other places. [~eedri] has offered more VMs for Foreman, but I know you guys are short on public IPs. Thus, could we get 1 more VM? specs: Network: same network config as <a href="https://ovirt-jira.atlassian.net/browse/OVIRT-1484">https://ovirt-jira.atlassian.net/browse/OVIRT-1484</a> Compute: 4Gb / 4CPU / 100Gb storage Keys: <a href="http://emeraldreverie.org/downloads/sshkey.pub">http://emeraldreverie.org/downloads/sshkey.pub</a> With this, we can then migrate the Jenkins master and run it alongside the puppetmaster. I won't request anything more until I see what other sponsors I can get to take the slaves off Rackspace :) Thanks! Greg</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100070)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYL..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1509355388-27719-154--
7 years
oVirt infra daily report - unstable production jobs - 489
by jenkins＠jenkins.phx.ovirt.org
------=_Part_220_1791111259.1509318002020
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Good morning!
Attached is the HTML page with the jenkins status report. You can see it also here:
- http://jenkins.ovirt.org/job/system_jenkins-report/489//artifact/exported...
Cheers,
Jenkins
------=_Part_220_1791111259.1509318002020
Content-Type: text/html; charset=us-ascii; name=upstream_report.html
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment; filename=upstream_report.html
Content-ID: <upstream_report.html>
<!DOCTYPE html><head><style type="text/css">
table.gridtable {
border-collapse: collapse;
table-layout:fixed;
width:1600px;
font-family: monospace;
font-size:13px;
}
.head {
font-size:20px;
font-family: arial;
}
.sub {
font-size:18px;
background-color:#e5e5e5;
font-family: arial;
}
pre {
font-family: monospace;
display: inline;
white-space: pre-wrap;
white-space: -moz-pre-wrap !important;
white-space: -pre-wrap;
white-space: -o-pre-wrap;
word-wrap: break-word;
}
</style>
</head>
<body>
<table class="gridtable" border=2>
<tr><th colspan=2 class=head>
RHEVM CI Jenkins Daily Report - 29/10/2017
</th></tr><tr><th colspan=2 class=sub>
<font color="blue"><a href="http://jenkins.ovirt.org/">00 Unstable Critical</a></font>
</th></tr>
<tr><td>
<a href="http://jenkins.ovirt.org/job/ovirt-system-tests_ansible-suite-master/">ovirt-system-tests_ansible-suite-master</a>
</td><td>
This job is automatically updated by jenkins job builder, any manual
change will be lost in the next update. If you want to make permanent
changes, check out the <a href="http://gerrit.ovirt.org/gitweb?p=jenkins.git;a=tree;h=refs/heads/master;h...">
jenkins</a> repo.
<!-- Managed by Jenkins Job Builder -->
</td></tr>
------=_Part_220_1791111259.1509318002020--
7 years
[JIRA] (OVIRT-1717) When a component build passes OST, tag its git hash
by eyal edri (oVirt JIRA)
This is a multi-part message in MIME format...
------------=_1509291601-21586-158
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
[ https://ovirt-jira.atlassian.net/browse/OVIRT-1717?page=com.atlassian.jir... ]
eyal edri commented on OVIRT-1717:
----------------------------------
This requires keeping a hash/mapping of each project rpm to a source repo, maintaining permissions for a specific user to tag any project ( and keeping track on new projects,etc.. ), and to what purpose?
I'm not sure what is the reason behind this request, but won't it be irrelevant once we move to patch gating? ( which should happen in the near future ), which means any merged patch is basically passing OST and tested?
> When a component build passes OST, tag its git hash
> ---------------------------------------------------
>
> Key: OVIRT-1717
> URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1717
> Project: oVirt - virtualization made easy
> Issue Type: New Feature
> Components: oVirt CI
> Reporter: danken
> Assignee: infra
> Labels: change-queue, standard-ci
>
> Tagging the source git would make it easier for me to tell if a specific git commit is already tested by OST and available for QA.
--
This message was sent by Atlassian Jira
(v1001.0.0-SNAPSHOT#100070)
------------=_1509291601-21586-158
Content-Type: text/html; charset="UTF-8"
Content-Disposition: inline
Content-Transfer-Encoding: 7bit
<html><body>
<pre>[ https://ovirt-jira.atlassian.net/browse/OVIRT-1717?page=com.atlassian.jir... ]</pre>
<h3>eyal edri commented on OVIRT-1717:</h3>
<p>This requires keeping a hash/mapping of each project rpm to a source repo, maintaining permissions for a specific user to tag any project ( and keeping track on new projects,etc.. ), and to what purpose?</p>
<p>I'm not sure what is the reason behind this request, but won't it be irrelevant once we move to patch gating? ( which should happen in the near future ), which means any merged patch is basically passing OST and tested?</p>
<blockquote><h3>When a component build passes OST, tag its git hash</h3>
<pre> Key: OVIRT-1717
URL: https://ovirt-jira.atlassian.net/browse/OVIRT-1717
Project: oVirt - virtualization made easy
Issue Type: New Feature
Components: oVirt CI
Reporter: danken
Assignee: infra
Labels: change-queue, standard-ci</pre>
<p>Tagging the source git would make it easier for me to tell if a specific git commit is already tested by OST and available for QA.</p></blockquote>
<p>— This message was sent by Atlassian Jira (v1001.0.0-SNAPSHOT#100070)</p>
<img src="https://u4043402.ct.sendgrid.net/wf/open?upn=i5TMWGV99amJbNxJpSp2-2BCmpYL..." alt="" width="1" height="1" border="0" style="height:1px !important;width:1px !important;border-width:0 !important;margin-top:0 !important;margin-bottom:0 !important;margin-right:0 !important;margin-left:0 !important;padding-top:0 !important;padding-bottom:0 !important;padding-right:0 !important;padding-left:0 !important;"/>
</body></html>
------------=_1509291601-21586-158--
7 years
[ OST Failure Report ] [ oVirt master ] [ 29-10-2017 ] [ 098_ovirt_provider_ovn.test_ovn_provider_rest ]
by Shlomo Ben David
Hi,
Link to suspected patches: https://gerrit.ovirt.org/#/c/83158/
Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3525/console
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/3525/artifa...
(Relevant) error snippet from the log:
<error>
File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
testMethod()
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129,
in wrapped_test
test()
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in
wrapper
return func(get_test_prefix(), *args, **kwargs)
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in
wrapper
prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in
wrapper
return func(get_test_prefix(), *args, **kwargs)
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/098_ovirt_provider_ovn.py",
line 441, in test_ovn_provider_rest
_remove_iface_from_vm(api, VM0_NAME, IFACE_NAME)
File
"/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/098_ovirt_provider_ovn.py",
line 400, in _remove_iface_from_vm
nic_service.remove()
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line
27808, in remove
self._internal_remove(headers, query, wait)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 262,
in _internal_remove
return future.wait() if wait else future
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 53,
in wait
return self._code(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 259,
in callback
self._check_fault(response)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 123,
in _check_fault
self._raise_error(response, body)
File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 109,
in _raise_error
raise error
'Fault reason is "Operation Failed". Fault detail is "[Cannot remove
Interface. The VM Network Interface is plugged to a running VM.]".
</error>
Best Regards,
Shlomi Ben-David | Software Engineer | Red Hat ISRAEL
RHCSA | RHCE | RHCVA
IRC: shlomibendavid (on #rhev-integ, #rhev-dev, #rhev-ci)
OPEN SOURCE - 1 4 011 && 011 4 1
7 years
