ovirt master system tests fail
by Daniel Belenky
Hi all,
test-repo ovirt experimental master job fails, and it seems that there is
an issue with 'add_host' phase under the '*bootstrap*' suite.
>From the logs, it seems that the suite was unable to fire up the host /
something is wrong with host
<error type="exceptions.RuntimeError" message="Host
lago-basic-suite-master-host1 is in non operational state
-------------------- >> begin captured logging << --------------------
lago.ssh: DEBUG: start task Get ssh client for
lago-basic-suite-master-host0 lago.ssh: DEBUG: Still got 100 tries for
lago-basic-suite-master-host0 lago.ssh: DEBUG: end task Get ssh client for
lago-basic-suite-master-host0 lago.ssh: DEBUG: Running aab0eff8 on
lago-basic-suite-master-host0: yum install -y iptables lago.ssh: DEBUG:
Command aab0eff8 on lago-basic-suite-master-host0 returned with 0 lago.ssh:
DEBUG: Command aab0eff8 on lago-basic-suite-master-host0 output: Loaded
plugins: fastestmirror Loading mirror speeds from cached hostfile * base:
centos.host-engine.com * extras: linux.mirrors.es.net * updates:
mirror.n5tech.com Package iptables-1.4.21-17.el7.x86_64 already installed
and latest version Nothing to do lago.ssh: DEBUG: start task Get ssh client
for lago-basic-suite-master-host1 lago.ssh: DEBUG: Still got 100 tries for
lago-basic-suite-master-host1 lago.ssh: DEBUG: end task Get ssh client for
lago-basic-suite-master-host1 lago.ssh: DEBUG: Running ab5c94f2 on
lago-basic-suite-master-host1: yum install -y iptables lago.ssh: DEBUG:
Command ab5c94f2 on lago-basic-suite-master-host1 returned with 0 lago.ssh:
DEBUG: Command ab5c94f2 on lago-basic-suite-master-host1 output: Loaded
plugins: fastestmirror Loading mirror speeds from cached hostfile * base:
mirror.n5tech.com * extras: ftp.osuosl.org * updates: mirrors.usc.edu
Package iptables-1.4.21-17.el7.x86_64 already installed and latest version
Nothing to do ovirtlago.testlib: ERROR: * Unhandled exception in <function
_host_is_up at 0x322e938> Traceback (most recent call last): File
"/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 217, in
assert_equals_within res = func() File
"/home/jenkins/workspace/test-repo_ovirt_experimental_master/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py",
line 162, in _host_is_up raise RuntimeError('Host %s is in non operational
state' % host.name()) RuntimeError: Host lago-basic-suite-master-host1 is
in non operational state --------------------- >> end captured logging <<
---------------------">
>From the engine.log, I found a timeout in the rpc call (but this error is
seen on jobs that success too, so might not be relevant(?))
2017-01-12 05:49:53,383-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand]
(org.ovirt.thread.pool-7-thread-2) [76b0383f] Command
'PollVDSCommand(HostName = lago-basic-suite-master-host1,
VdsIdVDSCommandParametersBase:{runAsync='true',
hostId='40eb11ba-e6ac-478a-b8b1-73b7892ace65'})' execution failed:
VDSGenericException: VDSNetworkException: Timeout during rpc call
2017-01-12 05:49:53,383-05 DEBUG
[org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand]
(org.ovirt.thread.pool-7-thread-2) [76b0383f] Exception:
org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException:
VDSGenericException: VDSNetworkException: Timeout during rpc call
... (the full error is very long, so I wont paste it here, its in the*
engine.log*)
2017-01-12 05:49:58,291-05 ERROR
[org.ovirt.engine.core.vdsbroker.vdsbroker.PollVDSCommand]
(org.ovirt.thread.pool-7-thread-1) [30b2ca77] Timeout waiting for VDSM
response: Internal timeout occured
In the host's vdsm.log, there are some errors too:
2017-01-12 05:51:48,336 ERROR (jsonrpc/0) [storage.StorageDomainCache]
looking for unfetched domain 380623d8-1e85-4831-9048-3d05932f3d3a
(sdc:151)
2017-01-12 05:51:48,336 ERROR (jsonrpc/0) [storage.StorageDomainCache]
looking for domain 380623d8-1e85-4831-9048-3d05932f3d3a (sdc:168)
2017-01-12 05:51:48,395 WARN (jsonrpc/0) [storage.LVM] lvm vgs
failed: 5 [] [' WARNING: Not using lvmetad because config setting
use_lvmetad=0.', ' WARNING: To avoid corruption, rescan devices to
make changes visible (pvscan --cache).', ' Volume group
"380623d8-1e85-4831-9048-3d05932f3d3a" not found', ' Cannot process
volume group 380623d8-1e85-4831-9048-3d05932f3d3a'] (lvm:377)
2017-01-12 05:51:48,398 ERROR (jsonrpc/0) [storage.StorageDomainCache]
domain 380623d8-1e85-4831-9048-3d05932f3d3a not found (sdc:157)
Traceback (most recent call last):
File "/usr/share/vdsm/storage/sdc.py", line 155, in _findDomain
dom = findMethod(sdUUID)
File "/usr/share/vdsm/storage/sdc.py", line 185, in _findUnfetchedDomain
raise se.StorageDomainDoesNotExist(sdUUID)
StorageDomainDoesNotExist: Storage domain does not exist:
(u'380623d8-1e85-4831-9048-3d05932f3d3a',)
and
2017-01-12 05:53:45,375 ERROR (JsonRpc (StompReactor))
[vds.dispatcher] SSL error receiving from
<yajsonrpc.betterAsyncore.Dispatcher connected ('::1', 43814, 0, 0) at
0x235a2d8>: unexpected eof (betterAsyncore:119)
Link to Jenkins
<http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_exp...>
Can someone please take a look?
Thanks,
*Daniel Belenky*
*RHV DevOps*
*Red Hat Israel*
7 years, 3 months
Introduction of cluster level 4.2 on master
by Martin Perina
Hi,
we have just merged a series of patches to engine which introduces cluster
level 4.2:
1. Adds cluster level 4.2 [1]
2. Sets version 4.2 to default data center and cluster for new
installations [1]
2. Bumps database upgrade scripts to 4.2 [2] - from now on every database
script merged to master has to start with 04_02 prefix
Please rebase all your existing patches to master on top of those changes
and rename db scripts accordingly.
Also please be aware that if you want to add a host to a 4.2 cluster, you
need quite recent VDSM version which includes patch [3] merged yesterday.
Please let me know if you have any issues.
Thanks
Martin Perina
[1] https://gerrit.ovirt.org/69757
[2] https://gerrit.ovirt.org/69758
[3] https://gerrit.ovirt.org/69759
7 years, 3 months
[oVirt 4.1 Localization Question #1] "No VM configuration"
by Yuko Katabami
Hi oVirt developers,
We have started translating 4.1 UI strings and would like to post our first
question.
File: CommonApplicationConstants
Resource ID: brokenVmConfiguration
String: No VM configuration
Question: Could anyone tell me where in the UI it is located? Is this used
as a part of an error message, or as a label?
Kind regards,
Yuko
7 years, 3 months
ovirt-system-tests 4.1 is failling due to VmDisksResource class
by Daniel Belenky
Hi all,
ovirt-system-tests are failing with the following error:
ERROR [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-1)
[] Can't find relative path for class
"org.ovirt.engine.api.resource.VmDisksResource", will return null
The error began at 4/1.
can someone take a look please?
Attached all the logs under /var/log/ovirt-engine , the error I've
mentioned above is seen in *engine.log*.
Can someone please take a look?
Thanks,
--
*Daniel Belenky*
*RHV DevOps*
*Red Hat Israel*
7 years, 3 months
(no subject)
by Daniel Belenky
On Thu, Jan 12, 2017 at 10:35 AM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
>
>
> On Thu, Jan 12, 2017 at 10:25 AM, Daniel Belenky <dbelenky(a)redhat.com>
> wrote:
>
>> Hi all,
>>
>> ovirt-system-tests are failing with the following error:
>>
>> ERROR [org.ovirt.engine.api.restapi.util.LinkHelper] (default task-1) [] Can't find relative path for class "org.ovirt.engine.api.resource.VmDisksResource", will return null
>>
>>
> This is a known issue ( https://bugzilla.redhat.com/
> show_bug.cgi?id=1410038 ) and is not a cause for failure.
>
>
>> The error began at 4/1.
>>
>
> This is really outdated. How did work yesterday?
>
It didn't. We just didn't get to this issue, but it's failing for a while.
>
>> can someone take a look please?
>>
>
> What is the test that is failing?
>
basic_sanity is failing on vm_run
Link
<http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt_4.1_system...>
to the failing job
Thanks,
*Daniel Belenky*
*RHV DevOps*
*Red Hat Israel*
7 years, 3 months
planned Jenkins restart
by Evgheni Dereveanchin
Hi everyone,
I'll be performing a planned Jenkins restart today.
No new builds will be scheduled during this maintenance period.
I will announce you once it is finished.
Regards,
Evgheni Dereveanchin
7 years, 3 months
PatternFly upgrade - how to handle JS dependencies
by Vojtech Szocs
Hello,
we'd like to upgrade the version of PatternFly (plus associated libraries,
namely Bootstrap and jQuery) used in oVirt UI.
Today, PatternFly stuff (PF + associated libraries) comes from `patternfly1`
package hosted at Copr repo:
https://copr.fedorainfracloud.org/coprs/patternfly/patternfly1/
Today, we require `patternfly1` as both Engine RPM build dependency and the
Engine devel. env. dependency.
To keep things simple, I'd like to propose the following approach:
- create oVirt specific package, e.g. `ovirt-patternfly`, hosted at Copr,
containing PatternFly + associated libraries (Bootstrap, jQuery) which
are intended *specifically* for use by oVirt UI (hence the ovirt prefix)
- discontinue maintenance of `patternfly1` package at Copr
- keep the existing approach: require `ovirt-patternfly` as both the Engine
build dependency and the Engine devel. env. dependency
An alternative approach would be to introduce Node.js as Engine devel. env.
dependency (use `npm install` to pull PF stuff), while using the existing
ovirt-engine-{nodejs,nodejs-modules} packages for Engine RPM build.
However, this alternative has some downsides, namely complication of devel.
env. just to fetch the PF stuff. I don't like the idea of complicating the
existing devel. env. just for the sake of fetching some 3rd party libs.
I'm wondering what others think about this.
Regards,
Vojtech
7 years, 3 months
Fwd: [oVirt Jenkins] test-repo_ovirt_experimental_master - Build #4658 - FAILURE!
by Piotr Kliczewski
Nir,
I saw similar issue on when I run system tests on my machine. Here is
piece of log::
2017-01-10 09:44:56,442 WARN (jsonrpc/3) [storage.HSM] getPV failed
for guid: 3600140527d13be6be704ff98fd8838c2 (hsm:1973)
Traceback (most recent call last):
File "/usr/share/vdsm/storage/hsm.py", line 1970, in _getDeviceList
pv = lvm.getPV(guid)
File "/usr/share/vdsm/storage/lvm.py", line 856, in getPV
raise se.InaccessiblePhysDev((pvName,))
InaccessiblePhysDev: Multipath cannot access physical device(s):
"devices=(u'3600140527d13be6be704ff98fd8838c2',)"
2017-01-10 09:44:57,493 ERROR (jsonrpc/4) [storage.StorageDomainCache]
domain d67c721e-d1f2-484f-a647-13a82db90b33 not found (sdc:157)
Traceback (most recent call last):
File "/usr/share/vdsm/storage/sdc.py", line 155, in _findDomain
dom = findMethod(sdUUID)
File "/usr/share/vdsm/storage/sdc.py", line 185, in _findUnfetchedDomain
raise se.StorageDomainDoesNotExist(sdUUID)
StorageDomainDoesNotExist: Storage domain does not exist:
(u'd67c721e-d1f2-484f-a647-13a82db90b33',)
Can you please take a look why are we seeing it?
Thanks,
Piotr
---------- Forwarded message ----------
From: <jenkins(a)jenkins.phx.ovirt.org>
Date: Tue, Jan 10, 2017 at 4:08 PM
Subject: [oVirt Jenkins] test-repo_ovirt_experimental_master - Build
#4658 - FAILURE!
To: infra(a)ovirt.org
Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/4658/,
Build Number: 4658,
Build Status: FAILURE
_______________________________________________
Infra mailing list
Infra(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/infra
7 years, 3 months
suspend_resume_vm fail on master experimental
by Daniel Belenky
Hi all,
*test-repo_ovirt_experimental_master* (link to Jenkins
<http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_exp...>)
job failed on basic_sanity scenario.
The job was triggered by https://gerrit.ovirt.org/#/c/69845/
>From looking at the logs, it seems that the reason is *VDSM*.
In the VDSM log, i see the following error:
2017-01-09 16:47:41,331 ERROR (JsonRpc (StompReactor))
[vds.dispatcher] SSL error receiving from
<yajsonrpc.betterAsyncore.Dispatcher connected ('::1', 34942, 0, 0) at
0x36b95f0>: unexpected eof (betterAsyncore:119)
Also, when looking at the MOM logs, I see the the following:
2017-01-09 16:43:39,508 - mom.vdsmInterface - ERROR - Cannot connect
to VDSM! [Errno 111] Connection refused
I've attached the full VDSM logs here in a zip file.
Can anyone please assist?
Thanks,
--
*Daniel Belenky*
*RHV DevOps*
*Red Hat Israel*
7 years, 3 months
oVirt 4.1 RC1 merge / branch / tag / bugzilla reminder
by Lev Veyde
All stable branch maintainers, please make sure to
merge all relevant open bugs until Monday morning 11:00 AM TLV time (10:00 AM CET).
For each package that need to be built (i.e oVirt product) please make sure every bug in MODIFIED has the right Target Release and Target Milestone.
A Target release should state the version of the package you're building and should include the same version you used for the tag you just used for this build. (e.g. for ovirt-engine, tag: ovirt-engine-4.1.0, tr: 4.1.0)
A list of bugs that require attention is here:
https://bugzilla.redhat.com/buglist.cgi?quicksearch=target_milestone%3A4....
Thanks in advance,
Lev Veyde.
7 years, 3 months
