ovirt-engine fails to build on Fedora 27
by Sandro Bonazzola
Hi,
can you please have a look at
http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-fc27-x86...
?
engine seems to fail building the rpm after the maven part completed,
during the install process at:
15:53:23 ++ echo ovirt-engine-aaa.pom
15:53:23 ++ sed -e 's/^ovirt-engine-//' -e 's/\.pom//'
15:53:23 + artifact_id=aaa
15:53:23 + '[' -f
/home/jenkins/workspace/ovirt-engine_master_build-artifacts-fc27-x86_64/ovirt-engine/rpmbuild/BUILDROOT/ovirt-engine-4.2.1.5-0.0.master.20180202142920.git0bc7f3edd2.fc27.x86_64/usr/share/java/ovirt-engine/aaa.jar
']'
15:53:23 + %add_maven_depmap -f backend JPP.ovirt-engine-aaa.pom
ovirt-engine/aaa.jar
15:53:23 /var/tmp/rpm-tmp.YIGxWv: line 161: fg: no job control
15:53:23 error: Bad exit status from /var/tmp/rpm-tmp.YIGxWv (%install)
Maybe some broken dep on backend around aaa.jar ?
--
SANDRO BONAZZOLA
ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig>
TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
6 years, 2 months
OST failures weekly update [28/01/2018-02/02/2018]
by Dafna Ron
Hello,
I would like to summarize the reported OST failures from the previous week.
01-02-2018
Test name: 001_upgrade_engine.test_initialize_engine
version: Ovirt master
Project otopi
Suite: master/upgrade
Cause: *https://gerrit.ovirt.org/#/c/86679/
<https://gerrit.ovirt.org/#/c/86679/> - core: Check Sequence before/after*
Fix:
*ON GOING ISSUE*
Test Name: 002_bootstrap.verify_add_all_hosts
Version: Ovirt Master
Project: vdsm
Suite: basic
Cause: *https://gerrit.ovirt.org/#/c/69668/
<https://gerrit.ovirt.org/#/c/69668/> - **gluster: Fix error when brick is
on a btrfs subvolume*
*Fix: No Fix updated*
*02*-02-2018
Test Name:
005_network_by_label.assign_hosts_network_label
Version: Master
Suite: master/upgrade
Project: ovirt-image-uploader
Cause: host object locked during api request
Fix: no fix for failure. re-triggered patch
Other incidents:
* We have several build-artifacts failing and this is causing change queue
to not run - the failure is caused due to clean-up on jobs which will
remove fc24
* We are still seeing CPU Unknown errors on the 4.2 branch
* there was a jenkins restart this week which cause a few jobs to fail
* Jenkins restart may also cause some failures on hosts due to uncompleted
clean-up - we are fixing any host found with this issue
Thanks,
Dafna
6 years, 2 months
[ OST Failure Report ] [ oVirt Master (ovirt-image-uploader) ] [ 02-02-2018 ] [005_network_by_label.assign_hosts_network_label ]
by Dafna Ron
Hi,
We failed test 005_network_by_label.assign_hosts_network_label for basic
suite in project ovirt-image-uploader.
The issue seems to be related to object lock on host and patch reported is
not related to the issue.
*Link and headline of suspected patches: Patch not related at all to error*
*Link to
Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5243/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5243/>Link
to all
logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5243/a...
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5243/artifact/>(Relevant)
error snippet from the log: <error><JsonRpcRequest id:
"4e9d7060-c523-4e66-9ace-dc8733954404", method: Host.getCapabilities,
params: {}>2018-02-02 08:11:40,131-05 INFO
[org.ovirt.engine.core.bll.network.host.LabelNicCommand] (default task-5)
[099e3ef6-4372-42a9-a360-1351ec66a64b] Running command: LabelNicCommand
internal: false. Entities affected : ID:
21b0c975-26f8-47a4-b5d0-cad0d41a9133 Type: VDSAction group
CONFIGURE_HOST_NETWORK with role type ADMIN2018-02-02 08:11:40,139-05 INFO
[org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default
task-5) [5c5509c0] Failed to Acquire Lock to object
'EngineLock:{exclusiveLocks='[HOST_NETWORK21b0c975-26f8-47a4-b5d0-cad0d41a9133=HOST_NETWORK]',
sharedLocks=''}'2018-02-02 08:11:40,139-05 WARN
[org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default
task-5) [5c5509c0] Validation of action 'HostSetupNetworks' failed for user
admin@internal-authz. Reasons:
VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,ACTION_TYPE_FAILED_SETUP_NETWORKS_OR_REFRESH_IN_PROGRESS2018-02-02
08:11:40,151-05 DEBUG
[org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
(default task-5) [5c5509c0] Compiled stored procedure. Call string is
[{call get_entity_snapshot_by_command_id(?)}]2018-02-02 08:11:40,151-05
DEBUG
[org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall]
(default task-5) [5c5509c0] SqlCall for procedure
[get_entity_snapshot_by_command_id] compiled2018-02-02 08:11:40,161-05
ERROR [org.ovirt.engine.core.bll.network.host.LabelNicCommand] (default
task-5) [5c5509c0] Transaction rolled-back for command
'org.ovirt.engine.core.bll.network.host.LabelNicCommand'.2018-02-02
08:11:40,165-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(default task-5) [5c5509c0] method: get, params:
[21b0c975-26f8-47a4-b5d0-cad0d41a9133], timeElapsed: 3ms2018-02-02
08:11:40,168-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(default task-5) [5c5509c0] EVENT_ID: LABEL_NIC_FAILED(1,137), Failed to
label network interface card eth0 with label NETWORK_LABEL on host
lago-basic-suite-master-host-1.2018-02-02 08:11:40,169-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(default task-5) [5c5509c0] method: runAction, params: [LabelNic,
LabelNicParameters:{commandId='aa8eb85e-8a30-4045-83f2-8f0516ec13c9',
user='null', commandType='Unknown'}], timeElapsed: 54ms2018-02-02
08:11:40,175-05 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
task-5) [] Operation Failed: [Cannot add Label. Another Setup Networks or
Host Refresh process in progress on the host. Please try later.]2018-02-02
08:11:40,230-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (SSL Stomp
Reactor) [27fdfc30]
MESSAGEcontent-length:10956destination:jms.topic.vdsm_responsescontent-type:application/jsonsubscription:37ef61dd-f4d0-4067-b563-1e2c04f2d2b8</error>*
6 years, 2 months
[ OST Failure Report ] [ oVirtMaster (otopi) ] [ 01-02-2018 ] [ 001_initialize_engine.initialize_engine/001_upgrade_engine.test_initialize_engine ]
by Dafna Ron
Hi,
We are failing initialize engine on both basic and upgrade suites.
Can you please check?
*Link and headline of suspected patches:
https://gerrit.ovirt.org/#/c/86679/ <https://gerrit.ovirt.org/#/c/86679/> -
*
*core: Check Sequence before/afterLink to
Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5187/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5187/>Link
to all
logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5187/a...
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5187/artifact/>(Relevant)
error snippet from the log: <error>2018-02-01 10:38:27,057-0500 DEBUG
otopi.plugins.otopi.dialog.human dialog.__logString:204
DIALOG:SEND Version: otopi-1.7.7_master
(otopi-1.7.7-0.0.master.20180201063428.git81ce9b7.el7.centos)2018-02-01
10:38:27,058-0500 ERROR otopi.context context.check:833 "before" parameter
of method
otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn.Plugin._misc_configure_provider
is a string, should probably be a tuple. Perhaps a missing comma?2018-02-01
10:38:27,058-0500 DEBUG otopi.plugins.otopi.dialog.human
dialog.__logString:204 DIALOG:SEND methodinfo: {'priority':
5000, 'name': None, 'before': 'osetup.ovn.provider.service.restart',
'after': ('osetup.pki.ca <http://osetup.pki.ca>.available',
'osetup.ovn.services.restart'), 'method': <bound method
?._misc_configure_provider of
<otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn.Plugin
object at 0x2edf6d0>>, 'condition': <bound method ?.<lambda> of
<otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn.Plugin
object at 0x2edf6d0>>, 'stage': 11}2018-02-01 10:38:27,059-0500 DEBUG
otopi.context context._executeMethod:143 method exceptionTraceback (most
recent call last): File
"/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in
_executeMethod method['method']() File
"/usr/share/otopi/plugins/otopi/core/misc.py", line 61, in _setup
self.context.checkSequence() File
"/usr/lib/python2.7/site-packages/otopi/context.py", line 844, in
checkSequence raise RuntimeError(_('Found bad "before" or "after"
parameters'))RuntimeError: Found bad "before" or "after"
parameters2018-02-01 10:38:27,059-0500 ERROR otopi.context
context._executeMethod:152 Failed to execute stage 'Environment setup':
Found bad "before" or "after" parameters</error>*
6 years, 2 months
[ OST Failure Report ] [ oVirt Master (vdsm) ] [ 01-02-2018 ] [ 002_bootstrap.verify_add_all_hosts ]
by Dafna Ron
Hi,
We failed cq test 002_bootstrap.verify_add_all_hosts for Master vdsm
project.
Looking at the log, vdsm cannot find master storage domain and engine puts
the host on non-operational state.
Although on the surface the patch seems to be related, the master storage
domain is iscsi whole the patch is related to gluster.
I do not think there is a connection between the patch and the failure but
can you please have a look to make sure?
*Link and headline of suspected patches:
https://gerrit.ovirt.org/#/c/69668/ <https://gerrit.ovirt.org/#/c/69668/> -
*
*gluster: Fix error when brick is on a btrfs subvolumeLink to
Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5180/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5180/>Link
to all
logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5180/a...
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5180/artifact/>(Relevant)
error snippet from the log: <error>vdsm: 2018-02-01 03:13:49,211-0500 INFO
(jsonrpc/4) [vdsm.api] START createStorageDomain(storageType=3,
sdUUID=u'077add35-9171-45d5-b6de-79cc5a853c36', domainName=u'iscsi',
typeSpecificArg=u'IdW3HG-K1Af-e0d3-u2O3-rGle-8fk5-ACNk6C',
domClass=1, domVersion=u'4', options=None) from=::ffff:192.168.201.4,58530,
flow_id=22d4ffd8, task_id=2ce6dd52-3d28-4532-abbf-d78d52af6cda
(api:46)2018-02-01 03:14:40,223-0500 INFO (jsonrpc/7) [vdsm.api] START
connectStoragePool(spUUID=u'2570c0c9-f872-4e49-964a-ee533a79c3f2',
hostID=1, msdUUID=u'077add35-9171-45d5-b6de-79cc5a853c36', masterVersion=1,
domainsMap={u'077add35-9171-45d5-b6de-79cc5a853c36': u'active'},
options=None) from=::ffff:192.168.201.4,36310, flow_id=19e9aa89,
task_id=878419a0-c5ce-4e35-aed5-b27d56b2886e (api:46)2018-02-01
03:14:40,225-0500 INFO (jsonrpc/7) [storage.StoragePoolMemoryBackend] new
storage pool master version 1 and domains map
{u'077add35-9171-45d5-b6de-79cc5a853c36': u'Active'}
(spbackends:449)2018-02-01 03:14:40,225-0500 INFO (jsonrpc/7)
[storage.StoragePool] updating pool 2570c0c9-f872-4e49-964a-ee533a79c3f2
backend from type NoneType instance 0x7f45919e3f20 to type
StoragePoolMemoryBackend instance 0x45411b0 (sp:157)2018-02-01
03:14:40,226-0500 INFO (jsonrpc/7) [storage.StoragePool] Connect host #1
to the storage pool 2570c0c9-f872-4e49-964a-ee533a79c3f2 with master
domain: 077add35-9171-45d5-b6de-79cc5a853c36 (ver = 1) (sp:692)2018-02-01
03:14:40,462-0500 INFO (jsonrpc/7) [vdsm.api] FINISH connectStoragePool
error=Cannot find master domain:
u'spUUID=2570c0c9-f872-4e49-964a-ee533a79c3f2,
msdUUID=077add35-9171-45d5-b6de-79cc5a853c36'
from=::ffff:192.168.201.4,36310, flow_id=19e9aa89,
task_id=878419a0-c5ce-4e35-aed5-b27d56b2886e (api:50)2018-02-01
03:14:40,462-0500 ERROR (jsonrpc/7) [storage.TaskManager.Task]
(Task='878419a0-c5ce-4e35-aed5-b27d56b2886e') Unexpected error
(task:875)Traceback (most recent call last): File
"/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in
_run return fn(*args, **kargs) File "<string>", line 2, in
connectStoragePool File
"/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in
method ret = func(*args, **kwargs) File
"/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1032, in
connectStoragePool spUUID, hostID, msdUUID, masterVersion, domainsMap)
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1094, in
_connectStoragePool res = pool.connect(hostID, msdUUID, masterVersion)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 704, in
connect self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1275, in
__rebuild self.setMasterDomain(msdUUID, masterVersion) File
"/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1488, in
setMasterDomain raise se.StoragePoolMasterNotFound(self.spUUID,
msdUUID)StoragePoolMasterNotFound: Cannot find master domain:
u'spUUID=2570c0c9-f872-4e49-964a-ee533a79c3f2,
msdUUID=077add35-9171-45d5-b6de-79cc5a853c36'2018-02-01 03:14:40,466-0500
INFO (jsonrpc/7) [storage.TaskManager.Task]
(Task='878419a0-c5ce-4e35-aed5-b27d56b2886e') aborting: Task is aborted:
"Cannot find master domain: u'spUUID=2570c0c9-f872-4e49-964a-ee533a79c3f2,
msdUUID=077add35-9171-45d5-b6de-79cc5a853c36'" - code 304
(task:1181)2018-02-01 03:14:40,467-0500 ERROR (jsonrpc/7)
[storage.Dispatcher] FINISH connectStoragePool error=Cannot find master
domain: u'spUUID=2570c0c9-f872-4e49-964a-ee533a79c3f2,
msdUUID=077add35-9171-45d5-b6de-79cc5a853c36' (dispatcher:82)2018-02-01
03:14:40,467-0500 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call
StoragePool.connect failed (error 304) in 0.25 seconds (__init__:573)*
*engine:*
2018-02-01 03:14:40,603-05 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [ba52086] EVENT_ID:
VDS_SET_NONOPERATIONAL_DOMAIN(522), Host lago-basic-suite-mast
er-host-0 cannot access the Storage Domain(s) <UNKNOWN> attached to the
Data Center test-dc. Setting Host state to Non-Operational.
2018-02-01 03:14:40,608-05 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [ba52086] EVENT_ID:
VDS_ALERT_FENCE_IS_NOT_CONFIGURED(9,000), Failed to verify Pow
er Management configuration for Host lago-basic-suite-master-host-0.
2018-02-01 03:14:40,610-05 WARN
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engineScheduled-Thread-70) [ba52086] EVENT_ID:
CONNECT_STORAGE_POOL_FAILED(995), Failed to connect Host lago-
basic-suite-master-host-0 to Storage Pool test-dc
*</error>*
6 years, 2 months