February 2018 - Devel - oVirt List Archives

ovirt-engine fails to build on Fedora 27
by Sandro Bonazzola 04 Feb '18

04 Feb '18

Hi, can you please have a look at http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-fc27-x86_6… ? engine seems to fail building the rpm after the maven part completed, during the install process at: 15:53:23 ++ echo ovirt-engine-aaa.pom 15:53:23 ++ sed -e 's/^ovirt-engine-//' -e 's/\.pom//' 15:53:23 + artifact_id=aaa 15:53:23 + '[' -f /home/jenkins/workspace/ovirt-engine_master_build-artifacts-fc27-x86_64/ovirt-engine/rpmbuild/BUILDROOT/ovirt-engine-4.2.1.5-0.0.master.20180202142920.git0bc7f3edd2.fc27.x86_64/usr/share/java/ovirt-engine/aaa.jar ']' 15:53:23 + %add_maven_depmap -f backend JPP.ovirt-engine-aaa.pom ovirt-engine/aaa.jar 15:53:23 /var/tmp/rpm-tmp.YIGxWv: line 161: fg: no job control 15:53:23 error: Bad exit status from /var/tmp/rpm-tmp.YIGxWv (%install) Maybe some broken dep on backend around aaa.jar ? -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 1

OST failures weekly update [28/01/2018-02/02/2018]
by Dafna Ron 02 Feb '18

02 Feb '18

Hello, I would like to summarize the reported OST failures from the previous week. 01-02-2018 Test name: 001_upgrade_engine.test_initialize_engine version: Ovirt master Project otopi Suite: master/upgrade Cause: *https://gerrit.ovirt.org/#/c/86679/ <https://gerrit.ovirt.org/#/c/86679/> - core: Check Sequence before/after* Fix: *ON GOING ISSUE* Test Name: 002_bootstrap.verify_add_all_hosts Version: Ovirt Master Project: vdsm Suite: basic Cause: *https://gerrit.ovirt.org/#/c/69668/ <https://gerrit.ovirt.org/#/c/69668/> - **gluster: Fix error when brick is on a btrfs subvolume* *Fix: No Fix updated* *02*-02-2018 Test Name: 005_network_by_label.assign_hosts_network_label Version: Master Suite: master/upgrade Project: ovirt-image-uploader Cause: host object locked during api request Fix: no fix for failure. re-triggered patch Other incidents: * We have several build-artifacts failing and this is causing change queue to not run - the failure is caused due to clean-up on jobs which will remove fc24 * We are still seeing CPU Unknown errors on the 4.2 branch * there was a jenkins restart this week which cause a few jobs to fail * Jenkins restart may also cause some failures on hosts due to uncompleted clean-up - we are fixing any host found with this issue Thanks, Dafna

1 0

[ OST Failure Report ] [ oVirt Master (ovirt-image-uploader) ] [ 02-02-2018 ] [005_network_by_label.assign_hosts_network_label ]
by Dafna Ron 02 Feb '18

02 Feb '18

Hi, We failed test 005_network_by_label.assign_hosts_network_label for basic suite in project ovirt-image-uploader. The issue seems to be related to object lock on host and patch reported is not related to the issue. *Link and headline of suspected patches: Patch not related at all to error* *Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5243/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5243/>Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5243/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5243/artifact/>(Relevant) error snippet from the log: <error><JsonRpcRequest id: "4e9d7060-c523-4e66-9ace-dc8733954404", method: Host.getCapabilities, params: {}>2018-02-02 08:11:40,131-05 INFO [org.ovirt.engine.core.bll.network.host.LabelNicCommand] (default task-5) [099e3ef6-4372-42a9-a360-1351ec66a64b] Running command: LabelNicCommand internal: false. Entities affected : ID: 21b0c975-26f8-47a4-b5d0-cad0d41a9133 Type: VDSAction group CONFIGURE_HOST_NETWORK with role type ADMIN2018-02-02 08:11:40,139-05 INFO [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-5) [5c5509c0] Failed to Acquire Lock to object 'EngineLock:{exclusiveLocks='[HOST_NETWORK21b0c975-26f8-47a4-b5d0-cad0d41a9133=HOST_NETWORK]', sharedLocks=''}'2018-02-02 08:11:40,139-05 WARN [org.ovirt.engine.core.bll.network.host.HostSetupNetworksCommand] (default task-5) [5c5509c0] Validation of action 'HostSetupNetworks' failed for user admin@internal-authz. Reasons: VAR__ACTION__SETUP,VAR__TYPE__NETWORKS,ACTION_TYPE_FAILED_SETUP_NETWORKS_OR_REFRESH_IN_PROGRESS2018-02-02 08:11:40,151-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-5) [5c5509c0] Compiled stored procedure. Call string is [{call get_entity_snapshot_by_command_id(?)}]2018-02-02 08:11:40,151-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-5) [5c5509c0] SqlCall for procedure [get_entity_snapshot_by_command_id] compiled2018-02-02 08:11:40,161-05 ERROR [org.ovirt.engine.core.bll.network.host.LabelNicCommand] (default task-5) [5c5509c0] Transaction rolled-back for command 'org.ovirt.engine.core.bll.network.host.LabelNicCommand'.2018-02-02 08:11:40,165-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-5) [5c5509c0] method: get, params: [21b0c975-26f8-47a4-b5d0-cad0d41a9133], timeElapsed: 3ms2018-02-02 08:11:40,168-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-5) [5c5509c0] EVENT_ID: LABEL_NIC_FAILED(1,137), Failed to label network interface card eth0 with label NETWORK_LABEL on host lago-basic-suite-master-host-1.2018-02-02 08:11:40,169-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-5) [5c5509c0] method: runAction, params: [LabelNic, LabelNicParameters:{commandId='aa8eb85e-8a30-4045-83f2-8f0516ec13c9', user='null', commandType='Unknown'}], timeElapsed: 54ms2018-02-02 08:11:40,175-05 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-5) [] Operation Failed: [Cannot add Label. Another Setup Networks or Host Refresh process in progress on the host. Please try later.]2018-02-02 08:11:40,230-05 DEBUG [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (SSL Stomp Reactor) [27fdfc30] MESSAGEcontent-length:10956destination:jms.topic.vdsm_responsescontent-type:application/jsonsubscription:37ef61dd-f4d0-4067-b563-1e2c04f2d2b8</error>*

1 0

oVirt Survey 2018 results
by Sandro Bonazzola 02 Feb '18

02 Feb '18

Thank you very much for having participated in oVirt Survey 2018! Results are now publicly available at http://bit.ly/2Ez909d We're now analyzing results for 4.3 planning. -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

[ OST Failure Report ] [ oVirtMaster (otopi) ] [ 01-02-2018 ] [ 001_initialize_engine.initialize_engine/001_upgrade_engine.test_initialize_engine ]
by Dafna Ron 02 Feb '18

02 Feb '18

Hi, We are failing initialize engine on both basic and upgrade suites. Can you please check? *Link and headline of suspected patches: https://gerrit.ovirt.org/#/c/86679/ <https://gerrit.ovirt.org/#/c/86679/> - * *core: Check Sequence before/afterLink to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5187/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5187/>Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5187/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5187/artifact/>(Relevant) error snippet from the log: <error>2018-02-01 10:38:27,057-0500 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:204 DIALOG:SEND Version: otopi-1.7.7_master (otopi-1.7.7-0.0.master.20180201063428.git81ce9b7.el7.centos)2018-02-01 10:38:27,058-0500 ERROR otopi.context context.check:833 "before" parameter of method otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn.Plugin._misc_configure_provider is a string, should probably be a tuple. Perhaps a missing comma?2018-02-01 10:38:27,058-0500 DEBUG otopi.plugins.otopi.dialog.human dialog.__logString:204 DIALOG:SEND methodinfo: {'priority': 5000, 'name': None, 'before': 'osetup.ovn.provider.service.restart', 'after': ('osetup.pki.ca <http://osetup.pki.ca>.available', 'osetup.ovn.services.restart'), 'method': <bound method ?._misc_configure_provider of <otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn.Plugin object at 0x2edf6d0>>, 'condition': <bound method ?.<lambda> of <otopi.plugins.ovirt_engine_setup.ovirt_engine.network.ovirtproviderovn.Plugin object at 0x2edf6d0>>, 'stage': 11}2018-02-01 10:38:27,059-0500 DEBUG otopi.context context._executeMethod:143 method exceptionTraceback (most recent call last): File "/usr/lib/python2.7/site-packages/otopi/context.py", line 133, in _executeMethod method['method']() File "/usr/share/otopi/plugins/otopi/core/misc.py", line 61, in _setup self.context.checkSequence() File "/usr/lib/python2.7/site-packages/otopi/context.py", line 844, in checkSequence raise RuntimeError(_('Found bad "before" or "after" parameters'))RuntimeError: Found bad "before" or "after" parameters2018-02-01 10:38:27,059-0500 ERROR otopi.context context._executeMethod:152 Failed to execute stage 'Environment setup': Found bad "before" or "after" parameters</error>*

2 1

[ OST Failure Report ] [ oVirt Master (vdsm) ] [ 01-02-2018 ] [ 002_bootstrap.verify_add_all_hosts ]
by Dafna Ron 01 Feb '18

01 Feb '18

Hi, We failed cq test 002_bootstrap.verify_add_all_hosts for Master vdsm project. Looking at the log, vdsm cannot find master storage domain and engine puts the host on non-operational state. Although on the surface the patch seems to be related, the master storage domain is iscsi whole the patch is related to gluster. I do not think there is a connection between the patch and the failure but can you please have a look to make sure? *Link and headline of suspected patches: https://gerrit.ovirt.org/#/c/69668/ <https://gerrit.ovirt.org/#/c/69668/> - * *gluster: Fix error when brick is on a btrfs subvolumeLink to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5180/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5180/>Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5180/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5180/artifact/>(Relevant) error snippet from the log: <error>vdsm: 2018-02-01 03:13:49,211-0500 INFO (jsonrpc/4) [vdsm.api] START createStorageDomain(storageType=3, sdUUID=u'077add35-9171-45d5-b6de-79cc5a853c36', domainName=u'iscsi', typeSpecificArg=u'IdW3HG-K1Af-e0d3-u2O3-rGle-8fk5-ACNk6C', domClass=1, domVersion=u'4', options=None) from=::ffff:192.168.201.4,58530, flow_id=22d4ffd8, task_id=2ce6dd52-3d28-4532-abbf-d78d52af6cda (api:46)2018-02-01 03:14:40,223-0500 INFO (jsonrpc/7) [vdsm.api] START connectStoragePool(spUUID=u'2570c0c9-f872-4e49-964a-ee533a79c3f2', hostID=1, msdUUID=u'077add35-9171-45d5-b6de-79cc5a853c36', masterVersion=1, domainsMap={u'077add35-9171-45d5-b6de-79cc5a853c36': u'active'}, options=None) from=::ffff:192.168.201.4,36310, flow_id=19e9aa89, task_id=878419a0-c5ce-4e35-aed5-b27d56b2886e (api:46)2018-02-01 03:14:40,225-0500 INFO (jsonrpc/7) [storage.StoragePoolMemoryBackend] new storage pool master version 1 and domains map {u'077add35-9171-45d5-b6de-79cc5a853c36': u'Active'} (spbackends:449)2018-02-01 03:14:40,225-0500 INFO (jsonrpc/7) [storage.StoragePool] updating pool 2570c0c9-f872-4e49-964a-ee533a79c3f2 backend from type NoneType instance 0x7f45919e3f20 to type StoragePoolMemoryBackend instance 0x45411b0 (sp:157)2018-02-01 03:14:40,226-0500 INFO (jsonrpc/7) [storage.StoragePool] Connect host #1 to the storage pool 2570c0c9-f872-4e49-964a-ee533a79c3f2 with master domain: 077add35-9171-45d5-b6de-79cc5a853c36 (ver = 1) (sp:692)2018-02-01 03:14:40,462-0500 INFO (jsonrpc/7) [vdsm.api] FINISH connectStoragePool error=Cannot find master domain: u'spUUID=2570c0c9-f872-4e49-964a-ee533a79c3f2, msdUUID=077add35-9171-45d5-b6de-79cc5a853c36' from=::ffff:192.168.201.4,36310, flow_id=19e9aa89, task_id=878419a0-c5ce-4e35-aed5-b27d56b2886e (api:50)2018-02-01 03:14:40,462-0500 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='878419a0-c5ce-4e35-aed5-b27d56b2886e') Unexpected error (task:875)Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in connectStoragePool File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1032, in connectStoragePool spUUID, hostID, msdUUID, masterVersion, domainsMap) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 1094, in _connectStoragePool res = pool.connect(hostID, msdUUID, masterVersion) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 704, in connect self.__rebuild(msdUUID=msdUUID, masterVersion=masterVersion) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1275, in __rebuild self.setMasterDomain(msdUUID, masterVersion) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1488, in setMasterDomain raise se.StoragePoolMasterNotFound(self.spUUID, msdUUID)StoragePoolMasterNotFound: Cannot find master domain: u'spUUID=2570c0c9-f872-4e49-964a-ee533a79c3f2, msdUUID=077add35-9171-45d5-b6de-79cc5a853c36'2018-02-01 03:14:40,466-0500 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='878419a0-c5ce-4e35-aed5-b27d56b2886e') aborting: Task is aborted: "Cannot find master domain: u'spUUID=2570c0c9-f872-4e49-964a-ee533a79c3f2, msdUUID=077add35-9171-45d5-b6de-79cc5a853c36'" - code 304 (task:1181)2018-02-01 03:14:40,467-0500 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH connectStoragePool error=Cannot find master domain: u'spUUID=2570c0c9-f872-4e49-964a-ee533a79c3f2, msdUUID=077add35-9171-45d5-b6de-79cc5a853c36' (dispatcher:82)2018-02-01 03:14:40,467-0500 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call StoragePool.connect failed (error 304) in 0.25 seconds (__init__:573)* *engine:* 2018-02-01 03:14:40,603-05 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [ba52086] EVENT_ID: VDS_SET_NONOPERATIONAL_DOMAIN(522), Host lago-basic-suite-mast er-host-0 cannot access the Storage Domain(s) <UNKNOWN> attached to the Data Center test-dc. Setting Host state to Non-Operational. 2018-02-01 03:14:40,608-05 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [ba52086] EVENT_ID: VDS_ALERT_FENCE_IS_NOT_CONFIGURED(9,000), Failed to verify Pow er Management configuration for Host lago-basic-suite-master-host-0. 2018-02-01 03:14:40,610-05 WARN [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-70) [ba52086] EVENT_ID: CONNECT_STORAGE_POOL_FAILED(995), Failed to connect Host lago- basic-suite-master-host-0 to Storage Pool test-dc *</error>*

1 0