- Devel - oVirt List Archives

OST Failure - Weekly update [17/02/2018-23/02/2018]
by Dafna Ron 23 Feb '18

23 Feb '18

Hello, I would like to update on this week's failures and OST current status. I am happy to say at the end of this week we do not have any failed changes. We had 5 reported failed changes this week: 1. replace vdsm stats with collectd virt plugin - https://gerrit.ovirt.org/#/c/87310/ - date: 20-02-2018 - version: Ovirt 4.2 - test failed: 003_00_metrics_bootstrap.metrics_and_log_collector - fix: fixed bug in initial validations - https://gerrit.ovirt.org/#/c/87919/ 2. replace vdsm stats with collectd virt plugin - https://gerrit.ovirt.org/#/c/87310/ - date: 21-02-2018 - version: Ovirt Master - test failed: 098_ovirt_provider_ovn.use_ovn_provider - NO FIX WAS REPORTED 3. ansible: End playbook based on initial validations - https://gerrit.ovirt.org/#/c/88062/ - date: 22-02-2018 - version: Ovirt Master - Test failed: 003_00_metrics_bootstrap.metrics_and_log_collector - Fix: bug reported: https://bugzilla.redhat.com/show_bug.cgi?id=1548087 4. momIF: change the way we connect to MOM - https://gerrit.ovirt.org/#/c/87944/ - date: 22-02-2018 - version: Ovirt Master - Test failed: 002_bootstrap.verify_add_hosts + 002_bootstrap.add_hosts - Fix: change was reverted until its fixed 5. Require collectd-virt plugin - https://gerrit.ovirt.org/#/c/87311/ - date: 22-02-2018 - version: Ovirt Master - Test: failed: 002_bootstrap.add_hosts - Fix: Require collectd-virt plugin - https://gerrit.ovirt.org/#/c/87311/ *Below you can see the chart for this week's resolved issues but cause of failure: * Code= regression of working components/functionalities Configurations - package related issues Other = failed build artifacts Infra = infrastructure/OST/Lago related issues [image: Inline image 1] [image: Inline image 2] *Below is a chart of resolved failures based on ovirt version* *[image: Inline image 3]* *[image: Inline image 4]* *Below is a chart showing failures by suite type: * * Suite type None means that it was a failure that did not result in a failed test such artificts related or packaging related. [image: Inline image 5] We are currently working on creating more statistics and defining the type of information it would be interesting to present. if there is anything specific that you would be interested in please let us know and we would add that to our plans. Many thanks, Dafna Ron

1 0

[ OST Failure Report ] [ oVirt Master (vdsm) ] [ 22-02-2018 ] [ 002_bootstrap.verify_add_hosts + 002_bootstrap.add_hosts ]
by Dafna Ron 22 Feb '18

22 Feb '18

Hi, We had two failed tests reported in vdsm project last evening the patch reported seems to be related to the issue. *Link and headline of suspected patches: momIF: change the way we connect to MOM - https://gerrit.ovirt.org/#/c/87944/ <https://gerrit.ovirt.org/#/c/87944/>Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5823/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5823/>Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5823/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5823/artifacts>(Relevant) error snippet from the log: <error>* 2018-02-21 14:15:47,576-0500 INFO (MainThread) [vdsm.api] FINISH prepareForShutdown return=None from=internal, task_id=7d37a33b-0215-40c0-a821-9b94707caca6 (api:52) 2018-02-21 14:15:47,576-0500 ERROR (MainThread) [vds] Exception raised (vdsmd:158) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/vdsmd.py", line 156, in run serve_clients(log) File "/usr/lib/python2.7/site-packages/vdsm/vdsmd.py", line 103, in serve_clients cif = clientIF.getInstance(irs, log, scheduler) File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 251, in getInstance cls._instance = clientIF(irs, log, scheduler) File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 121, in __init__ self.mom = MomClient(config.get("mom", "socket_path")) File "/usr/lib/python2.7/site-packages/vdsm/momIF.py", line 51, in __init__ raise MomNotAvailableError() MomNotAvailableError *</error>*

2 2

Let's get oVirt in Stack Overflow's Open Source Advertising again
by Allon Mureinik 22 Feb '18

22 Feb '18

Hi all, With Eldan's help, we have an advert for oVirt [1] suggested on Stack Overflow's traditional Open Source Advertising campaign [2]. Like always, we need the upvotes to get it up there. Want to see oVirt on Stack Overflow's sidebar again? Hope over and upvote. -Allon [1] https://meta.stackoverflow.com/a/363689/2422776 [2] https://meta.stackoverflow.com/q/362773/2422776

1 0

[VDSM] Failing CI check-patch on master, el7: Missing ppc64 repos
by Edward Haas 22 Feb '18

22 Feb '18

Hello, VDSM check-patch on master started to fail, looks like some repos vanished. http://jenkins.ovirt.org/job/vdsm_master_check-patch-el7-x86_64/21853/conso… Can someone have a look? (Do we need ppc64? Why it was removed?) Thanks, Edy.

4 3

[ OST Failure Report ] [ oVirt Master (ovirt-engine-metrics) ] [ 22-02-2018 ] [ 003_00_metrics_bootstrap.metrics_and_log_collector ]
by Dafna Ron 22 Feb '18

22 Feb '18

hi, We are failing test 003_00_metrics_bootstrap.metrics_and_log_collector for basic suite. *Link and headline of suspected patches: * *ansible: End playbook based on initial validations - https://gerrit.ovirt.org/#/c/88062/ <https://gerrit.ovirt.org/#/c/88062/>Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5829/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5829/>Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5829/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5829/artifacts>(Relevant) error snippet from the log: <error>* /var/tmp: drwxr-x--x. root abrt system_u:object_r:abrt_var_cache_t:s0 abrt -rw-------. root root unconfined_u:object_r:user_tmp_t:s0 rpm-tmp.aLitM7 -rw-------. root root unconfined_u:object_r:user_tmp_t:s0 rpm-tmp.G2r7IM -rw-------. root root unconfined_u:object_r:user_tmp_t:s0 rpm-tmp.kVymZE -rw-------. root root unconfined_u:object_r:user_tmp_t:s0 rpm-tmp.uPDvvU drwx------. root root system_u:object_r:tmp_t:s0 systemd-private-cd49c74726d5463f8d6f6502380e5e12-chronyd.service-i1T5IE drwx------. root root system_u:object_r:tmp_t:s0 systemd-private-cd49c74726d5463f8d6f6502380e5e12-systemd-timedated.service-lhoUsS /var/tmp/abrt: -rw-------. root root system_u:object_r:abrt_var_cache_t:s0 last-via-server /var/tmp/systemd-private-cd49c74726d5463f8d6f6502380e5e12-chronyd.service-i1T5IE: drwxrwxrwt. root root system_u:object_r:tmp_t:s0 tmp /var/tmp/systemd-private-cd49c74726d5463f8d6f6502380e5e12-chronyd.service-i1T5IE/tmp: /var/tmp/systemd-private-cd49c74726d5463f8d6f6502380e5e12-systemd-timedated.service-lhoUsS: drwxrwxrwt. root root system_u:object_r:tmp_t:s0 tmp /var/tmp/systemd-private-cd49c74726d5463f8d6f6502380e5e12-systemd-timedated.service-lhoUsS/tmp: /var/yp: ) 2018-02-22 07:24:05::DEBUG::__main__::251::root:: STDERR(/bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/6babba93-09c8-4846-9ccb-07728f72eecb/master/tasks/bd563276-5092-4d28-86c4-63aa6c0b4344.temp: No such file or directory ) 2018-02-22 07:24:05::ERROR::__main__::832::root:: Failed to collect logs from: lago-basic-suite-master-host-0; /bin/ls: cannot open directory /rhev/data-center/mnt/blockSD/6babba93-09c8-4846-9ccb-07728f72eecb/master/tasks/bd563276-5092-4d28-86c4-63aa6c0b4344.temp: No such file or directory *</error>*

2 1

[ OST Failure Report ] [ oVirt Master (ovirt-host) ] [ 22-02-2018 ] [002_bootstrap.add_hosts ]
by Dafna Ron 22 Feb '18

22 Feb '18

hi, we have a failed test for ovirt-host in upgrade suite. *Link and headline of suspected patches: * *Require collectd-virt plugin - https://gerrit.ovirt.org/#/c/87311/ <https://gerrit.ovirt.org/#/c/87311/>Link to Job: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5827/ <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5827/>Link to all logs:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5827/art… <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5827/artifacts>(Relevant) error snippet from the log: <error>* 2018-02-22 05:38:47,587-05 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (VdsDeploy) [546df98d] EVENT_ID: VDS_INSTALL_IN_PROGRESS(509), Installing Host lago-upgrade-from-release-suite-master-host0. Setting kernel arguments. 2018-02-22 05:38:47,858-05 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] Swallowing exception as preferring stderr 2018-02-22 05:38:47,859-05 ERROR [org.ovirt.engine.core.uutils.ssh.SSHDialog] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] SSH error running command root@lago-upgrade-from-release-suite-master-host0:'umask 0077; MYTMP="$(TMPDIR="${OVIRT_TMPDIR}" mktemp -d -t ovirt-XXXXXXXXXX)"; trap "chmod -R u+rwX \"${MYTMP}\" > /dev/null 2>&1; rm -fr \"${MYTMP}\" > /dev/null 2>&1" 0; tar --warning=no-timestamp -C "${MYTMP}" -x && "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:True': RuntimeException: Unexpected error during execution: bash: line 1: 1395 Segmentation fault "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:True 2018-02-22 05:38:47,859-05 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (VdsDeploy) [546df98d] Error during deploy dialog 2018-02-22 05:38:47,860-05 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] Error during host lago-upgrade-from-release-suite-master-host0 install 2018-02-22 05:38:47,864-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] EVENT_ID: VDS_INSTALL_IN_PROGRESS_ERROR(511), An error has occurred during installation of Host lago-upgrade-from-release-suite-master-host0: Unexpected error during execution: bash: line 1: 1395 Segmentation fault "${MYTMP}"/ovirt-host-deploy DIALOG/dialect=str:machine DIALOG/customization=bool:True . 2018-02-22 05:38:47,864-05 ERROR [org.ovirt.engine.core.bll.hostdeploy.VdsDeployBase] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] Error during host lago-upgrade-from-release-suite-master-host0 install, preferring first exception: Unexpected connection termination 2018-02-22 05:38:47,864-05 ERROR [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] Host installation failed for host '65c048e5-a97a-422d-88b8-abe1fd925602', 'lago-upgrade-from-release-suite-master-host0': Unexpected connection termination 2018-02-22 05:38:47,869-05 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] START, SetVdsStatusVDSCommand(HostName = lago-upgrade-from-release-suite-master-host0, SetVdsStatusVDSCommandParameters:{hostId='65c048e5-a97a-422d-88b8-abe1fd925602', status='InstallFailed', nonOperationalReason='NONE', stopSpmFailureLogged='false', maintenanceReason='null'}), log id: 1973b09c 2018-02-22 05:38:47,898-05 INFO [org.ovirt.engine.core.vdsbroker.SetVdsStatusVDSCommand] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] FINISH, SetVdsStatusVDSCommand, log id: 1973b09c 2018-02-22 05:38:47,911-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] EVENT_ID: VDS_INSTALL_FAILED(505), Host lago-upgrade-from-release-suite-master-host0 installation failed. Unexpected connection termination. 2018-02-22 05:38:47,920-05 INFO [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand] (EE-ManagedThreadFactory-engine-Thread-1) [546df98d] Lock freed to object 'EngineLock:{exclusiveLocks='[65c048e5-a97a-422d-88b8-abe1fd925602=VDS]', sharedLocks=''}' *</error>*

1 0

ovirt-engine-api-model jenkins jobs
by Sandro Bonazzola 22 Feb '18

22 Feb '18

Hi, I'm investigating on a failure[2 <http://jenkins.ovirt.org/job/ovirt-engine-api-model_master_build-artifacts-…>] on jenkins jobs for ovirt-engine-api-model[1 <http://jenkins.ovirt.org/search/?q=ovirt-engine-api-model_master_build-arti…>] on el7. Before I further invest time on it, I'd like to understand the purpose of the job. As far as I can tell the project is building some documentation in different formats and a tarball with the sources: asciidoctor.css 28.92 KB model.adoc 1.24 MB model.csv 264.77 KB model.html 3.37 MB model.json 7.39 MB model.xml 6.09 MB ovirt-engine-api-model-4.3.8.20180213gita1f6d9a.tar.gz 246.48 KB I think that some way the output may be used by http://ovirt.github.io/ovirt-engine-api-model/ but I don't feel like this is really done. I also see that artifacts are published on https://search.maven.org/#artifactdetails%7Corg.ovirt.engine.api%7Cmodel%7C… <https://search.maven.org/#artifactdetails%7Corg.ovirt.engine.api%7Cmodel%7C…> but this is also not done by the jenkins jobs. I see this artifacts is required in ovirt-engine pom.xml: <model.version>4.3.7</model.version> So I would expect the job to prepare a rpm including this jar and pom to be required in ovirt-engine spec file, at least if we follow Fedora packaging guidelines. As is, this jobs seams meaningless. What should we do here? Drop the job? Fix it for building rpms? [1] http://jenkins.ovirt.org/search/?q=ovirt-engine-api-model_master_build-arti… [2] http://jenkins.ovirt.org/job/ovirt-engine-api-model_master_build-artifacts-… -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

2 2

CentOS Virt SIG - oVirt 4.3 repositories
by Sandro Bonazzola 21 Feb '18

21 Feb '18

Hi, tags have been created for oVirt 4.3 development. I'm going to cross tag dependencies from oVirt 4.2 to oVirt 4.3 in the next weeks. Once done, I'll push patches to oVirt System Test master and oVirt Release master switching from oVirt 4.2 to oVirt 4.3. Repositories that will contain 4.3 related stuff while I'm working on it: https://cbs.centos.org/repos/virt7-ovirt-43-candidate https://cbs.centos.org/repos/virt7-ovirt-common-candidate Once I'll be sure master works with these, I'll tag for testing switching to: https://cbs.centos.org/repos/virt7-ovirt-43-testing https://cbs.centos.org/repos/virt7-ovirt-common-testing Once repoclosure passes, I'll ask to start signing and publishing rpms for testing getting the rpms into final testing destination: https://buildlogs.centos.org/centos/7/virt/$basearch/ovirt-4.3/ This will be the repo that will be added to OST and ovirt-release. If new dependencies needs to be added to 4.3 repo during this process, please let me know. Thanks -- SANDRO BONAZZOLA ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>

1 0

[ OST Failure Report ] [ oVirt $VER (ovirt-log-collector) ] [ 21-02-2018 ] [ 098_ovirt_provider_ovn.use_ovn_provider ]
by Dafna Ron 21 Feb '18

21 Feb '18

Hi, We failed test on duplicate network name. I don't think that the change is related. I can see that we are importing network_1 and then trying to create a network by the same name. I don't think there was a change made to ovn before this failure that could have cause this to fail but adding Dan in case he is aware of any changes. *Link and headline of suspected patches: failed change: * inventory: handle to postgresql dumps from sosreport - https://gerrit.ovirt.org/#/c/88010/ *Change reported as root cause of failure: *collector: fix list function - https://gerrit.ovirt.org/#/c/87798/ *Link to Job:http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5805 <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5805>Link to all logs:* http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/5805/artifacts *(Relevant) error snippet from the log: <error>* 2018-02-20 23:51:03,179-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-19) [4a0a588a-e606-4151-936a-f66a8c580a78] Compiled stored procedure. Call string is [{call getdcidbyexternalnetworkid(?)}] 2018-02-20 23:51:03,179-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.PostgresDbEngineDialect$PostgresSimpleJdbcCall] (default task-19) [4a0a588a-e606-4151-936a-f66a8c580a78] SqlCall for procedure [GetDcIdByExternalNetworkId] compiled 2018-02-20 23:51:03,180-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-19) [4a0a588a-e606-4151-936a-f66a8c580a78] method: runQuery, params: [GetAllExternalNetworksOnProvider, IdQueryParameters:{refresh='false', filtered='false'}], timeElapsed: 266ms 2018-02-20 23:51:03,187-05 INFO [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default task-19) [4a0a588a-e606-4151-936a-f66a8c580a78] Lock Acquired to object 'EngineLock:{exclusiveLocks='[network_1=NETWORK, 94fc81b9-fbf2-4262-a877-32a066a573c2=PROVIDER]', sharedLocks=''}' 2018-02-20 23:51:03,190-05 WARN [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default task-19) [4a0a588a-e606-4151-936a-f66a8c580a78] Validation of action 'AddNetwork' failed for user admin@internal-authz. Reasons: VAR__TYPE__NETWORK,VAR__ACTION__ADD,ACTION_TYPE_FAILED_NETWORK_NAME_IN_USE,$NetworkName network_1 2018-02-20 23:51:03,190-05 INFO [org.ovirt.engine.core.bll.network.dc.AddNetworkCommand] (default task-19) [4a0a588a-e606-4151-936a-f66a8c580a78] Lock freed to object 'EngineLock:{exclusiveLocks='[network_1=NETWORK, 94fc81b9-fbf2-4262-a877-32a066a573c2=PROVIDER]', sharedLocks=''}' 2018-02-20 23:51:03,191-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-19) [4a0a588a-e606-4151-936a-f66a8c580a78] method: runAction, params: [AddNetwork, AddNetworkStoragePoolParameters:{commandId='41c44658-04e5-4aeb-9f65-cc2ad76e0c42', user='null', commandType='Unknown'}], timeElapsed: 10ms 2018-02-20 23:51:03,196-05 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-19) [] Operation Failed: [Cannot add Network. The name of the logical network 'network_1' is already used by an existing logical network in the same data-center. -Please choose a different name.] 2018-02-20 23:51:03,346-05 DEBUG [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] method: get, params: [da0bebdb-cb83-415f-bcba-8920126e31a4], timeElapsed: 6ms 2018-02-20 23:51:03,346-05 DEBUG [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] vdsManager::refreshVdsStats entered, host='lago-basic-suite-master-host-1'(da0bebdb-cb83-415f-bcba-8920126e31a4) 2018-02-20 23:51:03,365-05 DEBUG [org.ovirt.engine.core.vdsbroker.vdsbroker.GetStatsAsyncVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] START, GetStatsAsyncVDSCommand(HostName = lago-basic-suite-master-host-1, VdsIdAndVdsVDSCommandParametersBase:{hostId='da0bebdb-cb83-415f-bcba-8920126e31a4', vds='Host[lago-basic-suite-master-host-1,da0bebdb-cb83-415f-bcba-8920126e31a4]'}), log id: 279de334 2018-02-20 23:51:03,365-05 DEBUG [org.ovirt.vdsm.jsonrpc.client.reactors.stomp.impl.Message] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [] SEND destination:jms.topic.vdsm_requests reply-to:jms.topic.vdsm_responses content-length:98 *</error>*

1 0

Documenting HA VMs on Wiki
by Milan Zamazal 20 Feb '18

20 Feb '18

Hi, to document and clarify some facts about HA VMs to users, especially regarding I/O errors, I work on a HOWTO document to be added to oVirt Wiki. If you know how HA VMs or related mechanisms (such as VM handling on I/O errors) work and would like to help review the document, please look at the pull request: https://github.com/oVirt/ovirt-site/pull/1530 Thanks, Milan

1 0