December 2020 - Devel - oVirt List Archives

OST failures on dnf_upgrade
by Marcin Sobczyk 14 Dec '20

14 Dec '20

Hi All, there's a known issue with OST failing right now during deployment on 'dnf_upgrade'. The problem is that 'mdevctl' package is missing in CentOS repos and new libvirt versions require it. While we wait for an official fix, we're rebuilding ost-images with the package included from a copr build. Hopefully OST will work in 1-2 hours. Sorry for the inconvenience. Regards, Marcin

2 1

Ovirt Engine migration to Wildfly 21
by Artur Socha 10 Dec '20

10 Dec '20

Hi, I am working at the moment on the migration of Ovirt Engine to Wildfly 21. I have encountered an issue with the http-remoting-connector which uses the 'default' connector. After the switch to Wildfly 21 this is no longer working. There is an error in server.log complaining about this service not being started ... however, I have not noticed that anything from the engine's core functionality is broken. There are even OST passing with this error. Does anyone know about any usages of remoting components in ovirt engine? Perhaps this could be removed as 'dead' configuration? Assuming, it is being used - I have a patch that seems to fix the problem [1]. The thing is that I still don't understand why this change is now required. 'Default' in reference configuration ie. standalone-full.xml is just http with redirect to https. Any help much appreciated. [1] https://gerrit.ovirt.org/#/c/ovirt-engine/+/112505/ -- Artur Socha Senior Software Engineer, RHV Red Hat

2 3

OST master fails on basic-suite-master.test-scenarios.test_004_basic_sanity.test_sparsify_disk1
by Galit Rosenthal 10 Dec '20

10 Dec '20

Hi It looks like basic suite master is failing on basic-suite-master.test-scenarios.test_004_basic_sanity.test_sparsify_disk1 (from pytest) [1] E ovirtsdk4.Error: Fault reason is "Operation Failed". Fault detail is "[Cannot sparsify Virtual Disk. Sparsifying is not supported for QCOW2 disk vm1_disk1.]". HTTP response code is 400. Can someone have a look? Thanks Galit [1] https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_bas… -- Galit Rosenthal SOFTWARE ENGINEER Red Hat <https://www.redhat.com/> galit(a)redhat.com T: 972-9-7692230 <https://www.redhat.com/>

2 1

Problems building CentOS 8.3-based engine image for OST
by Marcin Sobczyk 09 Dec '20

09 Dec '20

Hi All, since CentOS 8.3 is out, I'm trying to build a new base image for OST, but there are problems on the engine side. The provisioning script we use to build the engine VM is here [1]. The build ends with errors: Error: Problems in request: missing groups or modules: javapackages-tools Last metadata expiration check: 0:00:02 ago on Mon Dec 7 16:00:56 2020. Error: Problem 1: cannot install the best candidate for the job - nothing provides apache-commons-compress needed by ovirt-engine-4.4.4.4-0.0.master.20201206151430.git1a096b0d4e7.el8.noarch - nothing provides apache-commons-jxpath needed by ovirt-engine-4.4.4.4-0.0.master.20201206151430.git1a096b0d4e7.el8.noarch Problem 2: package ovirt-engine-extension-aaa-ldap-setup-1.4.3-0.289.202010220206.el8.noarch requires ovirt-engine-extension-aaa-ldap = 1.4.3-0.289.202010220206.el8, but none of the providers can be installed - package ovirt-engine-extension-aaa-ldap-1.4.3-0.289.202010220206.el8.noarch requires slf4j-jdk14, but none of the providers can be installed - conflicting requests - package slf4j-jdk14-1.7.25-4.module_el8.3.0+454+67dccca4.noarch is filtered out by modular filtering Please advise. Thanks, Marcin [1] https://gerrit.ovirt.org/gitweb?p=ost-images.git;a=blob_plain;f=el8-provisi…

4 7

ovirt-engine has been tagged (ovirt-engine-4.4.4.4)
by Tal Nisan 09 Dec '20

09 Dec '20

2 1

OST images moved to CentOS 8.3
by Marcin Sobczyk 09 Dec '20

09 Dec '20

Hi, the prebuilt OST images have been migrated to CentOS 8.3. If you find any issues with the new images, please let me know. There is a known issue of 'test_098_ovirt_provider_ovn' failing, but it's not related to CentOS 8.3 and should be fixed as soon as we get a new engine build. We also have a new layer of 'ost-images-el8-he-installed' that contains 'ovirt-hosted-engine-appliance' installed and is beneficial to HE suites. Finally, somehow related to yesterday's announcement, fyi we already have nightly-built OST images based on CentOS Stream since quite some time. For those of you interested in trying them out it should be as simple as: dnf install ost-images-el8stream-engine-installed ost-images-el8stream-host-installed export OST_IMAGES_DISTRO=el8stream ./run_suite.sh basic-suite-master Regards, Marcin

1 0

ost check patch
by Eitan Raviv 08 Dec '20

08 Dec '20

Hi, Since yesterday check patch jobs [1][2] are failing for me on emulator not found... kvm executable not found Any hint appreciated. Thanks [1] https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_sta… [2] https://jenkins.ovirt.org/blue/organizations/jenkins/ovirt-system-tests_sta…

4 5

global_setup.sh fail: "docker.service: Failed at step EXEC spawning /usr/bin/dockerd-current: Permission denied"
by Nir Soffer 07 Dec '20

07 Dec '20

Seems that all builds fail now because the global setup script fails to setup the slave. Some examples: - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25522/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25520/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25519/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25516/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25515/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25514/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25513/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25511/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25510/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25509/ - https://jenkins.ovirt.org/job/vdsm_standard-check-patch/25508/ Log from the failed step: https://jenkins.ovirt.org/blue/rest/organizations/jenkins/pipelines/vdsm_st… [2020-12-07T18:09:18.284Z] Dec 07 15:11:08 vm0148.workers-phx.ovirt.org dockerd-current[1342]: time="2020-12-07T15:11:08.426348650Z" level=warning msg="d10f349413a9ce2b5f6cb33d7d721d133b323aba1bf00021ce4d714059b00813 cleanup: failed to unmount secrets: invalid argument" [2020-12-07T18:09:18.284Z] Dec 07 15:11:39 vm0148.workers-phx.ovirt.org dockerd-current[1342]: time="2020-12-07T15:11:39.112783512Z" level=warning msg="55ff0f77d37781237a199a8ba86bc544932f10b1018a3e614bd967b8b5956fd5 cleanup: failed to unmount secrets: invalid argument" [2020-12-07T18:09:18.284Z] Dec 07 15:11:40 vm0148.workers-phx.ovirt.org dockerd-current[1342]: time="2020-12-07T15:11:40.654819711Z" level=info msg="Layer sha256:9d23de0c6d4682039041ca8bb2825278a71c411a22799aeb7b47739514a731d0 cleaned up" [2020-12-07T18:09:18.284Z] Dec 07 15:11:41 vm0148.workers-phx.ovirt.org dockerd-current[1342]: time="2020-12-07T15:11:41.290807003Z" level=info msg="Layer sha256:9d23de0c6d4682039041ca8bb2825278a71c411a22799aeb7b47739514a731d0 cleaned up" [2020-12-07T18:09:18.284Z] Dec 07 15:11:45 vm0148.workers-phx.ovirt.org systemd[1]: docker.service: Main process exited, code=exited, status=2/INVALIDARGUMENT [2020-12-07T18:09:18.284Z] -- Subject: Unit process exited [2020-12-07T18:09:18.284Z] -- Defined-By: systemd [2020-12-07T18:09:18.284Z] -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- An ExecStart= process belonging to unit docker.service has exited. [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- The process' exit code is 'exited' and its exit status is 2. [2020-12-07T18:09:18.284Z] Dec 07 15:11:45 vm0148.workers-phx.ovirt.org systemd[1]: docker.service: Failed with result 'exit-code'. [2020-12-07T18:09:18.284Z] -- Subject: Unit failed [2020-12-07T18:09:18.284Z] -- Defined-By: systemd [2020-12-07T18:09:18.284Z] -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- The unit docker.service has entered the 'failed' state with result 'exit-code'. [2020-12-07T18:09:18.284Z] Dec 07 15:12:30 vm0148.workers-phx.ovirt.org systemd[1]: Starting Docker Application Container Engine... [2020-12-07T18:09:18.284Z] -- Subject: A start job for unit docker.service has begun execution [2020-12-07T18:09:18.284Z] -- Defined-By: systemd [2020-12-07T18:09:18.284Z] -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- A start job for unit docker.service has begun execution. [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- The job identifier is 109048. [2020-12-07T18:09:18.284Z] Dec 07 15:12:30 vm0148.workers-phx.ovirt.org systemd[4524]: docker.service: Failed to execute command: Permission denied [2020-12-07T18:09:18.284Z] Dec 07 15:12:30 vm0148.workers-phx.ovirt.org systemd[4524]: docker.service: Failed at step EXEC spawning /usr/bin/dockerd-current: Permission denied [2020-12-07T18:09:18.284Z] -- Subject: Process /usr/bin/dockerd-current could not be executed [2020-12-07T18:09:18.284Z] -- Defined-By: systemd [2020-12-07T18:09:18.284Z] -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- The process /usr/bin/dockerd-current could not be executed and failed. [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- The error number returned by this process is ERRNO. [2020-12-07T18:09:18.284Z] Dec 07 15:12:30 vm0148.workers-phx.ovirt.org systemd[1]: docker.service: Main process exited, code=exited, status=203/EXEC [2020-12-07T18:09:18.284Z] -- Subject: Unit process exited [2020-12-07T18:09:18.284Z] -- Defined-By: systemd [2020-12-07T18:09:18.284Z] -- Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- An ExecStart= process belonging to unit docker.service has exited. [2020-12-07T18:09:18.284Z] -- [2020-12-07T18:09:18.284Z] -- The process' exit code is 'exited' and its exit status is 203. ...

1 0

ovirt-system-tests: engine fqdn, dns domain name, etc
by Yedidyah Bar David 07 Dec '20

07 Dec '20

Hi all, Current [1] is broken - it misses the domain name. After spending several hours on this last few days, I decided to give up for now and just fix by appending hard-coded 'lago.local'. I am testing locally, and if it passes, I'll push to gerrit. Going forward, I'd like to have as few as possible guesses and hard-coded strings that must match: - Engine fqdn should be hard-coded, arbitrary (but not "engine", as explained), ideally in a single place, propagated from there as needed - storage server (used in HE suites) should also be hard-coded and not guessed from the engine machine name like now - Same for domain name ("lago.local") - Since it's hard to read lago init file and parse it to get stuff from it without actually using lago (and even when using it, e.g. I failed to get dns domain name), we should decide on these things before starting lago and inject them into the lagoinitfile (like we do with other stuff via env). - Also, and to better (IMO) match what I think should happen in reality (for real setups), it should all be done on DNS level (dnsmasq of libvirt, controlled by lago, AFAIU), and we should stop playing with /etc/hosts. If there is agreement, I might decide to work on this. Or we can at least open a bug/ticket somewhere :-). For some of my thoughts/attempts along the way, you might want to see [2][3][4]. All are WIP, none work, probably none even is worth the time of a real review. Thoughts/opinions/ideas are most welcome! Thanks and best regards, [1] https://gerrit.ovirt.org/c/ovirt-system-tests/+/112488 [2] https://github.com/didib/ovirt-system-tests/tree/ostengine [3] https://github.com/didib/ovirt-system-tests/tree/fqdn-from-env [4] https://github.com/didib/ovirt-system-tests/tree/engine-fqdn-env -- Didi

1 0

planned Jenkins restart
by Evgheni Dereveanchin 02 Dec '20

02 Dec '20

Hi everyone, I'll be performing a planned Jenkins restart within the next hour. No new CI jobs will be scheduled during this maintenance period. I will inform you once it is back online. -- Regards, Evgheni Dereveanchin

2 2

Storage OST Failure
by Steven Rosenberg 02 Dec '20

02 Dec '20

Dear Personnel, It seems that the OST is failing due to a storage issue [1]. The failure is on the test_verify_and_remove_cloned_vm test which is not changed on the patch [2] . The engine log prints the error: 2020-12-01 13:07:48,297+01 ERROR [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-1) [] An error occurred while fetching unregistered disks from Storage Domain id '26d55c27-bf61-4f01-b5a3-d9a6a01c3e8a'2020-12-01 13:07:48,298+01 DEBUG [org.ovirt.engine.core.bll.storage.domain.AttachStorageDomainToPoolCommand] (default task-1) [] Skipping format update for domain '26d55c27-bf61-4f01-b5a3-d9a6a01c3e8a' (type 'ImportExport') and returns the following error: 2020-12-01 13:17:46,784+01 ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-2) [] Operation Failed: [Cannot remove VM. The VM is performing an operation on a Snapshot. Please wait for the operation to finish, and try again.] Maybe there is a recent change causing a regression here? [1] https://jenkins.ovirt.org/job/ovirt-system-tests_standard-check-patch/14137/ [2] https://github.com/oVirt/ovirt-system-tests/blob/master/basic-suite-master/… With Best Regards. Steven.

5 5

ovirt-engine has been tagged (ovirt-engine-4.4.4.3)
by Tal Nisan 02 Dec '20

02 Dec '20

1 0