February 2020 - Devel - oVirt List Archives

[VDSM] Started getting virNetTLSContextCheckCertTimes:154 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem has expired
by Edward Haas 11 Jan '24

11 Jan '24

Hi, I have a VM which acts as an oVirt host with VDSM installed. I use it for testing without connecting it to Engine. Recently VDSM fails to come up and I see this error: Sep 26 17:58:18 localhost libvirtd: 2018-09-26 14:58:18.978+0000: 12176: error : virNetTLSContextCheckCertTimes:154 : The server certificate /etc/pki/vdsm/certs/vdsmcert.pem has expired Any hint on how to generate or fetch a new certificate without the help of Engine? Thanks, Edy.

3 5

Error: Adding new Host to ovirt-engine
by Ahmad Khiet 14 Aug '23

14 Aug '23

Hi, Can't add new host to ovirt engine, because the following error: 2019-06-12 12:23:09,664 p=4134 u=engine | TASK [ovirt-host-deploy-facts : Set facts] ************************************* 2019-06-12 12:23:09,684 p=4134 u=engine | ok: [10.35.1.17] => { "ansible_facts": { "ansible_python_interpreter": "/usr/bin/python2", "host_deploy_vdsm_version": "4.40.0" }, "changed": false } 2019-06-12 12:23:09,697 p=4134 u=engine | TASK [ovirt-provider-ovn-driver : Install ovs] ********************************* 2019-06-12 12:23:09,726 p=4134 u=engine | fatal: [10.35.1.17]: FAILED! => {} MSG: The conditional check 'cluster_switch == "ovs" or (ovn_central is defined and ovn_central | ipaddr and ovn_engine_cluster_version is version_compare('4.2', '>='))' failed. The error was: The ipaddr filter requires python's netaddr be installed on the ansible controller The error appears to be in '/home/engine/apps/engine/share/ovirt-engine/playbooks/roles/ovirt-provider-ovn-driver/tasks/configure.yml': line 3, column 5, but may be elsewhere in the file depending on the exact syntax problem. The offending line appears to be: - block: - name: Install ovs ^ here 2019-06-12 12:23:09,728 p=4134 u=engine | PLAY RECAP ********************************************************************* 2019-06-12 12:23:09,728 p=4134 u=engine | 10.35.1.17 : ok=3 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0 whats missing!? Thanks -- Ahmad Khiet Red Hat <https://www.redhat.com/> akhiet(a)redhat.com M: +972-54-6225629 <https://red.ht/sig>

7 10

Purging inactive maintainers from vdsm-master-maintainers
by Nir Soffer 21 Jun '21

21 Jun '21

I want to remove inactive contributors from vdsm-master-maintainers. I suggest the simple rule of 2 years of inactivity for removing from this group, based on git log. See the list below for current status: https://gerrit.ovirt.org/#/admin/groups/106,members Thoughts? Nir

6 11

Error Java SDK Issue??
by Geschwentner, Patrick 02 Apr '21

02 Apr '21

Dear Ladies and Gentlemen! I am currently working with the java-sdk and I encountered a problem. If I would like to retrieve the disk details, I get the following error: Disk currDisk = ovirtConnection.followLink(diskAttachment.disk()); The Error is occurring in this line: [cid:image001.png@01D44537.AF127FD0] The getResponst looks quiet ok. (I inspected: [cid:image002.png@01D44537.AF127FD0] and it looks ok). Error: wrong number of arguments The code is quiet similar to what you published on github (https://github.com/oVirt/ovirt-engine-sdk-java/blob/master/sdk/src/test/jav… ). Can you confirm the defect? Best regards Patrick

6 11

Why is vdsm enabled by default?
by Yedidyah Bar David 02 Apr '20

02 Apr '20

If I do e.g.: 1. Install CentOS 2. yum install ovirt-releaseSOMETHING 3. yum install vdsm Then reboot the machine, vdsm starts, and for this, it does all kinds of things to the system (such as configure various services using vdsm-tool etc.). Are we sure we want/need this? Why would we want vdsm configured/running at all at this stage, before being added to an engine? In particular, if (especially during development) we have a bug in this configuration process, and then fix it, it might not be enough to upgrade vdsm - the tooling will then also have to fix the changes done by the buggy previous version, or require a full machine reinstall. Thanks and best regards, -- Didi

7 13

Container-based CI backend is now available for use
by Barak Korren 26 Mar '20

26 Mar '20

A little less then a month ago I sent an email to this list telling you all about ongoing work to create a new container-based backend for the oVirt CI system. I'm pleased to announce that we have managed to finally merged an initial set of patches implementing that backend yesterday, and it is now available for general use. *What? Were? How do I use it?* Documentation about how to use the new backend is now available in read the docs <https://ovirt-infra-docs.readthedocs.io/en/latest/CI/STDCI-Containers/index…> . *Wait! I needed it to do X which it doesn't!* For the time being the new backend lacks some features that some may consider to be essential, such as automated cloning of patch source code and build artifact collection. We already have implemented patches providing a substantial amount of additional functionality, and hopefully we will be able to merge them soon. Following is a list of those patches and the features they implement: 1. Automated source cloning support: - 104213 <https://gerrit.ovirt.org/104213>: Implement STDCI DSL support for initContainers - 104590 <https://gerrit.ovirt.org/104590>: STDCI DSL: Add the `decorate` option - 104668 <https://gerrit.ovirt.org/104668>: Document source cloning extension for containers 2. Artifact collection support - 104690 <https://gerrit.ovirt.org/104690>: Added NFS server container image - 104273 <https://gerrit.ovirt.org/104273>: STDCI PODS: Unique UID for each job build's POD - 104756 <https://gerrit.ovirt.org/104756>: pipeline-loader: refactor: separate podspec func - 104757 <https://gerrit.ovirt.org/104757>: pipeline-loader: refactor: Use podspec struct def - 104766 <https://gerrit.ovirt.org/104766>: STDCI PODs: Add artifact collection logic - 105522 <https://gerrit.ovirt.org/105522>: Documented artifact collection in containers 3. Extended log collection - 104842 <https://gerrit.ovirt.org/104842>: STDCI PODs: Add POD log collection - 105523 <https://gerrit.ovirt.org/105523>: Documented log collection in containers 4. Privileged container support - 104786 <https://gerrit.ovirt.org/104786>: STDCI DSL: Enable privileged containers 5. Support for using containers in gating jobs: - 104804 <https://gerrit.ovirt.org/104804>: standard-stage: refactor: move DSL to a library - 104811 <https://gerrit.ovirt.org/104811>: gate: Support getting suits from STDCI DSL 6. Providing the `extra_sources` file to OST suit containers: - 104843 <https://gerrit.ovirt.org/104843>: stdci_runner: Create extra_sources for PODs 7. Support for mirror injection and upstream source cloning - 104917 <https://gerrit.ovirt.org/104917>: Added a container with STDCI tools - 104918 <https://gerrit.ovirt.org/104918>: decorate.py: Add script - 104989 <https://gerrit.ovirt.org/104989>: STDCI DSL: Use `tools` container for `decorate` - 104994 <https://gerrit.ovirt.org/104994>: stdci_runner: Inject mirrors in PODs As you can see, we have quite a big pile of reviews to do, as always, help is very welcome... Regards, Barak. -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted

2 4

[Bug 1808271] unboundid-ldapsdk-5.0.0 is available
by Sandro Bonazzola 28 Feb '20

28 Feb '20

Hi, unboundid-ldapsdk-5.0.0 is available upstream. Release notes are available here: https://github.com/pingidentity/ldapsdk/releases most notable change is the licensing, including ASL along with GPL and LGPL. The library is used by aaa-ldap extension. I've not yet built it for Fedora Rawhide, and I'm not prioritizing this unless the new release content / licensing change is useful for oVirt scope. Please let me know if there's interest in getting this new release available for oVirt consumption. -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

2 1

net_mlx5 libibverbs.so.1 noise
by Yedidyah Bar David 27 Feb '20

27 Feb '20

Hi all, I upgraded my CentOS8 engine machine, and now whenever I run a shell (e.g. ssh, or another window in tmux), I get: net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) net_mlx5: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory net_mlx5: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx5) PMD: net_mlx4: cannot load glue library: libibverbs.so.1: cannot open shared object file: No such file or directory PMD: net_mlx4: cannot initialize PMD due to missing run-time dependency on rdma-core libraries (libibverbs, libmlx4) Not sure if this is a new known issue. It happened in the past, and was fixed, but apparently got broken again. Now commented about this on the old bug: https://bugzilla.redhat.com/show_bug.cgi?id=1642945#c9 Workaround: dnf install libibverbs.so.1 libmlx5 Best regards, -- Didi

3 2

Need steps to downgrade engine from 4.4 to 4.3.
by Ritesh Chikatwar 26 Feb '20

26 Feb '20

Hello all.. Is there any steps to downgrade engine from 4.4 to 4.3 Any help will appreciate. Ritesh

2 1

[CQ 20589] engine reports storage domain does not exist
by Sandro Bonazzola 25 Feb '20

25 Feb '20

https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20589/ Engine fails at: 2020-02-25 02:53:53,144-05 DEBUG [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetStorageDomainInfoVDSCommand] (default task-1) [56f93c90-3868-43f8-920f-bc1fccc72a27] Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to HSMGetStorageDomainInfoVDS, error = Storage domain does not exist: ('8c9f3762-3bf1-48fa-9237-b6587d0268ab',), code = 358 https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20589/artifa… corresponding failure in VDSM is: 2020-02-25 02:53:53,128-0500 ERROR (jsonrpc/7) [storage.TaskManager.Task] (Task='752b540e-fbfc-4602-8eeb-3c357fb7f5a2') Unexpected error (task:874) Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/vdsm/storage/task.py", line 881, in _run return fn(*args, **kargs) File "<decorator-gen-129>", line 2, in getStorageDomainInfo File "/usr/lib/python3.6/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 2752, in getStorageDomainInfo dom = self.validateSdUUID(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/hsm.py", line 309, in validateSdUUID sdDom = sdCache.produce(sdUUID=sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 115, in produce domain.getRealDomain() File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 139, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 156, in _findDomain return findMethod(sdUUID) File "/usr/lib/python3.6/site-packages/vdsm/storage/sdc.py", line 186, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) vdsm.storage.exception.StorageDomainDoesNotExist: Storage domain does not exist: ('8c9f3762-3bf1-48fa-9237-b6587d0268ab',) 2020-02-25 02:53:53,128-0500 INFO (jsonrpc/7) [storage.TaskManager.Task] (Task='752b540e-fbfc-4602-8eeb-3c357fb7f5a2') aborting: Task is aborted: "value=Storage domain does not exist: ('8c9f3762-3bf1-48fa-9237-b6587d0268ab',) abortedcode=358" (task:1184) 2020-02-25 02:53:53,128-0500 ERROR (jsonrpc/7) [storage.Dispatcher] FINISH getStorageDomainInfo error=Storage domain does not exist: ('8c9f3762-3bf1-48fa-9237-b6587d0268ab',) (dispatcher:83) 2020-02-25 02:53:53,129-0500 INFO (jsonrpc/7) [jsonrpc.JsonRpcServer] RPC call StorageDomain.getInfo failed (error 358) in 0.28 seconds (__init__:312) https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20589/artifa… Corresponding var log messages: Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: Attached scsi generic sg8 type 0 Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: alua: port group 00 state A non-preferred supports TOlUSNA Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: scsi 3:0:0:1: Direct-Access LIO-ORG lun1_bdev 4.0 PQ: 0 ANSI: 5 Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: [sdi] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB) Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: scsi 3:0:0:1: alua: supports implicit and explicit TPGS Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: scsi 3:0:0:1: alua: device naa.600140541e20cc9dba94e4fa46a57322 port group 0 rel port 1 Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: [sdi] Write Protect is off Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: [sdi] Write cache: enabled, read cache: enabled, supports DPO and FUA Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: Attached scsi generic sg9 type 0 Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: alua: port group 00 state A non-preferred supports TOlUSNA Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: [sdj] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB) Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:0: [sdf] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: [sdj] Write Protect is off Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: [sdj] Write cache: enabled, read cache: enabled, supports DPO and FUA Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:4: [sdg] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:3: [sdh] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:2: [sdi] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 kernel: sd 3:0:0:1: [sdj] Attached SCSI disk Feb 25 02:53:52 lago-basic-suite-master-host-0 systemd[1]: Started Session c67 of user root. Feb 25 02:53:52 lago-basic-suite-master-host-0 systemd[1]: Starting LVM event activation on device 8:128... Feb 25 02:53:52 lago-basic-suite-master-host-0 lvm[30164]: pvscan[30164] PV /dev/sdi online, VG 8c9f3762-3bf1-48fa-9237-b6587d0268ab is complete. Feb 25 02:53:52 lago-basic-suite-master-host-0 lvm[30164]: pvscan[30164] VG 8c9f3762-3bf1-48fa-9237-b6587d0268ab skip autoactivation. Feb 25 02:53:52 lago-basic-suite-master-host-0 systemd[1]: Started LVM event activation on device 8:128. https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20589/artifa… -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

2 1

pad.engineering.redhat.com not working
by Eyal Shenitzky 25 Feb '20

25 Feb '20

Hi, $SUBJECT isn't working. Does someone know what is the problem? -- Regards, Eyal Shenitzky

1 0

python-ovirt-engine-sdk4 failing to build for 4.3 pipeline
by Sandro Bonazzola 24 Feb '20

24 Feb '20

Hi, looking at jenkins failures, SDK fails to build on patch https://gerrit.ovirt.org/#/c/106716/ The patch is clearly not the root cause of the failure, you can see at https://jenkins.ovirt.org/job/python-ovirt-engine-sdk4_4.3_build-artifacts-… *00:05:28.534* [INFO] --- exec-maven-plugin:1.2.1:exec (test) @ python-sdk ---*00:05:28.886* 4.3.3.dev20200224+gitcb0b9d0*00:05:28.886* usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]*00:05:28.886* or: setup.py --help [cmd1 cmd2 ...]*00:05:28.886* or: setup.py --help-commands*00:05:28.886* or: setup.py cmd --help*00:05:28.886* *00:05:28.886* error: invalid command 'nosetests' Ori, can you please have a look? -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

[CQ 20569] failing to resolve copr repos
by Sandro Bonazzola 24 Feb '20

24 Feb '20

Hi, looking at last failure on OST. Job: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20569/ looking at engine dnf log: https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/20569/artifa… I see: 2020-02-24T18:49:14Z DEBUG error: Curl error (6): Couldn't resolve host name for https://copr-be.cloud.fedoraproject.org/results/networkmanager/NetworkManag… [Could not resolve host: copr-be.cloud.fedoraproject.org] (https://copr-be.cloud.fedoraproject.org/results/networkmanager/NetworkManag…) 2020-02-24T18:49:14Z DEBUG Cannot download 'https://copr-be.cloud.fedoraproject.org/results/networkmanager/NetworkManag…': Cannot download repomd.xml: Curl error (6): Couldn't resolve host name for https://copr-be.cloud.fedoraproject.org/results/networkmanager/NetworkManag… [Could not resolve host: copr-be.cloud.fedoraproject.org] 2020-02-24T18:49:14Z ERROR Failed to download metadata for repo 'ovirt-master-copr-NetworkManager-1.22-el8' 2020-02-24T18:49:14Z DDEBUG Cleaning up. 2020-02-24T18:49:14Z SUBDEBUG Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/dnf/repo.py", line 552, in load ret = self._repo.load() File "/usr/lib64/python3.6/site-packages/libdnf/repo.py", line 394, in load return _repo.Repo_load(self) RuntimeError: Failed to download metadata for repo 'ovirt-master-copr-NetworkManager-1.22-el8' Is OST suffering of network connectivity? -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

OST Failure: libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable
by Gobinda Das 24 Feb '20

24 Feb '20

Hi, One of the OST suite (hc-basic-suite-4.3) is failing with error: libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable Full log: * Create network lago-hc-basic-suite-4-3-net-management: ERROR (in 0:00:11)*07:42:57* # Start nets: ERROR (in 0:00:11)*07:42:57* @ Start Prefix: ERROR (in 0:00:11)*07:42:57* Error occured, aborting*07:42:57* Traceback (most recent call last):*07:42:57* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 969, in main*07:42:57* cli_plugins[args.verb].do_run(args)*07:42:57* File "/usr/lib/python2.7/site-packages/lago/plugins/cli.py", line 184, in do_run*07:42:57* self._do_run(**vars(args))*07:42:57* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 573, in wrapper*07:42:57* return func(*args, **kwargs)*07:42:57* File "/usr/lib/python2.7/site-packages/lago/utils.py", line 584, in wrapper*07:42:57* return func(*args, prefix=prefix, **kwargs)*07:42:57* File "/usr/lib/python2.7/site-packages/lago/cmd.py", line 271, in do_start*07:42:57* prefix.start(vm_names=vm_names)*07:42:57* File "/usr/lib/python2.7/site-packages/lago/sdk_utils.py", line 50, in wrapped*07:42:57* return func(*args, **kwargs)*07:42:57* File "/usr/lib/python2.7/site-packages/lago/prefix.py", line 1323, in start*07:42:57* self.virt_env.start(vm_names=vm_names)*07:42:57* File "/usr/lib/python2.7/site-packages/lago/virt.py", line 341, in start*07:42:57* net.start()*07:42:57* File "/usr/lib/python2.7/site-packages/lago/providers/libvirt/network.py", line 115, in start*07:42:57* net = self.libvirt_con.networkCreateXML(self._libvirt_xml())*07:42:57* File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4216, in networkCreateXML*07:42:57* if ret is None:raise libvirtError('virNetworkCreateXML() failed', conn=self)*07:42:57* libvirtError: COMMAND_FAILED: INVALID_IPV: 'ipv6' is not a valid backend or is unavailable*07:42:57* + on_exit Any one has idea what could be the issue? -- Thanks, Gobinda

2 1

Proposing Benny Zlotnik as an Engine Storage maintainer
by Tal Nisan 24 Feb '20

24 Feb '20

Hi everyone, Benny joined the Storage team in November 2016 and since then played a key role in investigating very complex customer bugs around oVirt engine as well as contributing to features such as DR, Cinberlib, new export/import mechanism and also rewrote and still maintains the LSM mechanism. Given his big contribution and knowledge around the engine I'd like to nominate Benny as an engine storage maintainer. Your thoughts please. Tal.

12 12

Fwd: Copr outage - details
by Sandro Bonazzola 20 Feb '20

20 Feb '20

FYI ---------- Forwarded message --------- Da: Miroslav Suchý <msuchy(a)redhat.com> Date: gio 20 feb 2020 alle ore 10:28 Subject: Copr outage - details To: Development discussions related to Fedora <devel(a)lists.fedoraproject.org > tl;dr: On Sunday 23rd February, there will be Copr outage. It will last the whole day. PPC64LE builder and chroots will be deactivated. The PPC64LE builders should be back in a matter of weeks. Hi. As previously announced, Fedora's infrastructure is moving to a different datacenter. For some servers, the move is trivial. Copr servers are different. Copr build system consists of four servers, plus four staging servers. Eight TB of repos, four TB of dist-git, and several small volumes. The original plan was to move to Washington D.C. to IAD2 datacenter by June. Copr is running in Fedora OpenStack, and this cloud has to be evacuated by the beginning of March to free an IP range. The plan was to move Copr to new hardware (thanks to Red Hat) and later move this HW to the new datacenter. That would mean two outages, where the second one lasted at least 15 days (!). We were looking for another option and we found it. We are going to move Copr to Amazon AWS, shutdown old VM on Fedora Cloud. Move the new HW to IAD2 datacenter and then move Copr from AWS to new HW in IAD2 - FYI, the final destination is still subject to change. This still means two outages, but they should be just a few hours. And web server with DNF repositories should be available all the time. The second outage, will happen in May or June. Here is a detailed schedule. We are going to update this table during migration. You can watch the progress during migration: https://docs.google.com/spreadsheets/d/1jrCgdhseZwi91CTRlo9Y5DNwfl9VHoZfjHP… Here is a short abstract: * we are doing constant rsync to the new location * we spin up staging and production instances in the new location * on Sunday morning we stop frontend and therefore accepting new jobs. The backend with DNF repos will still be operational. * we do final rsync (~6 hours) * around 13:00 UTC we switch DNS to the new location * we then enable all services * once we confirm that everything is operational, the outage will be over There are several caveats: * After we enable services on Sunday 13:00 UTC you may see some failures. Be asured that we will swiftly address them. * Once we get out of Fedora Cloud, we lost access to PPC64LE builders. We are going to deactivate those chroots just before the migration. After a few weeks, we should get it back. ETA is unknown. The worst-case scenario is in June 2020. We will be aiming to bring it back as soon as possible. * Any small issue can easily change the schedule by hours. E.g., just simple 'chown -R' on backend runs ~4 hours. There are going to be three Copr engineers and one fedora-infrastructure member available whole Sunday. If you experienced a problem, do not hesitate to contact us. We are on #fedora-buildsys on Freenode. The link to the outage ticket is: https://pagure.io/fedora-infrastructure/issue/8668 -- Miroslav Suchy, RHCARed Hat, Associate Manager ABRT/Copr, #brno, #fedora-buildsys _______________________________________________ devel mailing list -- devel(a)lists.fedoraproject.org To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

1 0

OST basic suite failing on el8 physical server
by Marcin Sobczyk 19 Feb '20

19 Feb '20

Hi all, I'm trying to run OST basic suite on CentOS 8 based physical server with lago running on py3. I'm currently stuck at 'verify_add_hosts' step. The engine continously tries to run: org.ovirt.engine.core.vdsbroker.vdsbroker.CollectVdsNetworkDataAfterInstallationVDSCommand but it ends up with a failure: 2020-02-18 05:29:01,386-05 DEBUG [org.ovirt.engine.core.dal.dbbroker.CustomSQLErrorCodeSQLExceptionTranslator] (EE-ManagedThreadFactory-engine-Thread-1) [39a236fd] Translating SQLException with SQL state '23505', error code '0', message [ERROR: duplicate key value violates unique constraint "name_server_pkey" Detail: Key (dns_resolver_configuration_id, address)=(98794c52-8364-4909-bbaf-a7ac16a1bf2a, 192.168.201.1) already exists. Here's part of my lago prefix status: [lago-basic-suite-master-host-0]: [NICs]: [eth0]: ip: 192.168.200.3 network: lago-basic-suite-master-net-management [eth1]: ip: 192.168.202.3 network: lago-basic-suite-master-net-storage [eth2]: ip: 192.168.201.2 network: lago-basic-suite-master-net-bonding [eth3]: ip: 192.168.201.3 network: lago-basic-suite-master-net-bonding distro: el8 [metadata]: deploy-scripts: $LAGO_PREFIX_PATH/scripts/_root_ovirt-system-tests_basic-suite-master_deploy-scripts_add_local_repo_no_ext_access.sh $LAGO_PREFIX_PATH/scripts/_root_ovirt-system-tests_basic-suite-master_deploy-scripts_setup_sar_stat.sh $LAGO_PREFIX_PATH/scripts/_root_ovirt-system-tests_basic-suite-master_deploy-scripts_setup_host_el7.sh $LAGO_PREFIX_PATH/scripts/_root_ovirt-system-tests_basic-suite-master_deploy-scripts_setup_1st_host_el7.sh root password: 123456 status: running [lago-basic-suite-master-host-1]: [NICs]: [eth0]: ip: 192.168.200.4 network: lago-basic-suite-master-net-management [eth1]: ip: 192.168.202.4 network: lago-basic-suite-master-net-storage [eth2]: ip: 192.168.201.4 network: lago-basic-suite-master-net-bonding [eth3]: ip: 192.168.201.5 network: lago-basic-suite-master-net-bonding So it seems the two bonding NICs indeed share the same dns resolver address - 192.168.201.1, but I suppose that's how it looks for all basic suite runs, right? Does anyone know what could be wrong in this setup and why the engine is complaining? Thanks, Marcin

1 0

do we even handle volume metadata in the engine?
by Fedor Gavrilov 19 Feb '20

19 Feb '20

Hi, It seems I was able to get my setup working for NFS storage, thanks for your advices! Now I'm afraid I am stuck again: what I need to do is to add validation for volume metadata to a certain command (to be exact, that VOLTYPE is LEAF). But I can't find examples of us dealing with this data in the engine part at all. I understand this is in the end executed by VDSM, but nevertheless - in what format are we even reading and writing volume metadata? Even class/method name would be helpful. Thanks, Fedor

3 5

ovirt-engine has been tagged (ovirt-engine-4.3.9.1)
by Tal Nisan 18 Feb '20

18 Feb '20

1 0

Engine-setup failure
by Ori Liel 17 Feb '20

17 Feb '20

Does anyone get this error while building the Engine? [ ERROR ] schema.sh: FATAL: Cannot execute sql command: --command=select 1; [ ERROR ] Failed to execute stage 'Misc configuration': Engine schema refresh failed Happened in a completely new setup, rebased from master, new database, even build directory manually deleted before running make. Any help would be appreciated, thanks

2 2

What should I do to support DPDK in ovirt, any instruction?
by lifuqiong＠sunyainfo.com 17 Feb '20

17 Feb '20

Hi All, I found that there is rarely topics about supporting dpdk in ovirt 4.2 in Internet except https://blogs.ovirt.org/2018/07/upgraded-dpdk-support-in-ovirt/; and I can't get information such as whether or not should I install dpdk or ovn manual? And I will get an error when I execute such cmds: ansible-playbook ovirt.dpdk-setup/tasks/main.yml ansible-playbook oVirt.dpdk-setup I have some experience about ovn and dpdk, but I can't make ovirt supporting dpdk according the instruction in https://blogs.ovirt.org/2018/07/upgraded-dpdk-support-in-ovirt/; Is there anybody helping me? Thank you. Mark

1 0

CI: test_qcow2_to_raw_preallocated is flaky
by Amit Bawer 16 Feb '20

16 Feb '20

Hi, This occurs on CI every now and then, taken from: https://jenkins.ovirt.org/job/vdsm_standard-check-patch/18302//artifact/che… Thanks ________ TestConvertPreallocation.test_qcow2_to_raw_preallocated[full] _________ self = <storage.qemuimg_test.TestConvertPreallocation object at 0x7f9bac7c3ef0> preallocation = 'full' @pytest.mark.parametrize("preallocation", [ qemuimg.PREALLOCATION.FALLOC, qemuimg.PREALLOCATION.FULL, ]) def test_qcow2_to_raw_preallocated(self, preallocation): virtual_size = 10 * MiB with namedTemporaryDir() as tmpdir: src = os.path.join(tmpdir, 'src') dst = os.path.join(tmpdir, 'dst') op = qemuimg.create(src, size=virtual_size, format="qcow2") op.run() op = qemuimg.convert(src, dst, srcFormat="qcow2", dstFormat="raw", preallocation=preallocation) op.run() > check_raw_preallocated_image(dst, virtual_size) storage/qemuimg_test.py:561: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ path = '/var/tmp/tmpxr0emprz/dst', virtual_size = 10485760 def check_raw_preallocated_image(path, virtual_size): image_stat = os.stat(path) assert image_stat.st_size == virtual_size > assert image_stat.st_blocks * 512 == virtual_size E assert (20488 * 512) == 10485760 E + where 20488 = os.stat_result(st_mode=33188, st_ino=411528, st_dev=2049, st_nlink=1, st_uid=0, st_gid=0, st_size=10485760, st_atime=1581845207, st_mtime=1581845207, st_ctime=1581845207).st_blocks storage/qemuimg_test.py:621: AssertionError

2 3

somehow, no permission to run VM
by Fedor Gavrilov 15 Feb '20

15 Feb '20

Hi, Have you ever seen an error like this: 2020-02-15 20:05:16,246+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (ForkJoinPool-1-worker-27) [] EVENT_ID: VM_DOWN_ERROR(119), VM vm0 is down with error. Exit message: internal error: child reported (status=125): unable to open /rhev/data-center/mnt/banana:_nfsshare/45e51eaa-b608-40b1-9db6-6e6a1b3ed428/images/9d440b9a-b033-4209-898b-6667a4605281/ead2017a-a7fe-4408-b8d4-1670cb767474: Permission denied. in the engine logs? I'm getting it on my env (master branch) and can't figure out why. I login as admin, I have a perfectly working host and storage machines, on which I created this VM and its disk. Without any actions in between, I try to run it and get this. Storage domain and data center statuses are updated as you expect when you put the host to maintenance and back, so it seems this is the only thing wrong. Do you know what that might be? Thanks, Fedor

3 2

Create VM snapshot with no disk
by scott.fitzgerald＠oracle.com 13 Feb '20

13 Feb '20

Hi, A VM Snapshot can be made using SnapshotBuilder builder = new SnapshotBuilder().vm(VM).name("Snap1").description("Test"); sysService.vmsService().vmService(VM.id()).snapshotsService().add().snapshot(builder).send(); It can be made with no memory by using a builder with persistMemorystate(false) How can one be made with no disk? .diskAttachments(emptyList) isnt working. TIA

3 5

please stop merging
by Dafna Ron 13 Feb '20

13 Feb '20

Hi all, Engine has been failing for a while and someone has also moved the broken package manually to the tested repo which broke all projects in cq. At this time, until we stabilise engine and get a new package into tested, all projects will continue failing cq and no new packages will be moved to the stable repo (tested). if you continue to merge, we not only risk adding more regression but it also makes it more difficult to debug and fix as there are a lot of changes in the queue waiting to be tested and very little resources to run it. Please stop merging on all projects until this is resolved. Thanks, Dafna

4 8