- Devel - oVirt List Archives

VDSM API schema inconsistencies
by Marcin Sobczyk 16 Oct '18

16 Oct '18

Hi, when I was working on BZ#1624306 it turned out, that even if I'd fix the help-printing code in vdsm-client, there's still a lot of inconsistencies in API's schema files. After some investigation, I found that there's a code for reporting schema inconsistencies, but it's turned off because it bloats the logs. There's also a "strict" mode that raises an exception each time an inconsistency is found, but it's also off because there's so many of them. I think that no one wants to maintain both: the actual code and schema files. My idea would be to make the schema derived from the code at build time as a long-term effort. Before we can do that though, we need to address the inconsistencies first. https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:v… This is a series of patches that helps fixing schema issues. What it does, is it adds a bunch of information from stack to vastly ease the process of fixing them. The logging level is also changed from 'WARNING' to 'DEBUG'. Here's an example of a logged entry: Host.setKsmTune With message: Required property pages_to_scan is not provided when calling Host.setKsmTune With backtrace: [ [ "_verify_object_type", { "call_arg_keys":[ "merge_across_nodes", "run" ], "schema_type_name":"KsmTuneParams" } ], [ "_verify_complex_type", { "schema_type_type":"object" } ] ] To make it work, a patch for OST's 'basic-master-suite' is on the way that switches 'schema.inconsistency' logger's logging level do 'DEBUG'. That way, we can find all reported errors in 'vdsm-schema-error.log' files. An initial report that groups reported errors by namespaces/methods has also been made. Please note, that an implementation that completely avoids error duplicates is really hard and IMHO not worth the effort. You can find the results here: * Host: https://pastebin.com/YM2XnvTV * Image: https://pastebin.com/GRc1yErL * LVMVolumeGroup: https://pastebin.com/R1276gTu * SDM: https://pastebin.com/P3tfYEDD * StorageDomain: https://pastebin.com/Q0DxKgF5 * StoragePool: https://pastebin.com/VXFWpfRC * VM: https://pastebin.com/tDC60c29 * Volume: https://pastebin.com/TYkr9Zsd * |jobs|: https://pastebin.com/jeXpYyz9 * |virt|: https://pastebin.com/nRREuEub Regards, Marcin

2 4

vdsm has been tagged (v4.20.43)
by Milan Zamazal 16 Oct '18

16 Oct '18

Vdsm 4.20.43 for oVirt 4.2.7 rc3

1 0

vdsm has been tagged (v4.30.1)
by Milan Zamazal 15 Oct '18

15 Oct '18

Vdsm 4.30.1 for the first 4.3 build from master

1 0

Upgrade to WildFly 14.0.1
by Martin Perina 12 Oct '18

12 Oct '18

Hi, WildFly 14 packages are already available for both CentOS 7 and FC28, both oVirt engine master and 4.2 already contain fixes for compatibility with both WildFly 13 and 14, but please update your development machines to contain both ovirt-engine-wildfly-14.0.1 and ovirt-engine-wildfly-overlay-14.0.1 packages. Patch adding requirement for WildFly 14 is prepared [1], it should be merged tomorrow (Thursday), backport to 4.2 will follow immediately. Anyway if you find any issues related to this WildFly upgrade, please report a bug on it asap, we need to have everything fixed in 4.2.7. Thanks Martin [1] https://gerrit.ovirt.org/94824 -- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.

1 1

diskunmap hooks and oVirt Node
by Sandro Bonazzola 11 Oct '18

11 Oct '18

Hi, we just got Bug 1638317 <https://bugzilla.redhat.com/show_bug.cgi?id=1638317> - "missing VDSM hook diskunmap in Node NG releases" opened as a bug, not an RFE, because "Pass discard from guest to underlying storage <https://www.ovirt.org/develop/release-management/features/storage/pass-disc…>" features introduced in oVirt 4.1 is not yet implemented for Cinder storage, for which Bug 1440230 <https://bugzilla.redhat.com/show_bug.cgi?id=1440230> - "[RFE] Allow "Pass discard from guest to underlying storage" for Cinder. " has been opened. I'm writing to people involved in the hook introduction ( https://gerrit.ovirt.org/#/c/29770/) to understand how safe is to include the hook in oVirt Node as default installed hook. I understand that the hook is going to add "discard=unmap" always, not only on Cinder. I don't know the implications of it being enabled other than supposedly fix the issue with Cinder storage. Looking at the feature page looks like this won't work with NFS storage, but other than not working, will it cause issues? I see Bug 1440230 <https://bugzilla.redhat.com/show_bug.cgi?id=1440230> is un-targeted, is there any plan to get it into oVirt 4.3? We support Cinder/Ceph since 3.6 and pass discard is supported since 4.1, not sure about what prevented the pass discard to be implemented for Cinder as well in 4.1. Can someone elaborate? Thanks, -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig> <https://www.redhat.com/en/events/red-hat-open-source-day-italia?sc_cid=701f…>

2 1

discussing the future of the upgrade suite in ost
by Dafna Ron 11 Oct '18

11 Oct '18

Hi All, I was reviewing the upgrade suites in ost and there are some issues that I am seeing in the suite tests-scenarios which I want to discuss and decide the future of. At it current state, I think we should remove the upgrade suite or most of the post test-scenarios as it is not testing what it should. The tests currently only test engine upgrade and basic sanity after the upgrade. This is problematic in a few ways: 1. upgrade should test the upgrade of rhv and not just a clean engine upgrade (i.e host, storage, vm). 2. as we have limited resources I do not think that the upgrade suite should be longer then the basic suite (and as we are currently running the basic suite after the upgrade it is longer) That brings me to question what should be essential to test in upgrade in the CI? I would also need someone in dev to volunteer and take ownership of the testing scenarios for upgrade - is there anyone that can help? Thanks, Dafna

4 4

Fwd: VDSM API schema inconsistencies
by Nir Soffer 10 Oct '18

10 Oct '18

Adding devel. ---------- Forwarded message --------- From: Nir Soffer <nsoffer(a)redhat.com> Date: Wed, 10 Oct 2018, 11:07 Subject: Re: VDSM API schema inconsistencies To: Piotr Kliczewski <pkliczew(a)redhat.com> Cc: Dan Kenigsberg <danken(a)redhat.com>, Edward Haas <ehaas(a)redhat.com>, Milan Zamazal <mzamazal(a)redhat.com>, Tomasz Baranski <tbaransk(a)redhat.com>, Francesco Romani <fromani(a)redhat.com>, Irit Goihman <igoihman(a)redhat.com>, Marcin Sobczyk <msobczyk(a)redhat.com> On Wed, 10 Oct 2018, 10:21 Piotr Kliczewski, <pkliczew(a)redhat.com> wrote: > +Nir > > On Wed, Oct 10, 2018 at 8:39 AM Marcin Sobczyk <msobczyk(a)redhat.com> > wrote: > >> Hi, >> I would love to get some feedback from you about it on the VDSM call >> today so if you have a moment, please take a look. >> Marcin >> >> ---------- Forwarded message --------- >> From: Marcin Sobczyk <msobczyk(a)redhat.com> >> Date: Mon, Oct 8, 2018 at 2:40 PM >> Subject: VDSM API schema inconsistencies >> To: devel <devel(a)ovirt.org> >> >> >> Hi, >> >> when I was working on BZ#1624306 it turned out, that even if I'd fix the >> help-printing code in vdsm-client, there's still a lot of inconsistencies >> in API's schema files. >> >> After some investigation, I found that there's a code for reporting >> schema inconsistencies, but it's turned off because it bloats the logs. >> > Right, an it should remain like this. In runtime we never want these warnings. There's also a "strict" mode that raises an exception each time an >> inconsistency is found, but it's also off because there's so many of them. >> > Same. Both are debugging aids for developers, to make it easy to check the schema. > I think that no one wants to maintain both: the actual code and schema >> files. >> > This is not a big issue. We rarely add or change APIs. But it is nice if we don't need to manualy make the schema correct. > My idea would be to make the schema derived from the code at build time as >> a long-term effort. >> > This cannot work, since clients depend on the schema, it should not change if when someone makes uintended change in the code. It should work in the opposite way, you change the schema, and generate new code frome schema. Threre are existing tools like grpc and thrift that work in this way. For long term we should drop our jsonrpc/stomp stack and move to grpc. For short term we can just fix the schema to reflect actual engine interaction. when engine and schema do not match, engine is usualy right. For responses, if engine can handle the response the code is right. Please do not make RPC code more complex and inefficient for improving schema issues detection. I would rather remove this code instead. Before we can do that though, we need to address the inconsistencies first. >> >> >> https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic:v… >> >> This is a series of patches that helps fixing schema issues. What it >> does, is it adds a bunch of information from stack to vastly ease the >> process of fixing them. The logging level is also changed from 'WARNING' to >> 'DEBUG'. Here's an example of a logged entry: >> >> Host.setKsmTune >> With message: Required property pages_to_scan is not provided when >> calling Host.setKsmTune >> With backtrace: [ >> [ >> "_verify_object_type", >> { >> "call_arg_keys":[ >> "merge_across_nodes", >> "run" >> ], >> "schema_type_name":"KsmTuneParams" >> } >> ], >> [ >> "_verify_complex_type", >> { >> "schema_type_type":"object" >> } >> ] >> ] >> >> To make it work, a patch for OST's 'basic-master-suite' is on the way >> that switches 'schema.inconsistency' logger's logging level do 'DEBUG'. >> That way, we can find all reported errors in 'vdsm-schema-error.log' files. >> >> An initial report that groups reported errors by namespaces/methods has >> also been made. Please note, that an implementation that completely avoids >> error duplicates is really hard and IMHO not worth the effort. You can find >> the results here: >> >> - Host: https://pastebin.com/YM2XnvTV >> - Image: https://pastebin.com/GRc1yErL >> - LVMVolumeGroup: https://pastebin.com/R1276gTu >> - SDM: https://pastebin.com/P3tfYEDD >> - StorageDomain: https://pastebin.com/Q0DxKgF5 >> - StoragePool: https://pastebin.com/VXFWpfRC >> - VM: https://pastebin.com/tDC60c29 >> - Volume: https://pastebin.com/TYkr9Zsd >> - |jobs|: https://pastebin.com/jeXpYyz9 >> - |virt|: https://pastebin.com/nRREuEub >> >> File a bug per vertical (infra, storage, virt) for these issues. Since we don't know about real user issues with these apis this must be incorrect schema. Nir

2 2

[VDSM] Do not use commands.execCmd(cmd, sync=False) or AsyncProcessOperation
by Nir Soffer 10 Oct '18

10 Oct '18

I just found that this patch was merged last month: https://gerrit.ovirt.org/c/94269/ Adding another use of commands.execCmd(cmd, sync=False) and AsyncProcessOperation. These APIs are deprecated and will be removed soon, please do merge any code using them. Also running commands anachronistically with a timeout is a bad idea in general. If the command is stuck on storage, we will created huge amount of commands. We have the same bad pattern in storage (iscsi/fcp rescan), it should be replaced by running command synchronically (sync=False) in a dedicated thread, and never running more then one command if the command get stuck. Marcin is working on the new improved infrastructure for running commands, based on https://gerrit.ovirt.org/c/83923. Nir

1 0

[ OST Failure Report ] [ oVirt master (vdsm) ] [ 09-10-2018 ] [ 005_network_by_label ]
by Daniel Belenky 09 Oct '18

09 Oct '18

Hi, The following patch is suspected to be failing OST: https://gerrit.ovirt.org/#/c/94754/2 Error snippet from vdsm log: 2018-10-09 08:51:50,799-0400 ERROR (Thread-7) [root] Shutdown by QEMU Guest Agent failed (vm:5269) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5260, in qemuGuestAgentShutdown self._dom.shutdownFlags(libvirt.VIR_DOMAIN_SHUTDOWN_GUEST_AGENT) File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 100, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/common/function.py", line 94, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2482, in shutdownFlags if ret == -1: raise libvirtError ('virDomainShutdownFlags() failed', dom=self) libvirtError: Guest agent is not responding: QEMU guest agent is not connected -- DANIEL BELENKY

2 1

BUG - Cannot install Host with empty password.
by Shawn Weeks 04 Oct '18

04 Oct '18

Is there a current open bug for https://bugzilla.redhat.com/show_bug.cgi?id=1057575. It's still happening in the current release but it should have been fixed back in 2016. Guide me fails with this error but add host works fine. Thanks Shawn

1 0