Fwd: VDSM API schema inconsistencies
by Nir Soffer
Adding devel.
---------- Forwarded message ---------
From: Nir Soffer <nsoffer(a)redhat.com>
Date: Wed, 10 Oct 2018, 11:07
Subject: Re: VDSM API schema inconsistencies
To: Piotr Kliczewski <pkliczew(a)redhat.com>
Cc: Dan Kenigsberg <danken(a)redhat.com>, Edward Haas <ehaas(a)redhat.com>,
Milan Zamazal <mzamazal(a)redhat.com>, Tomasz Baranski <tbaransk(a)redhat.com>,
Francesco Romani <fromani(a)redhat.com>, Irit Goihman <igoihman(a)redhat.com>,
Marcin Sobczyk <msobczyk(a)redhat.com>
On Wed, 10 Oct 2018, 10:21 Piotr Kliczewski, <pkliczew(a)redhat.com> wrote:
> +Nir
>
> On Wed, Oct 10, 2018 at 8:39 AM Marcin Sobczyk <msobczyk(a)redhat.com>
> wrote:
>
>> Hi,
>> I would love to get some feedback from you about it on the VDSM call
>> today so if you have a moment, please take a look.
>> Marcin
>>
>> ---------- Forwarded message ---------
>> From: Marcin Sobczyk <msobczyk(a)redhat.com>
>> Date: Mon, Oct 8, 2018 at 2:40 PM
>> Subject: VDSM API schema inconsistencies
>> To: devel <devel(a)ovirt.org>
>>
>>
>> Hi,
>>
>> when I was working on BZ#1624306 it turned out, that even if I'd fix the
>> help-printing code in vdsm-client, there's still a lot of inconsistencies
>> in API's schema files.
>>
>> After some investigation, I found that there's a code for reporting
>> schema inconsistencies, but it's turned off because it bloats the logs.
>>
> Right, an it should remain like this. In runtime
we never want these warnings.
There's also a "strict" mode that raises an exception each time an
>> inconsistency is found, but it's also off because there's so many of them.
>>
>
Same.
Both are debugging aids for developers, to
make it easy to check the schema.
> I think that no one wants to maintain both: the actual code and schema
>> files.
>>
>
This is not a big issue. We rarely add or
change APIs. But it is nice if we don't need to
manualy make the schema correct.
> My idea would be to make the schema derived from the code at build time as
>> a long-term effort.
>>
>
This cannot work, since clients depend on the schema, it should not change
if when someone
makes uintended change in the code.
It should work in the opposite way, you change the schema, and generate new
code frome schema.
Threre are existing tools like grpc and thrift that
work in this way. For long term we should drop
our jsonrpc/stomp stack and move to grpc.
For short term we can just fix the schema to
reflect actual engine interaction. when engine
and schema do not match, engine is usualy
right. For responses, if engine can handle the
response the code is right.
Please do not make RPC code more complex
and inefficient for improving schema issues
detection. I would rather remove this code
instead.
Before we can do that though, we need to address the inconsistencies first.
>>
>>
>> https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic...
>>
>> This is a series of patches that helps fixing schema issues. What it
>> does, is it adds a bunch of information from stack to vastly ease the
>> process of fixing them. The logging level is also changed from 'WARNING' to
>> 'DEBUG'. Here's an example of a logged entry:
>>
>> Host.setKsmTune
>> With message: Required property pages_to_scan is not provided when
>> calling Host.setKsmTune
>> With backtrace: [
>> [
>> "_verify_object_type",
>> {
>> "call_arg_keys":[
>> "merge_across_nodes",
>> "run"
>> ],
>> "schema_type_name":"KsmTuneParams"
>> }
>> ],
>> [
>> "_verify_complex_type",
>> {
>> "schema_type_type":"object"
>> }
>> ]
>> ]
>>
>> To make it work, a patch for OST's 'basic-master-suite' is on the way
>> that switches 'schema.inconsistency' logger's logging level do 'DEBUG'.
>> That way, we can find all reported errors in 'vdsm-schema-error.log' files.
>>
>> An initial report that groups reported errors by namespaces/methods has
>> also been made. Please note, that an implementation that completely avoids
>> error duplicates is really hard and IMHO not worth the effort. You can find
>> the results here:
>>
>> - Host: https://pastebin.com/YM2XnvTV
>> - Image: https://pastebin.com/GRc1yErL
>> - LVMVolumeGroup: https://pastebin.com/R1276gTu
>> - SDM: https://pastebin.com/P3tfYEDD
>> - StorageDomain: https://pastebin.com/Q0DxKgF5
>> - StoragePool: https://pastebin.com/VXFWpfRC
>> - VM: https://pastebin.com/tDC60c29
>> - Volume: https://pastebin.com/TYkr9Zsd
>> - |jobs|: https://pastebin.com/jeXpYyz9
>> - |virt|: https://pastebin.com/nRREuEub
>>
>>
File a bug per vertical (infra, storage, virt) for
these issues. Since we don't know about real user issues with these apis
this must be incorrect schema.
Nir
6 years, 2 months
[VDSM] Do not use commands.execCmd(cmd, sync=False) or AsyncProcessOperation
by Nir Soffer
I just found that this patch was merged last month:
https://gerrit.ovirt.org/c/94269/
Adding another use of commands.execCmd(cmd, sync=False) and
AsyncProcessOperation.
These APIs are deprecated and will be removed soon, please do merge any
code using
them.
Also running commands anachronistically with a timeout is a bad idea in
general.
If the command is stuck on storage, we will created huge amount of commands.
We have the same bad pattern in storage (iscsi/fcp rescan), it should be
replaced by
running command synchronically (sync=False) in a dedicated thread, and never
running more then one command if the command get stuck.
Marcin is working on the new improved infrastructure for running commands,
based
on https://gerrit.ovirt.org/c/83923.
Nir
6 years, 2 months
[ OST Failure Report ] [ oVirt master (vdsm) ] [ 09-10-2018 ] [ 005_network_by_label ]
by Daniel Belenky
Hi,
The following patch is suspected to be failing OST:
https://gerrit.ovirt.org/#/c/94754/2
Error snippet from vdsm log:
2018-10-09 08:51:50,799-0400 ERROR (Thread-7) [root] Shutdown by QEMU
Guest Agent failed (vm:5269)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 5260,
in qemuGuestAgentShutdown
self._dom.shutdownFlags(libvirt.VIR_DOMAIN_SHUTDOWN_GUEST_AGENT)
File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py", line 100, in f
ret = attr(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/common/libvirtconnection.py",
line 131, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/common/function.py",
line 94, in wrapper
return func(inst, *args, **kwargs)
File "/usr/lib64/python2.7/site-packages/libvirt.py", line 2482, in
shutdownFlags
if ret == -1: raise libvirtError ('virDomainShutdownFlags()
failed', dom=self)
libvirtError: Guest agent is not responding: QEMU guest agent is not connected
--
DANIEL BELENKY
6 years, 2 months
Adding Fedora 28 host fail with "NotImplementedError: Packager install not implemented"
by Nir Soffer
Trying to add Fedora 28 host to engine master, host installation fails
immediately with:
2018-10-02 20:34:45,440+0300 DEBUG otopi.plugins.otopi.dialog.machine
dialog.__logString:204 DIALOG:SEND **%EventStart STAGE
internal_packages METHOD otopi.plugins.ovirt_host_deploy.vd
sm.vdsmid.Plugin._packages (None)
2018-10-02 20:34:45,443+0300 DEBUG otopi.context context._executeMethod:143
method exception
Traceback (most recent call last):
File "/tmp/ovirt-qh2D5sbRmG/pythonlib/otopi/context.py", line 133, in
_executeMethod
method['method']()
File
"/tmp/ovirt-qh2D5sbRmG/otopi-plugins/ovirt-host-deploy/vdsm/vdsmid.py",
line 84, in _packages
self.packager.install(('dmidecode',))
File "/tmp/ovirt-qh2D5sbRmG/pythonlib/otopi/packager.py", line 102, in
install
raise NotImplementedError(_('Packager install not implemented'))
NotImplementedError: Packager install not implemented
2018-10-02 20:34:45,444+0300 ERROR otopi.context context._executeMethod:152
Failed to execute stage 'Environment packages setup': Packager install not
implemented
engine version:
4.3.0_master (efabde5a36cbf85b43a60ead37e42943c35c2741)
Is this a known issue? any workaround?
Nir
6 years, 2 months
Re: Issue with vdsm-hook-vmfex-dev = 4.20.40-11.git6d99cc8.el7 shipping on u/s 4.2
by Eyal Edri
Adding devel and infra as well.
On Wed, Oct 3, 2018 at 3:10 PM Michael Burman <mburman(a)redhat.com> wrote:
> Hi
>
> I think we have an issue with vdsm-hook-vmfex-dev on 4.2 u/s
>
> When i try to update i get this Dependency Resolution
>
> Error: Package: vdsm-4.20.40-11.git6d99cc8.el7.x86_64 (ovirt-4.2-snapshot)
> Requires: vdsm-hook-vmfex-dev = 4.20.40-11.git6d99cc8.el7
> Installed: vdsm-hook-vmfex-dev-4.20.40-10.giteffb1ef.el7.noarch
> (@ovirt-4.2-snapshot)
> vdsm-hook-vmfex-dev = 4.20.40-10.giteffb1ef.el7
> Available: vdsm-hook-vmfex-dev-4.20.23-1.el7.noarch
> (ovirt-4.2-centos-ovirt42)
> vdsm-hook-vmfex-dev = 4.20.23-1.el7
> Available: vdsm-hook-vmfex-dev-4.20.39.1-1.el7.noarch
> (ovirt-4.2-centos-ovirt42)
> vdsm-hook-vmfex-dev = 4.20.39.1-1.el7
> Error: Package: vdsm-hook-vmfex-dev-4.20.40-10.giteffb1ef.el7.noarch
> (@ovirt-4.2-snapshot)
> Requires: vdsm = 4.20.40-10.giteffb1ef.el7
> Removing: vdsm-4.20.40-10.giteffb1ef.el7.x86_64
> (@ovirt-4.2-snapshot)
> vdsm = 4.20.40-10.giteffb1ef.el7
> Updated By: vdsm-4.20.40-11.git6d99cc8.el7.x86_64
> (ovirt-4.2-snapshot)
> vdsm = 4.20.40-11.git6d99cc8.el7
> Available: vdsm-4.20.23-1.el7.x86_64 (ovirt-4.2-centos-ovirt42)
> vdsm = 4.20.23-1.el7
> Available: vdsm-4.20.39.1-1.el7.x86_64
> (ovirt-4.2-centos-ovirt42)
> vdsm = 4.20.39.1-1.el7
>
>
> https://resources.ovirt.org/pub/ovirt-4.2-snapshot/rpm/el7Server/noarch/
>
> Is missing the correct version and this is the problem
>
> Thanks,
> --
>
> Michael Burman
>
> Senior Quality engineer - rhv network - redhat israel
>
> Red Hat
>
> <https://www.redhat.com>
>
> mburman(a)redhat.com M: 0545355725 IM: mburman
> <https://red.ht/sig>
>
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/>
<https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
6 years, 2 months
[ENGINE] make install-dev fails, trying to install outside of ENGINE_PREFIX
by Nir Soffer
Trying to make install-dev with engine master:
$ git clean -dxf
$ git log | head -1
commit efabde5a36cbf85b43a60ead37e42943c35c2741
The build failes, trying to install stuff out of ENGINE_PREFIX:
$ make install-dev ENGINE_PREFIX=/home/nsoffer/ovirt-engine
...
make copy-recursive SOURCEDIR=packaging/sys-etc
TARGETDIR="/usr/local/etc"...
make[2]: Entering directory `/home/nsoffer/src/ovirt-engine'
( cd "packaging/sys-etc" && find . -type d -printf '%P\n' ) | while read d;
do \
install -d -m 755 "/usr/local/etc/${d}"; \
done
install: cannot change permissions of ‘/usr/local/etc/logrotate.d’: No such
file or directory
Is this know issue? any workaround?
Thanks,
Nir
6 years, 2 months
