On Mon, Oct 8, 2018 at 3:41 PM Marcin Sobczyk <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> wrote: Hi, when I was working on BZ#1624306 it turned out, that even if I'd fix the help-printing code in vdsm-client, there's still a lot of inconsistencies in API's schema files. After some investigation, I found that there's a code for reporting schema inconsistencies, but it's turned off because it bloats the logs. There's also a "strict" mode that raises an exception each time an inconsistency is found, but it's also off because there's so many of them. I think that no one wants to maintain both: the actual code and schema files. My idea would be to make the schema derived from the code at build time as a long-term effort. Before we can do that though, we need to address the inconsistencies first. The schema is supposed to be the specification and the code its implementation. Generating the schema from the code does not sound right to me. https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic... This is a series of patches that helps fixing schema issues. What it does, is it adds a bunch of information from stack to vastly ease the process of fixing them. The logging level is also changed from 'WARNING' to 'DEBUG'. Here's an example of a logged entry: Host.setKsmTune With message: Required property pages_to_scan is not provided when calling Host.setKsmTune With backtrace: [         [                 "_verify_object_type",                 {                         "call_arg_keys":[                                 "merge_across_nodes",                                 "run"                         ], "schema_type_name":"KsmTuneParams"                 }         ],         [                 "_verify_complex_type",                 {                         "schema_type_type":"object"                 }         ] ] To make it work, a patch for OST's 'basic-master-suite' is on the way that switches 'schema.inconsistency' logger's logging level do 'DEBUG'. That way, we can find all reported errors in 'vdsm-schema-error.log' files. Will we be able to detect the caller?

An initial report that groups reported errors by namespaces/methods has also been made. Please note, that an implementation that completely avoids error duplicates is really hard and IMHO not worth the effort. You can find the results here: * Host: https://pastebin.com/YM2XnvTV We will need to cover the network stuff in here, thanks. * Image: https://pastebin.com/GRc1yErL * LVMVolumeGroup: https://pastebin.com/R1276gTu * SDM: https://pastebin.com/P3tfYEDD * StorageDomain: https://pastebin.com/Q0DxKgF5 * StoragePool: https://pastebin.com/VXFWpfRC * VM: https://pastebin.com/tDC60c29 * Volume: https://pastebin.com/TYkr9Zsd * |jobs|: https://pastebin.com/jeXpYyz9 * |virt|: https://pastebin.com/nRREuEub Regards, Marcin _______________________________________________ Devel mailing list -- devel(a)ovirt.org <mailto:devel@ovirt.org> To unsubscribe send an email to devel-leave(a)ovirt.org <mailto:devel-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/NW67DZSIMQT...

On 10/10/18 12:53 PM, Marcin Sobczyk wrote: On 10/10/18 9:57 AM, Edward Haas wrote: On Mon, Oct 8, 2018 at 3:41 PM Marcin Sobczyk <msobczyk(a)redhat.com&gt; wrote: > Hi, > > when I was working on BZ#1624306 it turned out, that even if I'd fix the > help-printing code in vdsm-client, there's still a lot of inconsistencies > in API's schema files. > > After some investigation, I found that there's a code for reporting > schema inconsistencies, but it's turned off because it bloats the logs. > There's also a "strict" mode that raises an exception each time an > inconsistency is found, but it's also off because there's so many of them. > > I think that no one wants to maintain both: the actual code and schema > files. My idea would be to make the schema derived from the code at build > time as a long-term effort. Before we can do that though, we need to > address the inconsistencies first. > The schema is supposed to be the specification and the code its implementation. Generating the schema from the code does not sound right to me. > > https://gerrit.ovirt.org/#/q/status:open+project:vdsm+branch:master+topic... > > This is a series of patches that helps fixing schema issues. What it > does, is it adds a bunch of information from stack to vastly ease the > process of fixing them. The logging level is also changed from 'WARNING' to > 'DEBUG'. Here's an example of a logged entry: > > Host.setKsmTune > With message: Required property pages_to_scan is not provided when > calling Host.setKsmTune > With backtrace: [ > [ > "_verify_object_type", > { > "call_arg_keys":[ > "merge_across_nodes", > "run" > ], > "schema_type_name":"KsmTuneParams" > } > ], > [ > "_verify_complex_type", > { > "schema_type_type":"object" > } > ] > ] > > To make it work, a patch for OST's 'basic-master-suite' is on the way > that switches 'schema.inconsistency' logger's logging level do 'DEBUG'. > That way, we can find all reported errors in 'vdsm-schema-error.log' files. > Will we be able to detect the caller? I will look at it. I can include the context string in the log. Example entries: With context string: from=::ffff:10.37.128.217,49550, flow_id=22456882 With context string: from=::1,51550 AFAIK you can trace back the caller by the 'flow_id'. Would that kind of information suffice?

An initial report that groups reported errors by namespaces/methods has > also been made. Please note, that an implementation that completely avoids > error duplicates is really hard and IMHO not worth the effort. You can find > the results here: > > - Host: https://pastebin.com/YM2XnvTV > > We will need to cover the network stuff in here, thanks. > > - Image: https://pastebin.com/GRc1yErL > - LVMVolumeGroup: https://pastebin.com/R1276gTu > - SDM: https://pastebin.com/P3tfYEDD > - StorageDomain: https://pastebin.com/Q0DxKgF5 > - StoragePool: https://pastebin.com/VXFWpfRC > - VM: https://pastebin.com/tDC60c29 > - Volume: https://pastebin.com/TYkr9Zsd > - |jobs|: https://pastebin.com/jeXpYyz9 > - |virt|: https://pastebin.com/nRREuEub > > Regards, Marcin > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/NW67DZSIMQT... >