[ovirt-users] Re: oVirt 4.2.5.1-1.el7 JSON-RPC statistics error

I think the upgrade may not have affected the mom package, because no new mom rpm version was released for a few months. I'm not sure why the import fails even if the file is there. As a workaround, mom can be configured to use the older (deprecated) json rpc interface to communicate with vdsm. There is no difference in functionality. This can be done by editing /etc/vdsm/mom.conf. Set the 'hypervisor-interface' config value to 'vdsmjsonrpcbulk' insted of 'vdsmjsonrpcclient'. On Wed, 25 Jul 2018 at 08:56, Maton, Brett <matonb(a)ltresources.co.uk&gt; wrote: > I upgraded my test cluster to 4.2.5.2-1 last night (hosts rebooted after > update) and I've started to see the error again. > > As before it seems to be limited to the one host, I've checked that > > /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/ > vdsmjsonrpcclientInterface.py > > exists, which it does but there are no matching .pyc / .pyo files which > suggests that it's not been used? > > File permissions and SELinux contexts are the same on all files in the > HypervisorInterfaces directory. > > 0644 root root system_u:object_r:lib_t:s0 > > vdsm / mom log from time period attached. > > > On 18 July 2018 at 16:10, Maton, Brett <matonb(a)ltresources.co.uk&gt; wrote: > >> Thanks, >> >> Cluster is all installed from pre-release, maybe I managed to get an >> iffy rpm.... >> >> On 18 July 2018 at 15:50, Andrej Krejcir <akrejcir(a)redhat.com&gt; wrote: >> >>> Yes, copying it from another host with mom version 0.5.12 is enough. >>> >>> I'm not sure why it was not installed. The file is in the rpm. >>> >>> On Wed, 18 Jul 2018 at 16:38, Maton, Brett <matonb(a)ltresources.co.uk&gt; >>> wrote: >>> >>>> Bingo >>>> >>>> How could that file not be installed/deployed ? >>>> >>>> >>>> Should I simply copy it from one of the other hosts to make the >>>> message go away ? >>>> >>>> On 18 July 2018 at 15:30, Andrej Krejcir <akrejcir(a)redhat.com&gt; wrote: >>>> >>>>> Maybe the python module is missing. Can you check if this file exists? >>>>> >>>>> /usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/ >>>>> vdsmjsonrpcclientInterface.py >>>>> >>>>> >>>>> Andrej >>>>> >>>>> On Wed, 18 Jul 2018 at 16:04, Maton, Brett <matonb(a)ltresources.co.uk&gt; >>>>> wrote: >>>>> >>>>>> I just checked the mom version, it's already at 0.5.12 >>>>>> >>>>>> # rpm -qa mom >>>>>> mom-0.5.12-1.el7.centos.noarch >>>>>> >>>>>> All the physical hosts are fully updated /patched. >>>>>> >>>>>> Regards, >>>>>> Brett >>>>>> >>>>>> On 18 July 2018 at 14:08, Andrej Krejcir <akrejcir(a)redhat.com&gt; >>>>>> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Looking at MOM logs, the problem seems to be that MOM is configured >>>>>>> to use 'vdsmjsonrpcclient' module to communicate with vdsm, but it >>>>>>> cannot find this module, probably because it is an older version. >>>>>>> >>>>>>> Updating MOM to version 0.5.12 should fix it. >>>>>>> >>>>>>> >>>>>>> Regards, >>>>>>> Andrej >>>>>>> >>>>>>> On Wed, 18 Jul 2018 at 14:41, Maton, Brett < >>>>>>> matonb(a)ltresources.co.uk&gt; wrote: >>>>>>> >>>>>>>> FWIW: >>>>>>>> >>>>>>>> This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R) >>>>>>>> Xeon(R) CPU E3-1220 V2 @ 3.10GHz >>>>>>>> >>>>>>>> Network is bonded fail-over. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Brett >>>>>>>> >>>>>>>> On 18 July 2018 at 13:15, Francesco Romani <fromani(a)redhat.com&gt; >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Thanks! >>>>>>>>> >>>>>>>>> >>>>>>>>> On 07/18/2018 02:11 PM, Maton, Brett wrote: >>>>>>>>> >>>>>>>>>> Sure no problem, mom log attached. >>>>>>>>>> >>>>>>>>> >>>>>>>>> It seems to me that MOM is restarting *very* often: >>>>>>>>> >>>>>>>>> 2018-07-18 06:07:04,909 - mom - INFO - MOM starting >>>>>>>>> 2018-07-18 06:07:20,407 - mom - INFO - MOM starting >>>>>>>>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting >>>>>>>>> 2018-07-18 06:07:30,658 - mom - INFO - MOM starting >>>>>>>>> >>>>>>>>> And I totally not expect this. Martin, Andrej, is this the >>>>>>>>> expected behaviour? >>>>>>>>> >>>>>>>>> >>>>>>>>> Bests, >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Francesco Romani >>>>>>>>> Senior SW Eng., Virtualization R&D >>>>>>>>> Red Hat >>>>>>>>> IRC: fromani github: @fromanirh >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>> >>>> >> >