I think the upgrade may not have affected the mom package, because no new mom rpm version was released for a few months.

I'm not sure why the import fails even if the file is there.

As a workaround, mom can be configured to use the older (deprecated) json rpc interface to communicate with vdsm. There is no difference in functionality.

This can be done by editing /etc/vdsm/mom.conf. Set the 'hypervisor-interface' config value to 'vdsmjsonrpcbulk' insted of 'vdsmjsonrpcclient'.

On Wed, 25 Jul 2018 at 08:56, Maton, Brett <matonb@ltresources.co.uk> wrote:

I upgraded my test cluster to 4.2.5.2-1 last night (hosts rebooted after update) and I've started to see the error again.

As before it seems to be limited to the one host,  I've checked that

/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py

exists, which it does but there are no matching .pyc / .pyo files which suggests that it's not been used?

File permissions and SELinux contexts are the same on all files in the HypervisorInterfaces directory.

0644 root root system_u:object_r:lib_t:s0

vdsm / mom log from time period attached.

On 18 July 2018 at 16:10, Maton, Brett <matonb@ltresources.co.uk> wrote:

Thanks,

  Cluster is all installed from pre-release, maybe I managed to get an iffy rpm....

On 18 July 2018 at 15:50, Andrej Krejcir <akrejcir@redhat.com> wrote:

Yes, copying it from another host with mom version 0.5.12 is enough.

I'm not sure why it was not installed. The file is in the rpm.

On Wed, 18 Jul 2018 at 16:38, Maton, Brett <matonb@ltresources.co.uk> wrote:

Bingo

How could that file not be installed/deployed ?

Should I simply copy it from one of the other hosts to make the message go away ?

On 18 July 2018 at 15:30, Andrej Krejcir <akrejcir@redhat.com> wrote:

Maybe the python module is missing. Can you check if this file exists?

/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmjsonrpcclientInterface.py

Andrej

On Wed, 18 Jul 2018 at 16:04, Maton, Brett <matonb@ltresources.co.uk> wrote:

I just checked the mom version, it's already at 0.5.12

# rpm -qa mom
mom-0.5.12-1.el7.centos.noarch

All the physical hosts are fully updated /patched.

Regards,

Brett

On 18 July 2018 at 14:08, Andrej Krejcir <akrejcir@redhat.com> wrote:

Hi,

Looking at MOM logs, the problem seems to be that MOM is configured to use 'vdsmjsonrpcclient' module to communicate with vdsm, but it cannot find this module, probably because it is an older version.

Updating MOM to version 0.5.12 should fix it.

Regards,

Andrej

On Wed, 18 Jul 2018 at 14:41, Maton, Brett <matonb@ltresources.co.uk> wrote:

FWIW:

This test cluster is 3x HP MicroServer Gen 8 16GB RAM, Intel(R) Xeon(R) CPU E3-1220 V2 @ 3.10GHz

Network is bonded fail-over.

Regards,

Brett

On 18 July 2018 at 13:15, Francesco Romani <fromani@redhat.com> wrote:

Thanks!


On 07/18/2018 02:11 PM, Maton, Brett wrote:

Sure no problem, mom log attached.

It seems to me that MOM is restarting *very* often:

2018-07-18 06:07:04,909 - mom - INFO - MOM starting
2018-07-18 06:07:20,407 - mom - INFO - MOM starting
2018-07-18 06:07:30,658 - mom - INFO - MOM starting
2018-07-18 06:07:30,658 - mom - INFO - MOM starting

And I totally not expect this. Martin, Andrej, is this the expected behaviour?


Bests,

--
Francesco Romani
Senior SW Eng., Virtualization R&D
Red Hat
IRC: fromani github: @fromanirh