https://jenkins.ovirt.org/job/ovirt-system-tests_hc-basic-suite-4.2/235/artifact/exported-artifacts/test_logs/hc-basic-suite-4.2/post-002_bootstrap.py/lago-hc-basic-suite-4-2-host0/_var_log/vdsm/supervdsm.log

In supervdsm:

MainProcess|jsonrpc/3::ERROR::2018-06-05 21:12:42,094::supervdsm_server::100::SuperVdsm.ServerCallback::(wrapper) Error in vdoVolumeList
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vdsm/supervdsm_server.py", line 98, in wrapper
    res = func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vdsm/gluster/thinstorage.py", line 109, in vdoVolumeList
    for vdo, data in iteritems(vdoData["VDOs"]):
  File "/usr/lib/python2.7/site-packages/six.py", line 599, in iteritems
    return d.iteritems(**kw)
AttributeError: 'str' object has no attribute 'iteritems'

Following this issue, HE VM goes up and down without logging anything meaningful on libvirt side:

Jun  5 21:17:04 lago-hc-basic-suite-4-2-host0 vdsm[19399]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished?
Jun  5 21:17:19 lago-hc-basic-suite-4-2-host0 vdsm[19399]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished?
Jun  5 21:17:20 lago-hc-basic-suite-4-2-host0 kvm: 1 guest now active
Jun  5 21:17:20 lago-hc-basic-suite-4-2-host0 kvm: 0 guests now active
Jun  5 21:17:20 lago-hc-basic-suite-4-2-host0 kvm: 1 guest now active
Jun  5 21:17:31 lago-hc-basic-suite-4-2-host0 kvm: 0 guests now active
Jun  5 21:17:33 lago-hc-basic-suite-4-2-host0 python: ansible-command Invoked with warn=True executable=None _uses_shell=False _raw_params=virt-copy-out -a /var/tmp/localvm363phZ/images/f57df841-51ed-45e3-b888-fa43d1c185e4/23e54499-3a21-453b-a816-c51ca11b9c7f /etc/hosts "/var/tmp/localvm363phZ" removes=None creates=None chdir=None stdin=None
Jun  5 21:17:33 lago-hc-basic-suite-4-2-host0 kvm: 1 guest now active
Jun  5 21:17:34 lago-hc-basic-suite-4-2-host0 vdsm[19399]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished?
Jun  5 21:17:43 lago-hc-basic-suite-4-2-host0 kvm: 0 guests now active
Jun  5 21:17:45 lago-hc-basic-suite-4-2-host0 python: ansible-lineinfile Invoked with directory_mode=None force=None remote_src=None backrefs=False insertafter=None path=/var/tmp/localvm363phZ/hosts owner=None follow=False line=None group=None insertbefore=None unsafe_writes=None create=False state=absent content=NOT_LOGGING_PARAMETER serole=None setype=None dest=/var/tmp/localvm363phZ/hosts selevel=None regexp=# hosted-engine-setup-/var/tmp/localvm363phZ$ validate=None src=None seuser=None delimiter=None mode=None firstmatch=False attributes=None backup=False
Jun  5 21:17:48 lago-hc-basic-suite-4-2-host0 python: ansible-command Invoked with warn=True executable=None _uses_shell=False _raw_params=virt-copy-in -a /var/tmp/localvm363phZ/images/f57df841-51ed-45e3-b888-fa43d1c185e4/23e54499-3a21-453b-a816-c51ca11b9c7f "/var/tmp/localvm363phZ"/hosts /etc removes=None creates=None chdir=None stdin=None
Jun  5 21:17:49 lago-hc-basic-suite-4-2-host0 kvm: 1 guest now active
Jun  5 21:17:49 lago-hc-basic-suite-4-2-host0 vdsm[19399]: ERROR failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished?
Jun  5 21:17:59 lago-hc-basic-suite-4-2-host0 kvm: 0 guests now active

SANDRO BONAZZOLA

ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D

Red Hat EMEA

sbonazzo@redhat.com