<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Tue, Jan 9, 2018 at 12:04 PM, Peter Hudec <span dir="ltr"><<a href="mailto:phudec@cnc.sk" target="_blank">phudec@cnc.sk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">quick fix is follow the<br>
<a href="https://gerrit.ovirt.org/#/c/84802/2/backend/manager/modules/utils/src/main/java/org/ovirt/engine/core/utils/ovf/IOvfBuilder.java" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>84802/2/backend/manager/<wbr>modules/utils/src/main/java/<wbr>org/ovirt/engine/core/utils/<wbr>ovf/IOvfBuilder.java</a><br>
<br>
and remove trailing '/' in<br>
/usr/lib/python2.7/site-<wbr>packages/ovirt_hosted_engine_<wbr>ha/lib/ovf/ovf2VmParams.py<br></blockquote><div><br></div><div>Adding Denis. Thanks for the logs!<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
<br>
On 09/01/2018 10:45, Peter Hudec wrote:<br>
> the old hypervisoer /oVirt 4.1.8/ got probblem to release the HE due<br>
> this exception. The HE is on the NFS store.<br>
><br>
> MainThread::INFO::2018-01-09<br>
> 10:40:28,497::upgrade::998::<wbr>ovirt_hosted_engine_ha.lib.<wbr>upgrade.StorageServer::(<wbr>upgrade_35_36)<br>
> Host configuration is already up-to-date<br>
> MainThread::INFO::2018-01-09<br>
> 10:40:28,498::config::493::<wbr>ovirt_hosted_engine_ha.agent.<wbr>hosted_engine.HostedEngine.<wbr>config::(refresh_vm_conf)<br>
> Reloading vm.conf from the shared storage domain<br>
> MainThread::INFO::2018-01-09<br>
> 10:40:28,498::config::416::<wbr>ovirt_hosted_engine_ha.agent.<wbr>hosted_engine.HostedEngine.<wbr>config::(_get_vm_conf_content_<wbr>from_ovf_store)<br>
> Trying to get a fresher copy of vm configuration from the OVF_STORE<br>
> MainThread::INFO::2018-01-09<br>
> 10:40:28,498::ovf_store::132::<wbr>ovirt_hosted_engine_ha.lib.<wbr>ovf.ovf_store.OVFStore::(<wbr>getEngineVMOVF)<br>
> Extracting Engine VM OVF from the OVF_STORE<br>
> MainThread::INFO::2018-01-09<br>
> 10:40:28,498::ovf_store::134::<wbr>ovirt_hosted_engine_ha.lib.<wbr>ovf.ovf_store.OVFStore::(<wbr>getEngineVMOVF)<br>
> OVF_STORE volume path:<br>
> /var/run/vdsm/storage/<wbr>3981424d-a55c-4f07-bff2-<wbr>aca316a95d1f/3513775f-d6b0-<wbr>4423-be19-bbeb79c72ad2/<wbr>7ee3f450-5976-48f8-b667-<wbr>27b48f6cf778<br>
> MainThread::INFO::2018-01-09<br>
> 10:40:28,517::config::435::<wbr>ovirt_hosted_engine_ha.agent.<wbr>hosted_engine.HostedEngine.<wbr>config::(_get_vm_conf_content_<wbr>from_ovf_store)<br>
> Found an OVF for HE VM, trying to convert<br>
> MainThread::ERROR::2018-01-09<br>
> 10:40:28,523::agent::205::<wbr>ovirt_hosted_engine_ha.agent.<wbr>agent.Agent::(_run_agent)<br>
> Traceback (most recent call last):<br>
> File<br>
> "/usr/lib/python2.7/site-<wbr>packages/ovirt_hosted_engine_<wbr>ha/agent/agent.py",<br>
> line 191, in _run_agent<br>
> return action(he)<br>
> File<br>
> "/usr/lib/python2.7/site-<wbr>packages/ovirt_hosted_engine_<wbr>ha/agent/agent.py",<br>
> line 64, in action_proper<br>
> return he.start_monitoring()<br>
> File<br>
> "/usr/lib/python2.7/site-<wbr>packages/ovirt_hosted_engine_<wbr>ha/agent/hosted_engine.py",<br>
> line 421, in start_monitoring<br>
> self._config.refresh_vm_conf()<br>
> File<br>
> "/usr/lib/python2.7/site-<wbr>packages/ovirt_hosted_engine_<wbr>ha/env/config.py",<br>
> line 496, in refresh_vm_conf<br>
> content_from_ovf = self._get_vm_conf_content_<wbr>from_ovf_store()<br>
> File<br>
> "/usr/lib/python2.7/site-<wbr>packages/ovirt_hosted_engine_<wbr>ha/env/config.py",<br>
> line 438, in _get_vm_conf_content_from_ovf_<wbr>store<br>
> conf = ovf2VmParams.confFromOvf(<wbr>heovf)<br>
> File<br>
> "/usr/lib/python2.7/site-<wbr>packages/ovirt_hosted_engine_<wbr>ha/lib/ovf/ovf2VmParams.py",<br>
> line 283, in confFromOvf<br>
> vmConf = toDict(ovf)<br>
> File<br>
> "/usr/lib/python2.7/site-<wbr>packages/ovirt_hosted_engine_<wbr>ha/lib/ovf/ovf2VmParams.py",<br>
> line 210, in toDict<br>
> vmParams['vmId'] = tree.find('Content/Section').<wbr>attrib[OVF_NS + 'id']<br>
> File "lxml.etree.pyx", line 2272, in lxml.etree._Attrib.__getitem__<br>
> (src/lxml/lxml.etree.c:55336)<br>
> KeyError: '{<a href="http://schemas.dmtf.org/ovf/envelope/1/}id" rel="noreferrer" target="_blank">http://schemas.dmtf.org/ovf/<wbr>envelope/1/}id</a>'<br>
><br>
><br>
> On 09/01/2018 10:18, Peter Hudec wrote:<br>
>> The HA is flapping between 3400 nad 0. ;(<br>
>> And I'm not able to migrate also any other Vm to this host.<br>
>><br>
>> Loggs fromthe /var/log/ovirt-hosted-engine-<wbr>ha/agent.log file<br>
>><br>
>> MainThread::INFO::2018-01-08<br>
>> 21:44:45,805::state_machine::<wbr>174::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(refresh)<br>
>> Host <a href="http://dipovirt03.cnc.sk" rel="noreferrer" target="_blank">dipovirt03.cnc.sk</a> (id 1): {'conf_on_shared_storage': True, 'extra':<br>
>> 'metadata_parse_version=1\<wbr>nmetadata_feature_version=1\<wbr>ntimestamp=8232312<br>
>> (Mon Jan 8 21:44:29<br>
>> 2018)\nhost-id=1\nscore=3400\<wbr>nvm_conf_refresh_time=8232316 (Mon Jan 8<br>
>> 21:44:33<br>
>> 2018)\nconf_on_shared_storage=<wbr>True\nmaintenance=False\<wbr>nstate=EngineUp\nstopped=<wbr>False\n',<br>
>> 'hostname': '<a href="http://dipovirt03.cnc.sk" rel="noreferrer" target="_blank">dipovirt03.cnc.sk</a>', 'host-id': 1, 'engine-status':<br>
>> {'health': 'good', 'vm': 'up', 'detail': 'up'}, 'score': 3400,<br>
>> 'stopped': False, 'maintenance': False, 'crc32': 'f28d4648',<br>
>> 'local_conf_timestamp': 8232316, 'host-ts': 8232312}<br>
>> MainThread::INFO::2018-01-08<br>
>> 21:44:45,805::state_machine::<wbr>174::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(refresh)<br>
>> Host <a href="http://dipovirt02.cnc.sk" rel="noreferrer" target="_blank">dipovirt02.cnc.sk</a> (id 3): {'conf_on_shared_storage': True, 'extra':<br>
>> 'metadata_parse_version=1\<wbr>nmetada...skipping...<br>
>> neVMOVF) OVF_STORE volume path:<br>
>> /var/run/vdsm/storage/<wbr>3981424d-a55c-4f07-bff2-<wbr>aca316a95d1f/3513775f-d6b0-<wbr>4423-be19-bbe<br>
>> b79c72ad2/7ee3f450-5976-48f8-<wbr>b667-27b48f6cf778<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:13,904::state_machine::<wbr>169::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(refresh)<br>
>> Global metadata: {'maintenance': False}<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:13,905::state_machine::<wbr>174::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(refresh)<br>
>> Host <a href="http://dipovirt03.cnc.sk" rel="noreferrer" target="_blank">dipovirt03.cnc.sk</a> (id 1): {'conf_on_shared_storage': True, 'extra':<br>
>> 'metadata_parse_version=1\<wbr>nmetadata_feature_version=1\<wbr>ntimestamp=38598<br>
>> (Tue Jan 9 09:23:33<br>
>> 2018)\nhost-id=1\nscore=3400\<wbr>nvm_conf_refresh_time=38598 (Tue Jan 9<br>
>> 09:23:34<br>
>> 2018)\nconf_on_shared_storage=<wbr>True\nmaintenance=False\<wbr>nstate=EngineUp\nstopped=<wbr>False\n',<br>
>> 'hostname': '<a href="http://dipovirt03.cnc.sk" rel="noreferrer" target="_blank">dipovirt03.cnc.sk</a>', 'alive': False, 'host-id': 1,<br>
>> 'engine-status': {'health': 'good', 'vm': 'up', 'detail': 'up'},<br>
>> 'score': 3400, 'stopped': False, 'maintenance': False, 'crc32':<br>
>> '4c1d1890', 'local_conf_timestamp': 38598, 'host-ts': 38598}<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:13,905::state_machine::<wbr>174::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(refresh)<br>
>> Host <a href="http://dipovirt02.cnc.sk" rel="noreferrer" target="_blank">dipovirt02.cnc.sk</a> (id 3): {'conf_on_shared_storage': True, 'extra':<br>
>> 'metadata_parse_version=1\<wbr>nmetadata_feature_version=1\<wbr>ntimestamp=40677<br>
>> (Tue Jan 9 09:24:11<br>
>> 2018)\nhost-id=3\nscore=3400\<wbr>nvm_conf_refresh_time=40677 (Tue Jan 9<br>
>> 09:24:11<br>
>> 2018)\nconf_on_shared_storage=<wbr>True\nmaintenance=False\<wbr>nstate=EngineDown\nstopped=<wbr>False\n',<br>
>> 'hostname': '<a href="http://dipovirt02.cnc.sk" rel="noreferrer" target="_blank">dipovirt02.cnc.sk</a>', 'alive': False, 'host-id': 3,<br>
>> 'engine-status': {'reason': 'vm not running on this host', 'health':<br>
>> 'bad', 'vm': 'down', 'detail': 'unknown'}, 'score': 3400, 'stopped':<br>
>> False, 'maintenance': False, 'crc32': '3bf104bc',<br>
>> 'local_conf_timestamp': 40677, 'host-ts': 40677}<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:13,905::state_machine::<wbr>177::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(refresh)<br>
>> Local (id 2): {'engine-health': {'reason': 'vm not running on this<br>
>> host', 'health': 'bad', 'vm': 'down', 'detail': 'unknown'}, 'bridge':<br>
>> True, 'mem-free': 39540.0, 'maintenance': False, 'cpu-load': 0.0432,<br>
>> 'gateway': 1.0, 'storage-domain': True}<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:13,905::states::775::<wbr>ovirt_hosted_engine_ha.agent.<wbr>hosted_engine.HostedEngine::(<wbr>consume)<br>
>> Another host already took over..<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:13,928::state_<wbr>decorators::88::ovirt_hosted_<wbr>engine_ha.agent.hosted_engine.<wbr>HostedEngine::(check)<br>
>> Timeout cleared while transitioning <class<br>
>> 'ovirt_hosted_engine_ha.agent.<wbr>states.EngineStarting'> -> <class<br>
>> 'ovirt_hosted_engine_ha.agent.<wbr>states.EngineForceStop'><br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:14,046::brokerlink::68::<wbr>ovirt_hosted_engine_ha.lib.<wbr>brokerlink.BrokerLink::(<wbr>notify)<br>
>> Success, was notification of state_transition<br>
>> (EngineStarting-<wbr>EngineForceStop) sent? sent<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:14,464::hosted_engine::<wbr>494::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(_monitoring_<wbr>loop)<br>
>> Current state EngineForceStop (score: 3400)<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:14,467::hosted_engine::<wbr>1002::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(_stop_engine_<wbr>vm)<br>
>> Shutting down vm using `/usr/sbin/hosted-engine --vm-poweroff`<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:15,198::hosted_engine::<wbr>1007::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(_stop_engine_<wbr>vm)<br>
>> stdout:<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:15,198::hosted_engine::<wbr>1008::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(_stop_engine_<wbr>vm)<br>
>> stderr: Command VM.destroy with args {'vmID':<br>
>> '9a8ea503-f598-433e-9751-<wbr>93aee3e7b347'} failed:<br>
>> (code=1, message=Virtual machine does not exist: {'vmId':<br>
>> u'9a8ea503-f598-433e-9751-<wbr>93aee3e7b347'})<br>
>><br>
>> MainThread::ERROR::2018-01-09<br>
>> 10:15:15,199::hosted_engine::<wbr>1013::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(_stop_engine_<wbr>vm)<br>
>> Failed to stop engine vm with /usr/sbin/hosted-engine --vm-poweroff:<br>
>> Command VM.destroy with args {'vmID':<br>
>> '9a8ea503-f598-433e-9751-<wbr>93aee3e7b347'} failed:<br>
>> (code=1, message=Virtual machine does not exist: {'vmId':<br>
>> u'9a8ea503-f598-433e-9751-<wbr>93aee3e7b347'})<br>
>><br>
>> MainThread::ERROR::2018-01-09<br>
>> 10:15:15,199::hosted_engine::<wbr>1019::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(_stop_engine_<wbr>vm)<br>
>> Failed to stop engine VM: Command VM.destroy with args {'vmID':<br>
>> '9a8ea503-f598-433e-9751-<wbr>93aee3e7b347'} failed:<br>
>> (code=1, message=Virtual machine does not exist: {'vmId':<br>
>> u'9a8ea503-f598-433e-9751-<wbr>93aee3e7b347'})<br>
>><br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:15,317::brokerlink::68::<wbr>ovirt_hosted_engine_ha.lib.<wbr>brokerlink.BrokerLink::(<wbr>notify)<br>
>> Success, was notification of state_transition<br>
>> (EngineForceStop-<wbr>ReinitializeFSM) sent? sent<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:15,356::hosted_engine::<wbr>494::ovirt_hosted_engine_ha.<wbr>agent.hosted_engine.<wbr>HostedEngine::(_monitoring_<wbr>loop)<br>
>> Current state ReinitializeFSM (score: 0)<br>
>> MainThread::INFO::2018-01-09<br>
>> 10:15:25,560::brokerlink::68::<wbr>ovirt_hosted_engine_ha.lib.<wbr>brokerlink.BrokerLink::(<wbr>notify)<br>
>> Success, was notification of state_transition<br>
>> (ReinitializeFSM-EngineDown) sent? sent<br>
>><br>
>> Peter<br>
>><br>
>> On 09/01/2018 09:35, Yedidyah Bar David wrote:<br>
>>><br>
>>> 3) Hosted Engine HA:<br>
>>> Hosted Engine HA on upgraded hosts is 3400, the same as on the 4.1<br>
>>> hosts. Is this good or bad?<br>
>>><br>
>>><br>
>>> It's good.<br>
>><br>
>><br>
><br>
><br>
<br>
<br>
--<br>
*Peter Hudec*<br>
Infraštruktúrny architekt<br>
<a href="mailto:phudec@cnc.sk">phudec@cnc.sk</a> <mailto:<a href="mailto:phudec@cnc.sk">phudec@cnc.sk</a>><br>
<br>
*CNC, a.s.*<br>
Borská 6, 841 04 Bratislava<br>
Recepcia: <a href="tel:%2B421%202%C2%A0%2035%20000%20100" value="+421235000100">+421 2 35 000 100</a><br>
<br>
Mobil:<a href="tel:%2B421%C2%A0905%20997%20203" value="+421905997203">+421 905 997 203</a><br>
*<a href="http://www.cnc.sk" rel="noreferrer" target="_blank">www.cnc.sk</a>* <http:///<a href="http://www.cnc.sk" rel="noreferrer" target="_blank">www.cnc.sk</a>><br>
<br>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Didi<br></div>
</div></div>