<div dir="ltr"><div><div>This one we say already. The cause of it that during network setup engine sends messages which may fail or partially arrive. <br></div>We used to have host level lock to protect this kind of situation but it seems like we do not have it anymore.<br><br></div>Previous failure was triggered by host monitoring now it was SpmStatus.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Mar 3, 2017 at 2:54 PM, Pavel Zhukov <span dir="ltr"><<a href="mailto:pzhukov@redhat.com" target="_blank">pzhukov@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>
Hi,<br>
<br>
Migration is failed because host is in Connecting state.<br>
Seems like another jsonrpc related issue (Unrecognized message received).<br>
<br>
Job: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/5654/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/<wbr>test-repo_ovirt_experimental_<wbr>master/5654/</a><br>
Logs: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/5654/artifact/exported-artifacts/*zip*/exported-artifacts.zip" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/<wbr>test-repo_ovirt_experimental_<wbr>master/5654/artifact/exported-<wbr>artifacts/*zip*/exported-<wbr>artifacts.zip</a><br>
<br>
--<br>
Pavel<br>
<br>
[LOGS SNIPPET]<br>
2017-03-03 06:00:40,882-05 DEBUG [org.ovirt.vdsm.jsonrpc.<wbr>client.reactors.Reactor] (SSL Stomp Reactor) [3155acab] Unable to process messages Unrecognized message received : org.ovirt.vdsm.jsonrpc.client.<wbr>ClientConnectionException: Unrecognized message received<br>
42965:2017-03-03 06:00:40,889-05 ERROR [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (DefaultQuartzScheduler4) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,<wbr>802), Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM lago-basic-suite-master-host1 command SpmStatusVDS failed: Unrecognized message received<br>
42966:2017-03-03 06:00:40,889-05 ERROR [org.ovirt.engine.core.<wbr>vdsbroker.vdsbroker.<wbr>SpmStatusVDSCommand] (DefaultQuartzScheduler4) [] Command 'SpmStatusVDSCommand(HostName = lago-basic-suite-master-host1, SpmStatusVDSCommandParameters:<wbr>{runAsync='true', hostId='bba0ec26-4856-4389-<wbr>982d-2ad68cb3f682', storagePoolId='8cc227da-70e7-<wbr>4557-aa01-6d8ddee6f847'})' execution failed: VDSGenericException: VDSNetworkException: Unrecognized message received<br>
42999:2017-03-03 06:00:40,925-05 WARN [org.ovirt.engine.core.dal.<wbr>dbbroker.auditloghandling.<wbr>AuditLogDirector] (DefaultQuartzScheduler4) [6c7fb340] EVENT_ID: SYSTEM_CHANGE_STORAGE_POOL_<wbr>STATUS_PROBLEMATIC_WITH_ERROR(<wbr>987), Correlation ID: 6c7fb340, Call Stack: null, Custom Event ID: -1, Message: Invalid status on Data Center test-dc. Setting Data Center status to Non Responsive (On host lago-basic-suite-master-host1, Error: Network error during communication with the Host.).<br>
43457:2017-03-03 06:00:44,466-05 ERROR [org.ovirt.engine.core.bll.<wbr>network.host.HostValidator] (default task-17) [14be0e43-97b8-4882-bbcc-<wbr>27392543fae6] Unable to setup network: operation can only be done when Host status is one of: Maintenance, Up, NonOperational; current status is Connecting<br>
43460:2017-03-03 06:00:44,476-05 ERROR [org.ovirt.engine.api.restapi.<wbr>resource.<wbr>AbstractBackendResource] (default task-17) [] Operation Failed: [Cannot setup Networks. Operation can be performed only when Host status is Maintenance, Up, NonOperational.]<br>
</blockquote></div><br></div>