<div dir="ltr">As long as we have networking issues, I don't think it's worth looking into other issues.<div>Example:</div><div><br></div><div><div>2017-01-10 10:04:57,844-05 DEBUG [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) [] Message received: {"jsonrpc":"2.0","error":{"code":"<a href="http://192.168.201.2:1671737550">192.168.201.2:1671737550</a>","message":"Vds timeout occured"},"id":null}</div><div>2017-01-10 10:04:57,849-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler7) [11ef43b9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Message timeout which can be caused by communication issues</div><div>2017-01-10 10:04:57,849-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler7) [11ef43b9] ERROR, GetStoragePoolInfoVDSCommand( GetStoragePoolInfoVDSCommandParameters: {runAsync='true', storagePoolId='33d94705-e33d-4a47-8e38-033ad2f7eec5', ignoreFailoverLimit='true'}), exception: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues, log id: 65d698d6</div><div>2017-01-10 10:04:57,850-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler7) [11ef43b9] Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues</div><div> at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:188) [vdsbroker.jar:]</div><div> at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:32) [vdsbroker.jar:]</div><div> at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.lambda$executeVDSCommand$0(IrsBrokerCommand.java:95) [vdsbroker.jar:]</div></div><div><br></div><div><br></div><div>why do we have so many of them all along (grep for 11ef43b9 found several of those).</div><div>Y.</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 10, 2017 at 6:10 PM, Piotr Kliczewski <span dir="ltr"><<a href="mailto:piotr.kliczewski@gmail.com" target="_blank">piotr.kliczewski@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Nir,<br>
<br>
I saw similar issue on when I run system tests on my machine. Here is<br>
piece of log::<br>
<br>
2017-01-10 09:44:56,442 WARN (jsonrpc/3) [storage.HSM] getPV failed<br>
for guid: 3600140527d13be6be704ff98fd883<wbr>8c2 (hsm:1973)<br>
Traceback (most recent call last):<br>
File "/usr/share/vdsm/storage/hsm.<wbr>py", line 1970, in _getDeviceList<br>
pv = lvm.getPV(guid)<br>
File "/usr/share/vdsm/storage/lvm.<wbr>py", line 856, in getPV<br>
raise se.InaccessiblePhysDev((<wbr>pvName,))<br>
InaccessiblePhysDev: Multipath cannot access physical device(s):<br>
"devices=(u'<wbr>3600140527d13be6be704ff98fd883<wbr>8c2',)"<br>
<br>
2017-01-10 09:44:57,493 ERROR (jsonrpc/4) [storage.StorageDomainCache]<br>
domain d67c721e-d1f2-484f-a647-<wbr>13a82db90b33 not found (sdc:157)<br>
Traceback (most recent call last):<br>
File "/usr/share/vdsm/storage/sdc.<wbr>py", line 155, in _findDomain<br>
dom = findMethod(sdUUID)<br>
File "/usr/share/vdsm/storage/sdc.<wbr>py", line 185, in _findUnfetchedDomain<br>
raise se.StorageDomainDoesNotExist(<wbr>sdUUID)<br>
StorageDomainDoesNotExist: Storage domain does not exist:<br>
(u'd67c721e-d1f2-484f-a647-<wbr>13a82db90b33',)<br>
<br>
Can you please take a look why are we seeing it?<br>
<br>
Thanks,<br>
Piotr<br>
<br>
<br>
---------- Forwarded message ----------<br>
From: <<a href="mailto:jenkins@jenkins.phx.ovirt.org">jenkins@jenkins.phx.ovirt.org</a><wbr>><br>
Date: Tue, Jan 10, 2017 at 4:08 PM<br>
Subject: [oVirt Jenkins] test-repo_ovirt_experimental_<wbr>master - Build<br>
#4658 - FAILURE!<br>
To: <a href="mailto:infra@ovirt.org">infra@ovirt.org</a><br>
<br>
<br>
Build: <a href="http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/4658/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/<wbr>test-repo_ovirt_experimental_<wbr>master/4658/</a>,<br>
Build Number: 4658,<br>
Build Status: FAILURE<br>
______________________________<wbr>_________________<br>
Infra mailing list<br>
<a href="mailto:Infra@ovirt.org">Infra@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/infra" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/infra</a><br>
______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org">Devel@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/devel</a><br>
</blockquote></div><br></div>