Fwd: [oVirt Jenkins] test-repo_ovirt_experimental_master - Build #4658 - FAILURE!
Nir, I saw similar issue on when I run system tests on my machine. Here is piece of log:: 2017-01-10 09:44:56,442 WARN (jsonrpc/3) [storage.HSM] getPV failed for guid: 3600140527d13be6be704ff98fd8838c2 (hsm:1973) Traceback (most recent call last): File "/usr/share/vdsm/storage/hsm.py", line 1970, in _getDeviceList pv = lvm.getPV(guid) File "/usr/share/vdsm/storage/lvm.py", line 856, in getPV raise se.InaccessiblePhysDev((pvName,)) InaccessiblePhysDev: Multipath cannot access physical device(s): "devices=(u'3600140527d13be6be704ff98fd8838c2',)" 2017-01-10 09:44:57,493 ERROR (jsonrpc/4) [storage.StorageDomainCache] domain d67c721e-d1f2-484f-a647-13a82db90b33 not found (sdc:157) Traceback (most recent call last): File "/usr/share/vdsm/storage/sdc.py", line 155, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 185, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'd67c721e-d1f2-484f-a647-13a82db90b33',) Can you please take a look why are we seeing it? Thanks, Piotr ---------- Forwarded message ---------- From: <jenkins@jenkins.phx.ovirt.org> Date: Tue, Jan 10, 2017 at 4:08 PM Subject: [oVirt Jenkins] test-repo_ovirt_experimental_master - Build #4658 - FAILURE! To: infra@ovirt.org Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/4658/, Build Number: 4658, Build Status: FAILURE _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra
As long as we have networking issues, I don't think it's worth looking into other issues. Example: 2017-01-10 10:04:57,844-05 DEBUG [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) [] Message received: {"jsonrpc":"2.0","error":{"code":"192.168.201.2:1671737550","message":"Vds timeout occured"},"id":null} 2017-01-10 10:04:57,849-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler7) [11ef43b9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Message timeout which can be caused by communication issues 2017-01-10 10:04:57,849-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler7) [11ef43b9] ERROR, GetStoragePoolInfoVDSCommand( GetStoragePoolInfoVDSCommandParameters: {runAsync='true', storagePoolId='33d94705-e33d-4a47-8e38-033ad2f7eec5', ignoreFailoverLimit='true'}), exception: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues, log id: 65d698d6 2017-01-10 10:04:57,850-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler7) [11ef43b9] Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:188) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:32) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.lambda$executeVDSCommand$0(IrsBrokerCommand.java:95) [vdsbroker.jar:] why do we have so many of them all along (grep for 11ef43b9 found several of those). Y. On Tue, Jan 10, 2017 at 6:10 PM, Piotr Kliczewski < piotr.kliczewski@gmail.com> wrote:
Nir,
I saw similar issue on when I run system tests on my machine. Here is piece of log::
2017-01-10 09:44:56,442 WARN (jsonrpc/3) [storage.HSM] getPV failed for guid: 3600140527d13be6be704ff98fd8838c2 (hsm:1973) Traceback (most recent call last): File "/usr/share/vdsm/storage/hsm.py", line 1970, in _getDeviceList pv = lvm.getPV(guid) File "/usr/share/vdsm/storage/lvm.py", line 856, in getPV raise se.InaccessiblePhysDev((pvName,)) InaccessiblePhysDev: Multipath cannot access physical device(s): "devices=(u'3600140527d13be6be704ff98fd8838c2',)"
2017-01-10 09:44:57,493 ERROR (jsonrpc/4) [storage.StorageDomainCache] domain d67c721e-d1f2-484f-a647-13a82db90b33 not found (sdc:157) Traceback (most recent call last): File "/usr/share/vdsm/storage/sdc.py", line 155, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 185, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'd67c721e-d1f2-484f-a647-13a82db90b33',)
Can you please take a look why are we seeing it?
Thanks, Piotr
---------- Forwarded message ---------- From: <jenkins@jenkins.phx.ovirt.org> Date: Tue, Jan 10, 2017 at 4:08 PM Subject: [oVirt Jenkins] test-repo_ovirt_experimental_master - Build #4658 - FAILURE! To: infra@ovirt.org
Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_ master/4658/, Build Number: 4658, Build Status: FAILURE _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Tue, Jan 10, 2017 at 7:24 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
As long as we have networking issues, I don't think it's worth looking into other issues. Example:
2017-01-10 10:04:57,844-05 DEBUG [org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker] (ResponseWorker) [] Message received: {"jsonrpc":"2.0","error":{"code":"192.168.201.2:1671737550","message":"Vds timeout occured"},"id":null} 2017-01-10 10:04:57,849-05 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler7) [11ef43b9] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: VDSM command failed: Message timeout which can be caused by communication issues 2017-01-10 10:04:57,849-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler7) [11ef43b9] ERROR, GetStoragePoolInfoVDSCommand( GetStoragePoolInfoVDSCommandParameters: {runAsync='true', storagePoolId='33d94705-e33d-4a47-8e38-033ad2f7eec5', ignoreFailoverLimit='true'}), exception: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues, log id: 65d698d6 2017-01-10 10:04:57,850-05 ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand] (DefaultQuartzScheduler7) [11ef43b9] Exception: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSNetworkException: VDSGenericException: VDSNetworkException: Message timeout which can be caused by communication issues at org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:188) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.GetStoragePoolInfoVDSCommand.executeIrsBrokerCommand(GetStoragePoolInfoVDSCommand.java:32) [vdsbroker.jar:] at org.ovirt.engine.core.vdsbroker.irsbroker.IrsBrokerCommand.lambda$executeVDSCommand$0(IrsBrokerCommand.java:95) [vdsbroker.jar:]
why do we have so many of them all along (grep for 11ef43b9 found several of those). Y.
They are side effect of the real issue.
On Tue, Jan 10, 2017 at 6:10 PM, Piotr Kliczewski <piotr.kliczewski@gmail.com> wrote:
Nir,
I saw similar issue on when I run system tests on my machine. Here is piece of log::
2017-01-10 09:44:56,442 WARN (jsonrpc/3) [storage.HSM] getPV failed for guid: 3600140527d13be6be704ff98fd8838c2 (hsm:1973) Traceback (most recent call last): File "/usr/share/vdsm/storage/hsm.py", line 1970, in _getDeviceList pv = lvm.getPV(guid) File "/usr/share/vdsm/storage/lvm.py", line 856, in getPV raise se.InaccessiblePhysDev((pvName,)) InaccessiblePhysDev: Multipath cannot access physical device(s): "devices=(u'3600140527d13be6be704ff98fd8838c2',)"
2017-01-10 09:44:57,493 ERROR (jsonrpc/4) [storage.StorageDomainCache] domain d67c721e-d1f2-484f-a647-13a82db90b33 not found (sdc:157) Traceback (most recent call last): File "/usr/share/vdsm/storage/sdc.py", line 155, in _findDomain dom = findMethod(sdUUID) File "/usr/share/vdsm/storage/sdc.py", line 185, in _findUnfetchedDomain raise se.StorageDomainDoesNotExist(sdUUID) StorageDomainDoesNotExist: Storage domain does not exist: (u'd67c721e-d1f2-484f-a647-13a82db90b33',)
Can you please take a look why are we seeing it?
Thanks, Piotr
---------- Forwarded message ---------- From: <jenkins@jenkins.phx.ovirt.org> Date: Tue, Jan 10, 2017 at 4:08 PM Subject: [oVirt Jenkins] test-repo_ovirt_experimental_master - Build #4658 - FAILURE! To: infra@ovirt.org
Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/4658/, Build Number: 4658, Build Status: FAILURE _______________________________________________ Infra mailing list Infra@ovirt.org http://lists.ovirt.org/mailman/listinfo/infra _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (2)
-
Piotr Kliczewski -
Yaniv Kaul