Dan, It looks like it was one of the calls triggered when vdsm was down: 2018-04-03 05:30:16,065-0400 INFO (mailbox-hsm) [storage.MailBox.HsmMailMonitor] HSM_MailMonitor sending mail to SPM - ['/usr/bin/dd', 'of=/rhev/data-center/ddb765d2-2137-437d-95f8-c46dbdbc7711/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=4096', 'count=1', 'seek=1'] (mailbox:387) 2018-04-03 05:31:22,441-0400 INFO (MainThread) [vds] (PID: 20548) I am the actual vdsm 4.20.23-28.gitd11ed44.el7.centos lago-basic-suite-4-2-host-0 (3.10.0-693.21.1.el7.x86_64) (vdsmd:149) which failed and caused timeout. Thanks, Piotr On Tue, Apr 3, 2018 at 1:57 PM, Dan Kenigsberg <danken@redhat.com> wrote:
On Tue, Apr 3, 2018 at 2:07 PM, Barak Korren <bkorren@redhat.com> wrote:
Test failed: [ 006_migrations.prepare_migration_attachments_ipv6 ]
Link to suspected patches:
(Patch seems unrelated - do we have sporadic communication issues arising in PST?) https://gerrit.ovirt.org/c/89737/1 - vdsm - automation: check-patch: attempt to install vdsm-gluster
Link to Job: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/1521/
Link to all logs: http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/ 1521/artifact/exported-artifacts/basic-suit-4.2-el7/ test_logs/basic-suite-4.2/post-006_migrations.py/
Error snippet from log:
<error>
Traceback (most recent call last): File "/usr/lib64/python2.7/unittest/case.py", line 369, in run testMethod() File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest self.test(*self.arg) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 129, in wrapped_test test() File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 59, in wrapper return func(get_test_prefix(), *args, **kwargs) File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 78, in wrapper prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ ovirt-system-tests/basic-suite-4.2/test-scenarios/006_migrations.py", line 139, in prepare_migration_attachments_ipv6 engine, host_service, MIGRATION_NETWORK, ip_configuration) File "/home/jenkins/workspace/ovirt-4.2_change-queue-tester/ ovirt-system-tests/basic-suite-4.2/test_utils/network_utils_v4.py", line 71, in modify_ip_config check_connectivity=True) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/services.py", line 36729, in setup_networks return self._internal_action(action, 'setupnetworks', None, headers, query, wait) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 299, in _internal_action return future.wait() if wait else future File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 55, in wait return self._code(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 296, in callback self._check_fault(response) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 132, in _check_fault self._raise_error(response, body) File "/usr/lib64/python2.7/site-packages/ovirtsdk4/service.py", line 118, in _raise_error raise error Error: Fault reason is "Operation Failed". Fault detail is "[Network error during communication with the Host.]". HTTP response code is 400.
The error occurred sometime in the interval
09:32:58 [basic-suit] @ Run test: 006_migrations.py: 09:33:55 [basic-suit] Error occured, aborting
and indeed
http://jenkins.ovirt.org/job/ovirt-4.2_change-queue-tester/ 1521/artifact/exported-artifacts/basic-suit-4.2-el7/ test_logs/basic-suite-4.2/post-006_migrations.py/lago- basic-suite-4-2-engine/_var_log/ovirt-engine/engine.log/*view*/
has Engine disconnected from the host at
2018-04-03 05:33:32,307-04 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.HostMonitoring] (EE-ManagedThreadFactory-engineScheduled-Thread-39) [] Unable to RefreshCapabilities: VDSNetworkException: VDSGenericException: VDSNetworkException: Vds timeout occured
Maybe Piotr can read more into it.