<div dir="ltr">FYI,<div><br></div><div>We're seeing a single test 'add hotplug disk' failing recently on OST, the test only failed once on the jobs [1] and was fixed on its own after one run, but now I see it also on check-patch [2] jobs.</div><div><br></div><div>Looking at the changelog, I can't find immediate suspects, and if this is a race it might make sense it got in at some point and didn't fail on the 1st time.</div><div><br></div><div>Anyone can have a look to help debug this? it also failed once on 4.0 job.</div><div><br></div><div>I see some exceptions on VDSM.LOG [3]</div><div><br></div><div>[1] <a href="http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experimental_master/4079/">http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experimental_master/4079/</a><br clear="all"><div>[2] <a href="http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-fc24-x86_64/322/">http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-fc24-x86_64/322/</a></div><div>[3]</div><div><br></div><div><div>2016-12-11 04:51:19,055 INFO (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call Host.getDeviceList succeeded in 0.39 seconds (__init__:515)</div><div>2016-12-11 04:51:19,106 INFO (jsonrpc/5) [dispatcher] Run and protect: getAllTasksInfo(spUUID=None, options=None) (logUtils:49)</div><div>2016-12-11 04:51:19,106 ERROR (jsonrpc/5) [storage.TaskManager.Task] (Task='ef8d6b6f-256c-4420-9998-bef8a140a20f') Unexpected error (task:870)</div><div>Traceback (most recent call last):</div><div> File "/usr/share/vdsm/storage/task.py", line 877, in _run</div><div> return fn(*args, **kargs)</div><div> File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 50, in wrapper</div><div> res = f(*args, **kwargs)</div><div> File "/usr/share/vdsm/storage/hsm.py", line 2193, in getAllTasksInfo</div><div> allTasksInfo = self._pool.getAllTasksInfo()</div><div> File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 77, in wrapper</div><div> raise SecureError("Secured object is not in safe state")</div><div>SecureError: Secured object is not in safe state</div><div>2016-12-11 04:51:19,109 INFO (jsonrpc/5) [storage.TaskManager.Task] (Task='ef8d6b6f-256c-4420-9998-bef8a140a20f') aborting: Task is aborted: u'Secured object is not in safe state' - code 100 (task:1175)</div><div>2016-12-11 04:51:19,109 ERROR (jsonrpc/5) [storage.Dispatcher] Secured object is not in safe state (dispatcher:80)</div><div>Traceback (most recent call last):</div><div> File "/usr/share/vdsm/storage/dispatcher.py", line 72, in wrapper</div><div> result = ctask.prepare(func, *args, **kwargs)</div><div> File "/usr/share/vdsm/storage/task.py", line 105, in wrapper</div><div> return m(self, *a, **kw)</div><div> File "/usr/share/vdsm/storage/task.py", line 1183, in prepare</div><div> raise self.error</div></div><div><br></div><div><br></div><div>second error:</div><div><br></div><div><div>2016-12-11 04:55:21,837 ERROR (libvirt/events) [vds] Error running VM callback (clientIF:543)</div><div>Traceback (most recent call last):</div><div> File "/usr/share/vdsm/clientIF.py", line 514, in dispatchLibvirtEvents</div><div> v.onLibvirtLifecycleEvent(event, detail, None)</div><div> File "/usr/share/vdsm/virt/vm.py", line 4318, in onLibvirtLifecycleEvent</div><div> elif detail == libvirt.VIR_DOMAIN_EVENT_SUSPENDED_POSTCOPY:</div><div>AttributeError: 'module' object has no attribute 'VIR_DOMAIN_EVENT_SUSPENDED_POSTCOPY'</div></div><div><br></div><div><br></div><div><br></div><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>Eyal Edri<br>Associate Manager</div><div>RHV DevOps<br>EMEA ENG Virtualization R&D<br>Red Hat Israel<br><br>phone: +972-9-7692018<br>irc: eedri (on #tlv #rhev-dev #rhev-integ)</div></div></div></div></div></div></div>
</div></div>