hot plug disk test failed on system tests

[Eyal Edri]

FYI,

We're seeing a single test 'add hotplug disk' failing recently on OST, the
test only failed once on the jobs [1] and was fixed on its own after one
run, but now I see it also on check-patch [2] jobs.

Looking at the changelog, I can't find immediate suspects, and if this is a
race it might make sense it got in at some point and didn't  fail on the
1st time.

Anyone can have a look to help debug this? it also failed once on 4.0 job.

I see some exceptions on VDSM.LOG [3]

[1]
http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experimental_master/4079/
[2]
http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-fc24-x86_64/322/
[3]

2016-12-11 04:51:19,055 INFO  (jsonrpc/0) [jsonrpc.JsonRpcServer] RPC call
Host.getDeviceList succeeded in 0.39 seconds (__init__:515)
2016-12-11 04:51:19,106 INFO  (jsonrpc/5) [dispatcher] Run and protect:
getAllTasksInfo(spUUID=None, options=None) (logUtils:49)
2016-12-11 04:51:19,106 ERROR (jsonrpc/5) [storage.TaskManager.Task]
(Task='ef8d6b6f-256c-4420-9998-bef8a140a20f') Unexpected error (task:870)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/task.py", line 877, in _run
    return fn(*args, **kargs)
  File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 50, in
wrapper
    res = f(*args, **kwargs)
  File "/usr/share/vdsm/storage/hsm.py", line 2193, in getAllTasksInfo
    allTasksInfo = self._pool.getAllTasksInfo()
  File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line
77, in wrapper
    raise SecureError("Secured object is not in safe state")
SecureError: Secured object is not in safe state
2016-12-11 04:51:19,109 INFO  (jsonrpc/5) [storage.TaskManager.Task]
(Task='ef8d6b6f-256c-4420-9998-bef8a140a20f') aborting: Task is aborted:
u'Secured object is not in safe state' - code 100 (task:1175)
2016-12-11 04:51:19,109 ERROR (jsonrpc/5) [storage.Dispatcher] Secured
object is not in safe state (dispatcher:80)
Traceback (most recent call last):
  File "/usr/share/vdsm/storage/dispatcher.py", line 72, in wrapper
    result = ctask.prepare(func, *args, **kwargs)
  File "/usr/share/vdsm/storage/task.py", line 105, in wrapper
    return m(self, *a, **kw)
  File "/usr/share/vdsm/storage/task.py", line 1183, in prepare
    raise self.error


second error:

2016-12-11 04:55:21,837 ERROR (libvirt/events) [vds] Error running VM
callback (clientIF:543)
Traceback (most recent call last):
  File "/usr/share/vdsm/clientIF.py", line 514, in dispatchLibvirtEvents
    v.onLibvirtLifecycleEvent(event, detail, None)
  File "/usr/share/vdsm/virt/vm.py", line 4318, in onLibvirtLifecycleEvent
    elif detail == libvirt.VIR_DOMAIN_EVENT_SUSPENDED_POSTCOPY:
AttributeError: 'module' object has no attribute
'VIR_DOMAIN_EVENT_SUSPENDED_POSTCOPY'




-- 
Eyal Edri
Associate Manager
RHV DevOps
EMEA ENG Virtualization R&D
Red Hat Israel

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/infra/attachments/20161211/fec6ae40/attachment.html>