It happened again in [1]
  1. 2016-11-20 10:48:12,106 ERROR (jsonrpc/2) [storage.TaskManager.Task] (Task='6c1ec6e7-fb37-465b-8e30-1613317683b2') Unexpected error (task:870)
  2. Traceback (most recent call last):
  3.   File "/usr/share/vdsm/storage/task.py", line 877, in _run
  4.     return fn(*args, **kargs)
  5.   File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 50, in wrapper
  6.     res = f(*args, **kwargs)
  7.   File "/usr/share/vdsm/storage/hsm.py", line 2205, in getAllTasksInfo
  8.     allTasksInfo = sp.getAllTasksInfo()
  9.   File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 77, in wrapper
  10.     raise SecureError("Secured object is not in safe state")
  11. SecureError: Secured object is not in safe state
  12. 2016-11-20 10:48:12,109 INFO  (jsonrpc/2) [storage.TaskManager.Task] (Task='6c1ec6e7-fb37-465b-8e30-1613317683b2') aborting: Task is aborted: u'Secured object is not in safe state' - code 100 (task:1175)
  13. 2016-11-20 10:48:12,110 ERROR (jsonrpc/2) [storage.Dispatcher] Secured object is not in safe state (dispatcher:80)
  14. Traceback (most recent call last):
  15.   File "/usr/share/vdsm/storage/dispatcher.py", line 72, in wrapper
  16.     result = ctask.prepare(func, *args, **kwargs)
  17.   File "/usr/share/vdsm/storage/task.py", line 105, in wrapper
  18.     return m(self, *a, **kw)
  19.   File "/usr/share/vdsm/storage/task.py", line 1183, in prepare
  20.     raise self.error
  21. SecureError: Secured object is not in safe state

http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/3506/artifact/exported-artifacts/basic_suite_master.sh-el7/exported-artifacts/test_logs/basic-suite-master/post-006_network_by_label.py/lago-basic-suite-master-host1/_var_log_vdsm/vdsm.log

The storage VM is running on the same VM as engine ( to save memory ) and its serving both NFS & ISCSI. 
Do you think running it on the same VM as engine might cause such issues? 



On Mon, Oct 17, 2016 at 11:45 PM, Adam Litke <alitke@redhat.com> wrote:
On 17/10/16 11:51 +0200, Piotr Kliczewski wrote:
Adam,

I see constant failures due to this and found:

2016-10-17 03:55:21,045 ERROR   (jsonrpc/3) [storage.TaskManager.Task]
Task=`8989d694-7099-449b-bd66-4d63786be089`::Unexpected error
(task:870)
Traceback (most recent call last):
 File "/usr/share/vdsm/storage/task.py", line 877, in _run
   return fn(*args, **kargs)
 File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 50, in wrapper
   res = f(*args, **kwargs)
 File "/usr/share/vdsm/storage/hsm.py", line 2212, in getAllTasksInfo
   allTasksInfo = sp.getAllTasksInfo()
 File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py",
line 77, in wrapper
   raise SecureError("Secured object is not in safe state")
SecureError: Secured object is not in safe state

This usually indicates that the SPM role has been lost which happens
most likely due to connection issues with the storage.  What is the
storage environment being used for the system tests?


Please take a look not sure whether it is related. You can find latest
build here [1]

Thanks,
Piotr

[1] http://jenkins.ovirt.org/job/ovirt_master_system-tests/668/

On Fri, Oct 14, 2016 at 11:22 AM, Evgheni Dereveanchin
<ederevea@redhat.com> wrote:
Hello,

We've got several cases today where system tests failed
when attempting to export templates:

http://jenkins.ovirt.org/job/ovirt_master_system-tests/655/testReport/junit/(root)/004_basic_sanity/template_export/

Related engine.log looks something like this:
https://paste.fedoraproject.org/449936/47643643/raw/

I could not find any obvious issues in SPM logs, could someone
please take a look to confirm what may be causing this issue?

Full logs from the test are available here:
http://jenkins.ovirt.org/job/ovirt_master_system-tests/655/artifact/

Regards,
Evgheni Dereveanchin
_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

--
Adam Litke

_______________________________________________
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel





--
Eyal Edri
Associate Manager
RHV DevOps
EMEA ENG Virtualization R&D
Red Hat Israel

phone: +972-9-7692018
irc: eedri (on #tlv #rhev-dev #rhev-integ)