On Tue, Feb 14, 2017 at 9:16 PM, Dusan Fodor <dfodor@redhat.com> wrote:

Hi,

storage task manager returns following error (vdsm.log):

2017-02-14 06:58:27,975-0500 ERROR (jsonrpc/1) [storage.TaskManager.Task] (Task='1bc5b2ca-a44e-4794-81f9-d2415f27ad48') Unexpected error (task:871) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 878, in _run return fn(*args, **kargs) File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 52, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/storage/hsm.py", line 2215, in getAllTasksInfo allTasksInfo = self._pool.getAllTasksInfo() File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 77, in wrapper raise SecureError("Secured object is not in safe state") SecureError: Secured object is not in safe state 2017-02-14 06:58:27,977-0500 INFO (jsonrpc/1) [storage.TaskManager.Task] (Task='1bc5b2ca-a44e-4794-81f9-d2415f27ad48') aborting: Task is aborted: u'Secured object is not in safe state' - code 100 (task:1176) 2017-02-14 06:58:27,978-0500 ERROR (jsonrpc/1) [storage.Dispatcher] Secured object is not in safe state (dispatcher:80) Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 72, in wrapper result = ctask.prepare(func, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 106, in wrapper return m(self, *a, **kw) File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 1184, in prepare raise self.error SecureError: Secured object is not in safe state

It doesn't make OST fail, but i believe it is not desired. Sadly, I'm not able to find patch that might be related, since all builds in history have this error. Link to build: http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experi... Link to vdsm.log: http://jenkins.ovirt.org/view/experimental%20jobs/job/test-repo_ovirt_experi...

Thanks for reporting this, This log is too noisy, but it may reveal real error in the caller, calling spm methods on a host which is not the spm. However, I'm not sure it is possible to avoid this situation. This should probably be a one line warning instead of this noisy exception. But the real issue is the caller of getAllTasksInfo - why is it calling this on a host which is not the SPM? this is probably engine bug, or maybe code using engine REST api incorrectly - maybe the test framework? We plan to clean the logs during 4.1.z. Nir