Fixed in commit 726e946257174926ea2591a1c4a3be2dae4297ea Author: Nir Soffer <nsoffer@redhat.com> Date: Thu Feb 23 16:45:45 2017 +0200 sp: Mark helper method as @unsecured In commit 7cf19dafd7cd (storage_mailbox: make inbox/outbox mailbox args), we added a helper that is used before the spm is started, but the helper was not marked as @unsecure. This cause the call to fail with: File "/usr/share/vdsm/storage/sp.py", line 485, in __createMailboxMonitor outbox = self._master_volume_path("inbox") File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 77, in wrapper raise SecureError("Secured object is not in safe state") SecureError: Secured object is not in safe state As this helper doesn't change the state of the storage pool, there is no reason to treat it as a secured method, which is the default for this class. Change-Id: Icf92b9474c9000840a5c15e3b91f2ced4d02aca2 Signed-off-by: Nir Soffer <nsoffer@redhat.com> Verified with http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_... Thanks for reporting this. Nir On Thu, Feb 23, 2017 at 5:03 PM, Nir Soffer <nsoffer@redhat.com> wrote:
On Thu, Feb 23, 2017 at 4:51 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Thu, Feb 23, 2017 at 4:43 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Thu, Feb 23, 2017 at 4:38 PM, Barak Korren <bkorren@redhat.com> wrote:
On 23 February 2017 at 16:35, Nir Soffer <nsoffer@redhat.com> wrote:
On Thu, Feb 23, 2017 at 9:37 AM, Barak Korren <bkorren@redhat.com> wrote:
Test failed: [ add_secondary_storage_domains ]
Note: - This may or may not be related to https://bugzilla.redhat.com/show_bug.cgi?id=1421945 The BZ talks about sporadic failures, while this seems to be happening consistently (for 6 runs so far)
Link to suspected patches: - https://gerrit.ovirt.org/70415 - https://gerrit.ovirt.org/69157
Why do you suspect these patches?
Because the test right before them passed. These are all the changes that caused the failing OST job to run.
Did you try to run the tests with the latest patches before these patches?
Yes, the test before them pass.
Your are correct, these patches are broken:
2017-02-22 16:13:00,745-0500 ERROR (jsonrpc/1) [storage.TaskManager.Task] (Task='4f670db2-70c2-4c21-96ff-114f57de70c0') Unexpected error (task:871) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 878, in _run return fn(*args, **kargs) File "/usr/lib/python2.7/site-packages/vdsm/logUtils.py", line 52, in wrapper res = f(*args, **kwargs) File "/usr/share/vdsm/storage/hsm.py", line 989, in connectStoragePool spUUID, hostID, msdUUID, masterVersion, domainsMap) File "/usr/share/vdsm/storage/hsm.py", line 1051, in _connectStoragePool res = pool.connect(hostID, msdUUID, masterVersion) File "/usr/share/vdsm/storage/sp.py", line 672, in connect self.__createMailboxMonitor() File "/usr/share/vdsm/storage/sp.py", line 485, in __createMailboxMonitor outbox = self._master_volume_path("inbox") File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 77, in wrapper raise SecureError("Secured object is not in safe state") SecureError: Secured object is not in safe state
It's very confusing that this error is sometimes harmless and sometimes isn't - how did you identify it as problematic?
It depends on the context.
Here we called __createMailboxMontior, which is something we call when creating an instance, and is marked as @unsecured.
This call is calling now a new helper introduced in 7cf19dafd7cd, but the helper was not marked as @unsecured. This will raise UnsecureError, which will fail the current flow.
We have another instance of this during upgrade domain flow - I think we have the same issue there, but this needs investigation.
Other errors means that a real secured method is called when a host is not hte spm. This may be bad client code, or unavoidable, since there is no race-free way to check that a host is the spm before calling a method on the spm.
Y.
I'm sending a fix.
Nir
-- Barak Korren bkorren@redhat.com RHCE, RHCi, RHV-DevOps Team https://ifireball.wordpress.com/
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel