Tal,
I see the bug is in post but the patch was not merged yet:
https://gerrit.ovirt.org/#/c/98191/
can you tell me when will we merge the patch? as I rather not add SkipTest
if this will be merged soon,
thanks,
Dafna
On Mon, Mar 4, 2019 at 10:42 AM Dafna Ron <dron(a)redhat.com> wrote:
> As I had another failure of this today I will be disabling this test
> until issue is resolved (
https://bugzilla.redhat.com/1684267)
>
> Thanks,
> Dafna
>
>
> On Thu, Feb 28, 2019 at 8:48 PM Nir Soffer <nsoffer(a)redhat.com> wrote:
>
>> On Thu, Feb 28, 2019 at 11:52 AM Dafna Ron <dron(a)redhat.com> wrote:
>>
>>> Hi,
>>>
>>> We have a failure on the project in basic suite, master branch. The
>>> recent failure was in patch:
>>>
https://gerrit.ovirt.org/#/c/98087/1 - Add pre-seed for ovirt-web-ui
>>>
>>> CQ is pointing at the below as the root cause (which was merged a while
>>> back):
>>>
https://gerrit.ovirt.org/#/c/97491/ - Add pre-seed for ovirt-web-ui
>>>
>>> Can you please check the issue as it seems both patches are changing
>>> the same thing and the project seem to be broken since
>>>
https://gerrit.ovirt.org/#/c/97491/3
>>>
>>> Latest failure:
>>>
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13171/
>>>
>>> Logs:
>>>
>>>
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13171/arti...
>>>
>>> errors from logs:
>>> Engine:
>>>
>>> 2019-02-27 13:37:28,479-05 ERROR
>>> [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [74283e25]
>>> Transaction rolled-back for command
>>> 'org.ovirt.engine.core.bll.UpdateVmCommand'.
>>> 2019-02-27 13:37:28,483-05 ERROR
>>> [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
>>> (default task-1) [74283e25] EVENT_ID: USER_FAILED_UPDATE_VM(58), Failed to
>>> update VM vm2 (User: admin@inter
>>> nal-authz).
>>> 2019-02-27 13:37:28,485-05 INFO
>>> [org.ovirt.engine.core.bll.UpdateVmCommand] (default task-1) [74283e25]
>>> Lock freed to object
'EngineLock:{exclusiveLocks='[vm2=VM_NAME]',
>>> sharedLocks='[3500eb82-e5e2-4e24-b41c-ea
>>> 02d9f6adee=VM]'}'
>>> 2019-02-27 13:37:28,485-05 DEBUG
>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>> (default task-1) [74283e25] method: runAction, params: [UpdateVm,
>>> VmManagementParametersBase:{commandId='340
>>> 59769-05b9-429e-8356-f6b9b9953f55', user='admin',
>>> commandType='UpdateVm',
vmId='3500eb82-e5e2-4e24-b41c-ea02d9f6adee'}],
>>> timeElapsed: 6618ms
>>> 2019-02-27 13:37:28,486-05 ERROR
>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>>> task-1) [] Operation Failed: []
>>> 2019-02-27 13:37:28,579-05 DEBUG
>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>> (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] method: get, params:
>>> [e29c0ba1-464c-4eb4-a8f2-c6933d9
>>> 9969a], timeElapsed: 3ms
>>>
>>>
>>> vdsm:
>>>
>>> 2019-02-27 13:37:21,987-0500 INFO (jsonrpc/1) [vdsm.api] FINISH
>>> lease_info error=No such lease 3500eb82-e5e2-4e24-b41c-ea02d9f6adee
>>> from=::ffff:192.168.201.4,43920,
>>> flow_id=117dec74-ad59-4b12-8148-b2c130337c10,
>>> task_id=9c297d41-0aa7-4c74-b268-b710e666bc6c (api:52)
>>> 2019-02-27 13:37:21,988-0500 ERROR (jsonrpc/1)
>>> [storage.TaskManager.Task]
(Task='9c297d41-0aa7-4c74-b268-b710e666bc6c')
>>> Unexpected error (task:875)
>>> Traceback (most recent call last):
>>> File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py",
line
>>> 882, in _run
>>> return fn(*args, **kargs)
>>> File "<string>", line 2, in lease_info
>>> File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line
50,
>>> in method
>>> ret = func(*args, **kwargs)
>>> File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py",
line
>>> 3702, in lease_info
>>> info = dom.lease_info(lease.lease_id)
>>> File "/usr/lib/python2.7/site-packages/vdsm/storage/sd.py", line
674,
>>> in lease_info
>>> return vol.lookup(lease_id)
>>> File "/usr/lib/python2.7/site-packages/vdsm/storage/xlease.py",
line
>>> 553, in lookup
>>> raise NoSuchLease(lease_id)
>>> NoSuchLease: No such lease 3500eb82-e5e2-4e24-b41c-ea02d9f6adee
>>>
>>
>> This is not an error of vdsm. Someone asked for information about a
>> non-existing lease,
>> and vdsm fail the request with the expected error:
>>
>> NoSuchLease: No such lease 3500eb82-e5e2-4e24-b41c-ea02d9f6adee
>>
>> But since the error is not a public error, vdsm logger handle it as
>> unexpected error and
>> create lot of noise in vdsm log.
>>
>> On engine side, this creates even worse noise, as we can see here:
>>
>>
https://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/13171/arti...
>> (look for 2019-02-27 13:37:21,996-05)
>>
>> All these noise could be avoided if vdsm was using a proper public error.
>> I filed
https://bugzilla.redhat.com/1684267
>>
>> But, this error is not related to the actual failing test, this is just
>> noise hiding the real problem.
>>
>>
Hi,
As Nir already said, This log is just noise, this is not the real problem.
I think that we already talk about this issue but I can't find the thread.
This lease failure in the log is expected and this is not an error please
ignore this in the next failure analysis.
From the log I can see the following error:
{"jsonrpc": "2.0", "id":
"6a42a9cf-e220-4916-ba0e-db2182978a5d",
"error": {"message": "Cannot find master domain:
u'spUUID=717575a9-7818-460d-ba3a-d5bdd8ef9ed3,
msdUUID=34311cc1-c4d2-4cfe-88b5-dd5ad72261d3'", "code": 304}}�
2019-02-27 13:35:14,175-05 DEBUG
[org.ovirt.vdsm.jsonrpc.client.internal.ResponseWorker]
(ResponseWorker) [] Message received: {"jsonrpc": "2.0",
"id":
"6a42a9cf-e220-4916-ba0e-db2182978a5d", "error": {"message":
"Cannot
find master domain: u'spUUID=717575a9-7818-460d-ba3a-d5bdd8ef9ed3,
msdUUID=34311cc1-c4d2-4cfe-88b5-dd5ad72261d3'", "code": 304}}
2019-02-27 13:35:14,182-05 ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(EE-ManagedThreadFactory-engine-Thread-23) [73da8d8b] EVENT_ID:
VDS_BROKER_COMMAND_FAILURE(10,802), VDSM
lago-basic-suite-master-host-1 command ConnectStoragePoolVDS failed:
Cannot find master domain:
u'spUUID=717575a9-7818-460d-ba3a-d5bdd8ef9ed3,
msdUUID=34311cc1-c4d2-4cfe-88b5-dd5ad72261d3'
2019-02-27 13:35:14,182-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [73da8d8b] Command
'org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand'
return value 'StatusOnlyReturn [status=Status [code=304,
message=Cannot find master domain:
u'spUUID=717575a9-7818-460d-ba3a-d5bdd8ef9ed3,
msdUUID=34311cc1-c4d2-4cfe-88b5-dd5ad72261d3']]'
2019-02-27 13:35:14,182-05 INFO
[org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand]
(EE-ManagedThreadFactory-engine-Thread-23) [73da8d8b] HostName =
lago-basic-suite-master-host-1
This failure occured before the lease info was requested (and failed as
expected).
Therefore the fix for -
will not solve
anything that related to the failure.
T hanks for reporting this.