<div dir="ltr"><div>I forgot to add the polling after the second snapshot as well, it was added in this patch: <a href="https://gerrit.ovirt.org/#/c/89577/">https://gerrit.ovirt.org/#/c/89577/</a><br><br></div>Sorry for the inconvenience<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 29, 2018 at 1:46 PM, Yedidyah Bar David <span dir="ltr"><<a href="mailto:didi@redhat.com" target="_blank">didi@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Wed, Mar 28, 2018 at 4:19 PM, Gal Ben Haim <<a href="mailto:gbenhaim@redhat.com">gbenhaim@redhat.com</a>> wrote:<br>
><br>
><br>
> On Wed, Mar 28, 2018 at 2:48 PM, Benny Zlotnik <<a href="mailto:bzlotnik@redhat.com">bzlotnik@redhat.com</a>> wrote:<br>
>><br>
>> <a href="https://gerrit.ovirt.org/#/c/89534/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>89534/</a><br>
>> <a href="https://gerrit.ovirt.org/#/c/89537/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>89537/</a><br>
><br>
><br>
> Merged.<br>
<br>
</span>Seems like it failed again similarly:<br>
<br>
08:05:07 vm1_snapshots_service.<wbr>snapshot_service(<a href="http://snapshot.id" rel="noreferrer" target="_blank">snapshot.id</a>).<wbr>remove()<br>
08:05:07 File<br>
"/usr/lib64/python2.7/site-<wbr>packages/ovirtsdk4/services.<wbr>py", line<br>
20230, in remove<br>
08:05:07 self._internal_remove(headers, query, wait)<br>
08:05:07 File<br>
"/usr/lib64/python2.7/site-<wbr>packages/ovirtsdk4/service.py"<wbr>, line 271,<br>
in _internal_remove<br>
08:05:07 return future.wait() if wait else future<br>
08:05:07 File<br>
"/usr/lib64/python2.7/site-<wbr>packages/ovirtsdk4/service.py"<wbr>, line 55, in<br>
wait<br>
08:05:07 return self._code(response)<br>
08:05:07 File<br>
"/usr/lib64/python2.7/site-<wbr>packages/ovirtsdk4/service.py"<wbr>, line 268,<br>
in callback<br>
08:05:07 self._check_fault(response)<br>
08:05:07 File<br>
"/usr/lib64/python2.7/site-<wbr>packages/ovirtsdk4/service.py"<wbr>, line 132,<br>
in _check_fault<br>
08:05:07 self._raise_error(response, body)<br>
08:05:07 File<br>
"/usr/lib64/python2.7/site-<wbr>packages/ovirtsdk4/service.py"<wbr>, line 118,<br>
in _raise_error<br>
08:05:07 raise error<br>
08:05:07 Error: Fault reason is "Operation Failed". Fault detail is<br>
"[Cannot remove Snapshot. Snapshot is currently being created for VM<br>
vm1.]". HTTP response code is 409.<br>
<br>
<a href="http://jenkins.ovirt.org/job/ovirt-system-tests_master_check-patch-el7-x86_64/4770/consoleFull" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/<wbr>ovirt-system-tests_master_<wbr>check-patch-el7-x86_64/4770/<wbr>consoleFull</a><br>
<br>
Can you please check? Thanks.<br>
<div><div class="h5"><br>
>><br>
>><br>
>><br>
>> On Wed, Mar 28, 2018 at 12:01 PM, Dafna Ron <<a href="mailto:dron@redhat.com">dron@redhat.com</a>> wrote:<br>
>>><br>
>>> Thanks Beni.<br>
>>><br>
>>> can you please paste the fix that you submit to the mail?<br>
>>><br>
>>> Thanks,<br>
>>> Dafna<br>
>>><br>
>>><br>
>>> On Wed, Mar 28, 2018 at 9:13 AM, Benny Zlotnik <<a href="mailto:bzlotnik@redhat.com">bzlotnik@redhat.com</a>><br>
>>> wrote:<br>
>>>><br>
>>>> This issue looks like it's caused by the DB lock being released before<br>
>>>> the engine memory lock, there was a similar race with the LSM test a few<br>
>>>> months back. I'll apply the same fix for the snapshot_cold_merge.<br>
>>>> .<br>
>>>><br>
>>>> On Tue, Mar 27, 2018 at 10:03 PM, Benny Zlotnik <<a href="mailto:bzlotnik@redhat.com">bzlotnik@redhat.com</a>><br>
>>>> wrote:<br>
>>>>><br>
>>>>> Looking into this<br>
>>>>><br>
>>>>> On Tue, Mar 27, 2018 at 6:45 PM, Dafna Ron <<a href="mailto:dron@redhat.com">dron@redhat.com</a>> wrote:<br>
>>>>>><br>
>>>>>> Hi,<br>
>>>>>><br>
>>>>>> We have a failure on 004_basic_sanity.disk_<wbr>operations. the reported<br>
>>>>>> change seems to me to be connected to the failure.<br>
>>>>>><br>
>>>>>> Link and headline of suspected patches:<br>
>>>>>><br>
>>>>>> core: introduce CreateSnapshotForVm -<br>
>>>>>> <a href="https://gerrit.ovirt.org/#/c/87671/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>87671/</a><br>
>>>>>><br>
>>>>>><br>
>>>>>> Link to Job:<br>
>>>>>><br>
>>>>>> <a href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/<wbr>ovirt-master_change-queue-<wbr>tester/6564/</a><br>
>>>>>><br>
>>>>>> Link to all logs:<br>
>>>>>><br>
>>>>>><br>
>>>>>> <a href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/6564/artifact/exported-artifacts/basic-suit-master-el7/test_logs/basic-suite-master/post-004_basic_sanity.py/" rel="noreferrer" target="_blank">http://jenkins.ovirt.org/job/<wbr>ovirt-master_change-queue-<wbr>tester/6564/artifact/exported-<wbr>artifacts/basic-suit-master-<wbr>el7/test_logs/basic-suite-<wbr>master/post-004_basic_sanity.<wbr>py/</a><br>
>>>>>><br>
>>>>>> (Relevant) error snippet from the log:<br>
>>>>>><br>
>>>>>> <error><br>
>>>>>><br>
>>>>>><br>
>>>>>> 2018-03-27 11:30:16,167-04 WARN<br>
>>>>>> [org.ovirt.engine.core.bll.<wbr>snapshots.<wbr>CreateSnapshotForVmCommand] (default<br>
>>>>>> task-11) [9f713a3d-82a9-4db8-8207-<wbr>bb69a0a2c550] Validation of action<br>
>>>>>> 'CreateSnapshotForVm' failed for user admin@internal-authz. Re<br>
>>>>>> asons:<br>
>>>>>> VAR__ACTION__CREATE,VAR__TYPE_<wbr>_SNAPSHOT,ACTION_TYPE_FAILED_<wbr>SNAPSHOT_IS_BEING_TAKEN_FOR_<wbr>VM,$VmName<br>
>>>>>> vm1<br>
>>>>>> 2018-03-27 11:30:16,176-04 DEBUG<br>
>>>>>> [org.ovirt.engine.core.common.<wbr>di.interceptor.<wbr>DebugLoggingInterceptor]<br>
>>>>>> (default task-11) [9f713a3d-82a9-4db8-8207-<wbr>bb69a0a2c550] method: runAction,<br>
>>>>>> params: [CreateSnapshotForVm, CreateSnapshotForVmParameters<br>
>>>>>> :{commandId='d8bfdb0e-5ae4-<wbr>4fed-8852-a0b29e377708', user='null',<br>
>>>>>> commandType='Unknown', vmId='65e7fea5-8b7d-4281-bd0d-<wbr>6a84de207a00'}],<br>
>>>>>> timeElapsed: 57ms<br>
>>>>>> 2018-03-27 11:30:16,186-04 ERROR<br>
>>>>>> [org.ovirt.engine.api.restapi.<wbr>resource.<wbr>AbstractBackendResource] (default<br>
>>>>>> task-11) [] Operation Failed: [Cannot create Snapshot. Snapshot is currently<br>
>>>>>> being created for VM vm1.]<br>
>>>>>> 2018-03-27 11:30:16,820-04 INFO<br>
>>>>>> [org.ovirt.engine.core.bll.<wbr>SerialChildCommandsExecutionCa<wbr>llback]<br>
>>>>>> (EE-ManagedThreadFactory-<wbr>engineScheduled-Thread-100)<br>
>>>>>> [16c08ce6-9a73-489f-a1f2-<wbr>56e21699f14a] Command 'CopyImageGroupVolumesData'<br>
>>>>>> (id: '4c4023<br>
>>>>>> b4-21e8-4a98-8ac3-<wbr>aba51967d160') waiting on child command id:<br>
>>>>>> '3d96b933-6e3b-4838-94bc-<wbr>db3ff0cadcdc' type:'CopyData' to complete<br>
>>>>>> 2018-03-27 11:30:16,833-04 DEBUG<br>
>>>>>> [org.ovirt.engine.core.common.<wbr>di.interceptor.<wbr>DebugLoggingInterceptor]<br>
>>>>>> (EE-ManagedThreadFactory-<wbr>engineScheduled-Thread-100)<br>
>>>>>> [16c08ce6-9a73-489f-a1f2-<wbr>56e21699f14a] method: get, params:<br>
>>>>>> [a85992e9-40b8-4c80-9c<br>
>>>>>> 93-3bf767f6efa4], timeElapsed: 8ms<br>
>>>>>><br>
>>>>>><br>
>>>>>> </error><br>
>>>>>><br>
>>>>>><br>
>>>>><br>
>>>><br>
>>><br>
>><br>
>><br>
>> ______________________________<wbr>_________________<br>
>> Infra mailing list<br>
>> <a href="mailto:Infra@ovirt.org">Infra@ovirt.org</a><br>
>> <a href="http://lists.ovirt.org/mailman/listinfo/infra" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/infra</a><br>
>><br>
><br>
><br>
><br>
> --<br>
> GAL bEN HAIM<br>
> RHV DEVOPS<br>
><br>
> ______________________________<wbr>_________________<br>
> Infra mailing list<br>
> <a href="mailto:Infra@ovirt.org">Infra@ovirt.org</a><br>
> <a href="http://lists.ovirt.org/mailman/listinfo/infra" rel="noreferrer" target="_blank">http://lists.ovirt.org/<wbr>mailman/listinfo/infra</a><br>
><br>
<br>
<br>
<br>
--<br>
</div></div>Didi<br>
</blockquote></div><br></div>