4:14 a.m.
This is a multi-part message in MIME format.
--------------B1FC8E7562A08FCCC455CECC
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Hi,
We had a failure on master basic suite for test
007_sd_reattach.deactivate_storage_domain.
The failure was that we failed to deactivate domain due to running tasks.
It does not seem to be related to the patch it was testing and I think
that the test itself needs to be modified to check there are no running
tasks.
Is there perhaps a way to query if there are running tasks before
running the command? can you please take a look at the test on OST?
**
*Link and headline of suspected patches:**Not related to error***
**
**
*Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/*
***
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
(Relevant) error snippet from the log:
<error>
*2017-12-06 20:13:23,166-05 WARN
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
admin@internal-authz. Reasons:
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
2017-12-06 20:13:23,167-05 INFO
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
a4-44fe-aa43-e96754a975ba=STORAGE]',
sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06
20:13:23,172-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method:
runAction, params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06
20:13:23,176-05 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
task-7) [] Operation Failed: [Cannot deactivate Storage while there are
running tasks on this Storage. -Please wait until tasks will finish and
try again.] *
*<error>
--------------B1FC8E7562A08FCCC455CECC
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="content-type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We had a failure on master basic suite for test
007_sd_reattach.deactivate_storage_domain. <br>
</p>
<p>The failure was that we failed to deactivate domain due to
running tasks. <br>
</p>
<p>It does not seem to be related to the patch it was testing and I
think that the test itself needs to be modified to check there are
no running tasks. <br>
</p>
<p>Is there perhaps a way to query if there are running tasks before
running the command? can you please take a look at the test on
OST? <br>
</p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link
and headline of suspected patches:</span><b
style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><b>
Not related to error</b></b></p>
</b></p>
<p><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link
to Job: <a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
</b><b style="font-weight:normal;"
id="docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">Link
to all logs: <a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">(Relevant)
error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><error></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;"><b>2017-12-06
20:13:23,166-05 WARN
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of action
'DeactivateStorageDomainWithOvfUpdate' fa
iled for user admin@internal-authz. Reasons:
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
2017-12-06 20:13:23,167-05 INFO
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to object
'EngineLock:{exclusiveLocks='[ea2fd992-8a
a4-44fe-aa43-e96754a975ba=STORAGE]',
sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}'
2017-12-06 20:13:23,172-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor] (default task-7)
[d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction, params:
[DeactivateStorageDomainWithOvfUpdate, DeactivateSto
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
user='null', commandType='Unknown'}], timeElapsed: 65ms
2017-12-06 20:13:23,176-05 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-7) []
Operation Failed: [Cannot deactivate Storage while there are running tasks on this
Storage.
-Please wait until tasks will finish and try again.]
</b>
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt;"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap;">
</span></p>
</b><error><br
class="Apple-interchange-newline">
</p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
</body>
</html>
--------------B1FC8E7562A08FCCC455CECC--
5:06 a.m.
CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is that
we might have tasks which will start running internally using scheduling
(like OVF_UPDATE) and we can't really know how much time every task will
take until it will end.
Even if we check that there are no running tasks it will not guarantee that
no task will start until you deactivate the storage domain.
I think that the best solution for it is an engine support to cancel
running tasks or block tasks from running.
On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com> wrote:
Hi,
We had a failure on master basic suite for test
007_sd_reattach.deactivate_storage_domain.
The failure was that we failed to deactivate domain due to running tasks.
It does not seem to be related to the patch it was testing and I think
that the test itself needs to be modified to check there are no running
tasks.
Is there perhaps a way to query if there are running tasks before running
the command? can you please take a look at the test on OST?
*Link and headline of suspected patches: Not related to error*
*Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
* Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
(Relevant) error snippet from the log: <error> 2017-12-06 20:13:23,166-05
WARN
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
admin@internal-authz. Reasons:
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
2017-12-06 20:13:23,167-05 INFO
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
a4-44fe-aa43-e96754a975ba=STORAGE]',
sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06
20:13:23,172-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction,
params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06
20:13:23,176-05 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
task-7) [] Operation Failed: [Cannot deactivate Storage while there are
running tasks on this Storage. -Please wait until tasks will finish and try
again.] *<error>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
5:12 a.m.
This is a multi-part message in MIME format.
--------------CCBA5F838197258179C525C5
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
Maor, I either need to get new glasses or a magnifier glass to read what
you wrote :-P
when you say running tasks - these are actually running tasks that may
be running because of other tests in ost - correct? wouldn't killing or
blocking those can cause other tests to fail?
On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
that we might have tasks which will start running internally using
scheduling (like OVF_UPDATE) and we can't really know how much time
every task will take until it will end.
Even if we check that there are no running tasks it will not
guaranteethat no task will start until you deactivate the storage domain.
I think that the best solution for it is an engine support to cancel
running tasks or block tasks from running.
On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com
<mailto:dron@redhat.com>> wrote:
Hi,
We had a failure on master basic suite for test
007_sd_reattach.deactivate_storage_domain.
The failure was that we failed to deactivate domain due to running
tasks.
It does not seem to be related to the patch it was testing and I
think that the test itself needs to be modified to check there are
no running tasks.
Is there perhaps a way to query if there are running tasks before
running the command? can you please take a look at the test on OST?
**
*Link and headline of suspected patches:**Not related to error***
**
**
*Link to Job:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
***
Link to all logs:
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
(Relevant) error snippet from the log:
<error>
*2017-12-06 20:13:23,166-05 WARN
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation
of action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
admin@internal-authz. Reasons:
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
2017-12-06 20:13:23,167-05 INFO
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed
to object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
a4-44fe-aa43-e96754a975ba=STORAGE]',
sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}'
2017-12-06 20:13:23,172-05 DEBUG
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method:
runAction, params: [DeactivateStorageDomainWithOvfUpdate,
DeactivateSto
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06
20:13:23,176-05 ERROR
[org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
(default task-7) [] Operation Failed: [Cannot deactivate Storage
while there are running tasks on this Storage. -Please wait until
tasks will finish and try again.] *
*<error>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org <mailto:Devel@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
--------------CCBA5F838197258179C525C5
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Maor, I either need to get new glasses
or a magnifier glass to read what you wrote :-P<br>
when you say running tasks - these are actually running tasks that
may be running because of other tests in ost - correct? wouldn't
killing or blocking those can cause other tests to fail? <br>
<br>
On 12/07/2017 11:06 AM, Maor Lipchuk wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJ1JNOc0dtvdbmXA3CXXx3VWkREuWck2hYmRzjfACwr9XJ8swQ@mail.gmail.com">
<div dir="ltr"><span
id="gmail-m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
<p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><span
style="color:rgb(0,0,0);font-family:monospace;white-space:pre-wrap">CANNOT_DEACTIVATE_DOMAIN_WITH_</span><wbr
style="font-family:monospace"><span
style="color:rgb(0,0,0);font-family:monospace;white-space:pre-wrap">TASKS
</span><font><font
color="#000000" size="1"
face="Arial"><span style="white-space:pre-wrap">is a known
issue, the problem is that we might have tasks which will start running internally using
scheduling (like OVF_UPDATE) and we can't really know how much time every task will
take until it will end.</span></font></font></span></p>
<p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><font><font
color="#000000" size="1"
face="Arial"><span style="white-space:pre-wrap">Even if we
check that there are no running tasks it will not </span></font><font
color="#000000"><span
style="white-space:pre-wrap">guarantee</span></font><font
color="#000000"><font size="1"
face="Arial"><span style="white-space:pre-wrap"> that no task
will start until you deactivate the storage
domain.</span></font></font></font></span></p>
<p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><font><font
color="#000000"><font size="1"
face="Arial"><span style="white-space:pre-wrap">I think that
the best solution for it is an engine support to cancel running tasks or block tasks from
running.</span></font></font></font></span></p>
</span></div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Dec 7, 2017 at 12:14 PM, Dafna
Ron <span dir="ltr"><<a
href="mailto:dron@redhat.com"
target="_blank"
moz-do-not-send="true">dron(a)redhat.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We had a failure on master basic suite for test
007_sd_reattach.deactivate_<wbr>storage_domain. <br>
</p>
<p>The failure was that we failed to deactivate domain due
to running tasks. <br>
</p>
<p>It does not seem to be related to the patch it was
testing and I think that the test itself needs to be
modified to check there are no running tasks. <br>
</p>
<p>Is there perhaps a way to query if there are running
tasks before running the command? can you please take a
look at the test on OST? <br>
</p>
<p><b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
</b></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b
style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
and headline of suspected patches:</span><b
style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><b>
Not related to error</b></b></b></p>
<b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
</b>
<p><b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
</b></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b
style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
to Job: <a class="m_-5329876534188665956moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
target="_blank"
moz-do-not-send="true">http://jenkins.ovirt.org/job/<wbr&...
<b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
</b><b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
to all logs: <a class="m_-5329876534188665956moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
target="_blank"
moz-do-not-send="true">http://jenkins.ovirt.org/job/<wbr&...
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">(Relevant)
error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><error></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><b>2017-12-06
20:13:23,166-05 WARN
[org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>DeactivateStorageDomainWithOvf<wbr>UpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-<wbr>3362f2f130a0] Validation of action
'<wbr>DeactivateStorageDomainWithOvf<wbr>Update' fa
iled for user admin@internal-authz. Reasons:
VAR__TYPE__STORAGE__DOMAIN,<wbr>VAR__ACTION__DEACTIVATE,ERROR_<wbr>CANNOT_DEACTIVATE_DOMAIN_WITH_<wbr>TASKS
2017-12-06 20:13:23,167-05 INFO
[org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>DeactivateStorageDomainWithOvf<wbr>UpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-<wbr>3362f2f130a0] Lock freed to object
'EngineLock:{exclusiveLocks='[<wbr>ea2fd992-8a
a4-44fe-aa43-e96754a975ba=<wbr>STORAGE]',
sharedLocks='[5e0a0183-0e25-<wbr>4f43-b5b0-0cfb5510248e=POOL]'}<wbr>'
2017-12-06 20:13:23,172-05 DEBUG
[org.ovirt.engine.core.common.<wbr>di.interceptor.<wbr>DebugLoggingInterceptor]
(default task-7) [d82880e8-1d40-4a3b-a1ba-<wbr>3362f2f130a0] method: runAction,
params: [<wbr>DeactivateStorageDomainWithOvf<wbr>Update, DeactivateSto
rageDomainWithOvfUpdateParamet<wbr>ers:{commandId='630c28e1-41ab-<wbr>43db-9755-a2bb870dbcb3',
user='null', commandType='Unknown'}], timeElapsed: 65ms
2017-12-06 20:13:23,176-05 ERROR
[org.ovirt.engine.api.restapi.<wbr>resource.<wbr>AbstractBackendResource]
(default task-7) [] Operation Failed: [Cannot deactivate Storage while there are running
tasks on this Storage.
-Please wait until tasks will finish and try again.]
</b>
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">
</span></p>
</b><error><br
class="m_-5329876534188665956Apple-interchange-newline">
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
</div>
<br>
______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org"
moz-do-not-send="true">Devel(a)ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/devel"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.ovirt.org/<wbr>mai...
</blockquote>
</div>
<br>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>
--------------CCBA5F838197258179C525C5--
5:22 a.m.
On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com> wrote:
Maor, I either need to get new glasses or a magnifier glass to read
what
you wrote :-P
when you say running tasks - these are actually running tasks that may be
running because of other tests in ost - correct? wouldn't killing or
blocking those can cause other tests to fail?
It might well be the OVF update. How can we, from the API, wait for those
tasks to complete? Or should we catch exception and retry?
Y.
On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is that
we might have tasks which will start running internally using scheduling
(like OVF_UPDATE) and we can't really know how much time every task will
take until it will end.
Even if we check that there are no running tasks it will not guarantee
that no task will start until you deactivate the storage domain.
I think that the best solution for it is an engine support to cancel
running tasks or block tasks from running.
On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com> wrote:
> Hi,
>
> We had a failure on master basic suite for test
> 007_sd_reattach.deactivate_storage_domain.
>
> The failure was that we failed to deactivate domain due to running tasks.
>
> It does not seem to be related to the patch it was testing and I think
> that the test itself needs to be modified to check there are no running
> tasks.
>
> Is there perhaps a way to query if there are running tasks before running
> the command? can you please take a look at the test on OST?
>
> *Link and headline of suspected patches: Not related to error*
>
> *Link to Job:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>
>
> * Link to all logs:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
> (Relevant) error snippet from the log: <error> 2017-12-06 20:13:23,166-05
> WARN
>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
> admin@internal-authz. Reasons:
>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
> 2017-12-06 20:13:23,167-05 INFO
>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
> a4-44fe-aa43-e96754a975ba=STORAGE]',
> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06
> 20:13:23,172-05 DEBUG
> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction,
> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
> user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06
> 20:13:23,176-05 ERROR
> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
> task-7) [] Operation Failed: [Cannot deactivate Storage while there are
> running tasks on this Storage. -Please wait until tasks will finish and try
> again.] *<error>
>
>
>
>
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
5:30 a.m.
I think that the maybe the QE can share their methods on how to avoid those
issues.
From what I remember, before deactivating storage domain they make
sure
that there are no running tasks related to
the storage domain.
On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com> wrote:
> Maor, I either need to get new glasses or a magnifier glass to read what
> you wrote :-P
> when you say running tasks - these are actually running tasks that may be
> running because of other tests in ost - correct? wouldn't killing or
> blocking those can cause other tests to fail?
>
It might well be the OVF update. How can we, from the API, wait for those
tasks to complete? Or should we catch exception and retry?
Y.
>
> On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>
> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
> that we might have tasks which will start running internally using
> scheduling (like OVF_UPDATE) and we can't really know how much time every
> task will take until it will end.
>
> Even if we check that there are no running tasks it will not guarantee
> that no task will start until you deactivate the storage domain.
>
> I think that the best solution for it is an engine support to cancel
> running tasks or block tasks from running.
>
> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com> wrote:
>
>> Hi,
>>
>> We had a failure on master basic suite for test
>> 007_sd_reattach.deactivate_storage_domain.
>>
>> The failure was that we failed to deactivate domain due to running
>> tasks.
>>
>> It does not seem to be related to the patch it was testing and I think
>> that the test itself needs to be modified to check there are no running
>> tasks.
>>
>> Is there perhaps a way to query if there are running tasks before
>> running the command? can you please take a look at the test on OST?
>>
>> *Link and headline of suspected patches: Not related to error*
>>
>> *Link to Job:
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>>
>>
>> * Link to all logs:
>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
>> (Relevant) error snippet from the log: <error> 2017-12-06 20:13:23,166-05
>> WARN
>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
>> admin@internal-authz. Reasons:
>>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
>> 2017-12-06 20:13:23,167-05 INFO
>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
>> a4-44fe-aa43-e96754a975ba=STORAGE]',
>> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}'
2017-12-06
>> 20:13:23,172-05 DEBUG
>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction,
>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>>
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
>> user='null', commandType='Unknown'}], timeElapsed: 65ms
2017-12-06
>> 20:13:23,176-05 ERROR
>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>> task-7) [] Operation Failed: [Cannot deactivate Storage while there are
>> running tasks on this Storage. -Please wait until tasks will finish and try
>> again.] *<error>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel
--
Regards,
Eyal Shenitzky
5:59 a.m.
On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz(a)redhat.com> wrote:
I think that the maybe the QE can share their methods on how to
avoid
those issues.
From what I remember, before deactivating storage domain they make sure
that there are no running tasks related to
the storage domain.
Looks like an easy fix is to wrap it with try, except sdk4.Error and let it
sit within the testlib.assert_true_within_short() loop.
Y.
On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
>
>
> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com> wrote:
>
>> Maor, I either need to get new glasses or a magnifier glass to read what
>> you wrote :-P
>> when you say running tasks - these are actually running tasks that may
>> be running because of other tests in ost - correct? wouldn't killing or
>> blocking those can cause other tests to fail?
>>
>
> It might well be the OVF update. How can we, from the API, wait for those
> tasks to complete? Or should we catch exception and retry?
> Y.
>
>
>>
>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>>
>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
>> that we might have tasks which will start running internally using
>> scheduling (like OVF_UPDATE) and we can't really know how much time every
>> task will take until it will end.
>>
>> Even if we check that there are no running tasks it will not guarantee
>> that no task will start until you deactivate the storage domain.
>>
>> I think that the best solution for it is an engine support to cancel
>> running tasks or block tasks from running.
>>
>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com> wrote:
>>
>>> Hi,
>>>
>>> We had a failure on master basic suite for test
>>> 007_sd_reattach.deactivate_storage_domain.
>>>
>>> The failure was that we failed to deactivate domain due to running
>>> tasks.
>>>
>>> It does not seem to be related to the patch it was testing and I think
>>> that the test itself needs to be modified to check there are no running
>>> tasks.
>>>
>>> Is there perhaps a way to query if there are running tasks before
>>> running the command? can you please take a look at the test on OST?
>>>
>>> *Link and headline of suspected patches: Not related to error*
>>>
>>> *Link to Job:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>>> <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>>>
>>>
>>> * Link to all logs:
>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>>>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
>>> (Relevant) error snippet from the log: <error> 2017-12-06
20:13:23,166-05
>>> WARN
>>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
>>> admin@internal-authz. Reasons:
>>>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
>>> 2017-12-06 20:13:23,167-05 INFO
>>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
>>> a4-44fe-aa43-e96754a975ba=STORAGE]',
>>> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}'
2017-12-06
>>> 20:13:23,172-05 DEBUG
>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method: runAction,
>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>>>
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
>>> user='null', commandType='Unknown'}], timeElapsed: 65ms
2017-12-06
>>> 20:13:23,176-05 ERROR
>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>>> task-7) [] Operation Failed: [Cannot deactivate Storage while there are
>>> running tasks on this Storage. -Please wait until tasks will finish and try
>>> again.] *<error>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel
>
--
Regards,
Eyal Shenitzky
1:31 p.m.
This is a multi-part message in MIME format.
--------------78AFAE881715BC23A10C5F16
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
I opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523813
Allon, can you please assign someone to help fix this test?
Please let me know if you need help from me.
Thanks!
Dafna
On 12/07/2017 11:59 AM, Yaniv Kaul wrote:
On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz(a)redhat.com
<mailto:eshenitz@redhat.com>> wrote:
I think that the maybe the QE can share their methods on how to
avoid those issues.
From what I remember, before deactivating storage domain they make
sure that there are no running tasks related to
the storage domain.
Looks like an easy fix is to wrap it with try, except sdk4.Error and
let it sit within the testlib.assert_true_within_short() loop.
Y.
On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul(a)redhat.com
<mailto:ykaul@redhat.com>> wrote:
On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com
<mailto:dron@redhat.com>> wrote:
Maor, I either need to get new glasses or a magnifier
glass to read what you wrote :-P
when you say running tasks - these are actually running
tasks that may be running because of other tests in ost -
correct? wouldn't killing or blocking those can cause
other tests to fail?
It might well be the OVF update. How can we, from the API,
wait for those tasks to complete? Or should we catch exception
and retry?
Y.
On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>
> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the
> problem is that we might have tasks which will start
> running internally using scheduling (like OVF_UPDATE) and
> we can't really know how much time every task will take
> until it will end.
>
> Even if we check that there are no running tasks it will
> not guaranteethat no task will start until you deactivate
> the storage domain.
>
> I think that the best solution for it is an engine
> support to cancel running tasks or block tasks from running.
>
>
> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron
> <dron(a)redhat.com <mailto:dron@redhat.com>> wrote:
>
> Hi,
>
> We had a failure on master basic suite for test
> 007_sd_reattach.deactivate_storage_domain.
>
> The failure was that we failed to deactivate domain
> due to running tasks.
>
> It does not seem to be related to the patch it was
> testing and I think that the test itself needs to be
> modified to check there are no running tasks.
>
> Is there perhaps a way to query if there are running
> tasks before running the command? can you please take
> a look at the test on OST?
>
> **
>
> *Link and headline of suspected patches:**Not related
> to error***
>
> **
>
> **
>
> *Link to Job:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>
> ***
>
> Link to all logs:
>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
>
>
> (Relevant) error snippet from the log:
>
> <error>
>
> *2017-12-06 20:13:23,166-05 WARN
>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
> (default task-7)
> [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
> action 'DeactivateStorageDomainWithOvfUpdate' fa iled
> for user admin@internal-authz. Reasons:
>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
> 2017-12-06 20:13:23,167-05 INFO
>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
> (default task-7)
> [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
> a4-44fe-aa43-e96754a975ba=STORAGE]',
>
sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}'
> 2017-12-06 20:13:23,172-05 DEBUG
>
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
> (default task-7)
> [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method:
> runAction, params:
> [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
> user='null', commandType='Unknown'}], timeElapsed:
> 65ms 2017-12-06 20:13:23,176-05 ERROR
> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
> (default task-7) [] Operation Failed: [Cannot
> deactivate Storage while there are running tasks on
> this Storage. -Please wait until tasks will finish
> and try again.] *
>
> *<error>
>
>
>
>
>
> _______________________________________________
> Devel mailing list
> Devel(a)ovirt.org <mailto:Devel@ovirt.org>
> http://lists.ovirt.org/mailman/listinfo/devel
> <http://lists.ovirt.org/mailman/listinfo/devel>
>
>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org <mailto:Devel@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
_______________________________________________
Devel mailing list
Devel(a)ovirt.org <mailto:Devel@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/devel
<http://lists.ovirt.org/mailman/listinfo/devel>
--
Regards,
Eyal Shenitzky
--------------78AFAE881715BC23A10C5F16
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">I opened a bug:
<a class="moz-txt-link-freetext"
href="https://bugzilla.redhat.com/show_bug.cgi?id=1523813">h...
Allon, can you please assign someone to help fix this test? <br>
Please let me know if you need help from me. <br>
<br>
Thanks!<br>
Dafna<br>
<br>
<br>
On 12/07/2017 11:59 AM, Yaniv Kaul wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAJgorsZ0iXzc0WpWZHBe0=NtY53cs_wGRY4X8KQqS3oa0YCX0A@mail.gmail.com">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Dec 7, 2017 at 1:30 PM, Eyal
Shenitzky <span dir="ltr"><<a
href="mailto:eshenitz@redhat.com" target="_blank"
moz-do-not-send="true">eshenitz(a)redhat.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">I think that the maybe the QE can share
their methods on how to avoid those issues.
<div>From what I remember, before deactivating storage
domain they make sure that there are no running tasks
related to</div>
<div>the storage domain.</div>
</div>
</blockquote>
<div><br>
</div>
<div>Looks like an easy fix is to wrap it with try, except
sdk4.Error and let it sit within the
testlib.assert_true_within_short() loop.</div>
<div>Y.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="gmail_extra">
<div>
<div class="h5"><br>
<div class="gmail_quote">On Thu, Dec 7, 2017 at 1:22
PM, Yaniv Kaul <span dir="ltr"><<a
href="mailto:ykaul@redhat.com"
target="_blank"
moz-do-not-send="true">ykaul(a)redhat.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Thu, Dec 7, 2017
at 1:12 PM, Dafna Ron <span
dir="ltr"><<a
href="mailto:dron@redhat.com"
target="_blank"
moz-do-not-send="true">dron(a)redhat.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF">
<div
class="m_8942236351456563473m_-5014669676915136678m_3333631383734454308moz-cite-prefix">Maor,
I either need to get new glasses or
a magnifier glass to read what you
wrote :-P<br>
when you say running tasks - these
are actually running tasks that may
be running because of other tests in
ost - correct? wouldn't killing or
blocking those can cause other tests
to fail? <br>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>It might well be the OVF update. How
can we, from the API, wait for those
tasks to complete? Or should we catch
exception and retry?</div>
<div>Y.</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF">
<div
class="m_8942236351456563473m_-5014669676915136678m_3333631383734454308moz-cite-prefix">
<div>
<div
class="m_8942236351456563473m_-5014669676915136678h5">
<br>
On 12/07/2017 11:06 AM, Maor
Lipchuk wrote:<br>
</div>
</div>
</div>
<div>
<div
class="m_8942236351456563473m_-5014669676915136678h5">
<blockquote type="cite">
<div dir="ltr"><span
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308gmail-m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
<p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><span
style="color:rgb(0,0,0);font-family:monospace;white-space:pre-wrap">CANNOT_DEACTIVATE_DOMAIN_WITH_</span><span
style="color:rgb(0,0,0);font-family:monospace;white-space:pre-wrap"><wbr>TASKS
</span><font><font
color="#000000"
size="1"
face="Arial"><span
style="white-space:pre-wrap">is a known issue, the problem is that we might
have tasks which will start running internally using scheduling (like OVF_UPDATE) and we
can't really know how much time every task will take until it will
end.</span></font></font></span></p>
<p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><font><font
color="#000000"
size="1"
face="Arial"><span
style="white-space:pre-wrap">Even if we check that there are no running tasks
it will not </span></font><font
color="#000000"><span
style="white-space:pre-wrap">guarantee</span></font><font
color="#000000"><font
size="1"
face="Arial"><span
style="white-space:pre-wrap"> that no task will start until you deactivate
the storage
domain.</span></font></font></font></span></p>
<p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><font><font
color="#000000"><font
size="1"
face="Arial"><span
style="white-space:pre-wrap">I think that the best solution for it is an
engine support to cancel running tasks or block tasks from
running.</span></font></font></font></span></p>
</span></div>
<div
class="gmail_extra"><br>
<div class="gmail_quote">On
Thu, Dec 7, 2017 at 12:14
PM, Dafna Ron <span
dir="ltr"><<a
href="mailto:dron@redhat.com"
target="_blank"
moz-do-not-send="true">dron(a)redhat.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div text="#000000"
bgcolor="#FFFFFF">
<p>Hi, <br>
</p>
<p>We had a failure on
master basic suite for
test
007_sd_reattach.deactivate_sto<wbr>rage_domain.
<br>
</p>
<p>The failure was that
we failed to
deactivate domain due
to running tasks. <br>
</p>
<p>It does not seem to
be related to the
patch it was testing
and I think that the
test itself needs to
be modified to check
there are no running
tasks. <br>
</p>
<p>Is there perhaps a
way to query if there
are running tasks
before running the
command? can you
please take a look at
the test on OST? <br>
</p>
<p><b
style="font-weight:normal"
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
</b></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b
style="font-weight:normal"
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
and headline of suspected patches:</span><b
style="font-weight:normal"
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><b>
Not related to
error</b></b></b></p>
<b
style="font-weight:normal"
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
</b>
<p><b
style="font-weight:normal"
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
</b></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b
style="font-weight:normal"
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
to Job: <a
class="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
target="_blank"
moz-do-not-send="true">http://jenkins.ovirt.org/job/o<wbr...
<b
style="font-weight:normal"
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
</b><b
style="font-weight:normal"
id="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
to all logs: <a
class="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
target="_blank"
moz-do-not-send="true">http://jenkins.ovirt.org/job/o<wbr...
<br>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">(Relevant)
error snippet from the log: </span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><error></span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><b>2017-12-06
20:13:23,166-05 WARN
[org.ovirt.engine.core.bll.sto<wbr>rage.domain.DeactivateStorageD<wbr>omainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f<wbr>2f130a0] Validation of action
'DeactivateStorageDomainWithOv<wbr>fUpdate' fa
iled for user admin@internal-authz. Reasons:
VAR__TYPE__STORAGE__DOMAIN,VAR<wbr>__ACTION__DEACTIVATE,ERROR_CAN<wbr>NOT_DEACTIVATE_DOMAIN_WITH_TAS<wbr>KS
2017-12-06 20:13:23,167-05 INFO
[org.ovirt.engine.core.bll.sto<wbr>rage.domain.DeactivateStorageD<wbr>omainWithOvfUpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f<wbr>2f130a0] Lock freed to object
'EngineLock:{exclusiveLocks='[<wbr>ea2fd992-8a
a4-44fe-aa43-e96754a975ba=STOR<wbr>AGE]',
sharedLocks='[5e0a0183-0e25-4f<wbr>43-b5b0-0cfb5510248e=POOL]'}'
2017-12-06 20:13:23,172-05 DEBUG
[org.ovirt.engine.core.common.<wbr>di.interceptor.DebugLoggingInt<wbr>erceptor]
(default task-7) [d82880e8-1d40-4a3b-a1ba-3362f<wbr>2f130a0] method: runAction,
params: [DeactivateStorageDomainWithOv<wbr>fUpdate, DeactivateSto
rageDomainWithOvfUpdateParamet<wbr>ers:{commandId='630c28e1-41ab-<wbr>43db-9755-a2bb870dbcb3',
user='null', commandType='Unknown'}], timeElapsed: 65ms
2017-12-06 20:13:23,176-05 ERROR
[org.ovirt.engine.api.restapi.<wbr>resource.AbstractBackendResour<wbr>ce]
(default task-7) [] Operation Failed: [Cannot deactivate Storage while there are running
tasks on this Storage.
-Please wait until tasks will finish and try again.]
</b>
</span></p>
<p dir="ltr"
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">
</span></p>
</b><error><br
class="m_8942236351456563473m_-5014669676915136678m_3333631383734454308m_-5329876534188665956Apple-interchange-newline">
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
</div>
<br>
______________________________<wbr>_________________<br>
Devel mailing list<br>
<a
href="mailto:Devel@ovirt.org"
target="_blank"
moz-do-not-send="true">Devel(a)ovirt.org</a><br>
<a
href="http://lists.ovirt.org/mailman/listinfo/devel"
rel="noreferrer"
target="_blank"
moz-do-not-send="true">http://lists.ovirt.org/mailman<wbr...
</blockquote>
</div>
<br>
</div>
</blockquote>
<p><br>
</p>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org"
target="_blank"
moz-do-not-send="true">Devel(a)ovirt.org</a><br>
<a
href="http://lists.ovirt.org/mailman/listinfo/devel"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.ovirt.org/mailman<wbr...
</blockquote>
</div>
<br>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Devel mailing list<br>
<a href="mailto:Devel@ovirt.org"
target="_blank"
moz-do-not-send="true">Devel(a)ovirt.org</a><br>
<a
href="http://lists.ovirt.org/mailman/listinfo/devel"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.ovirt.org/mailman<wbr...
</blockquote>
</div>
<br>
<br clear="all">
<div><br>
</div>
</div>
</div>
<span class="HOEnZb"><font
color="#888888">-- <br>
<div class="m_8942236351456563473gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">Regards,
<div>Eyal Shenitzky</div>
</div>
</div>
</font></span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<p><br>
</p>
</body>
</html>
--------------78AFAE881715BC23A10C5F16--
2:39 p.m.
On Fri, Dec 8, 2017 at 9:31 PM, Dafna Ron <dron(a)redhat.com> wrote:
I opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523813
I'm optimistically hoping https://gerrit.ovirt.org/#/c/85195/ will fix it.
Not sure.
Y.
Allon, can you please assign someone to help fix this test?
Please let me know if you need help from me.
Thanks!
Dafna
On 12/07/2017 11:59 AM, Yaniv Kaul wrote:
On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz(a)redhat.com>
wrote:
> I think that the maybe the QE can share their methods on how to avoid
> those issues.
> From what I remember, before deactivating storage domain they make sure
> that there are no running tasks related to
> the storage domain.
>
Looks like an easy fix is to wrap it with try, except sdk4.Error and let
it sit within the testlib.assert_true_within_short() loop.
Y.
>
> On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
>
>>
>>
>> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com> wrote:
>>
>>> Maor, I either need to get new glasses or a magnifier glass to read
>>> what you wrote :-P
>>> when you say running tasks - these are actually running tasks that may
>>> be running because of other tests in ost - correct? wouldn't killing or
>>> blocking those can cause other tests to fail?
>>>
>>
>> It might well be the OVF update. How can we, from the API, wait for
>> those tasks to complete? Or should we catch exception and retry?
>> Y.
>>
>>
>>>
>>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>>>
>>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
>>> that we might have tasks which will start running internally using
>>> scheduling (like OVF_UPDATE) and we can't really know how much time
every
>>> task will take until it will end.
>>>
>>> Even if we check that there are no running tasks it will not guarantee
>>> that no task will start until you deactivate the storage domain.
>>>
>>> I think that the best solution for it is an engine support to cancel
>>> running tasks or block tasks from running.
>>>
>>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> We had a failure on master basic suite for test
>>>> 007_sd_reattach.deactivate_storage_domain.
>>>>
>>>> The failure was that we failed to deactivate domain due to running
>>>> tasks.
>>>>
>>>> It does not seem to be related to the patch it was testing and I think
>>>> that the test itself needs to be modified to check there are no running
>>>> tasks.
>>>>
>>>> Is there perhaps a way to query if there are running tasks before
>>>> running the command? can you please take a look at the test on OST?
>>>>
>>>> *Link and headline of suspected patches: Not related to error*
>>>>
>>>> *Link to Job:
>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>>>>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>>>>
>>>>
>>>> * Link to all logs:
>>>>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>>>>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
>>>> (Relevant) error snippet from the log: <error> 2017-12-06
20:13:23,166-05
>>>> WARN
>>>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation of
>>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
>>>> admin@internal-authz. Reasons:
>>>>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
>>>> 2017-12-06 20:13:23,167-05 INFO
>>>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed to
>>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
>>>> a4-44fe-aa43-e96754a975ba=STORAGE]',
>>>> sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}'
2017-12-06
>>>> 20:13:23,172-05 DEBUG
>>>> [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method:
runAction,
>>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>>>>
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
>>>> user='null', commandType='Unknown'}], timeElapsed: 65ms
2017-12-06
>>>> 20:13:23,176-05 ERROR
>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default
>>>> task-7) [] Operation Failed: [Cannot deactivate Storage while there are
>>>> running tasks on this Storage. -Please wait until tasks will finish and
try
>>>> again.] *<error>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>> _______________________________________________
>> Devel mailing list
>> Devel(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>>
>
>
>
> --
> Regards,
> Eyal Shenitzky
>
3:20 p.m.
On Fri, Dec 8, 2017 at 10:39 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
On Fri, Dec 8, 2017 at 9:31 PM, Dafna Ron <dron(a)redhat.com> wrote:
> I opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523813
>
I'm optimistically hoping https://gerrit.ovirt.org/#/c/85195/ will fix it.
Not sure.
Keeps failing with:
Operation Failed: [Cannot deactivate Storage. The relevant Storage Domain's
status is Maintenance.]
Which is strange:
1. I do check if the SD is in Maint. mode before trying to deactive and the
test is supposed to be skipped if it is.
2. Why can't I deactive a SD when in Maint. mode?
Probably missing something here.
Y.
Y.
>
> Allon, can you please assign someone to help fix this test?
> Please let me know if you need help from me.
>
> Thanks!
> Dafna
>
>
>
> On 12/07/2017 11:59 AM, Yaniv Kaul wrote:
>
>
>
> On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz(a)redhat.com>
> wrote:
>
>> I think that the maybe the QE can share their methods on how to avoid
>> those issues.
>> From what I remember, before deactivating storage domain they make sure
>> that there are no running tasks related to
>> the storage domain.
>>
>
> Looks like an easy fix is to wrap it with try, except sdk4.Error and let
> it sit within the testlib.assert_true_within_short() loop.
> Y.
>
>
>>
>> On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
>>
>>>
>>>
>>> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com> wrote:
>>>
>>>> Maor, I either need to get new glasses or a magnifier glass to read
>>>> what you wrote :-P
>>>> when you say running tasks - these are actually running tasks that may
>>>> be running because of other tests in ost - correct? wouldn't killing
or
>>>> blocking those can cause other tests to fail?
>>>>
>>>
>>> It might well be the OVF update. How can we, from the API, wait for
>>> those tasks to complete? Or should we catch exception and retry?
>>> Y.
>>>
>>>
>>>>
>>>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>>>>
>>>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
>>>> that we might have tasks which will start running internally using
>>>> scheduling (like OVF_UPDATE) and we can't really know how much time
every
>>>> task will take until it will end.
>>>>
>>>> Even if we check that there are no running tasks it will not guarantee
>>>> that no task will start until you deactivate the storage domain.
>>>>
>>>> I think that the best solution for it is an engine support to cancel
>>>> running tasks or block tasks from running.
>>>>
>>>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com>
wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> We had a failure on master basic suite for test
>>>>> 007_sd_reattach.deactivate_storage_domain.
>>>>>
>>>>> The failure was that we failed to deactivate domain due to running
>>>>> tasks.
>>>>>
>>>>> It does not seem to be related to the patch it was testing and I
>>>>> think that the test itself needs to be modified to check there are
no
>>>>> running tasks.
>>>>>
>>>>> Is there perhaps a way to query if there are running tasks before
>>>>> running the command? can you please take a look at the test on OST?
>>>>>
>>>>> *Link and headline of suspected patches: Not related to error*
>>>>>
>>>>> *Link to Job:
>>>>> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>>>>>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>>>>>
>>>>>
>>>>> * Link to all logs:
>>>>>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>>>>>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
>>>>> (Relevant) error snippet from the log: <error> 2017-12-06
20:13:23,166-05
>>>>> WARN
>>>>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation
of
>>>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for
user
>>>>> admin@internal-authz. Reasons:
>>>>>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
>>>>> 2017-12-06 20:13:23,167-05 INFO
>>>>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed
to
>>>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
>>>>> a4-44fe-aa43-e96754a975ba=STORAGE]',
>>>>>
sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06
>>>>> 20:13:23,172-05 DEBUG
>>>>>
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method:
runAction,
>>>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>>>>>
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
>>>>> user='null', commandType='Unknown'}], timeElapsed:
65ms 2017-12-06
>>>>> 20:13:23,176-05 ERROR
>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
(default
>>>>> task-7) [] Operation Failed: [Cannot deactivate Storage while there
are
>>>>> running tasks on this Storage. -Please wait until tasks will finish
and try
>>>>> again.] *<error>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Devel mailing list
>>>>> Devel(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>> _______________________________________________
>>> Devel mailing list
>>> Devel(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>
>>
>>
>>
>> --
>> Regards,
>> Eyal Shenitzky
>>
>
>
>
4:38 a.m.
On Fri, Dec 8, 2017 at 11:20 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
On Fri, Dec 8, 2017 at 10:39 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
>
>
> On Fri, Dec 8, 2017 at 9:31 PM, Dafna Ron <dron(a)redhat.com> wrote:
>
>> I opened a bug: https://bugzilla.redhat.com/show_bug.cgi?id=1523813
>>
>
> I'm optimistically hoping https://gerrit.ovirt.org/#/c/85195/ will fix
> it.
> Not sure.
>
Keeps failing with:
Operation Failed: [Cannot deactivate Storage. The relevant Storage
Domain's status is Maintenance.]
Which is strange:
1. I do check if the SD is in Maint. mode before trying to deactive and
the test is supposed to be skipped if it is.
I've added a comment in the patch,
I suspect it is related to the fact that we fetch the domain from
storage_domains and not attached_storage_domains
2. Why can't I deactive a SD when in Maint. mode?
I assume that the engine fails since that means that the storage domain is
not connected to the Host and the execute phase of maintenance performs
operations like update ovf store which depend on the storage domain to be
connected.
Probably missing something here.
Y.
Y.
>
>
>>
>> Allon, can you please assign someone to help fix this test?
>> Please let me know if you need help from me.
>>
>> Thanks!
>> Dafna
>>
>>
Thanks Dafna, besides the fix which should be done in the OST, here is an
old old bug which might also become useful:
https://bugzilla.redhat.com/879248
<https://bugzilla.redhat.com/show_bug.cgi?id=879248>
>>
>> On 12/07/2017 11:59 AM, Yaniv Kaul wrote:
>>
>>
>>
>> On Thu, Dec 7, 2017 at 1:30 PM, Eyal Shenitzky <eshenitz(a)redhat.com>
>> wrote:
>>
>>> I think that the maybe the QE can share their methods on how to avoid
>>> those issues.
>>> From what I remember, before deactivating storage domain they make sure
>>> that there are no running tasks related to
>>> the storage domain.
>>>
>>
>> Looks like an easy fix is to wrap it with try, except sdk4.Error and let
>> it sit within the testlib.assert_true_within_short() loop.
>> Y.
>>
>>
>>>
>>> On Thu, Dec 7, 2017 at 1:22 PM, Yaniv Kaul <ykaul(a)redhat.com> wrote:
>>>
>>>>
>>>>
>>>> On Thu, Dec 7, 2017 at 1:12 PM, Dafna Ron <dron(a)redhat.com> wrote:
>>>>
>>>>> Maor, I either need to get new glasses or a magnifier glass to read
>>>>> what you wrote :-P
>>>>> when you say running tasks - these are actually running tasks that
>>>>> may be running because of other tests in ost - correct? wouldn't
killing or
>>>>> blocking those can cause other tests to fail?
>>>>>
>>>>
>>>> It might well be the OVF update. How can we, from the API, wait for
>>>> those tasks to complete? Or should we catch exception and retry?
>>>> Y.
>>>>
>>>>
>>>>>
>>>>> On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
>>>>>
>>>>> CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
>>>>> that we might have tasks which will start running internally using
>>>>> scheduling (like OVF_UPDATE) and we can't really know how much
time every
>>>>> task will take until it will end.
>>>>>
>>>>> Even if we check that there are no running tasks it will not
>>>>> guarantee that no task will start until you deactivate the storage
>>>>> domain.
>>>>>
>>>>> I think that the best solution for it is an engine support to cancel
>>>>> running tasks or block tasks from running.
>>>>>
>>>>> On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com>
wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> We had a failure on master basic suite for test
>>>>>> 007_sd_reattach.deactivate_storage_domain.
>>>>>>
>>>>>> The failure was that we failed to deactivate domain due to
running
>>>>>> tasks.
>>>>>>
>>>>>> It does not seem to be related to the patch it was testing and I
>>>>>> think that the test itself needs to be modified to check there
are no
>>>>>> running tasks.
>>>>>>
>>>>>> Is there perhaps a way to query if there are running tasks
before
>>>>>> running the command? can you please take a look at the test on
OST?
>>>>>>
>>>>>> *Link and headline of suspected patches: Not related to error*
>>>>>>
>>>>>> *Link to Job:
>>>>>>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
>>>>>>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*
>>>>>>
>>>>>>
>>>>>> * Link to all logs:
>>>>>>
http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
>>>>>>
<http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...
>>>>>> (Relevant) error snippet from the log: <error> 2017-12-06
20:13:23,166-05
>>>>>> WARN
>>>>>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0]
Validation of
>>>>>> action 'DeactivateStorageDomainWithOvfUpdate' fa iled for
user
>>>>>> admin@internal-authz. Reasons:
>>>>>>
VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
>>>>>> 2017-12-06 20:13:23,167-05 INFO
>>>>>>
[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
>>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock
freed to
>>>>>> object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
>>>>>> a4-44fe-aa43-e96754a975ba=STORAGE]',
>>>>>>
sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}' 2017-12-06
>>>>>> 20:13:23,172-05 DEBUG
>>>>>>
[org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
>>>>>> (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method:
runAction,
>>>>>> params: [DeactivateStorageDomainWithOvfUpdate, DeactivateSto
>>>>>>
rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
>>>>>> user='null', commandType='Unknown'}],
timeElapsed: 65ms 2017-12-06
>>>>>> 20:13:23,176-05 ERROR
>>>>>> [org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
(default
>>>>>> task-7) [] Operation Failed: [Cannot deactivate Storage while
there are
>>>>>> running tasks on this Storage. -Please wait until tasks will
finish and try
>>>>>> again.] *<error>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Devel mailing list
>>>>>> Devel(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Devel mailing list
>>>>> Devel(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Devel mailing list
>>>> Devel(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/devel
>>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Eyal Shenitzky
>>>
>>
>>
>>
>
2337
days inactive
2340
days old
9 comments
4 participants
participants (4)
-
Dafna Ron -
Eyal Shenitzky -
Maor Lipchuk -
Yaniv Kaul