Re: [ovirt-devel] [ OST Failure Report ] [ oVirtMaster ] [ 07-11-2017 ] [007_sd_reattach.deactivate_storage_domain ]

Thursday, 7 December 2017

This is a multi-part message in MIME format.
--------------CCBA5F838197258179C525C5
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

Maor, I either need to get new glasses or a magnifier glass to read what
you wrote :-P
when you say running tasks - these are actually running tasks that may
be running because of other tests in ost - correct? wouldn't killing or
blocking those can cause other tests to fail?

On 12/07/2017 11:06 AM, Maor Lipchuk wrote:
...

 CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS is a known issue, the problem is
 that we might have tasks which will start running internally using
 scheduling (like OVF_UPDATE) and we can't really know how much time
 every task will take until it will end.

 Even if we check that there are no running tasks it will not
 guaranteethat no task will start until you deactivate the storage domain.

 I think that the best solution for it is an engine support to cancel
 running tasks or block tasks from running.

 On Thu, Dec 7, 2017 at 12:14 PM, Dafna Ron <dron(a)redhat.com
 <mailto:dron@redhat.com>> wrote:

     Hi,

     We had a failure on master basic suite for test
     007_sd_reattach.deactivate_storage_domain.

     The failure was that we failed to deactivate domain due to running
     tasks.

     It does not seem to be related to the patch it was testing and I
     think that the test itself needs to be modified to check there are
     no running tasks.

     Is there perhaps a way to query if there are running tasks before
     running the command? can you please take a look at the test on OST?

     **

     *Link and headline of suspected patches:**Not related to error***

     **

     **

     *Link to Job:
     http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/
     <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/>*

     ***

     Link to all logs:
     http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifact/
     <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4319/artifa...

     (Relevant) error snippet from the log:

     <error>

     *2017-12-06 20:13:23,166-05 WARN

[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
     (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Validation
     of action 'DeactivateStorageDomainWithOvfUpdate' fa iled for user
     admin@internal-authz. Reasons:

VAR__TYPE__STORAGE__DOMAIN,VAR__ACTION__DEACTIVATE,ERROR_CANNOT_DEACTIVATE_DOMAIN_WITH_TASKS
     2017-12-06 20:13:23,167-05 INFO

[org.ovirt.engine.core.bll.storage.domain.DeactivateStorageDomainWithOvfUpdateCommand]
     (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] Lock freed
     to object 'EngineLock:{exclusiveLocks='[ea2fd992-8a
     a4-44fe-aa43-e96754a975ba=STORAGE]',
     sharedLocks='[5e0a0183-0e25-4f43-b5b0-0cfb5510248e=POOL]'}'
     2017-12-06 20:13:23,172-05 DEBUG
     [org.ovirt.engine.core.common.di.interceptor.DebugLoggingInterceptor]
     (default task-7) [d82880e8-1d40-4a3b-a1ba-3362f2f130a0] method:
     runAction, params: [DeactivateStorageDomainWithOvfUpdate,
     DeactivateSto

rageDomainWithOvfUpdateParameters:{commandId='630c28e1-41ab-43db-9755-a2bb870dbcb3',
     user='null', commandType='Unknown'}], timeElapsed: 65ms 2017-12-06
     20:13:23,176-05 ERROR
     [org.ovirt.engine.api.restapi.resource.AbstractBackendResource]
     (default task-7) [] Operation Failed: [Cannot deactivate Storage
     while there are running tasks on this Storage. -Please wait until
     tasks will finish and try again.] *

     *<error>

     _______________________________________________
     Devel mailing list
     Devel(a)ovirt.org <mailto:Devel@ovirt.org>
     http://lists.ovirt.org/mailman/listinfo/devel
     <http://lists.ovirt.org/mailman/listinfo/devel>

--------------CCBA5F838197258179C525C5
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 7bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">Maor, I either need to get new glasses
      or a magnifier glass to read what you wrote :-P<br>
      when you say running tasks - these are actually running tasks that
      may be running because of other tests in ost - correct? wouldn't
      killing or blocking those can cause other tests to fail? <br>
      <br>
      On 12/07/2017 11:06 AM, Maor Lipchuk wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAJ1JNOc0dtvdbmXA3CXXx3VWkREuWck2hYmRzjfACwr9XJ8swQ@mail.gmail.com">
      <div dir="ltr"><span
id="gmail-m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
          <p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><span
style="color:rgb(0,0,0);font-family:monospace;white-space:pre-wrap">CANNOT_DEACTIVATE_DOMAIN_WITH_</span><wbr
                style="font-family:monospace"><span
style="color:rgb(0,0,0);font-family:monospace;white-space:pre-wrap">TASKS
</span><font><font
                  color="#000000" size="1"
face="Arial"><span style="white-space:pre-wrap">is a known
issue, the problem is that we might have tasks which will start running internally using
scheduling (like OVF_UPDATE) and we can't really know how much time every task will
take until it will end.</span></font></font></span></p>
          <p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><font><font
                  color="#000000" size="1"
face="Arial"><span style="white-space:pre-wrap">Even if we
check that there are no running tasks it will not </span></font><font
                  color="#000000"><span
style="white-space:pre-wrap">guarantee</span></font><font
                  color="#000000"><font size="1"
face="Arial"><span style="white-space:pre-wrap"> that no task
will start until you deactivate the storage
domain.</span></font></font></font></span></p>
          <p
style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="background-color:transparent;vertical-align:baseline"><font><font
                  color="#000000"><font size="1"
face="Arial"><span style="white-space:pre-wrap">I think that
the best solution for it is an engine support to cancel running tasks or block tasks from
running.</span></font></font></font></span></p>
        </span></div>
      <div class="gmail_extra"><br>
        <div class="gmail_quote">On Thu, Dec 7, 2017 at 12:14 PM, Dafna
          Ron <span dir="ltr">&lt;<a
href="mailto:dron@redhat.com"
              target="_blank"
moz-do-not-send=&quot;true&quot;&gt;dron(a)redhat.com&lt;/a&gt;&amp;gt;&lt;/span&gt;
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div text="#000000" bgcolor="#FFFFFF">
              <p>Hi, <br>
              </p>
              <p>We had a failure on master basic suite for test
                007_sd_reattach.deactivate_<wbr>storage_domain. <br>
              </p>
              <p>The failure was that we failed to deactivate domain due
                to running tasks. <br>
              </p>
              <p>It does not seem to be related to the patch it was
                testing and I think that the test itself needs to be
                modified to check there are no running tasks. <br>
              </p>
              <p>Is there perhaps a way to query if there are running
                tasks before running the command? can you please take a
                look at the test on OST? <br>
              </p>
              <p><b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
                </b></p>
              <p dir="ltr"

style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b
                  style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
and headline of suspected patches:</span><b
                    style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><b>
                      Not related to error</b></b></b></p>
              <b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
              </b>
              <p><b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
                </b></p>
              <p dir="ltr"

style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b
                  style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
to Job: <a class="m_-5329876534188665956moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
target="_blank"
moz-do-not-send="true">http://jenkins.ovirt.org/job/<wbr&...
              <b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8">
              </b><b style="font-weight:normal"
id="m_-5329876534188665956docs-internal-guid-5859b7a1-3073-5414-18b7-2d31991889e8"><br>
                <p dir="ltr"

style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link
to all logs: <a class="m_-5329876534188665956moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...
target="_blank"
moz-do-not-send="true">http://jenkins.ovirt.org/job/<wbr&...
                <br>
                <p dir="ltr"

style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">(Relevant)
error snippet from the log: </span></p>
                <p dir="ltr"

style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">&lt;error&gt;</span></p>
                <p dir="ltr"

style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">
</span></p>
                <p dir="ltr"

style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><b>2017-12-06
20:13:23,166-05 WARN 
[org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>DeactivateStorageDomainWithOvf<wbr>UpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-<wbr>3362f2f130a0] Validation of action
'<wbr>DeactivateStorageDomainWithOvf<wbr>Update' fa
iled for user admin@internal-authz. Reasons:
VAR__TYPE__STORAGE__DOMAIN,<wbr>VAR__ACTION__DEACTIVATE,ERROR_<wbr>CANNOT_DEACTIVATE_DOMAIN_WITH_<wbr>TASKS
2017-12-06 20:13:23,167-05 INFO 
[org.ovirt.engine.core.bll.<wbr>storage.domain.<wbr>DeactivateStorageDomainWithOvf<wbr>UpdateCommand]
(default task-7) [d82880e8-1d40-4a3b-a1ba-<wbr>3362f2f130a0] Lock freed to object
'EngineLock:{exclusiveLocks='[<wbr>ea2fd992-8a
a4-44fe-aa43-e96754a975ba=<wbr>STORAGE]',
sharedLocks='[5e0a0183-0e25-<wbr>4f43-b5b0-0cfb5510248e=POOL]'}<wbr>'
2017-12-06 20:13:23,172-05 DEBUG
[org.ovirt.engine.core.common.<wbr>di.interceptor.<wbr>DebugLoggingInterceptor]
(default task-7) [d82880e8-1d40-4a3b-a1ba-<wbr>3362f2f130a0] method: runAction,
params: [<wbr>DeactivateStorageDomainWithOvf<wbr>Update, DeactivateSto
rageDomainWithOvfUpdateParamet<wbr>ers:{commandId='630c28e1-41ab-<wbr>43db-9755-a2bb870dbcb3',
user='null', commandType='Unknown'}], timeElapsed: 65ms
2017-12-06 20:13:23,176-05 ERROR
[org.ovirt.engine.api.restapi.<wbr>resource.<wbr>AbstractBackendResource]
(default task-7) [] Operation Failed: [Cannot deactivate Storage while there are running
tasks on this Storage.
-Please wait until tasks will finish and try again.]
</b>
</span></p>
                <p dir="ltr"

style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span
style="font-size:11pt;font-family:Arial;color:#000000;background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">
</span></p>
              </b>&lt;error&gt;<br
                class="m_-5329876534188665956Apple-interchange-newline">
              <p><br>
              </p>
              <p><br>
              </p>
              <p><br>
              </p>
            </div>
            <br>
            ______________________________<wbr>_________________<br>
            Devel mailing list<br>
            <a href="mailto:Devel@ovirt.org"
moz-do-not-send=&quot;true&quot;&gt;Devel(a)ovirt.org&lt;/a&gt;&lt;br&gt;
            <a href="http://lists.ovirt.org/mailman/listinfo/devel"
              rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.ovirt.org/<wbr>mai...
          </blockquote>
        </div>
        <br>
      </div>
    </blockquote>
    <p><br>
    </p>
  </body>
</html>

--------------CCBA5F838197258179C525C5--

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Re: [ovirt-devel] [ OST Failure Report ] [ oVirtMaster ] [ 07-11-2017 ] [007_sd_reattach.deactivate_storage_domain ]