Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ 14-12-2017 ] [ 004_basic_sanity.disk_operations ]

Thursday, 14 December 2017

This is a multi-part message in MIME format.
--------------7C4A69F5EC47C0DE9F67DB09
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

On 12/14/2017 01:12 PM, Michal Skrivanek wrote:
...

> On 14 Dec 2017, at 13:00, Dafna Ron <dron(a)redhat.com
> <mailto:dron@redhat.com>> wrote:
>
> Hi,
>
> We have a failure on basic suite on test:
> 004_basic_sanity.disk_operations
>
> I think that we query a snapshot that was already deleted
> successfully and report the snapshot as gone. and that is because of
> a different error in update vm query which happens before.
>
> *
> Link and headline of suspected patches:
> https://gerrit.ovirt.org/#/c/85168/ - core: Prevent retry lease
> hotplag in case of failure.
>
> Link to Job:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4393
>
> Link to all logs:
> http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4393/artifact/
>
> (Relevant) error snippet from the log:
> <error>
> *
>
> vdsm: **
> **
> 2017-12-14 02:34:45,222-0500 ERROR (jsonrpc/7)
> [jsonrpc.JsonRpcServer] Internal server error (__init__:611)
> Traceback (most recent call last):
>   File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line
> 606, in _handle_request
>     res = method(**params)
>   File "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line
> 201, in _dynamicMethod
>     result = fn(*methodArgs)
>   File "<string>", line 2, in getAllVmStats
>   File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line
> 48, in method
>     ret = func(*args, **kwargs)
>   File "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1342, in
> getAllVmStats
>     statsList = self._cif.getAllVmStats()
>   File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 518,
> in getAllVmStats
>     return [v.getStats() for v in self.vmContainer.values()]
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1699,
> in getStats
>     oga_stats = self._getGuestStats()
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1895,
> in _getGuestStats
>     self._update_guest_disk_mapping()
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1909,
> in _update_guest_disk_mapping
>     self._sync_metadata()
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 4995,
> in _sync_metadata
>     self._md_desc.dump(self._dom)
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/metadata.py", line
> 477, in dump
>     dom.setMetadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT,
>   File "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py",
> line 47, in __getattr__
>     % self.vmid)
> NotConnectedError: VM '7cab7e5a-cb12-4977-ac4f-65218532df7e' was not
> defined yet or was undefined
> **

 it doesn’t seem to be relevant to this failure, but it deserves a fix
 nevertheless
 Francesco?

Smells like a race on shutdown.

-- 
Francesco Romani
Senior SW Eng., Virtualization R&D
Red Hat
IRC: fromani github: @fromanirh

--------------7C4A69F5EC47C0DE9F67DB09
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <br>
    <div class="moz-cite-prefix">On 12/14/2017 01:12 PM, Michal
      Skrivanek wrote:<br>
    </div>
    <blockquote type="cite"
      cite="mid:E8C90687-357B-46C4-902D-2FB5077F49B9@redhat.com">
      <meta http-equiv="Content-Type" content="text/html;
charset=utf-8">
      <br class="">
      <div>
        <blockquote type="cite" class="">
          <div class="">On 14 Dec 2017, at 13:00, Dafna Ron &lt;<a
              href="mailto:dron@redhat.com" class=""
              moz-do-not-send=&quot;true&quot;&gt;dron(a)redhat.com&lt;/a&gt;&amp;gt;
wrote:</div>
          <br class="Apple-interchange-newline">
          <div class="">
            <meta http-equiv="content-type" content="text/html;
              charset=utf-8" class="">
            <div text="#000000" bgcolor="#FFFFFF"
class="">
              <p class="">Hi, <br class="">
              </p>
              <p class="">We have a failure on basic suite on test:
                004_basic_sanity.disk_operations<br class="">
              </p>
              I think that we query a snapshot that was already deleted
              successfully and report the snapshot as gone. and that is
              because of a different error in update vm query which
              happens before. <br class="">
              <br class="">
              <b style="font-weight:normal;"
                id="docs-internal-guid-5859b7a1-54dd-82fe-8fc8-8fab5e981868"
                class="">
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span
style="font-size: 11pt; font-family: Arial; background-color: transparent;
font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration:
none; vertical-align: baseline; white-space: pre-wrap;" class="">Link
and headline of suspected patches: </span></div>
                <a class="moz-txt-link-freetext"
                  href="https://gerrit.ovirt.org/#/c/85168/"

moz-do-not-send="true">https://gerrit.ovirt.org/#/c/85168/&l...
                - core: Prevent retry lease hotplag in case of failure.<br
                  class="">
                <br class="">
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span
style="font-size: 11pt; font-family: Arial; background-color: transparent;
font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration:
none; vertical-align: baseline; white-space: pre-wrap;" class="">Link to
Job:</span></div>
                <a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...

moz-do-not-send="true">http://jenkins.ovirt.org/job/ovirt-ma...
                  class="">
                <br class="">
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span
style="font-size: 11pt; font-family: Arial; background-color: transparent;
font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration:
none; vertical-align: baseline; white-space: pre-wrap;" class="">Link to
all logs:</span></div>
                <tt><a class="moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/...

moz-do-not-send="true">http://jenkins.ovirt.org/job/ovirt-ma...
                  class="">
                <br class="">
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span
style="font-size: 11pt; font-family: Arial; background-color: transparent;
font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration:
none; vertical-align: baseline; white-space: pre-wrap;"
class="">(Relevant) error snippet from the log: </span></div>
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span
style="font-size: 11pt; font-family: Arial; background-color: transparent;
font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps:
normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration:
none; vertical-align: baseline; white-space: pre-wrap;"
class="">&lt;error&gt;</span></div>
                <b class=""><br class="">
                  <br class="">
                  vdsm: </b></b></div>
          </div>
        </blockquote>
        <blockquote type="cite" class="">
          <div class="">
            <div text="#000000" bgcolor="#FFFFFF"
class=""><b
                style="font-weight:normal;"
                id="docs-internal-guid-5859b7a1-54dd-82fe-8fc8-8fab5e981868"
                class=""><b class=""> <br
class="">
                  2017-12-14 02:34:45,222-0500 ERROR (jsonrpc/7)
                  [jsonrpc.JsonRpcServer] Internal server error
                  (__init__:611)<br class="">
                  Traceback (most recent call last):<br class="">
                    File
                  "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py",
                  line 606, in _handle_request<br class="">
                      res = method(**params)<br class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py",
                  line 201, in _dynamicMethod<br class="">
                      result = fn(*methodArgs)<br class="">
                    File "&lt;string&gt;", line 2, in
getAllVmStats<br
                    class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/common/api.py",
                  line 48, in method<br class="">
                      ret = func(*args, **kwargs)<br class="">
                    File "/usr/lib/python2.7/site-packages/vdsm/API.py",
                  line 1342, in getAllVmStats<br class="">
                      statsList = self._cif.getAllVmStats()<br class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/clientIF.py",
                  line 518, in getAllVmStats<br class="">
                      return [v.getStats() for v in
                  self.vmContainer.values()]<br class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py",
                  line 1699, in getStats<br class="">
                      oga_stats = self._getGuestStats()<br class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py",
                  line 1895, in _getGuestStats<br class="">
                      self._update_guest_disk_mapping()<br class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py",
                  line 1909, in _update_guest_disk_mapping<br class="">
                      self._sync_metadata()<br class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py",
                  line 4995, in _sync_metadata<br class="">
                      self._md_desc.dump(self._dom)<br class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/metadata.py",
                  line 477, in dump<br class="">

                  dom.setMetadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT,<br
                    class="">
                    File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py",
                  line 47, in __getattr__<br class="">
                      % self.vmid)<br class="">
                  NotConnectedError: VM
                  '7cab7e5a-cb12-4977-ac4f-65218532df7e' was not defined
                  yet or was undefined<br class="">
                </b></b></div>
          </div>
        </blockquote>
        <div><br class="">
        </div>
        it doesn’t seem to be relevant to this failure, but it deserves
        a fix nevertheless</div>
      <div>Francesco?</div>
      <div><br class="">
      </div>
    </blockquote>
    <br>
    Smells like a race on shutdown.<br>
    <br>
    <pre class="moz-signature" cols="72">-- 
Francesco Romani
Senior SW Eng., Virtualization R&amp;D
Red Hat
IRC: fromani github: @fromanirh</pre>
  </body>
</html>

--------------7C4A69F5EC47C0DE9F67DB09--

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Re: [ovirt-devel] [ OST Failure Report ] [ oVirt Master ] [ 14-12-2017 ] [ 004_basic_sanity.disk_operations ]