<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 14 Dec 2017, at 13:22, Francesco Romani &lt;<a href="mailto:fromani@redhat.com" class="">fromani@redhat.com</a>&gt; wrote:</div><br class="Apple-interchange-newline"><div class="">
  
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8" class="">
  
  <div text="#000000" bgcolor="#FFFFFF" class="">
    <br class="">
    <div class="moz-cite-prefix">On 12/14/2017 01:12 PM, Michal
      Skrivanek wrote:<br class="">
    </div>
    <blockquote type="cite" cite="mid:E8C90687-357B-46C4-902D-2FB5077F49B9@redhat.com" class="">
      <meta http-equiv="Content-Type" content="text/html; charset=utf-8" class="">
      <br class="">
      <div class="">
        <blockquote type="cite" class="">
          <div class="">On 14 Dec 2017, at 13:00, Dafna Ron &lt;<a href="mailto:dron@redhat.com" class="" moz-do-not-send="true">dron@redhat.com</a>&gt; wrote:</div>
          <br class="Apple-interchange-newline">
          <div class="">
            <meta http-equiv="content-type" content="text/html;
              charset=utf-8" class="">
            <div text="#000000" bgcolor="#FFFFFF" class=""><p class="">Hi, <br class="">
              </p><p class="">We have a failure on basic suite on test:
                004_basic_sanity.disk_operations<br class="">
              </p>
              I think that we query a snapshot that was already deleted
              successfully and report the snapshot as gone. and that is
              because of a different error in update vm query which
              happens before. <br class="">
              <br class="">
              <b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-54dd-82fe-8fc8-8fab5e981868" class="">
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span style="font-size: 11pt; font-family: Arial; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;" class="">Link and headline of suspected patches: </span></div>
                <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/85168/" moz-do-not-send="true">https://gerrit.ovirt.org/#/c/85168/</a>
                - core: Prevent retry lease hotplag in case of failure.<br class="">
                <br class="">
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span style="font-size: 11pt; font-family: Arial; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;" class="">Link to Job:</span></div>
                <a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4393" moz-do-not-send="true">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4393</a><br class="">
                <br class="">
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span style="font-size: 11pt; font-family: Arial; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;" class="">Link to all logs:</span></div>
                <tt class=""><a class="moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4393/artifact/" moz-do-not-send="true">http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/4393/artifact/</a></tt><br class="">
                <br class="">
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span style="font-size: 11pt; font-family: Arial; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;" class="">(Relevant) error snippet from the log: </span></div>
                <div style="line-height: 1.38; margin-top: 0pt;
                  margin-bottom: 0pt;" class=""><span style="font-size: 11pt; font-family: Arial; background-color: transparent; font-weight: 400; font-style: normal; font-variant-ligatures: normal; font-variant-caps: normal; font-variant-east-asian: normal; font-variant-position: normal; text-decoration: none; vertical-align: baseline; white-space: pre-wrap;" class="">&lt;error&gt;</span></div>
                <b class=""><br class="">
                  <br class="">
                  vdsm:&nbsp;</b></b></div>
          </div>
        </blockquote>
        <blockquote type="cite" class="">
          <div class="">
            <div text="#000000" bgcolor="#FFFFFF" class=""><b style="font-weight:normal;" id="docs-internal-guid-5859b7a1-54dd-82fe-8fc8-8fab5e981868" class=""><b class=""> <br class="">
                  2017-12-14 02:34:45,222-0500 ERROR (jsonrpc/7)
                  [jsonrpc.JsonRpcServer] Internal server error
                  (__init__:611)<br class="">
                  Traceback (most recent call last):<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py",
                  line 606, in _handle_request<br class="">
                  &nbsp;&nbsp;&nbsp; res = method(**params)<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py",
                  line 201, in _dynamicMethod<br class="">
                  &nbsp;&nbsp;&nbsp; result = fn(*methodArgs)<br class="">
                  &nbsp; File "&lt;string&gt;", line 2, in getAllVmStats<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/common/api.py",
                  line 48, in method<br class="">
                  &nbsp;&nbsp;&nbsp; ret = func(*args, **kwargs)<br class="">
                  &nbsp; File "/usr/lib/python2.7/site-packages/vdsm/API.py",
                  line 1342, in getAllVmStats<br class="">
                  &nbsp;&nbsp;&nbsp; statsList = self._cif.getAllVmStats()<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/clientIF.py",
                  line 518, in getAllVmStats<br class="">
                  &nbsp;&nbsp;&nbsp; return [v.getStats() for v in
                  self.vmContainer.values()]<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py",
                  line 1699, in getStats<br class="">
                  &nbsp;&nbsp;&nbsp; oga_stats = self._getGuestStats()<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py",
                  line 1895, in _getGuestStats<br class="">
                  &nbsp;&nbsp;&nbsp; self._update_guest_disk_mapping()<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py",
                  line 1909, in _update_guest_disk_mapping<br class="">
                  &nbsp;&nbsp;&nbsp; self._sync_metadata()<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py",
                  line 4995, in _sync_metadata<br class="">
                  &nbsp;&nbsp;&nbsp; self._md_desc.dump(self._dom)<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/metadata.py",
                  line 477, in dump<br class="">
                  &nbsp;&nbsp;&nbsp;
                  dom.setMetadata(libvirt.VIR_DOMAIN_METADATA_ELEMENT,<br class="">
                  &nbsp; File
                  "/usr/lib/python2.7/site-packages/vdsm/virt/virdomain.py",
                  line 47, in __getattr__<br class="">
                  &nbsp;&nbsp;&nbsp; % self.vmid)<br class="">
                  NotConnectedError: VM
                  '7cab7e5a-cb12-4977-ac4f-65218532df7e' was not defined
                  yet or was undefined<br class="">
                </b></b></div>
          </div>
        </blockquote>
        <div class=""><br class="">
        </div>
        it doesn’t seem to be relevant to this failure, but it deserves
        a fix nevertheless</div>
      <div class="">Francesco?</div>
      <div class=""><br class="">
      </div>
    </blockquote>
    <br class="">
    Smells like a race on shutdown.<br class=""></div></div></blockquote><div><br class=""></div>and suspend/resume</div><div>looks harmless, but we want to clear up as many tracebacks as we can:)</div><div><br class=""><blockquote type="cite" class=""><div class=""><div text="#000000" bgcolor="#FFFFFF" class="">
    <br class="">
    <pre class="moz-signature" cols="72">-- 
Francesco Romani
Senior SW Eng., Virtualization R&amp;D
Red Hat
IRC: fromani github: @fromanirh</pre>
  </div>

</div></blockquote></div><br class=""></body></html>