Re: [ovirt-devel] [ OST Failure Report ] [ oVirt master ] [ 28-09-2017 ] [ 003_00_metrics_bootstrap.configure_metrics ]

29 Sep 2017


      This is a multi-part message in MIME format.
--------------BF66E80773BC0C05F2D86FA2
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

On 09/29/2017 11:35 AM, Yaniv Kaul wrote:
...
On Fri, Sep 29, 2017 at 1:19 PM, Dafna Ron <dron@redhat.com
<mailto:dron@redhat.com>> wrote:
Hi,
We had failure in OST last night which seems to be related to two
    fluentd configuration patchs.
I found that the error is actually in the deployment and can be
    found in the lago log.
I am not sure it's related but I am also seeing unexpected errors
    in the vdsm log for getDeviceList:
    http://pastebin.test.redhat.com/520514
    <http://pastebin.test.redhat.com/520514>
We apparently calling getDeviceList + checkStatus=true with no guid.
**
*Link to suspected patches: *
*
**https://gerrit.ovirt.org/#/c/82329/
    <https://gerrit.ovirt.org/#/c/82329/> * *
**
*https://gerrit.ovirt.org/#/c/81551/
    <https://gerrit.ovirt.org/#/c/81551/>*
*
Please revert https://gerrit.ovirt.org/#/c/82329/ and see if it helps.
You could fix it, but I don't know what is the right value for the
num. of threads needed.
Y.
https://gerrit.ovirt.org/#/c/82379/
...
*
Link to Job:
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/
    <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/>*
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/
    <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/>*
Link to all logs:
*http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/artifact/
    <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/artifact/>**
**http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/artifact/
    <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/artifact/>**
*
    *
*(Relevant) error snippet from the log: *
*
<error>
TASK [ovirt_fluentd/client : Install non-ssl client]
    ***************************
    fatal: [lago-basic-suite-master-host-1]: FAILED! => {"changed":
    false, "failed": true, "msg": "AnsibleUndefinedVariable:
    'fluentd_num_threads' is undefined"}
    fatal: [lago-basic-suite-master-host-0]: FAILED! => {"changed":
    false, "failed": true, "msg": "AnsibleUndefinedVariable:
    'fluentd_num_threads' is undefined"}
    fatal: [localhost]: FAILED! => {"changed": false, "failed": true,
    "msg": "AnsibleUndefinedVariable: 'fluentd_num_threads' is undefined"}
</error>
*
_______________________________________________
    Devel mailing list
    Devel@ovirt.org <mailto:Devel@ovirt.org>
    http://lists.ovirt.org/mailman/listinfo/devel
    <http://lists.ovirt.org/mailman/listinfo/devel>
--------------BF66E80773BC0C05F2D86FA2
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <div class="moz-cite-prefix">On 09/29/2017 11:35 AM, Yaniv Kaul
      wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAJgorsaX_dmPmEcYf9hr98RQnZ4VE2LatCoHL=F9UGLPVmXsbQ@mail.gmail.com">
      <div dir="ltr"><br>
        <div class="gmail_extra"><br>
          <div class="gmail_quote">On Fri, Sep 29, 2017 at 1:19 PM,
            Dafna Ron <span dir="ltr"><<a
                href="mailto:dron@redhat.com" target="_blank"
                moz-do-not-send="true">dron@redhat.com</a>></span>
            wrote:<br>
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
              0.8ex;border-left:1px solid
              rgb(204,204,204);padding-left:1ex">
              <div bgcolor="#FFFFFF">
                <p>Hi, <br>
                </p>
                <p>We had failure in OST last night which seems to be
                  related to two fluentd configuration patchs. <br>
                </p>
                <p>I found that the error is actually in the deployment
                  and can be found in the lago log. <br>
                </p>
                <p>I am not sure it's related but I am also seeing
                  unexpected errors in the vdsm log for getDeviceList: <a
class="gmail-m_817976734258523958moz-txt-link-freetext"
                    href="http://pastebin.test.redhat.com/520514"
                    target="_blank" moz-do-not-send="true">http://pastebin.test.redhat.<wbr>com/520514</a></p>
                <p>We apparently calling getDeviceList +
                  checkStatus=true with no guid. <br>
                </p>
                <p><br>
                </p>
                <p><b style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c">
                  </b></p>
                <p dir="ltr"
                  style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b
                    style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link to suspected patches: 
</span></b></p>
                <b style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c">
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><b><a class="gmail-m_817976734258523958moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/82329/" target="_blank" moz-do-not-send="true">https://gerrit.ovirt.org/#/c/<wbr>82329/</a> </b>
</span></b></p>
                </b><b style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c">
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><b><a class="gmail-m_817976734258523958moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/81551/" target="_blank" moz-do-not-send="true">https://gerrit.ovirt.org/#/c/<wbr>81551/</a></b></span></p>
                </b></div>
            </blockquote>
            <div><br>
            </div>
            <div>Please revert <a
                href="https://gerrit.ovirt.org/#/c/82329/"
                moz-do-not-send="true">https://gerrit.ovirt.org/#/c/82329/</a>
              and see if it helps.</div>
            <div>You could fix it, but I don't know what is the right
              value for the num. of threads needed.</div>
            <div>Y.</div>
            <div> </div>
          </div>
        </div>
      </div>
    </blockquote>
    <br>
    <a class="moz-txt-link-freetext" href="https://gerrit.ovirt.org/#/c/82379/">https://gerrit.ovirt.org/#/c/82379/</a><br>
    <br>
    <blockquote type="cite"
cite="mid:CAJgorsaX_dmPmEcYf9hr98RQnZ4VE2LatCoHL=F9UGLPVmXsbQ@mail.gmail.com">
      <div dir="ltr">
        <div class="gmail_extra">
          <div class="gmail_quote">
            <blockquote class="gmail_quote" style="margin:0px 0px 0px
              0.8ex;border-left:1px solid
              rgb(204,204,204);padding-left:1ex">
              <div bgcolor="#FFFFFF"><b style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c">
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">
</span></p>
                  <br>
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link to Job:</span></p>
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><b><a class="gmail-m_817976734258523958moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/" target="_blank" moz-do-not-send="true">http://jenkins.ovirt.org/job/<wbr>ovirt-master_change-queue-<wbr>tester/2932/</a></b></span></p>
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><b><a class="gmail-m_817976734258523958moz-txt-link-freetext" href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/" target="_blank" moz-do-not-send="true">http://jenkins.ovirt.org/job/<wbr>ovirt-master_change-queue-<wbr>tester/2933/</a></b>
</span></p>
                  <br>
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">Link to all logs:</span></p>
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">
</span></p>
                  <b><a
                      class="gmail-m_817976734258523958moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2932/artifact/"
                      target="_blank" moz-do-not-send="true">http://jenkins.ovirt.org/job/<wbr>ovirt-master_change-queue-<wbr>tester/2932/artifact/</a></b></b>
                <p><b style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c"><b><a
class="gmail-m_817976734258523958moz-txt-link-freetext"
href="http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/2933/artifact/"
                        target="_blank" moz-do-not-send="true">http://jenkins.ovirt.org/job/<wbr>ovirt-master_change-queue-<wbr>tester/2933/artifact/</a></b></b></p>
                <p><b style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c"><br>
                  </b></p>
                <p dir="ltr"
                  style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><b
                    style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">(Relevant) error snippet from the log: </span></b></p>
                <b style="font-weight:normal"
id="gmail-m_817976734258523958docs-internal-guid-5859b7a1-cd16-1d43-c8eb-3860974f6d9c">
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><error></span></p>
                  <br>
                  TASK [ovirt_fluentd/client : Install non-ssl client]
                  ***************************<br>
                  fatal: [lago-basic-suite-master-host-<wbr>1]: FAILED!
                  => {"changed": false, "failed": true, "msg":
                  "AnsibleUndefinedVariable: 'fluentd_num_threads' is
                  undefined"}<br>
                  fatal: [lago-basic-suite-master-host-<wbr>0]: FAILED!
                  => {"changed": false, "failed": true, "msg":
                  "AnsibleUndefinedVariable: 'fluentd_num_threads' is
                  undefined"}<br>
                  fatal: [localhost]: FAILED! => {"changed": false,
                  "failed": true, "msg": "AnsibleUndefinedVariable:
                  'fluentd_num_threads' is undefined"}<br>
                  <br>
                  <br>
                  <p dir="ltr"
                    style="line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"></error></span></p>
                </b><br
                  class="gmail-m_817976734258523958Apple-interchange-newline">
              </div>
              <br>
              ______________________________<wbr>_________________<br>
              Devel mailing list<br>
              <a href="mailto:Devel@ovirt.org" moz-do-not-send="true">Devel@ovirt.org</a><br>
              <a href="http://lists.ovirt.org/mailman/listinfo/devel"
                rel="noreferrer" target="_blank" moz-do-not-send="true">http://lists.ovirt.org/<wbr>mailman/listinfo/devel</a><br>
            </blockquote>
          </div>
          <br>
        </div>
      </div>
    </blockquote>
    <p><br>
    </p>
  </body>
</html>

--------------BF66E80773BC0C05F2D86FA2--