<div dir="ltr"><div><div><div><div>Hello, <br><br></div>I would like to update on this week's failures and OST current status.<br></div>This week I am sending the report a bit earlier due to the holidays. <br><br></div>We had a few failures this week but I am glad to report that the issues were fixed quickly. <br><br></div>on 23-03-2018 we started getting a random failure which seems to be an issue with the ost test hotplug_cpu<div>This fix was submitted by Milan to fix the issue: <a href="https://gerrit.ovirt.org/89589" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/89589</a><br><br></div><div>On the 27-03-2018 we had a failure on basic_sanity.disk_<wbr>operations<b style="font-weight:normal" id="gmail-m_4379645091331860573gmail-docs-internal-guid-e8ba4241-681a-ed3b-8fdc-10a6411cfc7b"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"><br></span></b></div><div><span style="font-weight:normal"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap">This seems to be related to locking race and was fixed by Benny on 3 different fixes: <br></span></span><span class="gmail-im"> <a href="https://gerrit.ovirt.org/#/c/89534/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>89534/</a><br></span><span class="gmail-im"><span class="gmail-im"> <a href="https://gerrit.ovirt.org/#/c/89537/" rel="noreferrer" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>89537/</a><br></span></span><a href="https://gerrit.ovirt.org/#/c/89577/" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>89577/</a><br><br>There was an issue with build-artifacts on the vdsm project which was an
infra failure as well as a code issue (2 different builds). this was
also resolved quickly. <br><br></div><div>Infra issues: <br><br></div><div>The ci team discovered and fixed a failure in container cleanup on the the hosts. Daniel submitted this patch: <a href="https://gerrit.ovirt.org/#/c/89402/">https://gerrit.ovirt.org/#/c/89402/</a> and it seemed to have resolved the issue. <br><br></div><div>Gal has been investigating an issue which causes failure due to lack of memory on the host. for now, Gal submitted a fix: <a href="https://gerrit.ovirt.org/#/c/89588/" target="_blank">https://gerrit.ovirt.org/#/c/<wbr>89588/</a> </div><div><br></div><div></div><div>Lastly we had a packaging issue on 4.2 which was resolved as well. <br><br><b>Below you can see the chart for this week's resolved issues but cause of failure:<br><br>Code</b> = regression of working components/functionalities <br><b>Infra</b> = infrastructure/OST Infrastructure/Lago related issues/Power outages<br><b>OST</b> Tests - package related issues, failed build artifacts <b><br></b><br><img src="cid:ii_jfcxyuzf0_1627356fa3041b91" width="208" height="167"><br><img src="cid:ii_jfcxzet01_16273575fdfe6c6c" width="534" height="364"><br><br><br><br><br><br><br><br><br><br><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b>Below is a chart of resolved failures based on ovirt version<br><br><img src="cid:ii_jfcy1aue2_1627358b7a286754" width="168" height="150"><br><br><img src="cid:ii_jfcy1ksy3_1627358ebd3b7f87" width="534" height="381"><br><br><br><br><br><br></b></span></span></font></b></b><br><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><b>Below is a chart showing failures by suite type: <br><br><img src="cid:ii_jfcy2mhh4_1627359a9e8d43c7" width="174" height="125"><br><img src="cid:ii_jfcy2tva5_1627359d0557efae" width="534" height="374"><br><br></b></b></span></span></b></span></span></font></b></b></b></span></span></font></b></b></div><div><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><b>Thanks, <br></b></b></span></span></b></span></span></font></b></b></b></span></span></font></b></b></div><div><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><b>Dafna<br><br></b></b></span></span></b></span></span></font></b></b></b></span></span></font></b></b></div><div><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><b><b><font size="2"><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><span style="font-family:arial,helvetica,sans-serif"><span style="color:rgb(53,53,53);font-size:small;font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-weight:400;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:pre-wrap;word-spacing:0px;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;display:inline;float:none"><b><b><br></b></b></span></span></b></span></span></font></b></b></b></span></span></font></b></b></div><div><b style="font-weight:normal" id="gmail-m_4379645091331860573gmail-docs-internal-guid-e8ba4241-681a-ed3b-8fdc-10a6411cfc7b"><span style="font-size:11pt;font-family:Arial;color:rgb(0,0,0);background-color:transparent;font-weight:400;font-style:normal;font-variant:normal;text-decoration:none;vertical-align:baseline;white-space:pre-wrap"></span></b></div></div>