OST Failure - Weekly update [/04/2018-20/04/2018]
Hi, I wanted to give a short status on this week's failures and OST current status. I am glad to report that the issue with CQ alerts was resolved thanks to Barak and Evgheni. You can read more about the issue and how it was resolved here: https://ovirt-jira.atlassian.net/browse/OVIRT-1974 Currently we have one on-going possible regression which was reported to the list and to Arik. the change reported: https://gerrit.ovirt.org/#/c/89852/ - examples: upload ova as a virtual machine template. you can view the details in this Jira: https://ovirt-jira.atlassian.net/browse/OFT-648 The majority of issues we had this week were failed-build artifacts for fc27. There were two different cases, one was reported to Francesco who was already working on a fix to the issue and the second started and resolved during the evening/night of Apr 26-27. You can see the Jira to these two issues here: https://ovirt-jira.atlassian.net/browse/OFT-605 https://ovirt-jira.atlassian.net/browse/OFT-612 There was an infra issue with Mirrors not being available for a few minutes. the issue was momentarily and was resolved on its own. https://ovirt-jira.atlassian.net/browse/OFT-606 *Below you can see the chart for this week's resolved issues but cause of failure:**Code* = regression of working components/functionalities *Infra* = infrastructure/OST Infrastructure/Lago related issues/Power outages *OST Tests* = package related issues, failed build artifacts *Below is a chart showing failures by suite type: * *Below is a chart showing failures by version type: * *Below you can see the number of reported failures by resolution status:* Thanks, Dafna
On Fri, Apr 27, 2018 at 7:34 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
I wanted to give a short status on this week's failures and OST current status.
I am glad to report that the issue with CQ alerts was resolved thanks to Barak and Evgheni. You can read more about the issue and how it was resolved here: https://ovirt-jira.atlassian.net/browse/OVIRT-1974
How was the VM2 high-performance with vNUMA and pinned to more than one host (a race) was solved?
Currently we have one on-going possible regression which was reported to the list and to Arik. the change reported: https://gerrit.ovirt.org/#/c/89852/ - examples: upload ova as a virtual machine template. you can view the details in this Jira: https://ovirt-jira.atlassian. net/browse/OFT-648
I don't understand how that *example* script could have caused this regression - in a complete different scenario (virt-sparsify fails because of libguestfs issue). Y.
The majority of issues we had this week were failed-build artifacts for fc27. There were two different cases, one was reported to Francesco who was already working on a fix to the issue and the second started and resolved during the evening/night of Apr 26-27. You can see the Jira to these two issues here:
https://ovirt-jira.atlassian.net/browse/OFT-605 https://ovirt-jira.atlassian.net/browse/OFT-612
There was an infra issue with Mirrors not being available for a few minutes. the issue was momentarily and was resolved on its own.
https://ovirt-jira.atlassian.net/browse/OFT-606
*Below you can see the chart for this week's resolved issues but cause of failure:**Code* = regression of working components/functionalities *Infra* = infrastructure/OST Infrastructure/Lago related issues/Power outages *OST Tests* = package related issues, failed build artifacts
*Below is a chart showing failures by suite type: *
*Below is a chart showing failures by version type: *
*Below you can see the number of reported failures by resolution status:* Thanks, Dafna
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Fri, Apr 27, 2018 at 5:57 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Fri, Apr 27, 2018 at 7:34 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
I wanted to give a short status on this week's failures and OST current status.
I am glad to report that the issue with CQ alerts was resolved thanks to Barak and Evgheni. You can read more about the issue and how it was resolved here: https://ovirt-jira.atlassian.net/browse/OVIRT-1974
How was the VM2 high-performance with vNUMA and pinned to more than one host (a race) was solved?
I was not seeing it all week, however, we had just had a failure for that today: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7169/
Currently we have one on-going possible regression which was reported to the list and to Arik. the change reported: https://gerrit.ovirt.org/#/c/89852/ - examples: upload ova as a virtual machine template. you can view the details in this Jira: https://ovirt-jira.atlassian.n et/browse/OFT-648
I don't understand how that *example* script could have caused this regression - in a complete different scenario (virt-sparsify fails because of libguestfs issue). Y.
I noticed the "example" but since it was reported as a failure I wanted to make sure nothing in the "example" caused the failure which is why I sent to the list and asked Arik to have a look.
The majority of issues we had this week were failed-build artifacts for fc27. There were two different cases, one was reported to Francesco who was already working on a fix to the issue and the second started and resolved during the evening/night of Apr 26-27. You can see the Jira to these two issues here:
https://ovirt-jira.atlassian.net/browse/OFT-605 https://ovirt-jira.atlassian.net/browse/OFT-612
There was an infra issue with Mirrors not being available for a few minutes. the issue was momentarily and was resolved on its own.
https://ovirt-jira.atlassian.net/browse/OFT-606
*Below you can see the chart for this week's resolved issues but cause of failure:**Code* = regression of working components/functionalities *Infra* = infrastructure/OST Infrastructure/Lago related issues/Power outages *OST Tests* = package related issues, failed build artifacts
*Below is a chart showing failures by suite type: *
*Below is a chart showing failures by version type: *
*Below you can see the number of reported failures by resolution status:* Thanks, Dafna
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On Mon, Apr 30, 2018 at 4:15 PM, Dafna Ron <dron@redhat.com> wrote:
On Fri, Apr 27, 2018 at 5:57 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Fri, Apr 27, 2018 at 7:34 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
I wanted to give a short status on this week's failures and OST current status.
I am glad to report that the issue with CQ alerts was resolved thanks to Barak and Evgheni. You can read more about the issue and how it was resolved here: https://ovirt-jira.atlassian.net/browse/OVIRT-1974
How was the VM2 high-performance with vNUMA and pinned to more than one host (a race) was solved?
I was not seeing it all week, however, we had just had a failure for that today: http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7169/
Currently we have one on-going possible regression which was reported to the list and to Arik. the change reported: https://gerrit.ovirt.org/#/c/89852/ - examples: upload ova as a virtual machine template. you can view the details in this Jira: https://ovirt-jira.atlassian.n et/browse/OFT-648
I don't understand how that *example* script could have caused this regression - in a complete different scenario (virt-sparsify fails because of libguestfs issue). Y.
I noticed the "example" but since it was reported as a failure I wanted to make sure nothing in the "example" caused the failure which is why I sent to the list and asked Arik to have a look.
No, that sdk-example could not have caused it.
The majority of issues we had this week were failed-build artifacts for fc27. There were two different cases, one was reported to Francesco who was already working on a fix to the issue and the second started and resolved during the evening/night of Apr 26-27. You can see the Jira to these two issues here:
https://ovirt-jira.atlassian.net/browse/OFT-605 https://ovirt-jira.atlassian.net/browse/OFT-612
There was an infra issue with Mirrors not being available for a few minutes. the issue was momentarily and was resolved on its own.
https://ovirt-jira.atlassian.net/browse/OFT-606
*Below you can see the chart for this week's resolved issues but cause of failure:**Code* = regression of working components/functionalities *Infra* = infrastructure/OST Infrastructure/Lago related issues/Power outages *OST Tests* = package related issues, failed build artifacts
*Below is a chart showing failures by suite type: *
*Below is a chart showing failures by version type: *
*Below you can see the number of reported failures by resolution status:* Thanks, Dafna
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 30 Apr 2018, at 15:29, Arik Hadas <ahadas@redhat.com> wrote: =20 =20 =20 On Mon, Apr 30, 2018 at 4:15 PM, Dafna Ron <dron@redhat.com = <mailto:dron@redhat.com>> wrote: =20 =20 On Fri, Apr 27, 2018 at 5:57 PM, Yaniv Kaul <ykaul@redhat.com = <mailto:ykaul@redhat.com>> wrote: =20 =20 On Fri, Apr 27, 2018 at 7:34 PM, Dafna Ron <dron@redhat.com = <mailto:dron@redhat.com>> wrote: Hi,=20 =20 I wanted to give a short status on this week's failures and OST = current status. =20 I am glad to report that the issue with CQ alerts was resolved thanks = to Barak and Evgheni.=20 You can read more about the issue and how it was resolved here:=20 https://ovirt-jira.atlassian.net/browse/OVIRT-1974 = <https://ovirt-jira.atlassian.net/browse/OVIRT-1974> =20 How was the VM2 high-performance with vNUMA and pinned to more than = one host (a race) was solved? =20 I was not seeing it all week, however, we had just had a failure for =
--Apple-Mail=_BAFEAA81-91D1-4E16-85D8-DC2085D92B5B Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 that today: = http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7169/ = <http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7169/>
=20 =20 =20 =20 =20 Currently we have one on-going possible regression which was reported = to the list and to Arik.=20
Dafna, how can we see that the error is consistent and triggered by this patch? = Are there other builds passing, after this failure? I see some green builds afterwards, with =E2=80=8B"(ovirt-engine)=E2=80=9D= , does it mean the error is not happening all the time? Thanks, michal
the change reported: https://gerrit.ovirt.org/#/c/89852/ = <https://gerrit.ovirt.org/#/c/89852/> - examples: upload ova as a = virtual machine template.=20 you can view the details in this Jira: = https://ovirt-jira.atlassian.net/browse/OFT-648 = <https://ovirt-jira.atlassian.net/browse/OFT-648> =20 I don't understand how that *example* script could have caused this = regression - in a complete different scenario (virt-sparsify fails = because of libguestfs issue). Y. =20 I noticed the "example" but since it was reported as a failure I = wanted to make sure nothing in the "example" caused the failure which is = why I sent to the list and asked Arik to have a look. =20 =20 No, that sdk-example could not have caused it. =20 =20 =20 =20 =20 The majority of issues we had this week were failed-build artifacts = for fc27. There were two different cases, one was reported to Francesco = who was already working on a fix to the issue and the second started and = resolved during the evening/night of Apr 26-27.=20 You can see the Jira to these two issues here:=20 =20 https://ovirt-jira.atlassian.net/browse/OFT-605 = <https://ovirt-jira.atlassian.net/browse/OFT-605> https://ovirt-jira.atlassian.net/browse/OFT-612 = <https://ovirt-jira.atlassian.net/browse/OFT-612> =20 There was an infra issue with Mirrors not being available for a few = minutes. the issue was momentarily and was resolved on its own.=20 =20 https://ovirt-jira.atlassian.net/browse/OFT-606 = <https://ovirt-jira.atlassian.net/browse/OFT-606> =20 Below you can see the chart for this week's resolved issues but cause = of failure: =20 Code =3D regression of working components/functionalities=20 Infra =3D infrastructure/OST Infrastructure/Lago related issues/Power = outages OST Tests =3D package related issues, failed build artifacts=20 =20 <image.png> <image.png> =20 Below is a chart showing failures by suite type:=20 =20 =20 <image.png> =20 <image.png> =20 =20 =20 Below is a chart showing failures by version type:=20 =20 <image.png> =20 <image.png> Below you can see the number of reported failures by resolution = status: =20 <image.png> =20 <image.png> =20 Thanks,=20 Dafna =20 =20 _______________________________________________ Devel mailing list Devel@ovirt.org <mailto:Devel@ovirt.org> http://lists.ovirt.org/mailman/listinfo/devel = <http://lists.ovirt.org/mailman/listinfo/devel> =20 =20 =20 _______________________________________________ Devel mailing list Devel@ovirt.org <mailto:Devel@ovirt.org> http://lists.ovirt.org/mailman/listinfo/devel = <http://lists.ovirt.org/mailman/listinfo/devel> =20 _______________________________________________ Devel mailing list Devel@ovirt.org <mailto:Devel@ovirt.org> http://lists.ovirt.org/mailman/listinfo/devel = <http://lists.ovirt.org/mailman/listinfo/devel>
<div class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: = Helvetica; font-size: 12px; font-style: normal; font-variant-caps: = normal; font-weight: normal; letter-spacing: normal; text-transform: = none; white-space: normal; word-spacing: 0px; text-decoration: none; = -webkit-text-stroke-width: 0px;"><br class=3D""></div></span><div = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-transform: none; = white-space: normal; word-spacing: 0px; text-decoration: none; = -webkit-text-stroke-width: 0px;">I don't understand how that *example* =
--Apple-Mail=_BAFEAA81-91D1-4E16-85D8-DC2085D92B5B Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><br = class=3D""><div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On 30 Apr 2018, at 15:29, Arik Hadas <<a = href=3D"mailto:ahadas@redhat.com" class=3D"">ahadas@redhat.com</a>> = wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = dir=3D"ltr" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: normal; = word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: = none;" class=3D""><div class=3D"gmail_extra"><br = class=3D"Apple-interchange-newline"><br class=3D""><div = class=3D"gmail_quote">On Mon, Apr 30, 2018 at 4:15 PM, Dafna Ron<span = class=3D"Apple-converted-space"> </span><span dir=3D"ltr" = class=3D""><<a href=3D"mailto:dron@redhat.com" target=3D"_blank" = class=3D"">dron@redhat.com</a>></span><span = class=3D"Apple-converted-space"> </span>wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin: 0px 0px = 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div = dir=3D"ltr" class=3D""><br class=3D""><div class=3D"gmail_extra"><br = class=3D""><div class=3D"gmail_quote"><span class=3D"">On Fri, Apr 27, = 2018 at 5:57 PM, Yaniv Kaul<span = class=3D"Apple-converted-space"> </span><span dir=3D"ltr" = class=3D""><<a href=3D"mailto:ykaul@redhat.com" target=3D"_blank" = class=3D"">ykaul@redhat.com</a>></span><span = class=3D"Apple-converted-space"> </span>wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin: 0px 0px = 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div = dir=3D"ltr" class=3D""><br class=3D""><div class=3D"gmail_extra"><br = class=3D""><div class=3D"gmail_quote"><span = class=3D"m_658206069794252933gmail-">On Fri, Apr 27, 2018 at 7:34 PM, = Dafna Ron<span class=3D"Apple-converted-space"> </span><span = dir=3D"ltr" class=3D""><<a href=3D"mailto:dron@redhat.com" = target=3D"_blank" class=3D"">dron@redhat.com</a>></span><span = class=3D"Apple-converted-space"> </span>wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin: 0px 0px = 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div = dir=3D"ltr" class=3D""><div class=3D""><div class=3D""><div = class=3D""><div class=3D""><div class=3D""><div class=3D""><div = class=3D""><div class=3D""><div class=3D"">Hi,<span = class=3D"Apple-converted-space"> </span><br class=3D""><br = class=3D""></div>I wanted to give a short status on this week's failures = and OST current status.<br class=3D""><br class=3D""></div>I am glad to = report that the issue with CQ alerts was resolved thanks to Barak and = Evgheni.<span class=3D"Apple-converted-space"> </span><br = class=3D""></div>You can read more about the issue and how it was = resolved here:<span class=3D"Apple-converted-space"> </span><br = class=3D""><a href=3D"https://ovirt-jira.atlassian.net/browse/OVIRT-1974" = target=3D"_blank" class=3D"">https://ovirt-jira.atlassian.n<wbr = class=3D"">et/browse/OVIRT-1974</a></div></div></div></div></div></div></d= iv></blockquote><div class=3D""><br class=3D""></div></span><div = class=3D"">How was the VM2 high-performance with vNUMA and pinned to = more than one host (a race) was = solved?</div></div></div></div></blockquote><div class=3D""><br = class=3D""></div></span><div class=3D"">I was not seeing it all week, = however, we had just had a failure for that today:<span = class=3D"Apple-converted-space"> </span><a = href=3D"http://jenkins.ovirt.org/job/ovirt-master_change-queue-tester/7169= /" target=3D"_blank" class=3D"">http://jenkins.ovirt.org/job/<wbr = class=3D"">ovirt-master_change-queue-<wbr class=3D"">tester/7169/</a><br = class=3D""><br class=3D""> <br class=3D""></div><span = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin: 0px 0px = 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div = dir=3D"ltr" class=3D""><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><span class=3D"m_658206069794252933gmail-"><div = class=3D""> </div><blockquote class=3D"gmail_quote" style=3D"margin: = 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div = dir=3D"ltr" class=3D""><div class=3D""><div class=3D""><div = class=3D""><div class=3D""><div class=3D""><div class=3D""><br = class=3D""><br class=3D""></div>Currently we have one<span = class=3D"Apple-converted-space"> </span><span = style=3D"background-color: rgb(255, 255, 0);" class=3D"">on-going<span = class=3D"Apple-converted-space"> </span></span>possible regression = which was reported to the list and to Arik.<span = class=3D"Apple-converted-space"> </span><br = class=3D""></div></div></div></div></div></div></blockquote></span></div><= /div></div></blockquote></span></div></div></div></blockquote></div></div>= </div></div></blockquote><div><br class=3D""></div>Dafna,</div><div>how = can we see that the error is consistent and triggered by this patch? Are = there other builds passing, after this failure?</div><div>I see some = green builds afterwards, with =E2=80=8B"(ovirt-engine)=E2=80=9D, = does it mean the error is not happening all the time?</div><div><br = class=3D""></div><div>Thanks,</div><div>michal</div><div><br = class=3D""></div><div><br class=3D""><blockquote type=3D"cite" = class=3D""><div class=3D""><div dir=3D"ltr" style=3D"text-align: start; = text-indent: 0px;" class=3D""><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin: = 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div = dir=3D"ltr" class=3D""><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><span class=3D""><blockquote class=3D"gmail_quote" = style=3D"margin: 0px 0px 0px 0.8ex; border-left-width: 1px; = border-left-style: solid; border-left-color: rgb(204, 204, 204); = padding-left: 1ex;"><div dir=3D"ltr" class=3D""><div = class=3D"gmail_extra"><div class=3D"gmail_quote"><span = class=3D"m_658206069794252933gmail-"><blockquote class=3D"gmail_quote" = style=3D"margin: 0px 0px 0px 0.8ex; border-left-width: 1px; = border-left-style: solid; border-left-color: rgb(204, 204, 204); = padding-left: 1ex;"><div dir=3D"ltr" class=3D""><div class=3D""><div = class=3D""><div class=3D""><div class=3D"">the change reported:<span = class=3D"Apple-converted-space"> </span><a = href=3D"https://gerrit.ovirt.org/#/c/89852/" target=3D"_blank" = class=3D"">https://gerrit.ovirt.org/#/c/8<wbr class=3D"">9852/</a><span = class=3D"Apple-converted-space"> </span>- examples: upload ova as a = virtual machine template.<span = class=3D"Apple-converted-space"> </span><br class=3D""></div><div = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-transform: none; = white-space: normal; word-spacing: 0px; text-decoration: none; = -webkit-text-stroke-width: 0px;">you can view the details in this = Jira:<span class=3D"Apple-converted-space"> </span><a = href=3D"https://ovirt-jira.atlassian.net/browse/OFT-648" target=3D"_blank"= class=3D"">https://ovirt-jira.atlassian.n<wbr = class=3D"">et/browse/OFT-648</a></div></div></div></div></div></blockquote= script could have caused this regression - in a complete different = scenario (virt-sparsify fails because of libguestfs issue).</div><div = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-transform: none; = white-space: normal; word-spacing: 0px; text-decoration: none; = -webkit-text-stroke-width: = 0px;">Y.</div></div></div></div></blockquote><div class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: = normal; letter-spacing: normal; text-transform: none; white-space: = normal; word-spacing: 0px; text-decoration: none; = -webkit-text-stroke-width: 0px;"><br class=3D""></div></span><div = class=3D"" style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; text-transform: none; = white-space: normal; word-spacing: 0px; text-decoration: none; = -webkit-text-stroke-width: 0px;">I noticed the "example" but since it = was reported as a failure I wanted to make sure nothing in the "example" = caused the failure which is why I sent to the list and asked Arik to = have a look. <span class=3D"Apple-converted-space"> </span><br = class=3D""></div></div></div></div></blockquote><div class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: = normal; letter-spacing: normal; text-transform: none; white-space: = normal; word-spacing: 0px; text-decoration: none; = -webkit-text-stroke-width: 0px;"><br class=3D""></div><div class=3D"" = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: = normal; letter-spacing: normal; text-transform: none; white-space: = normal; word-spacing: 0px; text-decoration: none; = -webkit-text-stroke-width: 0px;">No, that sdk-example could not have = caused it.</div><div class=3D"" style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-transform: none; white-space: normal; word-spacing: 0px; = text-decoration: none; -webkit-text-stroke-width: = 0px;"> </div><blockquote class=3D"gmail_quote" style=3D"caret-color: = rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: = normal; font-variant-caps: normal; font-weight: normal; letter-spacing: = normal; text-transform: none; white-space: normal; word-spacing: 0px; = text-decoration: none; -webkit-text-stroke-width: 0px; margin: 0px 0px = 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><div = dir=3D"ltr" class=3D""><div class=3D"gmail_extra"><div = class=3D"gmail_quote"><div class=3D""></div><div class=3D""><div = class=3D"h5"><div class=3D""><br class=3D""></div><blockquote = class=3D"gmail_quote" style=3D"margin: 0px 0px 0px 0.8ex; = border-left-width: 1px; border-left-style: solid; border-left-color: = rgb(204, 204, 204); padding-left: 1ex;"><div dir=3D"ltr" class=3D""><div = class=3D"gmail_extra"><div class=3D"gmail_quote"><div = class=3D""> </div><blockquote class=3D"gmail_quote" style=3D"margin: = 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; = border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><span = class=3D"m_658206069794252933gmail-"><div dir=3D"ltr" class=3D""><div = class=3D""><div class=3D""><div class=3D""><div class=3D""><br = class=3D""></div><br class=3D""></div>The majority of issues we had this = week were failed-build artifacts for fc27. There were two different = cases, one was reported to Francesco who was already working on a fix to = the issue and the second started and resolved during the evening/night = of Apr 26-27.<span class=3D"Apple-converted-space"> </span><br = class=3D""></div>You can see the Jira to these two issues here:<span = class=3D"Apple-converted-space"> </span><br class=3D""><br = class=3D""><a href=3D"https://ovirt-jira.atlassian.net/browse/OFT-605" = target=3D"_blank" class=3D"">https://ovirt-jira.atlassian.n<wbr = class=3D"">et/browse/OFT-605</a><br class=3D""><a = href=3D"https://ovirt-jira.atlassian.net/browse/OFT-612" target=3D"_blank"= class=3D"">https://ovirt-jira.atlassian.n<wbr = class=3D"">et/browse/OFT-612</a><br class=3D""><br class=3D""></div>There = was an infra issue with Mirrors not being available for a few minutes. = the issue was momentarily and was resolved on its own.<span = class=3D"Apple-converted-space"> </span><br class=3D""><br = class=3D""><a href=3D"https://ovirt-jira.atlassian.net/browse/OFT-606" = target=3D"_blank" class=3D"">https://ovirt-jira.atlassian.n<wbr = class=3D"">et/browse/OFT-606</a><br class=3D""><div class=3D""><br = class=3D""></div><div class=3D""><span class=3D""><b class=3D"">Below = you can see the chart for this week's resolved issues but cause of = failure:<br class=3D""><br class=3D""></b><b class=3D""><b = class=3D"">Code</b></b><span = class=3D"Apple-converted-space"> </span>=3D regression of working = components/functionalities<span = class=3D"Apple-converted-space"> </span><b class=3D""><br = class=3D""><b class=3D"">Infra</b></b><span = class=3D"Apple-converted-space"> </span>=3D infrastructure/OST = Infrastructure/Lago related issues/Power outages<b class=3D""><br = class=3D""><b class=3D"">OST</b><span = class=3D"Apple-converted-space"> </span>Tests</b><span = class=3D"Apple-converted-space"> </span>=3D package related issues, = failed build artifacts<span = class=3D"Apple-converted-space"> </span><b class=3D""><b = class=3D""><br class=3D""><br class=3D""><span = id=3D"cid:ii_jgi4lomc0_16307c545a68a8e6"><image.png></span><br = class=3D""><span = id=3D"cid:ii_jgi4lx1u1_16307c56fc17ef48"><image.png></span><br = class=3D""></b></b></span><b class=3D""><b class=3D""><b class=3D""><b = class=3D""><br class=3D""></b></b><b class=3D""><b class=3D""><font = size=3D"2" class=3D""><span style=3D"font-family: arial, helvetica, = sans-serif;" class=3D""><span style=3D"color: rgb(53, 53, 53); = font-style: normal; font-variant-ligatures: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: pre-wrap; = word-spacing: 0px; background-color: rgb(255, 255, 255); display: = inline; float: none;" class=3D""><b class=3D""><b class=3D""><b = class=3D""><font size=3D"2" class=3D""><span style=3D"font-family: = arial, helvetica, sans-serif;" class=3D""><span style=3D"color: rgb(53, = 53, 53); font-style: normal; font-variant-ligatures: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = pre-wrap; word-spacing: 0px; background-color: rgb(255, 255, 255); = display: inline; float: none;" class=3D""><b class=3D""><span = style=3D"font-family: arial, helvetica, sans-serif;" class=3D""><span = style=3D"color: rgb(53, 53, 53); font-size: small; font-style: normal; = font-variant-ligatures: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: pre-wrap; word-spacing: 0px; = background-color: rgb(255, 255, 255); display: inline; float: none;" = class=3D""><b class=3D""><b class=3D"">Below is a chart showing failures = by suite type: <br class=3D""><br class=3D""><br class=3D""><span = id=3D"cid:ii_jgi4mxzc2_16307c62a4279fde"><image.png></span><br = class=3D""><br class=3D""><span = id=3D"cid:ii_jgi4n8he3_16307c65e8983f5e"><image.png></span><br = class=3D""><br class=3D""><br = class=3D""></b></b></span></span></b></span></span></font></b></b></b></sp= an></span></font></b></b></b></b><b class=3D""><b class=3D""><b = class=3D""><b class=3D""><br class=3D""></b></b><b class=3D""><b = class=3D""><font size=3D"2" class=3D""><span style=3D"font-family: = arial, helvetica, sans-serif;" class=3D""><span style=3D"color: rgb(53, = 53, 53); font-style: normal; font-variant-ligatures: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = pre-wrap; word-spacing: 0px; background-color: rgb(255, 255, 255); = display: inline; float: none;" class=3D""><b class=3D""><b class=3D""><b = class=3D""><font size=3D"2" class=3D""><span style=3D"font-family: = arial, helvetica, sans-serif;" class=3D""><span style=3D"color: rgb(53, = 53, 53); font-style: normal; font-variant-ligatures: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = pre-wrap; word-spacing: 0px; background-color: rgb(255, 255, 255); = display: inline; float: none;" class=3D""><b class=3D""><span = style=3D"font-family: arial, helvetica, sans-serif;" class=3D""><span = style=3D"color: rgb(53, 53, 53); font-size: small; font-style: normal; = font-variant-ligatures: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: pre-wrap; word-spacing: 0px; = background-color: rgb(255, 255, 255); display: inline; float: none;" = class=3D""><b class=3D""><b class=3D"">Below is a chart showing failures = by version type: <br class=3D""><br class=3D""><span = id=3D"cid:ii_jgi4opfq4_16307c76a5a4b850"><image.png></span><br = class=3D""><br class=3D""><span = id=3D"cid:ii_jgi4oxdu5_16307c7922068fd2"><image.png></span><br = class=3D""></b></b></span></span></b></span></span></font></b></b></b></sp= an></span></font></b></b></b></b><b class=3D"">Below you can see the = number of reported failures by resolution status:<br class=3D""><br = class=3D""><span = id=3D"cid:ii_jgi4qvwg6_16307c8f9a0d4dee"><image.png></span><br = class=3D""><br class=3D""><span = id=3D"cid:ii_jgi4r2n87_16307c91a04775d4"><image.png></span><br = class=3D""></b><br class=3D""></div><div class=3D"">Thanks,<span = class=3D"Apple-converted-space"> </span><br class=3D""></div><div = class=3D"">Dafna<br class=3D""><b class=3D""><br class=3D""></b></div><div= class=3D""><b class=3D""><b class=3D""><b class=3D""><b class=3D""><font = size=3D"2" class=3D""><span style=3D"font-family: arial, helvetica, = sans-serif;" class=3D""><span style=3D"color: rgb(53, 53, 53); = font-style: normal; font-variant-ligatures: normal; font-variant-caps: = normal; font-weight: 400; letter-spacing: normal; text-align: start; = text-indent: 0px; text-transform: none; white-space: pre-wrap; = word-spacing: 0px; background-color: rgb(255, 255, 255); display: = inline; float: none;" class=3D""><b class=3D""><b class=3D""><b = class=3D""><font size=3D"2" class=3D""><span style=3D"font-family: = arial, helvetica, sans-serif;" class=3D""><span style=3D"color: rgb(53, = 53, 53); font-style: normal; font-variant-ligatures: normal; = font-variant-caps: normal; font-weight: 400; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = pre-wrap; word-spacing: 0px; background-color: rgb(255, 255, 255); = display: inline; float: none;" class=3D""><b class=3D""><span = style=3D"font-family: arial, helvetica, sans-serif;" class=3D""><span = style=3D"color: rgb(53, 53, 53); font-size: small; font-style: normal; = font-variant-ligatures: normal; font-variant-caps: normal; font-weight: = 400; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: pre-wrap; word-spacing: 0px; = background-color: rgb(255, 255, 255); display: inline; float: none;" = class=3D""></span></span></b></span></span></font></b></b></b></span></spa= n></font></b></b></b></b></div></div><br = class=3D""></span>______________________________<wbr = class=3D"">_________________<br class=3D"">Devel mailing list<br = class=3D""><a href=3D"mailto:Devel@ovirt.org" target=3D"_blank" = class=3D"">Devel@ovirt.org</a><br class=3D""><a = href=3D"http://lists.ovirt.org/mailman/listinfo/devel" rel=3D"noreferrer" = target=3D"_blank" class=3D"">http://lists.ovirt.org/mailman<wbr = class=3D"">/listinfo/devel</a><br class=3D""></blockquote></div><br = class=3D""></div></div></blockquote></div></div></div><br = class=3D""></div></div><br class=3D"">______________________________<wbr = class=3D"">_________________<br class=3D"">Devel mailing list<br = class=3D""><a href=3D"mailto:Devel@ovirt.org" = class=3D"">Devel@ovirt.org</a><br class=3D""><a = href=3D"http://lists.ovirt.org/mailman/listinfo/devel" rel=3D"noreferrer" = target=3D"_blank" class=3D"">http://lists.ovirt.org/<wbr = class=3D"">mailman/listinfo/devel</a><br class=3D""></blockquote></div><br= class=3D""></div></div><span style=3D"caret-color: rgb(0, 0, 0); = font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = text-align: start; text-indent: 0px; text-transform: none; white-space: = normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; = text-decoration: none; float: none; display: inline !important;" = class=3D"">_______________________________________________</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: = normal; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><span = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: = normal; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none; float: none; = display: inline !important;" class=3D"">Devel mailing list</span><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: = normal; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><a = href=3D"mailto:Devel@ovirt.org" style=3D"font-family: Helvetica; = font-size: 12px; font-style: normal; font-variant-caps: normal; = font-weight: normal; letter-spacing: normal; orphans: auto; text-align: = start; text-indent: 0px; text-transform: none; white-space: normal; = widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; = -webkit-text-stroke-width: 0px;" class=3D"">Devel@ovirt.org</a><br = style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: = 12px; font-style: normal; font-variant-caps: normal; font-weight: = normal; letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; = -webkit-text-stroke-width: 0px; text-decoration: none;" class=3D""><a = href=3D"http://lists.ovirt.org/mailman/listinfo/devel" = style=3D"font-family: Helvetica; font-size: 12px; font-style: normal; = font-variant-caps: normal; font-weight: normal; letter-spacing: normal; = orphans: auto; text-align: start; text-indent: 0px; text-transform: = none; white-space: normal; widows: auto; word-spacing: 0px; = -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" = class=3D"">http://lists.ovirt.org/mailman/listinfo/devel</a></div></blockq= uote></div><br class=3D""></body></html>= --Apple-Mail=_BAFEAA81-91D1-4E16-85D8-DC2085D92B5B--
On Mon, Apr 30, 2018 at 3:55 PM, Michal Skrivanek < michal.skrivanek@redhat.com> wrote:
On 30 Apr 2018, at 15:29, Arik Hadas <ahadas@redhat.com> wrote:
On Mon, Apr 30, 2018 at 4:15 PM, Dafna Ron <dron@redhat.com> wrote:
On Fri, Apr 27, 2018 at 5:57 PM, Yaniv Kaul <ykaul@redhat.com> wrote:
On Fri, Apr 27, 2018 at 7:34 PM, Dafna Ron <dron@redhat.com> wrote:
Hi,
I wanted to give a short status on this week's failures and OST current status.
I am glad to report that the issue with CQ alerts was resolved thanks to Barak and Evgheni. You can read more about the issue and how it was resolved here: https://ovirt-jira.atlassian.net/browse/OVIRT-1974
How was the VM2 high-performance with vNUMA and pinned to more than one host (a race) was solved?
I was not seeing it all week, however, we had just had a failure for that today: http://jenkins.ovirt.org/job/ovirt-master_change-queu e-tester/7169/
Currently we have one on-going possible regression which was reported to the list and to Arik.
Dafna, how can we see that the error is consistent and triggered by this patch? Are there other builds passing, after this failure? I see some green builds afterwards, with "(ovirt-engine)”, does it mean the error is not happening all the time?
Thanks, michal
Hi Michal, if I think the change may be related and I reported it to the developer, and yet the developer thinks that his change was not causing the failure, a way to debug it is to re-add this change to CQ and if it fails again on the same test the developer should take a closer look. CQ tries to isolate failures so that they would not break the tests which is why you are seeing green builds after the failure. I re-added this change today at 11:30 and indeed it passed: http://jenkins.ovirt.org/view/Change%20queue%20jobs/job/ovirt-master_change-... Thanks, Dafna
the change reported: https://gerrit.ovirt.org/#/c/89852/ - examples:
upload ova as a virtual machine template. you can view the details in this Jira: https://ovirt-jira.atlas sian.net/browse/OFT-648
I don't understand how that *example* script could have caused this regression - in a complete different scenario (virt-sparsify fails because of libguestfs issue). Y.
I noticed the "example" but since it was reported as a failure I wanted to make sure nothing in the "example" caused the failure which is why I sent to the list and asked Arik to have a look.
No, that sdk-example could not have caused it.
The majority of issues we had this week were failed-build artifacts for fc27. There were two different cases, one was reported to Francesco who was already working on a fix to the issue and the second started and resolved during the evening/night of Apr 26-27. You can see the Jira to these two issues here:
https://ovirt-jira.atlassian.net/browse/OFT-605 https://ovirt-jira.atlassian.net/browse/OFT-612
There was an infra issue with Mirrors not being available for a few minutes. the issue was momentarily and was resolved on its own.
https://ovirt-jira.atlassian.net/browse/OFT-606
*Below you can see the chart for this week's resolved issues but cause of failure:**Code* = regression of working components/functionalities *Infra* = infrastructure/OST Infrastructure/Lago related issues/Power outages *OST Tests* = package related issues, failed build artifacts
*<image.png><image.png>*
*Below is a chart showing failures by suite type: <image.png><image.png>*
*Below is a chart showing failures by version type: <image.png><image.png>*
*Below you can see the number of reported failures by resolution status:<image.png><image.png>* Thanks, Dafna
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (4)
-
Arik Hadas -
Dafna Ron -
Michal Skrivanek -
Yaniv Kaul