On Fri, 1 Mar 2019, 16:23 Greg Sheremeta, <gshereme@redhat.com> wrote:


On Fri, Mar 1, 2019 at 9:20 AM Dafna Ron <dron@redhat.com> wrote:


On Fri, Mar 1, 2019 at 2:07 PM Dan Kenigsberg <danken@redhat.com> wrote:


On Fri, 1 Mar 2019, 15:48 Dafna Ron, <dron@redhat.com> wrote:
Hi Dan,

our intention is never to skip tests and I would not merge a SkipTest patch if someone actually shows interest in resolving this.
if you notice, this was reported a few days ago, after CI determined that this is not an infra related issue and since then, but aside from Michal asking for logs and getting a link to them, no other communication from dev that suggest that this is being checked was sent - even after I added [URGENT] and asked if there is any update.
As we have done in the past, when I see that there is no cooperation from the developers to resolve a sporadic test failure, then I add a skip test until someone in dev has time to resolve the issue

I believe that this is wrong. You should not merge a Skip before having a phone call with the relevant team lead.

If we had any info on who to speak with we would probably do so - in cases like this, where there are multiple projects failures and its sporadic, its hard for us to pin someone down.
Also, please note that the CI team is here to maintain the infrastructure that allows developers to run the tests and I don't think that we should be chasing after the developers to fix regressions (either in the code or in the tests) and that this should be dev's responsibility.

I can see both sides of the argument, but developers need to do a better job of being more responsive to Dafna. She is always extremely diligent about reporting these, including follow-ups, and sometimes she doesn't get replies.

Note that I want her to do less, not more: identify a suspected team lead, and tell him/her it is their job to fix it. That's all.

Phone call? I don't answer my phone unless it's my wife (phone spam in the US is unrelenting).

So she should IRC/WhatsApp/gChat you if she thinks you are responsible for a failure.

I think the bigger issue is that everyone (ok, most people) filters devel and other lists into a label and hardly checks it. IMO that needs to stop.

True. I think that late-evening phone call can help make it happen.

Or maybe we need a dedicated ost-failures list that people don't filter.

The same people who filter out URGENT messages from Dafna would easily filter a new list. It might be easier for them. But none of them should be a maintainer or team lead.

That sounds a little better to me than a single dev contact.

Each *test* has a natural dev contact. Host devices are virt, so the contract here is Michal+Ryan until proven differently or suspected differently by Dafna.


Greg
 

I think that it may be time to have a dev contact which would be helping to debug and assign issues such as these and I will raise that in a different thread.

- not ideal but having different project fail sporadically is not ideal as well

As I see that you guys are now responding I have no reason to merge the patch - and that is what we all want.

I can say that since its effecting more then one project, and from past experience, its either related to the OST test itself or to the API/SDK.
I think that Evgheni ruled out an infra issue and suggested that the issue is actually related to the API - so not sure its strictly related to Ryan.

I'll trust you here. Though since the failing test is Ryan's, it is Ryan's job to find what breaks it, and pass the buck.

If you believe it is a bug in oVirt ask, then you should call mperina, too.


Thanks,
Dafna



On Fri, Mar 1, 2019 at 1:27 PM Dan Kenigsberg <danken@redhat.com> wrote:
I think this would be a bad idea, Dafna.

OST exists to find bug, not to skip tests.
Michal said that he be working on it. It's time to call him (or Ryan or Shmuel)  to hear his finding, not to kill his tests.

If you do skip his test, you should file an oVirt bug about it.

On Fri, 1 Mar 2019, 11:46 Dafna Ron, <dron@redhat.com> wrote:
Hi,

As this is failing on average 3 patches a day (from different projects) and it seems no one is working on debugging this issue, I will add a patch to skip this test today.

Thanks,
Dafna


On Thu, Feb 28, 2019 at 2:12 PM Dafna Ron <dron@redhat.com> wrote:
Hi,

We are still getting random failures on this test. 
Is anyone working on this?

Thanks,
Dafna


On Wed, Feb 27, 2019 at 9:10 PM Nir Soffer <nsoffer@redhat.com> wrote:
On Wed, Feb 27, 2019 at 4:31 PM Dan Kenigsberg <danken@redhat.com> wrote:

 

On Wed, Feb 27, 2019 at 4:24 PM Michal Skrivanek <michal.skrivanek@redhat.com> wrote:


On 27 Feb 2019, at 14:07, Galit Rosenthal <grosenth@redhat.com> wrote:

Hi, 

we are failing basic suite master (vdsm, ovirt-engine-sdk ...) 

It happens sporadically.

We disqualify that this is an infra issue.

Can you please have a look at the issue? 


ERROR: 
<testcase classname="002_bootstrap" name="get_host_devices" time="0.077">
    <error type="exceptions.RuntimeError" message="Could not find block_vda_1 device in host devices: "><![CDATA[Traceback (most recent call last):
  File "/usr/lib64/python2.7/unittest/case.py", line 369, in run
    testMethod()
  File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 142, in wrapped_test
    test()
  File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 60, in wrapper
    return func(get_test_prefix(), *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/ovirtlago/testlib.py", line 79, in wrapper
    prefix.virt_env.engine_vm().get_api(api_ver=4), *args, **kwargs
  File "/home/jenkins/workspace/ovirt-master_change-queue-tester/ovirt-system-tests/basic-suite-master/test-scenarios/002_bootstrap.py", line 1016, in get_host_devices
    raise RuntimeError('Could not find block_vda_1 device in host devices: {}'.format(device_list))
RuntimeError: Could not find block_vda_1 device in host devices: 
]]></error>

Thanks,
Galit

Examples, Full logs:

where can I find full logs? It would be interesting to see hosts’ logs as well as engine.log

Thanks,
michal
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/HF4MB2VMJKBO4O277K46HLPBUPYEXGHW/

_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/YY4MXJ44Q6TJQRFFD6KFSR4FV7OCYATW/
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/KUHLEKMAVQ7NPX6EZKCMY65AIZUWBR7G/
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/6HHUAPUKLVLWSAFC4C3ZW5ZOQICL77GB/
_______________________________________________
Devel mailing list -- devel@ovirt.org
To unsubscribe send an email to devel-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/ERLW4GNWISM5VJGPQ2DUVH6FRQMK3DJH/


--

GREG SHEREMETA

SENIOR SOFTWARE ENGINEER - TEAM LEAD - RHV UX

Red Hat NA

gshereme@redhat.com    IRC: gshereme