discussing the future of the upgrade suite in ost
Hi All, I was reviewing the upgrade suites in ost and there are some issues that I am seeing in the suite tests-scenarios which I want to discuss and decide the future of. At it current state, I think we should remove the upgrade suite or most of the post test-scenarios as it is not testing what it should. The tests currently only test engine upgrade and basic sanity after the upgrade. This is problematic in a few ways: 1. upgrade should test the upgrade of rhv and not just a clean engine upgrade (i.e host, storage, vm). 2. as we have limited resources I do not think that the upgrade suite should be longer then the basic suite (and as we are currently running the basic suite after the upgrade it is longer) That brings me to question what should be essential to test in upgrade in the CI? I would also need someone in dev to volunteer and take ownership of the testing scenarios for upgrade - is there anyone that can help? Thanks, Dafna
On Wed, Oct 3, 2018 at 3:51 PM Dafna Ron <dron@redhat.com> wrote:
Hi All,
I was reviewing the upgrade suites in ost and there are some issues that I am seeing in the suite tests-scenarios which I want to discuss and decide the future of.
At it current state, I think we should remove the upgrade suite or most of the post test-scenarios as it is not testing what it should.
Nothing at all, of what it should? Or not enough?
The tests currently only test engine upgrade and basic sanity after the upgrade. This is problematic in a few ways:
1. upgrade should test the upgrade of rhv and not just a clean engine upgrade (i.e host, storage, vm).
This sounds to me like missing functionality, not a reason to remove it altogether.
2. as we have limited resources I do not think that the upgrade suite should be longer then the basic suite (and as we are currently running the basic suite after the upgrade it is longer)
How often do we run it? If it's a significant burden on the CI systems, perhaps run it only once per day, or even less.
That brings me to question what should be essential to test in upgrade in the CI?
That's a very good question, but not sure it's related to the previous point. If you think we need more functionality, and I think I agree, open a bug. If current suite causes too much load, run it less frequently.
I would also need someone in dev to volunteer and take ownership of the testing scenarios for upgrade - is there anyone that can help?
I guess I can try looking after the current suite (mainly testing engine-setup's upgrade functionality). Not sure about new functionality (hosts, storage etc).
Thanks, Dafna
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/IMWTTWFIGJM3OB...
-- Didi
On Mon, Oct 8, 2018 at 1:00 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Wed, Oct 3, 2018 at 3:51 PM Dafna Ron <dron@redhat.com> wrote:
Hi All,
I was reviewing the upgrade suites in ost and there are some issues that
I am seeing in the suite tests-scenarios which I want to discuss and decide the future of.
At it current state, I think we should remove the upgrade suite or most
of the post test-scenarios as it is not testing what it should.
Nothing at all, of what it should? Or not enough?
The tests currently only test engine upgrade and basic sanity after the
upgrade.
This is problematic in a few ways:
1. upgrade should test the upgrade of rhv and not just a clean engine upgrade (i.e host, storage, vm).
This sounds to me like missing functionality, not a reason to remove it altogether.
2. as we have limited resources I do not think that the upgrade suite should be longer then the basic suite (and as we are currently running the basic suite after the upgrade it is longer)
How often do we run it? If it's a significant burden on the CI systems, perhaps run it only once per day, or even less.
We currently gate with it on CQ along with the basic suite. We we think the suite in its current status isn't that beneficial, we can move it to nightly and move it from CQ, but then we won't be gating upgrade issues from reaching to QE, is that what we want? Sandro, do you want to move the upgrade suite to nightly mode until it'll have more functionality?
That brings me to question what should be essential to test in upgrade
in the CI?
That's a very good question, but not sure it's related to the previous point.
If you think we need more functionality, and I think I agree, open a bug.
If current suite causes too much load, run it less frequently.
I would also need someone in dev to volunteer and take ownership of the testing scenarios for upgrade - is there anyone that can help?
I guess I can try looking after the current suite (mainly testing engine-setup's upgrade functionality). Not sure about new functionality (hosts, storage etc).
Thanks, Dafna
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/IMWTTWFIGJM3OB...
-- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/GTUFYRH23OKAAE...
-- Eyal edri MANAGER RHV/CNV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On Mon, Oct 8, 2018 at 1:41 PM Eyal Edri <eedri@redhat.com> wrote:
On Mon, Oct 8, 2018 at 1:00 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Wed, Oct 3, 2018 at 3:51 PM Dafna Ron <dron@redhat.com> wrote:
Hi All,
I was reviewing the upgrade suites in ost and there are some issues
that I am seeing in the suite tests-scenarios which I want to discuss and decide the future of.
At it current state, I think we should remove the upgrade suite or most
of the post test-scenarios as it is not testing what it should.
Nothing at all, of what it should? Or not enough?
The tests currently only test engine upgrade and basic sanity after the
upgrade.
This is problematic in a few ways:
1. upgrade should test the upgrade of rhv and not just a clean engine upgrade (i.e host, storage, vm).
This sounds to me like missing functionality, not a reason to remove it altogether.
2. as we have limited resources I do not think that the upgrade suite should be longer then the basic suite (and as we are currently running the basic suite after the upgrade it is longer)
How often do we run it? If it's a significant burden on the CI systems, perhaps run it only once per day, or even less.
We currently gate with it on CQ along with the basic suite. We we think the suite in its current status isn't that beneficial, we can move it to nightly and move it from CQ, but then we won't be gating upgrade issues from reaching to QE, is that what we want?
Sandro, do you want to move the upgrade suite to nightly mode until it'll have more functionality?
I plan to add more tests, which comes from infra team responsibilities (like upgrade of the host, upgrade the whole cluster, ...). I will post a design document till end of week ...
That brings me to question what should be essential to test in upgrade
in the CI?
That's a very good question, but not sure it's related to the previous point.
If you think we need more functionality, and I think I agree, open a bug.
If current suite causes too much load, run it less frequently.
I would also need someone in dev to volunteer and take ownership of the testing scenarios for upgrade - is there anyone that can help?
I guess I can try looking after the current suite (mainly testing engine-setup's upgrade functionality). Not sure about new functionality (hosts, storage etc).
Thanks, Dafna
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/IMWTTWFIGJM3OB...
-- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/GTUFYRH23OKAAE...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/KLXZDWIQOXSTFU...
-- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o.
Thanks Martin. Please ping me when you have it and we can take it from there. Thanks, Dafna On Mon, Oct 8, 2018 at 3:22 PM Martin Perina <mperina@redhat.com> wrote:
On Mon, Oct 8, 2018 at 1:41 PM Eyal Edri <eedri@redhat.com> wrote:
On Mon, Oct 8, 2018 at 1:00 PM Yedidyah Bar David <didi@redhat.com> wrote:
On Wed, Oct 3, 2018 at 3:51 PM Dafna Ron <dron@redhat.com> wrote:
Hi All,
I was reviewing the upgrade suites in ost and there are some issues
that I am seeing in the suite tests-scenarios which I want to discuss and decide the future of.
At it current state, I think we should remove the upgrade suite or
most of the post test-scenarios as it is not testing what it should.
Nothing at all, of what it should? Or not enough?
The tests currently only test engine upgrade and basic sanity after
the upgrade.
This is problematic in a few ways:
1. upgrade should test the upgrade of rhv and not just a clean engine upgrade (i.e host, storage, vm).
This sounds to me like missing functionality, not a reason to remove it altogether.
2. as we have limited resources I do not think that the upgrade suite should be longer then the basic suite (and as we are currently running the basic suite after the upgrade it is longer)
How often do we run it? If it's a significant burden on the CI systems, perhaps run it only once per day, or even less.
We currently gate with it on CQ along with the basic suite. We we think the suite in its current status isn't that beneficial, we can move it to nightly and move it from CQ, but then we won't be gating upgrade issues from reaching to QE, is that what we want?
Sandro, do you want to move the upgrade suite to nightly mode until it'll have more functionality?
I plan to add more tests, which comes from infra team responsibilities (like upgrade of the host, upgrade the whole cluster, ...). I will post a design document till end of week ...
That brings me to question what should be essential to test in upgrade
in the CI?
That's a very good question, but not sure it's related to the previous point.
If you think we need more functionality, and I think I agree, open a bug.
If current suite causes too much load, run it less frequently.
I would also need someone in dev to volunteer and take ownership of the testing scenarios for upgrade - is there anyone that can help?
I guess I can try looking after the current suite (mainly testing engine-setup's upgrade functionality). Not sure about new functionality (hosts, storage etc).
Thanks, Dafna
_______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/IMWTTWFIGJM3OB...
-- Didi _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/GTUFYRH23OKAAE...
--
Eyal edri
MANAGER
RHV/CNV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ) _______________________________________________ Devel mailing list -- devel@ovirt.org To unsubscribe send an email to devel-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/KLXZDWIQOXSTFU...
-- Martin Perina Associate Manager, Software Engineering Red Hat Czech s.r.o. _______________________________________________ Infra mailing list -- infra@ovirt.org To unsubscribe send an email to infra-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/infra@ovirt.org/message/XR76IAH2RA6JKZ...
participants (4)
-
Dafna Ron -
Eyal Edri -
Martin Perina -
Yedidyah Bar David