November 2016 - Infra - oVirt List Archives

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by eyal edri [Administrator] (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] eyal edri [Administrator] commented on OVIRT-858: ------------------------------------------------- So it's not really Up testing right? It's performance testing, which in this case is running on the US itself, but as you said, more performance tests might be coming soon... I would say it might be actually good candidate for a performance suit on oVirt system tests. But I think it's best if we'll do a short meeting over it and hear your plans for future tests so we can decide on the best solution, I guess Roy should be there as well, and he knows about system tests, so it will be easier. On Nov 21, 2016 8:30 PM, "Greg Sheremeta (oVirt JIRA)" < > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by Greg Sheremeta (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] Greg Sheremeta commented on OVIRT-858: -------------------------------------- Exactly. It's a selenium loop that does the exact same flow through oVirt say 400 times, and then makes sure performance doesn't degrade over time. See http://imgur.com/a/XcGib for an example result. In that case, I compared a new rpc patch to master. But sometimes I want to compare two patches. Perhaps a better phrasing is "stability and performance" or "memory leak testing" or "long time browser open performance" I dunno > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by eyal edri [Administrator] (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] eyal edri [Administrator] commented on OVIRT-858: ------------------------------------------------- can you elaborate on "running copy of oVirt"? It means that you built engine + webadmin from a patch and installed it + run it? so you now have a running oVirt instance which then you run the tests on? (selenium style sort of thing?) > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by Greg Sheremeta (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] Greg Sheremeta commented on OVIRT-858: -------------------------------------- I check out the code, run the script manually against a patched running copy of oVirt, and then I make a graph with those results. It's not automated at all, as you can see. > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by eyal edri [Administrator] (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] eyal edri [Administrator] commented on OVIRT-858: ------------------------------------------------- It would help if you can give an example or flow of test that you will run in that repo, what will trigger it, what resources the test will need ( VM? testing framework? special hardware ?) Will it be checking source code or run on a live oVirt instance/UI? > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by Greg Sheremeta (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] Greg Sheremeta commented on OVIRT-858: -------------------------------------- > If the tests are for generating various metrics and collecting data (And have no yes/no conclusions) the those should probably be collected in a new test suit. This one. > Either way, we can help to add it, just tell us the need/use case and we'll continue from there I think neither of those cases, at this point. Although in the long run, I could see us setting up a suite that performance tests the engine and fails the patch if performance is decreased by some threshold. Probably something [~rgolan(a)redhat.com] should lead, though. For now, I really just want a directory to stick a 200 line helper script in. :) > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by eyal edri [Administrator] (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] eyal edri [Administrator] commented on OVIRT-858: ------------------------------------------------- [~gshereme(a)redhat.com] Having these tests under automation dir inside ovirt-engine repo will give you great control and verification on any code that gets into engine, and can potentially block new bad code from getting in, same way any unit tests are running today inside check-patch.sh or check-merge.sh You'll get the full advantage of standard CI (using .packages files for e.g to install any deps you need), you can checkout VDSM tests, they are running it in check-patch and check-merged. You can also limit it to run only on frontend/* files, see examples for it on ovirt-engine check-patch.sh where we check for dal/db files to run dao tests. If your tests are more "heavy" and require not only engine but other projects and you might need to run VMs / hosts as part of it, you should then consider using ovirt-system-tests (and Lago below it). Either way, we can help to add it, just tell us the need/use case and we'll continue from there > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[oVirt Jenkins] test-repo_ovirt_experimental_master - Build #3535 - FAILURE!
by jenkins＠jenkins.phx.ovirt.org 21 Nov '16

21 Nov '16

------=_Part_729_856520325.1479747861235 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Build: http://jenkins.ovirt.org/job/test-repo_ovirt_experimental_master/3535/, Build Number: 3535, Build Status: FAILURE ------=_Part_729_856520325.1479747861235--

1 1

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by Barak Korren (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] Barak Korren commented on OVIRT-858: ------------------------------------ {quote} This is meant to test only the UI. It's used in conjunction with vdsmfake (typically) so that only the UI / engine are under perf test. However, it could be used down the line to do full stack tests – it's webdriver {quote} Well, tests that get put inside a project repo are typically meant to be run per-patch (via check_patch.sh) or pre-merge (via check_merged.sh once we get merge-gating to work). If those are meant to run in one of these scenarios, then by all means, put them in the engine repo. If the tests are not meant to be used are part of the decision-making process for merging a patch to a particular project, then they go in ovirt-system-tests. {quote} I've looked into ovirt-system-tests in the past, but I'm afraid I don't have time right now to look deeply and figure out where to put this. Can you make a guess and let me know which directory I can use? Perhaps a new one? {quote} Well, infrastructure-style files (like the web driver itself) that will be shared by multipile tests can probably go in a new directory, probably under 'common'. With regard to particular tests, that depends on what is the purpose and output of the tests: If those tests should be counted towards the conclusion that "oVirt is stable" then they should probably be collected in numbered files under the `basic-suit-*/test-scenarios` directories. If the tests are for generating various metrics and collecting data (And have no yes/no conclusions) the those should probably be collected in a new test suit. > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0

[JIRA] (OVIRT-858) new gerrit repo: ovirt-ui-tests
by Greg Sheremeta (oVirt JIRA) 21 Nov '16

21 Nov '16

[ https://ovirt-jira.atlassian.net/browse/OVIRT-858?page=com.atlassian.jira.p… ] Greg Sheremeta commented on OVIRT-858: -------------------------------------- This is meant to test only the UI. It's used in conjunction with vdsmfake (typically) so that *only* the UI / engine are under perf test. However, it could be used down the line to do full stack tests -- it's webdriver :) I've looked into ovirt-system-tests in the past, but I'm afraid I don't have time right now to look deeply and figure out where to put this. Can you make a guess and let me know which directory I can use? Perhaps a new one? > new gerrit repo: ovirt-ui-tests > ------------------------------- > > Key: OVIRT-858 > URL: https://ovirt-jira.atlassian.net/browse/OVIRT-858 > Project: oVirt - virtualization made easy > Issue Type: By-EMAIL > Reporter: Greg Sheremeta > Assignee: infra > > Hi, > Please create a new gerrit repo called ovirt-ui-tests. > Description: functional tests for oVirt UI, focused around performance > testing > +2 rights for me, awels, vszocs, oourfali > Please mirror it to github. > Let me know if you need more information. > -- > Greg Sheremeta, MBA > Red Hat, Inc. > Sr. Software Engineer > gshereme(a)redhat.com -- This message was sent by Atlassian JIRA (v1000.552.6#100018)

1 0