ovirt-system-tests run specific scenario/debugging
Is it possible to run a specific scenario without having to run back through spin up/tear down? I want to rapidly debug a `test-scenarios/00#_something.py` and the bootstrap scripts (001,002) take a really long time. Also is it possible to attach to a debugger within the test-scenario with pdb? I didnt have luck and it looks like its abstracted away and not executed as a regular python file in a way that i can get to an interactive debugger
AFAIK It is not possible ATM, but we were considering adding that feature when we move from python-nose to PyTest [1]. Hmm, maybe it will be possible using the export / import feature, but will probably need some custom code for it. So you'll run basic suite, get your env up and running and then export it to a tar.gz file. next time you want to test something, you import it back and run only the test you want. We've been working on a project called 'ovirt demo tool' [2] which will officially replace ovirt-live using Lago images, it is still under development, but should work for basic suite if you want to try it. The manual job [3] should also allow saving the run as an image as an option [3], by setting the 'create images' option to 'yes'. [1] https://ovirt-jira.atlassian.net/browse/OST-8 [2] https://docs.google.com/document/d/1MZdpHAjNWlpPFhOXFfMavZbg6Kh386PRBg6jG6xs... [3] http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_... On Thu, Sep 14, 2017 at 8:59 PM, Marc Young <3vilpenguin@gmail.com> wrote:
Is it possible to run a specific scenario without having to run back through spin up/tear down?
I want to rapidly debug a `test-scenarios/00#_something.py` and the bootstrap scripts (001,002) take a really long time.
Also is it possible to attach to a debugger within the test-scenario with pdb? I didnt have luck and it looks like its abstracted away and not executed as a regular python file in a way that i can get to an interactive debugger
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Eyal edri ASSOCIATE MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
On 14 September 2017 at 20:59, Marc Young <3vilpenguin@gmail.com> wrote:
Is it possible to run a specific scenario without having to run back through spin up/tear down?
You can snapshot the environment after Lago finishes the bootstrap stages with `lago snapshot` and then roll back to the snapshot when you need to. You just need to intialize the environment manually with 'lago init', 'lago ovirt reposetup ...' 'lago deploy' and then run the initial tests one by one with 'lago ovirt runtest' instead of letting `runsuit.sh` do it all for you. Alternatively you could possibly remove some of the symlinks from the 'test-scenarious' directory to leave just the ones you need and then use `runsuit.sh` to get the the desired state before snapshotting.
I want to rapidly debug a `test-scenarios/00#_something.py` and the bootstrap scripts (001,002) take a really long time.
Also is it possible to attach to a debugger within the test-scenario with pdb? I didnt have luck and it looks like its abstracted away and not executed as a regular python file in a way that i can get to an interactive debugger
'lago ovirt runtest' just runs nose, but its probably redirecting I/O in a way that would not let you interact with a debugger. But you can try... 'runsuit.sh' adds its own layers of redirection, so working through it will probably be more challenging. -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
Hi, Rerunning a specific python test file is possible, though it takes few manual steps. Take a look at [1]. With regard to the debugger, it is possible to run the tests without 'lago ovirt runtest' commands at all, directly with Lago as a Python library, though it isn't fully used in OST. Basically you would have to export the same environment variables as described in [1], and then use the same imports and decorators as found in OST(mainly the testlib.with_ovirt_prefix decorator), with that in hand you can call the python file however you'd like. Of course this is all good for debugging, but less for OST(as you need the suite logic: log collections, order of tests, etc). [1] http://lists.ovirt.org/pipermail/lago-devel/20170402/000650.html On Thu, Sep 14, 2017 at 8:59 PM, Marc Young <3vilpenguin@gmail.com> wrote:
Is it possible to run a specific scenario without having to run back through spin up/tear down?
I want to rapidly debug a `test-scenarios/00#_something.py` and the bootstrap scripts (001,002) take a really long time.
Also is it possible to attach to a debugger within the test-scenario with pdb? I didnt have luck and it looks like its abstracted away and not executed as a regular python file in a way that i can get to an interactive debugger
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Im attempting to run via the lago commands re Barak's suggestion, but I'm running into ssh timeouts (as it seems like maybe a failing deployment) myoung at dev-vm in ~/repos/github/ovirt-system-tests/vagrant on master* $ lago deploy @ Deploy environment: # [Thread-2] Deploy VM lago-vagrant-host-0: # [Thread-1] Deploy VM lago-vagrant-engine: * [Thread-2] Wait for ssh connectivity: * [Thread-1] Wait for ssh connectivity: * [Thread-2] Wait for ssh connectivity: ERROR (in 0:28:21) # [Thread-2] Deploy VM lago-vagrant-host-0: ERROR (in 0:28:21) With no changes, using `run_suite.sh`, however, it works fine: ...snip.... /home/myoung/repos/github/ovirt-system-tests @ Deploy oVirt environment: # Deploy environment: * [Thread-2] Deploy VM lago-vagrant-engine: * [Thread-3] Deploy VM lago-vagrant-host-0: * [Thread-3] Deploy VM lago-vagrant-host-0: Success (in 0:00:44) * [Thread-2] Deploy VM lago-vagrant-engine: Success (in 0:13:16) # Deploy environment: Success (in 0:13:16) @ Deploy oVirt environment: Success (in 0:13:16) /home/myoung/repos/github/ovirt-system-tests Running test scenario 000_check_repo_closure.py @ Run test: 000_check_repo_closure.py: ....snip.... and even continues to a full suite success: @ Collect artifacts: Success (in 0:00:04) /home/myoung/repos/github/ovirt-system-tests run_suite.sh::main::SUCCESS:: /home/myoung/repos/github/ovirt-system-tests/vagrant - All tests passed :) Is there a lago command I'm missing/some sort of configuration change needed that I'm missing to run the lago commands manually? I've been trying to piece together the manual commands from run_suite.sh but I dont see it doing anything that I'm outright missing. On Sat, Sep 16, 2017 at 2:27 PM, Nadav Goldin <ngoldin@virtual-gate.net> wrote:
Hi, Rerunning a specific python test file is possible, though it takes few manual steps. Take a look at [1].
With regard to the debugger, it is possible to run the tests without 'lago ovirt runtest' commands at all, directly with Lago as a Python library, though it isn't fully used in OST. Basically you would have to export the same environment variables as described in [1], and then use the same imports and decorators as found in OST(mainly the testlib.with_ovirt_prefix decorator), with that in hand you can call the python file however you'd like.
Of course this is all good for debugging, but less for OST(as you need the suite logic: log collections, order of tests, etc).
[1] http://lists.ovirt.org/pipermail/lago-devel/20170402/000650.html
Is it possible to run a specific scenario without having to run back
On Thu, Sep 14, 2017 at 8:59 PM, Marc Young <3vilpenguin@gmail.com> wrote: through
spin up/tear down?
I want to rapidly debug a `test-scenarios/00#_something.py` and the bootstrap scripts (001,002) take a really long time.
Also is it possible to attach to a debugger within the test-scenario with pdb? I didnt have luck and it looks like its abstracted away and not executed as a regular python file in a way that i can get to an interactive debugger
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
On 19 September 2017 at 06:35, Marc Young <3vilpenguin@gmail.com> wrote:
Is there a lago command I'm missing/some sort of configuration change needed that I'm missing to run the lago commands manually? I've been trying to piece together the manual commands from run_suite.sh but I dont see it doing anything that I'm outright missing.
Did you run 'lago init', and 'lago start' before running 'lago deploy'? You'd probably see these SSH errors if you just run 'lago deploy' after 'lago init' because the VMs are not up yet. You'll also need to run 'lago ovirt reposetup ...' to make OST deploy work, and the parameters you need to pass it may not be trivial. Practically, just removing the test scenarios you don't need would probably be much easier. As Eyal mentioned before, we're working on 'demotool' which will allow you to download a full working oVirt Lago environment. -- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
On Tue, Sep 19, 2017 at 9:05 AM, Barak Korren <bkorren@redhat.com> wrote:
On 19 September 2017 at 06:35, Marc Young <3vilpenguin@gmail.com> wrote:
Is there a lago command I'm missing/some sort of configuration change
needed
that I'm missing to run the lago commands manually? I've been trying to piece together the manual commands from run_suite.sh but I dont see it doing anything that I'm outright missing.
Did you run 'lago init', and 'lago start' before running 'lago deploy'?
You'd probably see these SSH errors if you just run 'lago deploy' after 'lago init' because the VMs are not up yet.
You'll also need to run 'lago ovirt reposetup ...' to make OST deploy work, and the parameters you need to pass it may not be trivial.
Practically, just removing the test scenarios you don't need would probably be much easier.
As Eyal mentioned before, we're working on 'demotool' which will allow you to download a full working oVirt Lago environment.
Adding Gal, I think the 'demo tool' feature might be working for basic suite already as I mentioned, but its not official/standardize yet, we're working on it currently and it will have stanard build / deploy flow which will allow to consume it in a much easier fashion. You can try it and see if it helps your need, try the following: 1. Run the manual job [1] and tick the 'create images' option on job parameters ( also provide link to custom yum repo / jenkins job with RPMs you want to test if you have any ) 2. Once the job is done, download the tar.gz file 3. Follow the intructions on [2], though there might be issues since its not official yet, but it basically comes down to: - tar -xzvf $your_image.tar.gz - lago init ( inside the lago workdir when you opened the files ) - lago ovirt start --with-vms - lago status or lago ovitrt status should show if the VMs are up Now you have a running oVirt env, which run all the tests in the basic suite, so you can try and use it or run api calls against it. Maybe use lago shell to connect to the one of the VMs if you need, etc... [1] http://jenkins.ovirt.org/view/oVirt%20system%20tests/job/ovirt-system-tests_... ( this only helps if the suite passes w/o errors ) [2] https://docs.google.com/document/d/1MZdpHAjNWlpPFhOXFfMavZbg6Kh386PRBg6jG6xs...
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
-- Eyal edri ASSOCIATE MANAGER RHV DevOps EMEA VIRTUALIZATION R&D Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)
Did you run 'lago init', and 'lago start' before running 'lago deploy'?
I missed `lago start` (its hard to find decisive commands across a single source of documentation, but that will very likely solve my issue). I was running lago init lago ovirt reposetup --reposync-yum-config reposync-config.repo lago deploy I think the 'demo tool' feature might be working for basic suite already as
I mentioned, but its not official/standardize yet, we're working on it currently and it will have stanard build / deploy flow which will allow to consume it in a much easier fashion.
Admittedly most of my issues have been largely due to deciphering commands/etc from misc docs and jenkins configs. Most things have worked very easily after figuring out the little details, but Im happy to try out the demo tool. On Tue, Sep 19, 2017 at 1:41 AM, Eyal Edri <eedri@redhat.com> wrote:
On Tue, Sep 19, 2017 at 9:05 AM, Barak Korren <bkorren@redhat.com> wrote:
On 19 September 2017 at 06:35, Marc Young <3vilpenguin@gmail.com> wrote:
Is there a lago command I'm missing/some sort of configuration change
needed
that I'm missing to run the lago commands manually? I've been trying to piece together the manual commands from run_suite.sh but I dont see it doing anything that I'm outright missing.
Did you run 'lago init', and 'lago start' before running 'lago deploy'?
You'd probably see these SSH errors if you just run 'lago deploy' after 'lago init' because the VMs are not up yet.
You'll also need to run 'lago ovirt reposetup ...' to make OST deploy work, and the parameters you need to pass it may not be trivial.
Practically, just removing the test scenarios you don't need would probably be much easier.
As Eyal mentioned before, we're working on 'demotool' which will allow you to download a full working oVirt Lago environment.
Adding Gal, I think the 'demo tool' feature might be working for basic suite already as I mentioned, but its not official/standardize yet, we're working on it currently and it will have stanard build / deploy flow which will allow to consume it in a much easier fashion.
You can try it and see if it helps your need, try the following:
1. Run the manual job [1] and tick the 'create images' option on job parameters ( also provide link to custom yum repo / jenkins job with RPMs you want to test if you have any ) 2. Once the job is done, download the tar.gz file 3. Follow the intructions on [2], though there might be issues since its not official yet, but it basically comes down to:
- tar -xzvf $your_image.tar.gz - lago init ( inside the lago workdir when you opened the files ) - lago ovirt start --with-vms - lago status or lago ovitrt status should show if the VMs are up
Now you have a running oVirt env, which run all the tests in the basic suite, so you can try and use it or run api calls against it. Maybe use lago shell to connect to the one of the VMs if you need, etc...
[1] http://jenkins.ovirt.org/view/oVirt%20system%20tests/ job/ovirt-system-tests_manual/ ( this only helps if the suite passes w/o errors ) [2] https://docs.google.com/document/d/1MZdpHAjNWlpPFhOXFfMavZbg6Kh38 6PRBg6jG6xslao/edit
-- Barak Korren RHV DevOps team , RHCE, RHCi Red Hat EMEA redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
--
Eyal edri
ASSOCIATE MANAGER
RHV DevOps
EMEA VIRTUALIZATION R&D
Red Hat EMEA <https://www.redhat.com/> <https://red.ht/sig> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted> phone: +972-9-7692018 <+972%209-769-2018> irc: eedri (on #tlv #rhev-dev #rhev-integ)
participants (4)
-
Barak Korren -
Eyal Edri -
Marc Young -
Nadav Goldin