<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<br>
<br>
<div class="moz-cite-prefix">On 10/08/2015 03:46, Chandra Shehkhar
Reddy Potula wrote:<br>
</div>
<blockquote cite="mid:55C848CD.5070608@linux.vnet.ibm.com"
type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<br>
<div class="moz-cite-prefix">On 08/06/2015 11:51 PM, Aline Manera
wrote:<br>
</div>
<blockquote cite="mid:55C3A5B2.2040505@linux.vnet.ibm.com"
type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<br>
<div class="moz-cite-prefix">On 06/08/2015 04:06, Chandra
Shehkhar Reddy Potula wrote:<br>
</div>
<blockquote cite="mid:55C30767.4010205@linux.vnet.ibm.com"
type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
Hi Aline,<br>
<br>
Thanks for your inputs !!!<br>
<br>
The tests we written so far are functional verification test
cases which can be ran against the Kimchi which is installed,
up and running and can validate REST API calls with the real
data. <br>
</blockquote>
<br>
What do you mean by a "Kimchi installed, up and running"? Is it
not assuming a local instance running from local source code?<br>
</blockquote>
I mean, in order to run FVT test suite, Kimchi must be installed
on one of the supported distro. Test suite may not be on the same
target machine where it is suppose to run but it can also be
triggered from different machine against the target machine.<br>
<blockquote cite="mid:55C3A5B2.2040505@linux.vnet.ibm.com"
type="cite"> <br>
<blockquote cite="mid:55C30767.4010205@linux.vnet.ibm.com"
type="cite"> <br>
About the current test cases under folder<b> /tests</b>, If I
understood correctly they are suppose to do unit testing,
hence majority of them uses mocking. I went through test cases
like test_authorization.py, test_exception.py, test_host.py,
test_plugin.py, test_server.py etc (by the way I excluded
mockmodel and model test files) and I saw the <b>import
kimchi.mockmodel, </b>which makes me feel that these cases
are designed for mock testing. They are not meant for
functional verification even though they are REST API calls.<br>
<br>
</blockquote>
<br>
The MockModel is to allow testing functionalities in which we
may depend on system configuration.<br>
<br>
For example, how test authorization if you don't know which
users and groups exists on the system? Even though you know the
username, what about the password? How will you test login?<br>
A similar scenario applies to other tests we use MockModel.<br>
<br>
Also MockModel also connects to libvirt to do the virtualization
work. The difference is it uses the libvirt Test driver instead
of the qemu one.<br>
And as the libvirt Test driver has some limitations we needed to
overwrite some functions. It is better to understand looking at
the code in mockmodel.py.<br>
<br>
Could you give me an example of a FVT test you have in mind?
Just to confirm we don't already have some of them upstream.<br>
</blockquote>
Exactly, Mock is an object that simulates the behavior of a real
object in controlled ways. Mock objects are used in unit testing.
<br>
With MockModel it still helps you in testing certain functionality
but on simulated data/environment.<br>
<br>
On the other hand Functional Verification Test - (also can be
called as Integration Test), helps testing a slice of
functionality on a real system. This will test many methods and
may interact with dependencies involved. <br>
<br>
We maintained a config file, which has a section to provide the
session details (user, host, ssl port num etc) of the target
machine against which test suite will run. End user have to update
the config file before executing the test suite.<br>
<br>
For example FVT test cases we developed:<br>
Test case to perform Virtual Machine life-cycle operations.<br>
Test case for debug report generations.<br>
Test case for repository creation/updation/deletion.<br>
Test case for create/modify/update various template scenario's.<br>
Test case to create/activate/de-activate/delete of network
interfaces like isolation, NAT and bridge network interfaces. <br>
Test case to create/activate/deactivate/delete of storage pools of
various types (DIR, SCSI, iSCSI, Logical) and storage volumes
operations (create/update/delete) on each of these storage pools.<br>
and so on....<br>
</blockquote>
<br>
Well, we've already had those tests in our current master branch.
Please, check all test_model* files under /tests directory. All
those tests are made on real Model environment.<br>
<br>
<blockquote cite="mid:55C848CD.5070608@linux.vnet.ibm.com"
type="cite">
<blockquote cite="mid:55C3A5B2.2040505@linux.vnet.ibm.com"
type="cite"> <br>
<blockquote cite="mid:55C30767.4010205@linux.vnet.ibm.com"
type="cite"> I agree that we could place functional
verification test suite under the<b> /tests</b> folder, but
not to mix with existing unit test cases. I do recommend below
structure inside <b>/tests.</b><br>
<br>
/tests <br>
<b>|</b><b><br>
</b><b>-----</b>/tests-unit<br>
<b>|</b><b><br>
</b><b>-----</b>/tests-fvt<br>
<br>
/ui/tests<br>
<b>|</b><b><br>
</b><b>-----</b>/tests-unit<br>
<b>|</b><b><br>
</b><b>-----</b>/tests -selinium<br>
</blockquote>
<br>
I'd only suggest to remove the second "tests-" as it seems
redundant for me =)<br>
<br>
/tests/unit/<br>
/tests/fvt<br>
<br>
/ui/tests/unit<br>
/ui/tests/selenium<br>
<br>
</blockquote>
Fine with me !!!<br>
<blockquote cite="mid:55C3A5B2.2040505@linux.vnet.ibm.com"
type="cite">
<blockquote cite="mid:55C30767.4010205@linux.vnet.ibm.com"
type="cite"> <br>
Hope this make sense !!!<br>
<br>
<br>
Regards<br>
Chandra<br>
<div class="moz-cite-prefix">On 08/06/2015 12:35 AM, Aline
Manera wrote:<br>
</div>
<blockquote cite="mid:55C25E94.7040600@linux.vnet.ibm.com"
type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<br>
<div class="moz-cite-prefix">On 04/08/2015 07:22, Chandra Sr
Potula wrote:<br>
</div>
<blockquote
cite="mid:OFCCF17497.DA79826C-ON65257E97.00350B74-65257E97.0038F4E2@in.ibm.com"
type="cite">
<p><font size="2" face="DejaVu Serif">We have developed
functional verification regression test suite for
Kimchi/Ginger in order to verify the REST API's calls
currently available on 1.5. </font><br>
<br>
<font size="2" face="DejaVu Serif">Our proposal is to
create the following folder structure in the main
stream of Kimchi/Ginger for FVT. From this week we
will be submitting the patches.</font><br>
<br>
<font size="2" face="DejaVu Serif"><b>tests-fvt</b></font><br>
<font size="2" face="DejaVu Serif"> |</font><br>
<font size="2" face="DejaVu Serif"> -----</font><font
size="2" face="DejaVu Serif"><b>tests-restapi</b></font><br>
<font size="2" face="DejaVu Serif"> |</font><br>
<font size="2" face="DejaVu Serif"> -----</font><font
size="2" face="DejaVu Serif"><b>tests-ui</b></font><br>
<br>
<font size="2" face="DejaVu Serif">Please provide
feedback if you have any !!!</font><br>
</p>
</blockquote>
<br>
We already have unit tests to cover the REST APIs. You can
check them under /tests<br>
You can add as many tests as you want there. Usually we are
on a way to remove test_model and test_mockmodel and
separate the tests per components: templates, networks,
guests, etc.<br>
About the directory structure, I don't think we need to
change it.<br>
<br>
For UI tests, we need to wait a little bit to do that
according to the new UI.<br>
Samuel will do unit tests and Selenium tests as soon as the
new UI components are getting ready.<br>
About the directory structure, we can create a new directory
under /ui, for example, ui/tests to handle all the unit
tests and ui/tests/selenium to handle the Selenium tests.<br>
<br>
<blockquote
cite="mid:OFCCF17497.DA79826C-ON65257E97.00350B74-65257E97.0038F4E2@in.ibm.com"
type="cite">
<p> <br>
<font size="2" face="DejaVu Serif">Thanks in advance.</font><br>
<font size="2" face="DejaVu Serif">Chandra</font><br>
<br>
<font size="3" face="serif"> </font> <br>
</p>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Kimchi-devel mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Kimchi-devel@ovirt.org">Kimchi-devel@ovirt.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/kimchi-devel">http://lists.ovirt.org/mailman/listinfo/kimchi-devel</a>
</pre>
</blockquote>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Kimchi-devel mailing list
<a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:Kimchi-devel@ovirt.org">Kimchi-devel@ovirt.org</a>
<a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/kimchi-devel">http://lists.ovirt.org/mailman/listinfo/kimchi-devel</a>
</pre>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<br>
</blockquote>
<br>
</body>
</html>