May 2016 - Devel - oVirt List Archives

Integration tests future (and very nice alternative for the DAO fixture file)
by Roman Mohr 04 Jul '16

04 Jul '16

Hi all, In [1] you can find some patches which are meant to improve the test writing experience in ovirt-engine. They provide the following things: A) Domain Object builders which can be used for creating and/or persisting domain objects [2] B) DAO testing without writing fixtures because of the builders C) Integration testing for commands in conjunction with a real database Arquillian, injectable commands and the builders [3] # How to run what? A) In normal unit tests just create a new instance of a builder and use it. This should help us to get rid of all the small createDefaultVm(), createHostWithX() helper methods in our tests. B) In dao tests just inject them and go ahead. The advantage of not using the fixture file is that we can now set up clean scenarios for every test in a setup method. See example 2 below on how easy it is to set up a new cluster. C) Arquillian integration tests need to be marked with "@Category(IntegrationTest.class)" and can inherit from TransactionalTestBase. The @Category annotation makes sure that the integration tests are only run when mvn clean verify -DskipITs=false is invoked. Note that these tests are then executed in the integration test phase of maven. For them we use the maven-failsafe-plugin[5] which will also make sure that the testing database is up to date. See [4] for more details. # Examples 1) Add a running VM to a host, persist everything to the database and load all VMs which are running on the host: VDS host = vdsBuilder.cluster(persistedCluster).persist(); vmBuilder.host(host).up().persist(); List<VM> vms = vmDao.getAllRunningForVds(host.getId()); 2) Add 10 hosts with 1 GB of RAM to a cluster, persist the hosts to the database in a DAO test: public class MyHostDaoTest extends BaseDaoTestCase { @Inject private VdsBuilder vdsBuilder; @Test public void createHosts() { VdsBuilder builder = vdsBuilder.cluster(persistedCluster).physicalMemory(1000); for (int x =0; x < 10; x++){ builder.id(Guid.newGuid()).persist(); } } } 3) Full integration test with arquillian and the database @Category(IntegrationTest.class) public class VmDaoIntegrationTest extends TransactionalTestBase { @Inject VmDao vmDao; private final Guid VM1_GUID = Guid.createGuidFromString("0fe4bc81-5999-4ab6-80f8-7a4a2d4bfacd"); @Deployment public static JavaArchive deploy(){ return createDeployment(); } @Test public void shouldFailOnExistingEntity() { vmBuilder.id(VM1_GUID).cluster(clusterBuilder.reset().persist()).persist(); // This uses assertThat from assertj: assertThat(vmDao.get(VM1_GUID)).isNotNull(); } } 4) Using the builders in a normal unit test without a database: VM vm = new VmBuilder().id(Guid.newGuid()).up().build(); # How to add your own Domain objects? There are just a few simple rules: 1) Your builder should extend org.ovirt.engine.core.builder.AbstractBuilder 2) Make sure that you only access DAOs injected into the builder during #prePersist() and #persist(). This allows to use the #build() method also without injections 3) #prePersist() should set all fields which are necessary to suffice database constraints. The fields should only be set if they are not already set before by the builder. When following this rule we can always persist new objects to the database by simply calling myBuilder.reset().persist(). 4) Mark your builder with @Repository to make them useable for our Spring DAO tests and our Arquillian integration tests. So have a look at the patches at [1] and let me know what you think about them. Best Regards and happy testing, Roman [1] https://gerrit.ovirt.org/#/q/topic:integration [2] https://gerrit.ovirt.org/#/c/47008/17 [3] https://gerrit.ovirt.org/#/c/47007/10 [4] https://gerrit.ovirt.org/#/c/47008/17 [5] https://maven.apache.org/surefire/maven-failsafe-plugin/

5 7

gerrit.ovirt.org projects
by Shlomo Ben David 08 Jun '16

08 Jun '16

Hi, I have a question about projects on gerrit.ovirt.org. I'm working on cleaning up the gerrit.ovirt.org and updating the replication process to github, I've noticed that the projects below weren't updated > ~6 months. Do you know which of the below projects is still relevant for replication (sync to github)? ovirt-register test chrooter gluster-nagios-monitoring ovirt-container-node ovirt-container-engine ovirt-engine-sdk-js vdsm-arch-dependencies ovirt-node-dbus-backend samples-portals jasperreports-server-rpm ovirt-node-tests ovirt-engine-sdk-tests Node ovirt-tools-common-python mediawiki-example Thanks in advanced, Shlomi Ben-David | Software Engineer | Red Hat ISRAEL Phone: +972-54-8008858 IRC: sbendavi OPEN SOURCE - 1 4 011 && 011 4 1

3 2

[ACTION REQUESTED] i18n default English text is now stored in properties files
by Scott Dickerson 01 Jun '16

01 Jun '16

Hi, In order to resolve bug [1] and prep [2], the default English text for I18N Constants and Messages have been moved to their corresponding properties files. Going forward, if a new constant or message needs to be added, please add the key method to the proper interface and then add the key plus English text to the interface's corresponding properties file. Checks will be added, in the next few days, to fail the build process if the GWT annotations (@DefaultStringValue, @DefaultMessage) are used. This change has been made to simplify the translation process [1] and to prepare for the replacement of the GWT i18n engine in 4.1 [2]. For example, in the webadmin project, the ApplicationConstants [3] interface previously looked like: package org.ovirt.engine.ui.webadmin; import org.ovirt.engine.ui.common.CommonApplicationConstants; public interface ApplicationConstants extends CommonApplicationConstants { @DefaultStringValue("oVirt Engine Web Administration") String applicationTitle(); @DefaultStringValue("About") String aboutPopupCaption(); @DefaultStringValue("oVirt Engine Version:") String ovirtVersionAbout(); Now, the interface looks like this [4]: package org.ovirt.engine.ui.webadmin; import org.ovirt.engine.ui.common.CommonApplicationConstants; public interface ApplicationConstants extends CommonApplicationConstants { String applicationTitle(); String aboutPopupCaption(); String ovirtVersionAbout(); With a new properties file created, ApplicationContants.properties [5], with this kind of content : # # # Bundle name: org.ovirt.engine.ui.webadmin.ApplicationConstants # # #Thu May 19 01:24:21 EDT 2016 aboutPopupCaption=About applicationTitle=oVirt Engine Web Administration ovirtVersionAbout=oVirt Engine Version\: Let me know if there are any questions. Regards, Scott -- Scott Dickerson Senior Software Engineer RHEV-M Engineering - UX Team Red Hat, Inc [1] - https://bugzilla.redhat.com/show_bug.cgi?id=1224423 [2] - https://bugzilla.redhat.com/show_bug.cgi?id=1287408 [3] - https://github.com/oVirt/ovirt-engine/blob/569695916d2325d0c1e1fdabb6f85cd9… [4] - https://github.com/oVirt/ovirt-engine/blob/master/frontend/webadmin/modules… [5] - https://github.com/oVirt/ovirt-engine/blob/master/frontend/webadmin/modules…

4 6

Interacting with VDSM storage without oVirt Engine
by Hayley Swimelar 01 Jun '16

01 Jun '16

Hello all, Like the title says, I need to work with VDSM's storage layer without involving the engine. Presently, I'm testing this with NFS domains and have been able to use vdsClient to create, attach, and activate a new storage domain, but I have not been able to create a new image or volume. Here are the commands I ran to get to this step, starting with a nfs volume already mounted on the machine I ran the commands on. ---- vdsClient -s host_name createStorageDomain 1 97338d5c-9b6f-1859-b827-e977ed082e53 cli_domain storage_server:/data vdsClient -s host_name attachStorageDomain 97338d5c-9b6f-1859-b827-e977ed082e53 00000001-0001-0001-0001-00000000033c vdsClient -s host_name activateStorageDomain 97338d5c-9b6f-1859-b827-e977ed082e53 00000001-0001-0001-0001-00000000033c vdsClient -s frodo createVolume 97338d5c-9b6f-1859-b827-e977ed082e53 00000001-0001-0001-0001-00000000033c 0288f410-71f1-4b7d-bdb7-e815a93e34ef 53690000000 5 1 2 96d7726c-6969-40a5-84bd-0925496b6051 cli_volume ---- The last command outputs a new UUID, but there is nothing under /rhev/data-center/00000001-0001-0001-0001-00000000033c/97338d5c-9b6f-1859-b827-e977ed082e53/images/ directory. If I pass createVolume an existing image/storage domain UUID, the command still outputs a new UUID, but nothing changes under the images directory. Is there a step that I am missing somewhere in this process? -- Hayley Swimelar LINBIT | Keeping the Digital World Running DRBD — Corosync — Pacemaker +1-503-573-1262 x212

6 12

Empty Dropdown Menus in Webadmin
by Phillip Bailey 01 Jun '16

01 Jun '16

Hi, Is anyone else experiencing empty dropdown menus in webadmin? I started having the problem after rebasing Friday afternoon. I rebased again this morning, but am still experiencing the same problem. I'm trying to determine whether this is something isolated to my environment, or a more widespread issue. I appreciate any advice any of you may be able to offer. Thanks! -Phillip Bailey

3 3

[VDSM] stuck tests in ci
by Nir Soffer 31 May '16

31 May '16

Hi all, I found another stuck build today: http://jenkins.ovirt.org/job/vdsm_master_check-patch-fc23-x86_64/1151/conso… 11:27:18 ------------------------------------------------------------------------------------------------------------------------------------------------ 11:27:18 TOTAL 40513 21121 48% 11:27:18 ---------------------------------------------------------------------- 11:27:18 Ran 2169 tests in 145.934s 11:27:18 11:27:18 OK (SKIP=88) 11:27:18 Exception AttributeError: "'NoneType' object has no attribute 'write'" in <bound method IOProcess.__del__ of <ioprocess.IOProcess object at 0x7fd7c9f2d3d0>> ignored [...] 11:27:18 Exception AttributeError: "'NoneType' object has no attribute 'write'" in <bound method IOProcess.__del__ of <ioprocess.IOProcess object at 0x7fd7c9f15550>> ignored 11:27:18 Exception in thread ioprocess communication (6533) (most likely raised during interpreter shutdown): 11:27:18 Traceback (most recent call last): 11:27:18 File "/usr/lib64/python2.7/threading.py", line 804, in __bootstrap_inner 11:27:18 File "/usr/lib64/python2.7/threading.py", line 757, in run 11:27:18 File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 180, in _communicate 11:27:18 <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'close' This seems smells like a non-daemon thread started by some code, blocking hte test process. I suspect ioprocess, starting such thread, looking into it. Meanwhile, please: - verify that all threads in actual code and in the tests are daemon threads - convert your threads to use vdsm.concurrent.thread instead of threading.Thread (daemon by default) - watch your builds and abort stuck builds David, we need a timeout in the ci, aborting the job after a project based timeout, maybe defined in the project yaml. Cheers, Nir

3 8

[vdsm] Master build fails on f23
by Piotr Kliczewski 31 May '16

31 May '16

vdsm build fails with: 17:18:54 Compiling './vdsm/clientIF.py'... 17:18:54 Makefile:984: recipe for target 'python3' failed 17:18:54 make: *** [python3] Error 1 17:18:54 Took 33 seconds Thanks, Piotr

1 0

oVirt Reports Portal localization
by Juliette Tux 31 May '16

31 May '16

Hello gentlemen, Russian localizators here :) Which files should we look into to be able to translate the Reports Portal interface/messages for oVirt ver 3.x ? Did not find anything relevant on Zanata in case you would like to redirect me over there :) -- С уважением, Дронова Юлия

2 2

Failing MacPoolDaoTests on master
by Roman Mohr 31 May '16

31 May '16

Hi, DAO tests are failing on master in org.ovirt.engine.core.dao.MacPoolDaoTest. Build logs are here: https://travis-ci.org/oVirt/ovirt-engine/builds/134082394 Best Regards, Roman

5 14

dao tests status in CI
by Eyal Edri 31 May '16

31 May '16

FYI, All ovirt-engine dao tests (3.6,4.0,master) were moved to new jenkins and the ones in the old one were disabled. The jobs are still running in "old" mode and not yamelized. Works is being done now to convert them to run in standard CI mode, under mock which should add much more stability to flexibility to them for any developer who would want to change/extend them. -- Eyal Edri Associate Manager RHEV DevOps EMEA ENG Virtualization R&D Red Hat Israel phone: +972-9-7692018 irc: eedri (on #tlv #rhev-dev #rhev-integ)

1 0