- Devel - oVirt List Archives

New VMStatus methods
by Shmuel Melamud 16 Dec '15

16 Dec '15

Hi! I've just posted the patch introducing new set of VM status checking methods. https://gerrit.ovirt.org/#/c/50590/ The methods are added to VMStatus class and delegated to VM class. The old methods are marked as deprecated and we plan to get rid of them soon. I highly recommend to use these methods everywhere and never to compare to VMStatus constants directly. There are methods to check for individual states and for meaningful sets - using them makes the code much more clear and understandable and it was the goal. If you cannot find a method that fits your needs, feel free to contact me, we can add it. Shmuel

1 0

New VM status functions - your comments?
by Shmuel Melamud 16 Dec '15

16 Dec '15

Hi! https://docs.google.com/a/redhat.com/spreadsheets/d/1yb-JdTAGh_4bOC6tL7KLz1… In the document above I've summarized the changes I plan to make to reorganize VM status checks in the Engine. The second page of the documents contains the list of the new functions and the statuses they will check. On the first page I've collected all the places I'm planning to change - class name, method name and line number. "New method" column is the new check that will be put in this place. Changing to new functions will often add/remove a state or two, that's why I'll be VERY pleased to see your comments. I've sorted the classes by topic, so it will be easy to look on those you're familiar with. Ask yourself questions: 1. If Up state is required, what about PoweringUp, PoweringDown, RebootInProgress? 2. If Up state is required, what about WaitingForLaunch? Paused? 3. What about migration? 4. What about hibernation? 5. If Down state is required, should it allow also ImageLocked and ImageIllegal? 6. If Down state is required, what about Suspended? Thank you in advance. Shmuel

2 1

QCOW2 support for base images vs GlusterFS support
by Christopher Pereira 16 Dec '15

16 Dec '15

Hi, 4:23 AM and this paradox is struggling my head... Can someone please confirm if oVirt supports QCOW2 *base* images? AFAIK, oVirt only supports QCOW2 for *snapshots* not base images. Base images must always be in RAW format and there is no way to create QCOW2 disks in Engine [1]. On the other hand, working with RAW images on a GlusterFS storage is inefficient because of [2]. in fact, sparse-files performance is unacceptable [3]. Please confirm, comment or ignore. [1] : https://bugzilla.redhat.com/show_bug.cgi?id=1282654 [2] : https://bugzilla.redhat.com/show_bug.cgi?id=1220173 [3] : For example.: A empty RAW disk image file with 2 TB virtual-size and less then 1 MB physical-size is treated by Gluster as a 2 TB physical-size file (TB's of useless zeros traversing the DataCenter's LAN during many hours).

1 0

[QE] oVirt 3.6.1 Hosted Engine test day
by Sandro Bonazzola 16 Dec '15

16 Dec '15

Hi, oVirt 3.6.1 RC4 included latest bits allowing to auto import Hosted Engine storage domain within the Engine. In order to have a wider coverage of the Hosted Engine workflow we've scheduled a Hosted Engine test day for next week on Monday, December 14th. Please join us testing Hosted Engine with oVirt 3.6.1 RC4, both on a clean install and on an upgrade from 3.5 flow. Thanks, -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

6 24

[ACTION REQUIRED] ovirt-engine build failing on master
by Sandro Bonazzola 15 Dec '15

15 Dec '15

*05:22:42* [INFO] [ERROR] Required argument 3 not present: {0} ({1} Sockets, {2} Kerne pro Socket)*05:22:42* [INFO] [ERROR] Errors in 'org/ovirt/engine/ui/uicompat/ConstantsManager.java'*05:22:42* [INFO] [ERROR] Line 38: Failed to resolve 'org.ovirt.engine.ui.uicompat.UIMessages' via deferred binding See http://jenkins.ovirt.org/job/ovirt-engine_master_build-artifacts-el7-x86_64… Please fix ASAP -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

6 11

AppErrors cleanup
by Allon Mureinik 15 Dec '15

15 Dec '15

Hi all, A recent bug [1] reported as part of the translation effort alerted me to the fact that we have a lot (and I mean a LOT - over 100 per file) of deprecated, unused keys in the various AppErrors files that serve no purpose and just take up space and waste translators time when they examine them. To make a long story short - I've just merged a patch to remove all these useless messages, and enforce via unit tests that EVERY key there should have a corresponding constant in the EngineMessage or EngineError enums. Many thanks to my reviewers! I know this was an tedious patch that couldn't have been too fun to review. -Allon [1] https://bugzilla.redhat.com/show_bug.cgi?id=1244766

5 6

Lago testing environment contribution
by Yaniv Bronheim 15 Dec '15

15 Dec '15

Hi list, If you're not aware yet about Lago you should. This our engine to test functional flows over variant distributions for ovirt-engine and vdsm This project is currently maintained by dcaro but it will love to have more contribution in reviews, tests and more. If you want to follow it I encourage to join the mailing list lago-devel(a)ovirt.org and help with reviews if possible ([2]) You can also checkout following links for more info In [3] you can find full readme about how to use it to run ovirt-engine tests. should be intuitive and quick. If not, please raise [1] https://gerrit.ovirt.org/#/admin/projects/lago [2] https://gerrit.ovirt.org/#/q/project:+lago+status:+open [3] https://github.com/ovirt/lago Hope you'll find it interesting and Lago will get more attention by our community Greetings, -- *Yaniv Bronhaim.*

4 5

oVirt Community Newsletter, November 2015
by Mikey Ariel 15 Dec '15

15 Dec '15

Following the release of oVirt 3.6, we’ve had some great stories from the community during November. Check out the newsletter for new videos and some important project milestones! ----------------- Software Releases ----------------- oVirt 3.6.1 First Release Candidate is now available for testing http://lists.ovirt.org/pipermail/announce/2015-November/000210.html oVirt 3.5.6 Second Release Candidate is now available for testing http://lists.ovirt.org/pipermail/announce/2015-November/000206.html ---------------- In the Community ---------------- oVirt Node roadmap is now available http://lists.ovirt.org/pipermail/devel/2015-November/011741.html Switching from HyperV to oVirt http://andrewwippler.com/2015/11/27/switching-from-hyper-v-to-ovirt/ oVirt 3.6 Community Release Party in Milan [Italian] http://www.ovirt-italia.it/2015/11/ovirt-36-e-le-proposte-per-ovirt-4-al.ht… oVirt 3.6 Released [Italian] http://www.miamammausalinux.org/2015/11/e-stata-rilasciata-ovirt-3-6-tutte-… Introducing Lago, a nested virtualization testing framework used by oVirt http://lago.readthedocs.org/en/latest/README.html ------------------------------------ Deep Dives and Technical Discussions ------------------------------------ First oVirt Node and oVirt Engine Integration Testing https://asciinema.org/a/29921 (video) RATATUI - TUI for simple oVirt DR http://www.screencast.com/users/chipchaderez/folders/Default/media/c22e68dc… (video) Experience the Virtualization: Virtualization Platform based on oVirt [Spanish] http://tv.rediris.es/es/jt2015/472/file/453.html (video) The oVirt Virtualization Platform [Czech] https://www.youtube.com/watch?v=IWlb79nce30 (video) -- Mikey Ariel Community Lead, oVirt www.ovirt.org "To be is to do" (Socrates) "To do is to be" (Jean-Paul Sartre) "Do be do be do" (Frank Sinatra) Mobile: +420-702-131-141 IRC: mariel / thatdocslady Twitter: @ThatDocsLady

1 0

Proposal: Hystrix for realtime command monitoring
by Roman Mohr 15 Dec '15

15 Dec '15

Hi All, I am contributing to the engine for three months now. While I dug into the code I started to wonder how to visualize what the engine is actually doing. To get better insights I added hystrix[1] to the engine. Hystrix is a circuit breaker library which was developed by Netflix and has one pretty interesting feature: Real time metrics for commands. In combination with hystrix-dashboard[2] it allows very interesting insights. You can easily get an overview of commands involved in operations, their performance and complexity. Look at [2] and the attachments in [5] and [6] for screenshots to get an Impression. I want to propose to integrate hystrix permanently because from my perspective the results were really useful and I also had some good experiences with hystrix in past projects. A first implementation can be found on gerrit[3]. # Where is it immediately useful? During development and QA. An example: I tested the hystrix integration on /api/vms and /api/hosts rest endpoints and immediately saw that the number of command exectuions grew lineary whit the number of vms and hosts. The bug reports [5] and [6] are the result. # How to monitor the engine? It is as easy as starting a hystrix-dashboard [2] with $ git clone https://github.com/Netflix/Hystrix.git $ cd Hystrix/hystrix-dashboard $ ../gradlew jettyRun and point the dashboard to https://<customer.engine.ip>/ovirt-engine/hystrix.stream. # Other possible benefits? * Live metrics at customer site for admins, consultants and support. * Historical metrics for analysis in addition to the log files. The metrics information is directly usable in graphite [7]. Therefore it would be possible to collect the json stream for a certain time period and analyze them later like in [4]. To do that someone just has to run curl --user admin@internal:engine http://localhost:8080/ovirt-engine/api/hystrix.stream > hystrix.stream for as long as necessary. The results can be analyzed later. # Possible architectural benefits? In addition to the live metrics we might also have use for the real hystrix features: * Circuit Breaker * Bulk execution of commands * De-dublication of commands (Caching) * Synchronous and asynchronous execution support * ... Our commands do already have a lot of features, so I don't think that there are some quick wins, but maybe there are interesting opportunities for infra. # Overhead? In [5] the netflix employees describe their results regarding the overhead of wrapping every command into a new instance of a hystrix command. They ran their tests on a standard 4-core Amazon EC2 server with a load of 60 request per second. When using threadpools they measured a mean overhead of less than one millisecond (so negligible). At the 90th percentile they measured an overhead of 3 ms. At the 99th percentile of about 9 ms. When configuring the hystrix commands to use semaphores instead of threadpools they are even faster. # How to integrate? A working implementation can be found on gerrit[3]. These patch sets wrap a hystrix command around every VdcAction, every VdcQuery and every VDSCommand. This just required four small modifications in the code base. # Security? In the provided patches the hystrix-metrics-servlet is accessible at /ovirt-engine/api/hystrix.stream. It is protected by basic auth but accessible for everyone who can authenticate. We should probably restrict it to admins. # Todo? 1) We do report failed actions with return values. Hystrix expects failing commands to throw an exception. So on the dashboard almost every command looks like a success. To overcome this, it would be pretty easy to throw an exception inside the command and catch it immediately after it leaves the hystrix wrapper. 2) Finetuning Do we want semaphores or a thread pool. When the thread pool, what size do we want? 3) Three unpackaged dependencies: archaius, hystrix-core, hystrix-contrib # References [1] https://github.com/Netflix/Hystrix [2] https://github.com/Netflix/Hystrix/tree/master/hystrix-dashboard [3] https://gerrit.ovirt.org/#/q/topic:hystrix [4] http://www.nurkiewicz.com/2015/02/storing-months-of-historical-metrics.html [5] https://github.com/Netflix/Hystrix/wiki/FAQ#what-is-the-processing-overhead… [5] https://bugzilla.redhat.com/show_bug.cgi?id=1268216 [6] https://bugzilla.redhat.com/show_bug.cgi?id=1268224 [7] http://graphite.wikidot.com

3 5

Debian support for vdsm and subpackages
by Yaniv Bronheim 14 Dec '15

14 Dec '15

We currently manage debian packaging files under https://gerrit.ovirt.org/#/q/project:releng-tools instead of in the project itself. imo it makes it harder, as developer won't update the debian folder while changing spec parts , which makes much more observation work for Simone.. But if we keep to manage debian that way we should remove the debian folder from cpopen, vdsm, pthreading to avoid duplication. Simone can you explain more how you manage to flow code changes and update the debian support currently? should I indeed drop the current debian code that we have in vdsm? -- *Yaniv Bronhaim.*

4 4