- Devel - oVirt List Archives

[Engine-devel] Requirement for Locking Mechanism
by Michael Kublin 29 Nov '11

29 Nov '11

Hi All, We are introducing a new short term locking mechanism at engine. The motivation is races which are occurring between different flows, the main problem is : we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which can not be fixed in appropriate way by engine or vdsm. The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity. The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that an internal short term lock will be released. The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403 (Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround) Regards Michael

5 6

[Engine-devel] Feedback on ovirt-engine-sdk
by Geert Jansen 28 Nov '11

28 Nov '11

Hi Michael, i just wrote a small program with the new Python SDK. I found it very easy to use, so great work. I do have a few points of feedback: * Probably we want all Exceptions derived from a single base class. That way, the user can intercept any ovirtsdk with a single try/except clause. Currently that's not possible, as all your exceptions derive directly from Exception. * I think the .get() method should be id-based not name based. Using "id" as a primary key has a number of advantages over name: - It is immutable, which makes e.g. renaming a lot easier. - It allows you to retrieve an object without a search query or a filter. - It matches how the API works as well. Also currently i think it's not possible to retrieve a resource by its ID, unless you use the id= keyword argument to list() which retrieves the whole collection and then filters out the matching id. * You may want to introduce some non-entity related operations, like ping(), and connect(). These allow you to better control when you connect, and allow an program to test if the connection details work. That helps client except the right exceptions at the right time, and display more relevant error messages. * There's two Python objects involved for each API object. For a VM for example, you have params.VM and brokers.VM. This distinction is made visible to the user, for example: template = api.templates.get('Blank') vm = params.VM() vm.name = 'foo' vm.template = template # ERROR The last line need to be: vm.template = params.Template(id=template.id) In my version of the Python API i used mix-in classes to add the OO behavior to the generated classes. It was easy, because PyXB had support for that. This gave me the single class behavior. I am not sure if GenerateDS generated code allows that. I'm pretty sure it could be done though, but you may need to do some Python magic. * Nitpick: should this just be called "ovirt" instead of "ovirtsdk"? Overall great work though! Regards, Geert

3 4

[Engine-devel] timeout when starting a new VM
by Geert Jansen 27 Nov '11

27 Nov '11

Hi, i've got a setup with 1500 guests, of which about 800 are running now. Things start getting a little slow. I'm trying to start more guests via the ovirt-engine-sdk, but i get this error: File "/usr/bin/labmgr", line 171, in func_start vm.start(action) File "/usr/lib/python2.6/site-packages/ovirt_engine_sdk-1.0_SNAPSHOT-py2.6.egg/ovirtsdk/infrastructure/brokers.py", line 2389, in start body=ParseHelper.toXml(action)) File "/usr/lib/python2.6/site-packages/ovirt_engine_sdk-1.0_SNAPSHOT-py2.6.egg/ovirtsdk/infrastructure/proxy.py", line 42, in request conn=self.getConnectionsPool().getConnection()) File "/usr/lib/python2.6/site-packages/ovirt_engine_sdk-1.0_SNAPSHOT-py2.6.egg/ovirtsdk/infrastructure/proxy.py", line 54, in __doRequest raise ConnectionError, str(e) ovirtsdk.infrastructure.errors.ConnectionError: [ERROR]::oVirt API connection failure, The read operation timed out Any ideas? Regards, Geert

2 1

[Engine-devel] SDK: multi threading
by Geert Jansen 27 Nov '11

27 Nov '11

Hi Michael, two questions about threading..: - Is the SDK thread safe? - Can I make multiple concurrent requests to the API by using multiple threads, or do the threads block each other? Regards, Geert

2 1

[Engine-devel] SDK: multiple instances of API?
by Geert Jansen 27 Nov '11

27 Nov '11

Hi Michael, when i try to create multiple instances of the API object like this: from ovirtsdk.api import API url = 'xxx' username = 'admin@internal' password = 'yyy' a1 = API(url, username, password) a2 = API(url, username, password) I get this error: Traceback (most recent call last): File "test.py", line 8, in <module> a2 = API(url, username, password) File "/home/geertj/Source/ovirt-engine-sdk/src/ovirtsdk/api.py", line 47, in __init__ Mode.R) File "/home/geertj/Source/ovirt-engine-sdk/src/ovirtsdk/infrastructure/contextmanager.py", line 53, in add raise ImmutableError(key) ovirtsdk.infrastructure.errors.ImmutableError: [ERROR]::'proxy' is immutable. Any ideas? Thanks, Geert

2 1

Re: [Engine-devel] (no subject)
by Saggi Mizrahi 27 Nov '11

27 Nov '11

On Wed 16 Nov 2011 02:48:40 PM IST, Maor wrote: > Hello all, > > The Quota feature description is published under the following links: > http://www.ovirt.org/wiki/Features/DetailedQuota > http://www.ovirt.org/wiki/Features/Quota > Notice that screens of UI mockups should be updated. > > Please feel free, to share your comments about it. > > Thank you, > Maor > > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel I can't see how the host is supposed to enforce and handle it. Pause the VM? Crash it? raise ENOMEM\ENOSPC in the guest? Also what about cases of KSM\QCow2, disk\memory overcommit. Disk Templates. Storage for hibernation disk. Temporary and shared disks. Shared disks between VMs owned by different users. Backup snapshots (should they count in the quota? They are transient) I also don't see how vcpu limiting is any good? I don't even know what it means. What happens in migration between hosts with different amount of physical CPUs? I also don't think CPU limiting is even a thing to be managed by a quota. There is no reason not to use 100% of the CPU if you are the only VM active. CPU scheduling should use a priority model and not a quota IMHO.

3 6

Re: [Engine-devel] [rhevm-api] Event notifications in RHEVM 3
by Itamar Heim 27 Nov '11

27 Nov '11

On 11/25/2011 02:48 PM, David Curylo wrote: > In our use case, we are trying to get real time notifications of events into our own application tier, so there would only be one or two of our application servers connected at a time. It need not be REST or HTTP - a persistent TCP connection will work just as well for us, the importance is that we are notified as quickly as possible. We have a similar need for real time notifications throughout our system and our architecture can be summarized as follows: > > Application server(s) --> AMQP topic exchange --> AMQP queue topic binding --> Web server(s) > > I will share more of our usage scenarios and a little about our architecture as it pertains to the discussion: > > When our application tier is notified of events, it performs other actions. For example, for Red Hat certification, we track the power off and power on events so we can provide a report of OS usage - polling every minute or so would work fine here. We also need to notify server monitoring systems that a system was powered off so they can stop monitoring rather than alerting of an outage - in this case, we need to be notified ASAP. Additionally, we need to notify any connected clients that a system has changed state (any event). Those clients are connected to web servers, which we notify by a message queuing infrastructure. All events are published to an AMQP topic exchange, to which web servers (and other systems) may subscribe based on their connected client's needs. The transport and scalability requirements no longer need to be handled by our application tier. > > The web servers, in the end, have to support a lot of client scenarios that include polling, long polling, sending HTTP requests, maintaining a TCP channel for streaming events, etc. We can easily scale up web servers, and we need to provide transports that are suitable for whatever our clients may need. > > I hope this helps explain our usage scenarios and suggest an additional technology to consider. (moving the discussion to the new upstream mailing list) do you care about all events, or specific ones? the event notifier is about sending registered events per user. maybe extend it to send the amqp events directly / extend the list of events it cares about? > > -----Original Message----- > From: Yaniv Kaul [mailto:ykaul@redhat.com] > Sent: Thursday, November 24, 2011 8:56 AM > To: Geert Jansen > Cc: Itamar Heim; rhevm-api(a)lists.fedorahosted.org; David Curylo > Subject: Re: [rhevm-api] Event notifications in RHEVM 3 > > On 11/24/2011 03:14 PM, Geert Jansen wrote: >> >> On 11/24/2011 01:18 PM, Itamar Heim wrote: >>> On 11/24/2011 01:37 PM, David Curylo wrote: >>>> I don't know the capabilities of this daemon. Can it do other notifications besides email, such as execute curl to send a request to my system? I would like to get notifications as close to real time as possible. >>> I think it's every 5 minutes right now, but can be configured (in 3.0). >> I don't believe though, that out notification service suits this use >> case. The use case is to get almost real-time notifications via an API >> when a change has been made. >> >> In my view, the best way to do this in HTTP/REST, is so-called long >> polling. It's not perfect, and not terribly scalable (i takes up one >> TCP connection, and in the simplest implementation, one app server slot). >> But on the other side, i do not see a need for 100s of clients polling >> a single RHEV systems. Typically, a customer may have 1-2 ISV packages >> on top of RHEV that need events. In my view, 10s of clients should be >> easily doable with long polling. > > - A single persistent TCP connection is "cheaper" to everyone than open-request-response-close-open-request-response-close .... > - This is widely used over HTTP with "100 continue" messages that never actually complete with a 200. It's neat and takes very little bandwidth, actually. > Y. > >> >> If we want to scale higher than this, probably a different >> architecture like e.g. a pub/sub architecture using a protocol >> optimized for binary messaging would make sense. But again, for now, i >> have not seen this need so for 3.1 i think we need to focus on the >> simplest solution that will satisfy 95% of the requirements. >> >> Regards, >> Geert >> _______________________________________________ >> rhevm-api mailing list >> rhevm-api(a)lists.fedorahosted.org >> https://fedorahosted.org/mailman/listinfo/rhevm-api >

1 0

[Engine-devel] Ovirt/VDSM installation Status
by Ronen Angluster 25 Nov '11

25 Nov '11

Hello, We successfully installed Ovirt-engine & VDSM (on the same host!) from the ovirt's repository. The Steps were as following: 1. add a repository to /etc/yum.repo.d (will be replaced by a RPM package containing all the needed information) 2. execute: "yum install -y ovirt-engine; yum install -y vdsm*" 3. disable NetworkManager and activating the network service 4. create a bridge interface named "engine" 5. change configuration in /etc/libvirt/qeum.conf /etc/libvirt/libvirtd.conf & /etc/vdsm/vdsm.conf 6. configure postgresql for first time installation 7. execute /usr/share/ovirt-engine/dbscripts/create_db_devel.sh 8. change vdc_options option 'EmulatedMachine' from 'rhel6.2.0' to 'pc-0.14' (known bug in ovirt-engine, patch already submitted) 9. start vdsm & jboss services After those steps were carried, we were able to: 1. add new data center, cluster & host 2. configure storage 3. create and run a new VM a wiki page containing all the necessary information will be delivered tomorrow. Ronen

5 5

[Engine-devel] Quota open issues following oVirt-engine-core meeting
by Maor 24 Nov '11

24 Nov '11

Hi all. Following the oVirt-engine-core meeting today, and the introduction of the Quota feature, a few open issues were discussed, and are described as follow: 1. Grace limit - We relinquish the grace limit property, since the threshold property should be good enough, for the alert functionality, the administrator needs. 2. Email notifications - The administrator will use the notification service for the Quota threshold event messages, to configure which users, will get notification email on Quota that exceeded the threshold. (We should consider using postponed notifications, to avoid flood). 3, Quota feature scope - Just to clarify, Quota is not designed from hardware point of view, but more from the billing POV. It will only be managed in the ovirt-engine scope. Regards, Maor Lipchuk

2 2

[Engine-devel] The ovirg-mange-domains tool makes super user by design?
by Juan Hernandez 24 Nov '11

24 Nov '11

Hello, I see that the "ovirt-manage-domains" tool updates the "users" and "permissions" table to make the user given in the "-user" option a super user. Is this by design? Shouldn't it just add the domain and leave the "permissions" and "users" table untouched? Thanks in advance, Juan Hernandez

2 1