[Engine-devel] Proposed next-generation vdsm API
by Adam Litke
Recently we've had some very productive discussions concerning the VDSM API. I
want to attempt to refocus the discussion around an emerging proposal and see if
we can agree on a sensible path forward.
Based on the discussion, I have identified the following requirements that
a new API for vdsm should have:
1.) Single API that can be consumed by ovirt-engine and ISVs
- We don't want to maintain multiple parallel APIs
- To develop a vendor ecosystem, we must have a robust external API to
vdsm
2.) Full vdsm capabilities are exposed without requiring ovirt-engine
- ovirt components should be modular and independently useful
- Some deployments might want to manage nodes without ovirt-engine
3.) Standardized protocol with low overhead
- Required for widespread adoption
4.) Support for asynchronous tasks and events
- Needed by ovirt-engine and other consumers
Based on these requirements, the following proposal has started to emerge:
Create a REST API that will provide all of the functionality that is currently
available via the xmlrpc interface (with the goal of deprecating xmlrpc once it
becomes mature enough). To support advanced clustering features that
ovirt-engine is planning, we'll write an QMF broker that can proxy the REST API
onto a message bus. ovirt-engine will interact with vdsm exclusively over this
bus but the REST API will be the principle API and the entry point for ISV apps.
A REST API provides a light-weight and standard way to access all of the vdsm
functionality.
The REST API will handle events by exposing a new 'events' collection at the api
root. REST users will use some sort of polling to collect these events. The
details of this interface are being worked on. Several ways for minimizing the
impact of polling have been discussed. The QMF broker can expose a
publish/subscribe model for events as appropriate.
Is this model an acceptable way to improve the vdsm API? I would like to hear
the opinions of ovirt-engine developers, vdsm developers, and other
stakeholders. Thanks for providing feedback on this proposal!
--
Adam Litke <agl(a)us.ibm.com>
IBM Linux Technology Center
12 years, 4 months
Re: [Engine-devel] Announcing a proof of concept REST API for VDSM
by Itamar Heim
On 11/29/2011 11:36 PM, Adam Litke wrote:
> On Tue, Nov 29, 2011 at 12:54:44PM -0800, Chris Wright wrote:
>> * Adam Litke (agl(a)us.ibm.com) wrote:
>>> On Tue, Nov 29, 2011 at 11:39:08AM -0800, Chris Wright wrote:
>>>> * Adam Litke (agl(a)us.ibm.com) wrote:
>>>>>
>>>>> https://github.com/aglitke/vdsm-rest/
>>>>>
>>>>> Today I am releasing a proof of concept implementation of a REST API for vdsm.
>>>>> You can find the code on github. My goal is to eventually replace the current
>>>>> xmlrpc interface with a REST API. Once completed, ovirt-engine could switch to
>>>>> this new API. The major advantages to making this change are: 1) VDSM will gain
>>>>> a structured API that conceptually, structurally, and functionally aligns with
>>>>> the ovirt-engine REST API, 2) this new API can be made public, thus providing an
>>>>> entry point for direct virtualization management@the node level.
>>>>
>>>> Adam, this looks like a nice PoC. I didn't see how API versioning is
>>>> handled. Any VDSM developers willing to review this work?
>>>
>>> Thanks for taking a look. I am not handling versioning yet. I think we can add
>>> a version field to the root API object. As for compatibility, we'll just have
>>> to decide on an API backwards-compat support policy. Would this be enough to
>>> handle versioning issues? We shouldn't need anything like capabilities because
>>> the API is discoverable.
>>
>> Right, that seems sensible.
>>
>> Did you find cases where RPC to REST resource mapping was difficult?
>
> I haven't yet fully implemented the current vdsm API but I suspect that certain
> calls (like the ones you mention below) will require some extensions to what I
> have available currently. The main missing piece is probably events and a nice
> polling API. Another big piece of work will be to rebase onto the newly
> redesigned storage model.
>
>> I could see something like migrate() plus migrateStatus() and
>> migrateCancel() needing to add some kind of operational state that to the
>> resource. And something like monitorCommand() which has both a possible
>> side-effect and some freefrom response...
>
> Hopefully monitorCommand will not be too bad, since vdsm should be asking
> libvirt for the VM details when they are needed. Of course we'll need to be
> testing to make sure we aren't keeping state around. Also, I would expect
> monitorCommand to 'taint' the VM in the eyes of the vdsm API (as it does for
> libvirt).
>
>>>> <snip>
>>>>> ovirt-engine wants to subscribe to asynchronous events. REST APIs do not
>>>>> typically support async events and instead rely on polling of resources. I am
>>>>> investigating what options are available for supporting async events via REST.
>>>>
>>>> I think typical is either polling or long polling. If it's a single
>>>> resource, then perhaps long polling would be fine (won't be a large
>>>> number of connections tied up if it's only a single resource).
>>>
>>> Not sure if this is what you are referring to, but I was thinking we could do a
>>> batch-polling mechanism where an API user passes in a list of task UUIDs and/or
>>> event URIs. The server would respond with the status of these resources in one
>>> response. I have some ideas on how we could wire up asynchronous events on the
>>> server side to reduce the amount of actual work that such a batch-polling
>>> operation would require.
>>
>> Oh, I just meant this:
>>
>> Polling (GET /event + 404 loop)
>> Long polling (GET + block ... can chew up a thread connection)
>
> Yep. And we can talk later about building an API for efficient, repeated
> polling. I wonder if the ovirt-engine guys could weigh in as to whether a REST
cc-ing engine-devel...
> interface with event polling would be acceptable to them. It is critical that
> we settle on an API that can become _the_ first-class vehicle for interacting
> with vdsm.
>
> Thanks!
>
i have two points for consideration around this:
1. as the api between ovirt-engine and vdsm, I had a preference for the
bus like nature of QMF, as it would allow multiple ovirt-engine to load
balance handling of messages from the queue, and multiple consumers for
some messages (say, history service picking up the stats in parallel to
engine picking them, rather than copying them from engine).
2. as node level api, i think a lightweight ovirt-engine managing a
single node and exposing the exact same REST API and behavior of the
multi-node ovirt engine would be easier to cosnume for someone that
wants to interact with a single node same way they would interact with
ovirt-engine.
12 years, 5 months
[Engine-devel] Requirement for Locking Mechanism
by Michael Kublin
Hi All,
We are introducing a new short term locking mechanism at engine.
The motivation is races which are occurring between different flows, the main problem is :
we did not update status of some entity and other flow was started , which is a main cause for different collisions and situation which
can not be fixed in appropriate way by engine or vdsm.
The current status is a workaround which is trying to reduce a race : in many places in the code, there is additional query to DB in order to check the appropriate status of entity.
The proposed solution is internal locking mechanism - which will lock an appropriate entity until its status will not be updated in DB, after that
an internal short term lock will be released.
The design for locking mechanism can be found here: http://www.ovirt.org/wiki/Features/DetailedLockMechanism
The patch which is introduce an implementation for mechanism can be found here: http://gerrit.ovirt.org/#change,403
(Using of new mechanism all around a code and integration with bll will be sent in the future, including cleaning of workaround)
Regards Michael
12 years, 5 months
[Engine-devel] Feedback on ovirt-engine-sdk
by Geert Jansen
Hi Michael,
i just wrote a small program with the new Python SDK. I found it very
easy to use, so great work. I do have a few points of feedback:
* Probably we want all Exceptions derived from a single base class.
That way, the user can intercept any ovirtsdk with a single try/except
clause. Currently that's not possible, as all your exceptions derive
directly from Exception.
* I think the .get() method should be id-based not name based. Using
"id" as a primary key has a number of advantages over name:
- It is immutable, which makes e.g. renaming a lot easier.
- It allows you to retrieve an object without a search query or a filter.
- It matches how the API works as well.
Also currently i think it's not possible to retrieve a resource by
its ID, unless you use the id= keyword argument to list() which
retrieves the whole collection and then filters out the matching id.
* You may want to introduce some non-entity related operations, like
ping(), and connect(). These allow you to better control when you
connect, and allow an program to test if the connection details work.
That helps client except the right exceptions at the right time, and
display more relevant error messages.
* There's two Python objects involved for each API object. For a VM
for example, you have params.VM and brokers.VM. This distinction is made
visible to the user, for example:
template = api.templates.get('Blank')
vm = params.VM()
vm.name = 'foo'
vm.template = template # ERROR
The last line need to be:
vm.template = params.Template(id=template.id)
In my version of the Python API i used mix-in classes to add the OO
behavior to the generated classes. It was easy, because PyXB had support
for that. This gave me the single class behavior. I am not sure if
GenerateDS generated code allows that. I'm pretty sure it could be done
though, but you may need to do some Python magic.
* Nitpick: should this just be called "ovirt" instead of "ovirtsdk"?
Overall great work though!
Regards,
Geert
12 years, 5 months
[Engine-devel] timeout when starting a new VM
by Geert Jansen
Hi,
i've got a setup with 1500 guests, of which about 800 are running now.
Things start getting a little slow. I'm trying to start more guests via
the ovirt-engine-sdk, but i get this error:
File "/usr/bin/labmgr", line 171, in func_start
vm.start(action)
File
"/usr/lib/python2.6/site-packages/ovirt_engine_sdk-1.0_SNAPSHOT-py2.6.egg/ovirtsdk/infrastructure/brokers.py",
line 2389, in start
body=ParseHelper.toXml(action))
File
"/usr/lib/python2.6/site-packages/ovirt_engine_sdk-1.0_SNAPSHOT-py2.6.egg/ovirtsdk/infrastructure/proxy.py",
line 42, in request
conn=self.getConnectionsPool().getConnection())
File
"/usr/lib/python2.6/site-packages/ovirt_engine_sdk-1.0_SNAPSHOT-py2.6.egg/ovirtsdk/infrastructure/proxy.py",
line 54, in __doRequest
raise ConnectionError, str(e)
ovirtsdk.infrastructure.errors.ConnectionError: [ERROR]::oVirt API
connection failure, The read operation timed out
Any ideas?
Regards,
Geert
12 years, 5 months
[Engine-devel] SDK: multi threading
by Geert Jansen
Hi Michael,
two questions about threading..:
- Is the SDK thread safe?
- Can I make multiple concurrent requests to the API by using
multiple threads, or do the threads block each other?
Regards,
Geert
12 years, 5 months
[Engine-devel] SDK: multiple instances of API?
by Geert Jansen
Hi Michael,
when i try to create multiple instances of the API object like this:
from ovirtsdk.api import API
url = 'xxx'
username = 'admin@internal'
password = 'yyy'
a1 = API(url, username, password)
a2 = API(url, username, password)
I get this error:
Traceback (most recent call last):
File "test.py", line 8, in <module>
a2 = API(url, username, password)
File "/home/geertj/Source/ovirt-engine-sdk/src/ovirtsdk/api.py",
line 47, in __init__
Mode.R)
File
"/home/geertj/Source/ovirt-engine-sdk/src/ovirtsdk/infrastructure/contextmanager.py",
line 53, in add
raise ImmutableError(key)
ovirtsdk.infrastructure.errors.ImmutableError: [ERROR]::'proxy' is
immutable.
Any ideas?
Thanks,
Geert
12 years, 5 months
Re: [Engine-devel] (no subject)
by Saggi Mizrahi
On Wed 16 Nov 2011 02:48:40 PM IST, Maor wrote:
> Hello all,
>
> The Quota feature description is published under the following links:
> http://www.ovirt.org/wiki/Features/DetailedQuota
> http://www.ovirt.org/wiki/Features/Quota
> Notice that screens of UI mockups should be updated.
>
> Please feel free, to share your comments about it.
>
> Thank you,
> Maor
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
I can't see how the host is supposed to enforce and handle it. Pause
the VM? Crash it? raise ENOMEM\ENOSPC in the guest?
Also what about cases of KSM\QCow2, disk\memory overcommit.
Disk Templates.
Storage for hibernation disk.
Temporary and shared disks.
Shared disks between VMs owned by different users.
Backup snapshots (should they count in the quota? They are transient)
I also don't see how vcpu limiting is any good? I don't even know what
it means. What happens in migration between hosts with different amount
of physical CPUs?
I also don't think CPU limiting is even a thing to be managed by a
quota. There is no reason not to use 100% of the CPU if you are the
only VM active. CPU scheduling should use a priority model and not a
quota IMHO.
12 years, 5 months
