- Devel - oVirt List Archives

[Engine-devel] [RFC] New Connection Management API
by Saggi Mizrahi 29 Jan '12

29 Jan '12

I have begun work at changing how API clients can control storage connections when interacting with VDSM. Currently there are 2 API calls: connectStorageServer() - Will connect to the storage target if the host is not already connected to it. disconnectStorageServer() - Will disconnect from the storage target if the host is connected to it. This API is very simple but is inappropriate when multiple clients and flows try to access the same storage. This is currently solved by trying to synchronize things inside rhevm. This is hard and convoluted. It also brings out issues with other clients using the VDSM API. Another problem is error recovery. Currently ovirt-engine(OE) has no way of monitoring the connections on all the hosts an if a connection disappears it's OE's responsibility to reconnect. I suggest a different concept where VDSM 'manages' the connections. VDSM receives a manage request with the connection information and from that point forward VDSM will try to keep this connection alive. If the connection fails VDSM will automatically try and recover. Every manage request will also have a connection ID(CID). This CID will be used when the same client asks to unamange the connection. When multiple requests for manage are received to the same connection they all have to have their own unique CID. By internally mapping CIDs to actual connections VDSM can properly disconnect when no CID is addressing the connection. This allows each client and even each flow to have it's own CID effectively eliminating connect\disconnect races. The change from (dis)connect to (un)manage also changes the semantics of the calls significantly. Whereas connectStorageServer would have returned when the storage is either connected or failed to connect, manageStorageServer will return once VDSM registered the CID. This means that the connection might not be active immediately as the VDSM tries to connect. The connection might remain down for a long time if the storage target is down or is having issues. This allows for VDSM to receive the manage request even if the storage is having issues and recover as soon as it's operational without user intervention. In order for the client to query the current state of the connections I propose getStorageConnectionList(). This will return a mapping of CID to connection status. The status contains the connection info (excluding credentials), whether the connection is active, whether the connection is managed (unamanged connection are returned with transient IDs), and, if the connection is down, the last error information. The same actual connection can return multiple times, once for each CID. For cases where an operation requires a connection to be active a user can poll the status of the CID. The user can then choose to poll for a certain amount of time or until an error appears in the error field of the status. This will give you either a timeout or a "try once" semantic depending on the flows needs. All connections that have been managed persist VDSM restart and will be managed until a corresponding unmanage command has been issued. There is no concept of temporary connections as "temporary" is flow dependent and VDSM can't accommodate all interpretation of "temporary". An ad-hoc mechanism can be build using the CID field. For instance a client can manage a connection with "ENGINE_FLOW101_CON1". If the flow got interrupted the client can clean all IDs with certain flow IDs. I think this API gives safety, robustness, and implementation freedom. Nitty Gritty: manageStorageServer =================== Synopsis: manageStorageServer(uri, connectionID): Parameters: uri - a uri pointing to a storage target (eg: nfs://server:export, iscsi://host/iqn;portal=1) connectionID - string with any char except "/". Description: Tells VDSM to start managing the connection. From this moment on VDSM will try and have the connection available when needed. VDSM will monitor the connection and will automatically reconnect on failure. Returns: Success code if VDSM was able to manage the connection. It usually just verifies that the arguments are sane and that the CID is not already in use. This doesn't mean the host is connected. ---- unmanageStorageServer ===================== Synopsis: unmanageStorageServer(connectionID): Parameters: connectionID - string with any char except "/". Descriptions: Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection. Returns: Success code if VDSM was able to unmanage the connection. It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList() ---- getStorageServerList ==================== Synopsis: getStorageServerList() Description: Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls. Results:VDSM was able to manage the connection. It usually just verifies that the arguments are sane and that the CID is not already in use. This doesn't mean the host is connected. ---- unmanageStorageServer ===================== Synopsis: unmanageStorageServer(connectionID): Parameters: connectionID - string with any char except "/". Descriptions: Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection. Returns: Success code if VDSM was able to unmanage the connection. It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList() ---- getStorageServerList ==================== Synopsis: getStorageServerList() Description: Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls. Results: A mapping between CIDs and the status. example return value (Actual key names may differ) {'conA': {'connected': True, 'managed': True, 'lastError': 0, 'connectionInfo': { 'remotePath': 'server:/export 'retrans': 3 'version': 4 }} 'iscsi_session_34': {'connected': False, 'managed': False, 'lastError': 339, 'connectionIfno': { 'hostname': 'dandylopn' 'portal': 1}} }

7 21

[Engine-devel] data modelling in fixtures.xml
by Yair Zaslavsky 25 Jan '12

25 Jan '12

Hi all, When you introduce new business entities + DAOs or change some behaviour of existing ones, besides providing tests, please try as much as possible to insert reasonable test data to fixtures.xml, as we use the same fixtures.xml file for all our DAO testing. Kind regards, Yair

3 4

[Engine-devel] latency issues with gerrit.ovirt.org
by Barak Azulay 25 Jan '12

25 Jan '12

I'm experiencing severe latency issues with gerrit.ovirt.org - am I the only one ? Thanks Barak Azulay

9 11

Re: [Engine-devel] Call for oVirt subproject release note beats!
by Steve Gordon 24 Jan '12

24 Jan '12

Hi all, Assuming we are still on schedule for a release this is the last call for release note items. We will discuss further in the weekly sync up meeting tomorrow. Steve ----- Original Message ----- > From: "Steve Gordon" <sgordon(a)redhat.com> > To: arch(a)ovirt.org > Sent: Monday, January 16, 2012 10:34:01 AM > Subject: Re: Call for oVirt subproject release note beats! > > Hi all, > > As discussed in last week's sync-up meeting I emailed the > contributors listed as oVirt subproject maintainers requesting > release note beats. So far only the oVirt engine section of the > release notes page has been populated (thanks Livnat!). The release > notes page is here: > > http://www.ovirt.org/wiki/Release_Notes > > Please take the time to update it with any features, resolved issues, > or known issues you are aware of that you think should be noted for > this first release of oVirt. Even one liners or a link to the > relevant feature page/bugzilla would be great, I intend to edit this > into a cohesive document before the actual release. > > Thanks, > > Steve > > ----- Original Message ----- > > From: "Steve Gordon" <sgordon(a)redhat.com> > > To: "Michael Pasternak" <mpastern(a)redhat.com>, "Einav Cohen" > > <ecohen(a)redhat.com>, "Itamar Heim" <iheim(a)redhat.com>, > > "Barak Azulay" <bazulay(a)redhat.com>, "Yaniv Dary" > > <ydary(a)redhat.com>, "Ofer Schreiber" <oschreib(a)redhat.com>, "Alan > > Pevec" <apevec(a)redhat.com>, "Livnat Peer" <lpeer(a)redhat.com>, "Ayal > > Baron" <abaron(a)redhat.com> > > Cc: "Karsten 'quaid' Wade" <kwade(a)redhat.com>, "Robyn Bergeron" > > <rbergero(a)redhat.com> > > Sent: Wednesday, January 11, 2012 11:51:09 AM > > Subject: Call for oVirt subproject release note beats! > > > > Hi all, > > > > If you are getting this email you are listed as the first > > maintainer > > of an oVirt subproject at > > http://www.ovirt.org/project/subprojects/. > > As we ramp up towards our initial formal release of the oVirt > > project we were hoping that you could provide a short run down of > > what your subproject is, and a brief blurb on any noteworthy > > features/resolved issues/known issues in it for this initial > > release. If you have bugzilla #s to include even better! > > > > This effort would assist us greatly in providing a release notes > > document for this initial release. Please add your input to the > > relevant section of this page on the wiki: > > > > http://www.ovirt.org/wiki/Release_Notes > > > > Note that the structure I have provided is just to be used as a > > guide > > for now, once we have see what we have we can discuss further and > > reformat appropriately. > > > > Thanks! > > > > Steve >

1 0

[Engine-devel] engine-core redundant instantiation of mappers
by Yair Zaslavsky 24 Jan '12

24 Jan '12

Hi all, I am refactoring now DiskImageDAODbFacadeImpl. As part of my work, I'm also defining there a Spring-JDBC mapper , similar to other ParameterizedRowMapper based mappers we have in code (i.e DbUserDAODbFacadeImpl has one) Looks like in our DAO getXXXX methods we instantiate these mapper objects (each method creates a new mapper object). Can anyone see a reason , in case a mapper object is stateless (and from what I see, they are) - why not to have a single instantiation per mapper type (i.e - Have a MapperUtils class, with static methods like MapperUtils.getDiskImageMapper() ? IMHO this can save us some unnecessary instanatiations and improve performance Yair

2 1

[Engine-devel] Engine UI only: cleaning EAR from deployment folder may be required.
by Asaf Shakarchi 23 Jan '12

23 Jan '12

Hello, Some maven re-factoring (for UI modules only) patch (http://gerrit.ovirt.org/#change,689) has just merged, You may encounter weird UI behaviours that might occur because some unnecessary data still exist in the deployed EAR folder. If so please follow these steps: 1) Clean then build (the entire engine) with Pgwt-admin,gwt-user profiles 2) Fully remove the ovirt.ear from your JBOSS deployment folder. 3) cd OVIRT_ENGINE/ear then 'mvn install -Pdep' Thanks, Regards, Asaf

2 2

[Engine-devel] running jboss as 7 with openjdk
by Jon Choate 22 Jan '12

22 Jan '12

I keep getting a message that it can't find sun.security.krb5.KrbException. Any ideas? Jboss-as-5 ran ovirt-engine just fine with openjdk.

2 2

Re: [Engine-devel] Fwd: VM Payload feature
by Hugh Brock 22 Jan '12

22 Jan '12

On Sat, Jan 21, 2012 at 11:54:02PM +0200, Itamar Heim wrote: > this is intended to cover the cloud forms requirement - please > review and make sure it is in the right direction. > (going with floppy/iso for this version, rather than IP/injection > which could be optional future injection methods). > please reply on thread upstream for the discussion. > > Thanks, > Itamar >From the Aeolus side I don't think we have a strong opinion which way we do this -- it's more up to the Deltacloud guys. Neither approach requires the guest agent to be installed, right? In any case I'm glad to see this is in the works. Take care, --Hugh > -------- Original Message -------- > Subject: [Engine-devel] VM Payload feature > Date: Wed, 18 Jan 2012 10:41:59 -0500 (EST) > From: Oved Ourfalli <ovedo(a)redhat.com> > To: engine-devel(a)ovirt.org > > Hey all, > > Continuing the discussion about Aeolus instance data injection to a > VM (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html) > we propose a new VM Payload feature. > > The following wiki page contains a description page of the feature. > http://www.ovirt.org/wiki/Features/VMPayload > > Please read and review. > There are several approaches there, and we wish to head your > opinions and thoughts about them. > > Once we agree on an approach, we will start designing. > > Thank you, > Oved > _______________________________________________ > Engine-devel mailing list > Engine-devel(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel -- == Hugh Brock, hbrock(a)redhat.com == == Engineering Manager, Cloud BU == == Aeolus Project: Manage virtual infrastructure across clouds. == == http://aeolusproject.org == "I know that you believe you understand what you think I said, but I’m not sure you realize that what you heard is not what I meant." --Robert McCloskey

2 1

[Engine-devel] Question about CloneVMFromSnapshot feature in context of shared disks and direct LUNs-based disks
by Yair Zaslavsky 22 Jan '12

22 Jan '12

Hi all, Following the upstream meeting dated Wednesday January 18th, 2012 - I presented the clone VM from snpashot feature and we discussed the feature behaviour. Two issues that were raised are the behaviour of the feature in context of shared disks and direct LUNs-based disks - On one hand, if we copy&collapse such images - this may yield to data corruption (think of a scenario where the source and destination VMs use the same disk). On the other hand - if we decide not to copy&collapse - the target VM will have missing VM and its state will not totally reflect the logical state. One of the solution raises is to mark such disks (at the destination) as unplugged, allowing the administrator the ability to plug them (understanding of course the consequences). I would like to receive inputs on this issue Kind regards, Yair

7 26

[Engine-devel] getDeviceList should report partioned devices, too
by Dan Kenigsberg 22 Jan '12

22 Jan '12

Hi Lists, One cannot create a PV on a partitioned device, and therefor such devices where not reported to Engine. This proved surprising to users who woder where their LUN disappeared. Vdsm should report all devices, and ovirt-engine should mark partitioned devices as unworthy of a PV. In the future, Vdsm may allow to forcefully remove a partition table from a device, to make it usable as a PV. Douglas (CCed) would take resposibilty on the Vdsm side. Initial patch at http://gerrit.ovirt.org/944 sports a backword-compatible API. Who's taking this on Engine? This involves GUI, too, as partitioned devices should probably be displayed greyed-out. Dan.

4 14