From masayag at redhat.com  Sun Jan  1 00:39:18 2012
From: masayag at redhat.com (Moti Asayag)
Date: Sat, 31 Dec 2011 19:39:18 -0500 (EST)
Subject: [Engine-devel] Task Manager Design Review
Message-ID: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>

The following is a new meeting request:

Subject: Task Manager Design Review 
Organizer: "Moti Asayag" <masayag at redhat.com> 

Location: asia-tlv at redhat.com 
Resources: asia-tlv at redhat.com 
Time: Monday, January 2, 2012, 2:00:00 PM - 3:00:00 PM GMT +02:00 Jerusalem
 
Invitees: engine-devel at ovirt.org; mgoldboi at redhat.com; ykaul at redhat.com 


*~*~*~*~*~*~*~*~*~*

This is an invitation for a design review for the Task Manager feature.
Task Manager Detailed design could be found here:
http://ovirt.org/wiki/Features/TaskManagerDetailed

You are invited to join.

== Conference Call Info ==

* Toll Free Dial-In Number (US & Canada): (800) 451-8679
* International dial-in listed below
* Conference Code: 9197544554

If you are not actively participating in the conversation
please put your line on mute to make it easier for all
participants to hear.

*6 -- Mute your line
#6 -- Unmute your line

Global Access Numbers Local:
Australia, Sydney Dial-In #: 0289852326
Austria, Vienna Dial-In #: 012534978196
Belgium, Brussels Dial-In #: 027920405
China Dial-In #: 4006205013
Denmark, Copenhagen Dial-In #: 32729215
Finland, Helsinki Dial-In #: 0923194436
France, Paris Dial-In #: 0170377140
Germany, Berlin Dial-In #: 030300190579
Ireland, Dublin Dial-In #: 014367793
Italy, Milan Dial-In #: 0236269529
Netherlands, Amsterdam Dial-In #: 0207975872
Norway, Oslo Dial-In #: 21033188
Singapore Dial-In #: 64840858
Spain, Barcelona Dial-In #: 935452328
Sweden, Stockholm Dial-In #: 0850513770
Switzerland, Geneva Dial-In #: 0225927881
United Kingdom Dial-In #: 02078970515
United Kingdom Dial-In #: 08445790676
United Kingdom, LocalCall Dial-In #: 08445790678
United States Dial-In #: 2127295016

 
Global Access Numbers Tollfree:
Argentina Dial-In #: 8004441016
Australia Dial-In #: 1800337169
Austria Dial-In #: 0800005898
Bahamas Dial-In #: 18002054776
Bahrain Dial-In #: 80004377
Belgium Dial-In #: 080048325
Brazil Dial-In #: 08008921002
Bulgaria Dial-In #: 008001100236
Chile Dial-In #: 800370228
Colombia Dial-In #: 018009134033
Costa Rica Dial-In #: 08000131048
Cyprus Dial-In #: 80095297
Czech Republic Dial-In #: 800700318
Denmark Dial-In #: 80887114
Dominican Republic Dial-In #: 18887512313
Estonia Dial-In #: 8000100232
Finland Dial-In #: 0800117116
France Dial-In #: 0805632867
Germany Dial-In #: 8006647541
Greece Dial-In #: 00800127562
Hong Kong Dial-In #: 800930349
Hungary Dial-In #: 0680016796
Iceland Dial-In #: 8008967
India Dial-In #: 0008006501533
Indonesia Dial-In #: 0018030179162
Ireland Dial-In #: 1800932401
Israel Dial-In #: 1809462557
Italy Dial-In #: 800985897
Jamaica Dial-In #: 18002050328
Japan Dial-In #: 0120934453
Korea (South) Dial-In #: 007986517393
Latvia Dial-In #: 80003339
Lithuania Dial-In #: 880030479
Luxembourg Dial-In #: 80026595
Malaysia Dial-In #: 1800814451
Mexico Dial-In #: 0018664590915
New Zealand Dial-In #: 0800888167
Norway Dial-In #: 80012994
Panama Dial-In #: 008002269184
Philippines Dial-In #: 180011100991
Poland Dial-In #: 008001210187
Portugal Dial-In #: 800814625
Russian Federation Dial-In #: 81080028341012
Saint Kitts and Nevis Dial-In #: 18002059252
Singapore Dial-In #: 8006162235
Slovak Republic Dial-In #: 0800001441
South Africa Dial-In #: 0800981148
Spain Dial-In #: 800300524
Sweden Dial-In #: 200896860
Switzerland Dial-In #: 800650077
Taiwan Dial-In #: 00801127141
Thailand Dial-In #: 001800656966
Trinidad and Tobago Dial-In #: 18002024615
United Arab Emirates Dial-In #: 8000650591
United Kingdom Dial-In #: 08006948057
United States Dial-In #: 8004518679
Uruguay Dial-In #: 00040190315
Venezuela Dial-In #: 08001627182
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20111231/cca7858c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: meeting.ics
Type: text/calendar
Size: 9674 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20111231/cca7858c/attachment.ics>

From masayag at redhat.com  Sun Jan  1 00:52:00 2012
From: masayag at redhat.com (Moti Asayag)
Date: Sat, 31 Dec 2011 19:52:00 -0500 (EST)
Subject: [Engine-devel] Task Manager Introduction and Scope for 3.1
Message-ID: <9fa1962c-d2bc-469c-adf3-92e36e8787d3@zmail14.collab.prod.int.phx2.redhat.com>

The following is a new meeting request:

Subject: Task Manager Introduction and Scope for 3.1 
Organizer: "Moti Asayag" <masayag at redhat.com> 

Location: "Asia-tlv" <asia-tlv at redhat.com> 
Resources: "Asia-tlv" <asia-tlv at redhat.com> 
Time: Sunday, January 1, 2012, 3:00:00 PM - 4:00:00 PM GMT +02:00 Jerusalem
 
Invitees: ykaul at redhat.com; mgoldboi at redhat.com; engine-devel at ovirt.org 


*~*~*~*~*~*~*~*~*~*

This meeting is about introducing the Task Manager feature for 3.1 and its scope.
The feature page could be found here: 
http://ovirt.org/wiki/Features/TaskManager
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20111231/04428d94/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: meeting.ics
Type: text/calendar
Size: 3057 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20111231/04428d94/attachment.ics>

From oliel at redhat.com  Sun Jan  1 13:15:18 2012
From: oliel at redhat.com (Ori Liel)
Date: Sun, 01 Jan 2012 08:15:18 -0500 (EST)
Subject: [Engine-devel] restapi - removing template from a specified
	storage-domain
In-Reply-To: <4EFC7D7C.4030803@redhat.com>
Message-ID: <e29d151e-7ec1-42dd-84f9-3107f7a67af1@zmail15.collab.prod.int.phx2.redhat.com>

I'm about to implement removing template from a specified storage-domain. 

A template is meta-data + disks. The meta-data is in the ovirt-engine database, the disks are (potentially) scattered across several storage-domains. Removing a template from a specified storage domain means removing that template's disks from that storage-domain. API-wise, it's better to enable the deletion at the single-disk level, otherwise rollback and all sorts of unnecessary complexities enter the picture. So what I would like to go forward with is the following API: 

DELETE /api/templates/{template_xxx}/disks/{disk_yyy}

   <storage_domain id="domain_zzz"/>

This means: "delete the disk 'disk_yyy' (which belongs to template 'template_xxx' from the storage-domain 'domain_zzz'. 

Any comments? 

Thanks, 

Ori. 


From ofrenkel at redhat.com  Sun Jan  1 15:59:59 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Sun, 01 Jan 2012 10:59:59 -0500 (EST)
Subject: [Engine-devel] restapi - removing template from a
	specified	storage-domain
In-Reply-To: <e29d151e-7ec1-42dd-84f9-3107f7a67af1@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <63a7fa0a-62f8-4266-9f39-639d99ea2139@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Ori Liel" <oliel at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Sunday, January 1, 2012 3:15:18 PM
> Subject: [Engine-devel] restapi - removing template from a specified	storage-domain
> 
> I'm about to implement removing template from a specified
> storage-domain.
> 
> A template is meta-data + disks. The meta-data is in the ovirt-engine
> database, the disks are (potentially) scattered across several
> storage-domains. Removing a template from a specified storage domain
> means removing that template's disks from that storage-domain.
> API-wise, it's better to enable the deletion at the single-disk
> level, otherwise rollback and all sorts of unnecessary complexities
> enter the picture. So what I would like to go forward with is the
> following API:
> 
> DELETE /api/templates/{template_xxx}/disks/{disk_yyy}
> 
>    <storage_domain id="domain_zzz"/>
> 
> This means: "delete the disk 'disk_yyy' (which belongs to template
> 'template_xxx' from the storage-domain 'domain_zzz'.
> 
> Any comments?

just that CURRENT backend only allows removing per storage domain(s),
this functionallity is in plan, though (if that was what you meant..)

> 
> Thanks,
> 
> Ori.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From iheim at redhat.com  Sun Jan  1 19:44:47 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sun, 01 Jan 2012 21:44:47 +0200
Subject: [Engine-devel] restapi - removing template from a
	specified	storage-domain
In-Reply-To: <63a7fa0a-62f8-4266-9f39-639d99ea2139@zmail07.collab.prod.int.phx2.redhat.com>
References: <63a7fa0a-62f8-4266-9f39-639d99ea2139@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F00B7AF.1080207@redhat.com>

On 01/01/2012 05:59 PM, Omer Frenkel wrote:
>
>
> ----- Original Message -----
>> From: "Ori Liel"<oliel at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Sunday, January 1, 2012 3:15:18 PM
>> Subject: [Engine-devel] restapi - removing template from a specified	storage-domain
>>
>> I'm about to implement removing template from a specified
>> storage-domain.
>>
>> A template is meta-data + disks. The meta-data is in the ovirt-engine
>> database, the disks are (potentially) scattered across several
>> storage-domains. Removing a template from a specified storage domain
>> means removing that template's disks from that storage-domain.
>> API-wise, it's better to enable the deletion at the single-disk
>> level, otherwise rollback and all sorts of unnecessary complexities
>> enter the picture. So what I would like to go forward with is the
>> following API:
>>
>> DELETE /api/templates/{template_xxx}/disks/{disk_yyy}
>>
>>     <storage_domain id="domain_zzz"/>
>>
>> This means: "delete the disk 'disk_yyy' (which belongs to template
>> 'template_xxx' from the storage-domain 'domain_zzz'.
>>
>> Any comments?
>
> just that CURRENT backend only allows removing per storage domain(s),
> this functionallity is in plan, though (if that was what you meant..)

I think the implementation for this can/should wait for the new support 
in the backend if makes more sense api-wise, rather than implementing 
based on the current limitations and later changing the api.
the question is with the move to multiple storage domains, will an 
atomic[1] verb to delete all disks from a specific storage domain (per 
vm/template) will be provided, or only per disk.

[1] well, delete is rollforward, so not sure i understand Ori's point 
about rollback. OTOH, the API should probably behave the same for 
move/copy of disks, etc.


From iheim at redhat.com  Sun Jan  1 21:08:18 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sun, 01 Jan 2012 23:08:18 +0200
Subject: [Engine-devel] Stable Device Addresses HLD/LLD for tomorrow
 meeting (15:00 GMT+1)
In-Reply-To: <60a8a798-2cc8-467c-8b05-c07414723048@zmail13.collab.prod.int.phx2.redhat.com>
References: <60a8a798-2cc8-467c-8b05-c07414723048@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F00CB42.4070404@redhat.com>

On 12/28/2011 06:52 PM, Eli Mesika wrote:
> High level design :
> http://www.ovirt.org/wiki/Features/Design/StableDeviceAddresses
>
> Low level design :
> http://www.ovirt.org/wiki/Features/Design/DetailedStableDeviceAddresses
> (probably will be modified until the meeting)


1. " hash -- holds the md5 like encryption indicating a change "

I don't think "encryption" is relevant here.

2. VMOldInfoManager

how will we know/flag when we expect it can be deleted?
("Old" is relative. what if you will need to create another OldOld 
class? maybe something more specific like 3_0, so you know you can 
delete it when you no longer have 3_0 clusters supported?)

3. "Addresses are kept in the database as a plain-text strings"
at some point we will want to expose this to users. maybe a better 
representation would be in order (could be handled later as part of 
upgrade process i guess).

4. video cards
you mention only spice - what happens to VNC monitors?

5. floppy/cd-rom
expect there will be more than one cd-rom (looks as the most probably 
solution to user data injection afaiu)

6. virtio-serial
you mention it in the beginning, but not afterwards wrt implementation 
details. there may be multiple of these (qemu-ga, ovirt-guest-agent, 
spice-vd-agent?)




> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From iheim at redhat.com  Sun Jan  1 21:38:45 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sun, 01 Jan 2012 23:38:45 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F00D265.9070604@redhat.com>

On 01/01/2012 02:39 AM, Moti Asayag wrote:
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

looks pretty comprehensive - a few questions:
1. can one see all audit log events related to a task?

2. "A command with VDSM tasks" - what about other types of tasks? what 
if after the vdsm tasks there is another part of logic, then another 
vdsm task (iirc, there was a legacy limitation of "one single async task 
in the end of the command", but this shouldn't be relevant any more).

3. "A command which invokes internal commands - By default, the internal 
command will not be presented as a step of the parent command."

examples of the non default behavior?

4. "A customized command - an asynchronous job which its termination is 
decided by an event other than tasks. There are few types of scenarios 
for it"
the examples are all described as internal implementation details - what 
are some actual relevant use cases/flows and what will the user see for 
them?

you also mention AddVdsCommand here - will it's internal steps be 
visible to the user as part of the monitored task?

5. requirements: what about internal tasks originated by the system (SPM 
election, live migration due to load balancing, fencing taking place, 
refresh of users from a directory, etc.)?


6. STEP table, start_time is "not null" - shouldn't it be nullable?

7. UX - no main tasks view?

8. can i define a step based on a previous step failure rather than 
success (i.e., step 2 will be run only if step 1 succeeds, step 3 will 
only run if step 1 failed, step 4 could be either dependent on any of 
them, or run after step 2 and 3 finished, whatever their result was.

9. REST API modeling?



From masayag at redhat.com  Mon Jan  2 00:02:55 2012
From: masayag at redhat.com (Moti Asayag)
Date: Mon, 02 Jan 2012 02:02:55 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F00D265.9070604@redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>
	<4F00D265.9070604@redhat.com>
Message-ID: <4F00F42F.4080405@redhat.com>

On 01/01/2012 11:38 PM, Itamar Heim wrote:
> On 01/01/2012 02:39 AM, Moti Asayag wrote:
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> looks pretty comprehensive - a few questions:
> 1. can one see all audit log events related to a task?
It means to apply a search query for the event log by job id.
The job id is associated with the audit log, therefore such query could
be comprised. This will probably be enabled from the global Tasks view.

> 
> 2. "A command with VDSM tasks" - what about other types of tasks? what
> if after the vdsm tasks there is another part of logic, then another
> vdsm task (iirc, there was a legacy limitation of "one single async task
> in the end of the command", but this shouldn't be relevant any more).
The Step entity is associated with external_system_type and external_id
which in this case are correlated to VDSM (external system type) and
VDSM task id (external id).
Having additional external systems which the backend invokes tasks on
will be supported if the task on the external system could be associated
with its representing Step and the task could be queried for it status.

VDSM Tasks which are created by a command will be polled only after the
command execution is over. If the command failed in between the creation
of Tasks due to some runtime exception, the command should be
rolled-back and the VDSM tasks should be cancelled and cleared. If
execution ends, by polling VDSM task status the step representing each
task should be updated with the task status. An example for such command
is HibernateVmCommand.


> 
> 3. "A command which invokes internal commands - By default, the internal
> command will not be presented as a step of the parent command."
> 
> examples of the non default behavior?
For example the MaintenanceNumberOfVdsCommand which invokes
MaintenanceVdsCommand which invokes InternalMigrateVmCommand.
Those are significant steps which we should expose to the user.

> 
> 4. "A customized command - an asynchronous job which its termination is
> decided by an event other than tasks. There are few types of scenarios
> for it"
> the examples are all described as internal implementation details - what
> are some actual relevant use cases/flows and what will the user see for
> them?
> 
> you also mention AddVdsCommand here - will it's internal steps be
> visible to the user as part of the monitored task?
AddVdsCommand is an example for the above as its execution is durable
due to the installation part that should be exposed as a step to the user.

Another example is the determination of completing the
MaintenanceVdsCommand by receiving an event from the host monitor for
host moved to maintenance. It is already a step created earlier, but its
completion will be due to this specific event.


> 
> 5. requirements: what about internal tasks originated by the system (SPM
> election, live migration due to load balancing, fencing taking place,
> refresh of users from a directory, etc.)?
It was discussed on the meeting held today. It was agreed to report for
specific internal action such as VM migration, Host fencing,... but not
for the event itself (in order to prevent from flooding the Tasks view
from frequent events).

> 
> 
> 6. STEP table, start_time is "not null" - shouldn't it be nullable?
No. The Step is created in adjust to the execution unit it describes. In
order to prevent from additional update, it should be created with
status Started and start time.

> 
> 7. UX - no main tasks view?
Eldan, Einav ?

> 
> 8. can i define a step based on a previous step failure rather than
> success (i.e., step 2 will be run only if step 1 succeeds, step 3 will
> only run if step 1 failed, step 4 could be either dependent on any of
> them, or run after step 2 and 3 finished, whatever their result was.
It requires introducing the "best effort step" (V2), a failure of a step
will not fail the entire job. Mixing it with customized command should
provide that behavior. However, the Steps are used to report on
execution units status and do not participate in handling the flow.

> 
> 9. REST API modeling?
On the way...
> 



From dlaor at redhat.com  Sun Jan  1 12:52:00 2012
From: dlaor at redhat.com (Dor Laor)
Date: Sun, 01 Jan 2012 14:52:00 +0200
Subject: [Engine-devel] MOM Stable Device Address DR
In-Reply-To: <4EFDC9EE.7060206@redhat.com>
References: <dff626eb-c7e1-436e-b8ab-33b910fd8f91@zmail13.collab.prod.int.phx2.redhat.com>
	<4EFDC9EE.7060206@redhat.com>
Message-ID: <4F0056F0.10605@redhat.com>

On 12/30/2011 04:25 PM, Yair Zaslavsky wrote:
> On 12/29/2011 09:32 PM, Eli Mesika wrote:
>> The following summarizes the DR meeting from today
>>
>> Attendees: Backend Team + Moran&  Haim from QA
>>
>> -----------------------------------------
>> =>  marks responsibility for action items.|
>> -----------------------------------------
>>
>> Issues/Action Items:
>>
>>   1) Video - memory allocation logic, check again why the logic of how much memory to allocate to a video device should move to backend. =>  (Eli, Igor)
>>   2) Generic-device - rename to vm_device (Eli)
>>   3) Verify hash is only on devices (not on all domxml) , check if can be expanded to VM scope (app list etc.) =>  (Eli, Igor)
>>   4) Floppy/CD - should be handled as vm_device that means we have to add to it also boot order =>  (Eli, Igor)
>>   5) Hot plug -  need to check for added/deleted devices, Open issue how to handle for managed device.
>>                same for any other device that is changed not via backend =>  (Eli, Igor)
> About this issue - a question I asked at the meeting is whether disks
> have some sort of unique identifier as MAC address in case of NIC. If
> such info can be obtained by engine-core, we will be able to handle a
> scenario of plugging back a managed disk that got plugged out.

Disks have UUID and kvm supports it for its devices (virtio uuid is 
relevantly new)

>
>>   6) How boot order affects GUI (order of NICs) - check with Einav if support can be added for that in UI  (for disks use only 1) (Eli,Einav)

How does boot order correlates to stable device address?
Note that KVM supports boot order priority setting per device basis.


>>   7) OVF - add new devices , addresses , coordinate with V2V =>  (Eli)
>>   8) Live snapshot , check if affects design =>  (Eli, Igor)
>>   9) Review class diagram of detailed design =>  (Eli)
>> 10) Do we have to persist indexes ? check if addresses are respected in spite of indexes.... =>  (Eli, Igor)
>>
>> 11) Update wiki pages =>  (Eli)
>>
>> Reference:
>> http://www.ovirt.org/wiki/Features/Design/StableDeviceAddresses
>> http://www.ovirt.org/wiki/Features/Design/DetailedStableDeviceAddresses
>>
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From iheim at redhat.com  Mon Jan  2 07:53:22 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 02 Jan 2012 09:53:22 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F00F42F.4080405@redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>
	<4F00D265.9070604@redhat.com> <4F00F42F.4080405@redhat.com>
Message-ID: <4F016272.5060807@redhat.com>

On 01/02/2012 02:02 AM, Moti Asayag wrote:
...

>>
>> 5. requirements: what about internal tasks originated by the system (SPM
>> election, live migration due to load balancing, fencing taking place,
>> refresh of users from a directory, etc.)?
> It was discussed on the meeting held today. It was agreed to report for
> specific internal action such as VM migration, Host fencing,... but not
> for the event itself (in order to prevent from flooding the Tasks view
> from frequent events).

I think SPM election/status is an interesting enough task that if it 
happens it should be documented (as well as its process/results).

>
>>
>>
>> 6. STEP table, start_time is "not null" - shouldn't it be nullable?
> No. The Step is created in adjust to the execution unit it describes. In
> order to prevent from additional update, it should be created with
> status Started and start time.

oh - I thought a job is pre-defined with its steps, then the backend 
runs them.
from your reply i understand the job/steps are just documentation of 
what already happened?


From masayag at redhat.com  Mon Jan  2 08:12:40 2012
From: masayag at redhat.com (Moti Asayag)
Date: Mon, 02 Jan 2012 10:12:40 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F016272.5060807@redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>
	<4F00D265.9070604@redhat.com> <4F00F42F.4080405@redhat.com>
	<4F016272.5060807@redhat.com>
Message-ID: <4F0166F8.3000804@redhat.com>

On 01/02/2012 09:53 AM, Itamar Heim wrote:
> On 01/02/2012 02:02 AM, Moti Asayag wrote:
> ...
> 
>>>
>>> 5. requirements: what about internal tasks originated by the system (SPM
>>> election, live migration due to load balancing, fencing taking place,
>>> refresh of users from a directory, etc.)?
>> It was discussed on the meeting held today. It was agreed to report for
>> specific internal action such as VM migration, Host fencing,... but not
>> for the event itself (in order to prevent from flooding the Tasks view
>> from frequent events).
> 
> I think SPM election/status is an interesting enough task that if it
> happens it should be documented (as well as its process/results).
> 
>>
>>>
>>>
>>> 6. STEP table, start_time is "not null" - shouldn't it be nullable?
>> No. The Step is created in adjust to the execution unit it describes. In
>> order to prevent from additional update, it should be created with
>> status Started and start time.
> 
> oh - I thought a job is pre-defined with its steps, then the backend
> runs them.
> from your reply i understand the job/steps are just documentation of
> what already happened?
Correct. The Command implementation will determine by code when to add a
step and how to decide it ended. A default implementation is provided
for all commands, except those which specified on the requirements to be
more detailed.

Doing the other way (let the flow definition execute the steps) means
implementing a business process manager (or using existing one - such as
jBPM). I don't think the backend as implemented now is capable of
adjusting to it and letting external service orchestrate the flow.



From yzaslavs at redhat.com  Mon Jan  2 09:05:20 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Mon, 02 Jan 2012 11:05:20 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F0166F8.3000804@redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>	<4F00D265.9070604@redhat.com>
	<4F00F42F.4080405@redhat.com>	<4F016272.5060807@redhat.com>
	<4F0166F8.3000804@redhat.com>
Message-ID: <4F017350.5010301@redhat.com>

On 01/02/2012 10:12 AM, Moti Asayag wrote:
> On 01/02/2012 09:53 AM, Itamar Heim wrote:
>> On 01/02/2012 02:02 AM, Moti Asayag wrote:
>> ...
>>
>>>>
>>>> 5. requirements: what about internal tasks originated by the system (SPM
>>>> election, live migration due to load balancing, fencing taking place,
>>>> refresh of users from a directory, etc.)?
>>> It was discussed on the meeting held today. It was agreed to report for
>>> specific internal action such as VM migration, Host fencing,... but not
>>> for the event itself (in order to prevent from flooding the Tasks view
>>> from frequent events).
>>
>> I think SPM election/status is an interesting enough task that if it
>> happens it should be documented (as well as its process/results).
>>
>>>
>>>>
>>>>
>>>> 6. STEP table, start_time is "not null" - shouldn't it be nullable?
>>> No. The Step is created in adjust to the execution unit it describes. In
>>> order to prevent from additional update, it should be created with
>>> status Started and start time.
>>
>> oh - I thought a job is pre-defined with its steps, then the backend
>> runs them.
>> from your reply i understand the job/steps are just documentation of
>> what already happened?
Actually, also of what is already happening, If I understand correctly.

> Correct. The Command implementation will determine by code when to add a
> step and how to decide it ended. A default implementation is provided
> for all commands, except those which specified on the requirements to be
> more detailed.
> 
> Doing the other way (let the flow definition execute the steps) means
> implementing a business process manager (or using existing one - such as
> jBPM). I don't think the backend as implemented now is capable of
> adjusting to it and letting external service orchestrate the flow.
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From masayag at redhat.com  Mon Jan  2 09:16:33 2012
From: masayag at redhat.com (Moti Asayag)
Date: Mon, 02 Jan 2012 11:16:33 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F017350.5010301@redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>	<4F00D265.9070604@redhat.com>
	<4F00F42F.4080405@redhat.com>	<4F016272.5060807@redhat.com>
	<4F0166F8.3000804@redhat.com> <4F017350.5010301@redhat.com>
Message-ID: <4F0175F1.6060703@redhat.com>

On 01/02/2012 11:05 AM, Yair Zaslavsky wrote:
> On 01/02/2012 10:12 AM, Moti Asayag wrote:
>> On 01/02/2012 09:53 AM, Itamar Heim wrote:
>>> On 01/02/2012 02:02 AM, Moti Asayag wrote:
>>> ...
>>>
>>>>>
>>>>> 5. requirements: what about internal tasks originated by the system (SPM
>>>>> election, live migration due to load balancing, fencing taking place,
>>>>> refresh of users from a directory, etc.)?
>>>> It was discussed on the meeting held today. It was agreed to report for
>>>> specific internal action such as VM migration, Host fencing,... but not
>>>> for the event itself (in order to prevent from flooding the Tasks view
>>>> from frequent events).
>>>
>>> I think SPM election/status is an interesting enough task that if it
>>> happens it should be documented (as well as its process/results).
>>>
>>>>
>>>>>
>>>>>
>>>>> 6. STEP table, start_time is "not null" - shouldn't it be nullable?
>>>> No. The Step is created in adjust to the execution unit it describes. In
>>>> order to prevent from additional update, it should be created with
>>>> status Started and start time.
>>>
>>> oh - I thought a job is pre-defined with its steps, then the backend
>>> runs them.
>>> from your reply i understand the job/steps are just documentation of
>>> what already happened?
> Actually, also of what is already happening, If I understand correctly.
Correct.
See
http://ovirt.org/wiki/Features/TaskManagerDetailed#Simple_Command_Invocation_Sequence_Diagram

> 
>> Correct. The Command implementation will determine by code when to add a
>> step and how to decide it ended. A default implementation is provided
>> for all commands, except those which specified on the requirements to be
>> more detailed.
>>
>> Doing the other way (let the flow definition execute the steps) means
>> implementing a business process manager (or using existing one - such as
>> jBPM). I don't think the backend as implemented now is capable of
>> adjusting to it and letting external service orchestrate the flow.
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From oliel at redhat.com  Mon Jan  2 09:29:52 2012
From: oliel at redhat.com (Ori Liel)
Date: Mon, 02 Jan 2012 04:29:52 -0500 (EST)
Subject: [Engine-devel] restapi - removing template from a
	specified	storage-domain
In-Reply-To: <4F00B7AF.1080207@redhat.com>
Message-ID: <d75e1892-0197-439b-ace1-07d02ee7c893@zmail15.collab.prod.int.phx2.redhat.com>

>
>
>----- Original Message -----
>From: "Itamar Heim" <iheim at redhat.com>
>To: "Omer Frenkel" <ofrenkel at redhat.com>
>Cc: "Ori Liel" <oliel at redhat.com>, engine-devel at ovirt.org, "Geert Jansen" <gjansen at redhat.com>
>Sent: Sunday, January 1, 2012 9:44:47 PM
>Subject: Re: [Engine-devel] restapi - removing template from a specified	storage-domain
>
>On 01/01/2012 05:59 PM, Omer Frenkel wrote:
>>
>>
>> ----- Original Message -----
>>> From: "Ori Liel"<oliel at redhat.com>
>>> To: engine-devel at ovirt.org
>>> Sent: Sunday, January 1, 2012 3:15:18 PM
>>> Subject: [Engine-devel] restapi - removing template from a specified	storage-domain
>>>
>>> I'm about to implement removing template from a specified
>>> storage-domain.
>>>
>>> A template is meta-data + disks. The meta-data is in the ovirt-engine
>>> database, the disks are (potentially) scattered across several
>>> storage-domains. Removing a template from a specified storage domain
>>> means removing that template's disks from that storage-domain.
>>> API-wise, it's better to enable the deletion at the single-disk
>>> level, otherwise rollback and all sorts of unnecessary complexities
>>> enter the picture. So what I would like to go forward with is the
>>> following API:
>>>
>>> DELETE /api/templates/{template_xxx}/disks/{disk_yyy}
>>>
>>>     <storage_domain id="domain_zzz"/>
>>>
>>> This means: "delete the disk 'disk_yyy' (which belongs to template
>>> 'template_xxx' from the storage-domain 'domain_zzz'.
>>>
>>> Any comments?
>>
>> just that CURRENT backend only allows removing per storage domain(s),
>> this functionallity is in plan, though (if that was what you meant..)
>
>I think the implementation for this can/should wait for the new support 
>in the backend if makes more sense api-wise, rather than implementing 
>based on the current limitations and later changing the api.
>the question is with the move to multiple storage domains, will an 
>atomic[1] verb to delete all disks from a specific storage domain (per 
>vm/template) will be provided, or only per disk.
>
>[1] well, delete is rollforward, so not sure i understand Ori's point 
>about rollback. OTOH, the API should probably behave the same for 
>move/copy of disks, etc.

What I meant by rollback issues is: how do we handle a scenario in which 
deletion of disk1 succeeds, but then deletion of disk2 fails? 

When you say 'delete is rollforward', I assume you mean that delete 
operations don't require roll-back, and that even if only partial deletion 
was done - that's ok (I'm verifying because I'm not familiar with the term 
roll-forward).

Assuming this is what you meant, I still prefer the per-disk deletion, 
because the scenario of partial success still requires extra handling: 

1) Backend would have to generate a complex message that disk1 was 
deleted but disk2 not. 

2) REST-API user might resend the same request to retry - and fail because
this time disk1 does not exist. 






From mlipchuk at redhat.com  Mon Jan  2 10:34:28 2012
From: mlipchuk at redhat.com (Maor)
Date: Mon, 02 Jan 2012 12:34:28 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F0175F1.6060703@redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>	<4F00D265.9070604@redhat.com>
	<4F00F42F.4080405@redhat.com>	<4F016272.5060807@redhat.com>
	<4F0166F8.3000804@redhat.com> <4F017350.5010301@redhat.com>
	<4F0175F1.6060703@redhat.com>
Message-ID: <4F018834.1090509@redhat.com>

Hi Moti,
I got a few questions regarding the Task Manager design:

Do we still need zombie tasks after this change, since I saw the user
can now end the process from the GUI?

What should be the behaviour when user will try to end step deleteVolume
from the GUI? How will it reflect the entire Job?

Referencing DB scope, Will step_name and external_system_type will be
reflected by enums? if they are, maybe its better to use int instead of
String in those fields.
Would not it be better to declare correlation_id and external_id as UUID
instead of String.

Does the task name is calculated on create, or should it be persisted in
the DB?

Besides that, looks very good!

Regards,
Maor Lipchuk



From yzaslavs at redhat.com  Mon Jan  2 11:38:40 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Mon, 02 Jan 2012 13:38:40 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F018834.1090509@redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>	<4F00D265.9070604@redhat.com>	<4F00F42F.4080405@redhat.com>	<4F016272.5060807@redhat.com>	<4F0166F8.3000804@redhat.com>
	<4F017350.5010301@redhat.com>	<4F0175F1.6060703@redhat.com>
	<4F018834.1090509@redhat.com>
Message-ID: <4F019740.3080508@redhat.com>

On 01/02/2012 12:34 PM, Maor wrote:
> Hi Moti,
> I got a few questions regarding the Task Manager design:
> 
> Do we still need zombie tasks after this change, since I saw the user
> can now end the process from the GUI?

I think we should still have a "safe-way" mechanism for the issue of
tasks that are not ended properly.
> 
> What should be the behaviour when user will try to end step deleteVolume
> from the GUI? How will it reflect the entire Job?
> 
> Referencing DB scope, Will step_name and external_system_type will be
> reflected by enums? if they are, maybe its better to use int instead of
> String in those fields.
We should carefully consider DB performance over readability/debugging
of DB. For example, JPA lets you define if the enum maps as string or
int to DB, and there is a reason why they let you choose :)

> Would not it be better to declare correlation_id and external_id as UUID
> instead of String.
+1
> 
> Does the task name is calculated on create, or should it be persisted in
> the DB?
> 
> Besides that, looks very good!
> 
> Regards,
> Maor Lipchuk
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From masayag at redhat.com  Mon Jan  2 10:48:37 2012
From: masayag at redhat.com (Moti Asayag)
Date: Mon, 02 Jan 2012 05:48:37 -0500 (EST)
Subject: [Engine-devel] Task Manager Design Review
Message-ID: <167fa5d9-e0b1-4718-adf7-dac5c0c8411a@zmail14.collab.prod.int.phx2.redhat.com>

The following meeting has been modified:

Subject: Task Manager Design Review 
Organizer: "Moti Asayag" <masayag at redhat.com> 

Location: "Europe-tlv" <europe-tlv at redhat.com> [MODIFIED]
Resources: "Europe-tlv" <europe-tlv at redhat.com> [MODIFIED]
Time: Monday, January 2, 2012, 3:30:00 PM - 4:30:00 PM GMT +02:00 Jerusalem [MODIFIED]
 
Required: engine-devel at ovirt.org; mgoldboi at redhat.com; ykaul at redhat.com; lpeer at redhat.com; ofrenkel at redhat.com; ovedo at redhat.com; gchaplik at redhat.com; mkolesni at redhat.com 
Optional: dfediuck at redhat.com 

*~*~*~*~*~*~*~*~*~*

This is an invitation for a design review for the Task Manager feature. 
Task Manager Detailed design could be found here: 
http://ovirt.org/wiki/Features/TaskManagerDetailed 

You are invited to join. 

== Conference Call Info == 

* Toll Free Dial-In Number (US & Canada): (800) 451-8679 
* International dial-in listed below 
* Conference Code: 9197544554 

If you are not actively participating in the conversation 
please put your line on mute to make it easier for all 
participants to hear. 

*6 -- Mute your line 
#6 -- Unmute your line 

Global Access Numbers Local: 
Australia, Sydney Dial-In #: 0289852326 
Austria, Vienna Dial-In #: 012534978196 
Belgium, Brussels Dial-In #: 027920405 
China Dial-In #: 4006205013 
Denmark, Copenhagen Dial-In #: 32729215 
Finland, Helsinki Dial-In #: 0923194436 
France, Paris Dial-In #: 0170377140 
Germany, Berlin Dial-In #: 030300190579 
Ireland, Dublin Dial-In #: 014367793 
Italy, Milan Dial-In #: 0236269529 
Netherlands, Amsterdam Dial-In #: 0207975872 
Norway, Oslo Dial-In #: 21033188 
Singapore Dial-In #: 64840858 
Spain, Barcelona Dial-In #: 935452328 
Sweden, Stockholm Dial-In #: 0850513770 
Switzerland, Geneva Dial-In #: 0225927881 
United Kingdom Dial-In #: 02078970515 
United Kingdom Dial-In #: 08445790676 
United Kingdom, LocalCall Dial-In #: 08445790678 
United States Dial-In #: 2127295016 


Global Access Numbers Tollfree: 
Argentina Dial-In #: 8004441016 
Australia Dial-In #: 1800337169 
Austria Dial-In #: 0800005898 
Bahamas Dial-In #: 18002054776 
Bahrain Dial-In #: 80004377 
Belgium Dial-In #: 080048325 
Brazil Dial-In #: 08008921002 
Bulgaria Dial-In #: 008001100236 
Chile Dial-In #: 800370228 
Colombia Dial-In #: 018009134033 
Costa Rica Dial-In #: 08000131048 
Cyprus Dial-In #: 80095297 
Czech Republic Dial-In #: 800700318 
Denmark Dial-In #: 80887114 
Dominican Republic Dial-In #: 18887512313 
Estonia Dial-In #: 8000100232 
Finland Dial-In #: 0800117116 
France Dial-In #: 0805632867 
Germany Dial-In #: 8006647541 
Greece Dial-In #: 00800127562 
Hong Kong Dial-In #: 800930349 
Hungary Dial-In #: 0680016796 
Iceland Dial-In #: 8008967 
India Dial-In #: 0008006501533 
Indonesia Dial-In #: 0018030179162 
Ireland Dial-In #: 1800932401 
Israel Dial-In #: 1809462557 
Italy Dial-In #: 800985897 
Jamaica Dial-In #: 18002050328 
Japan Dial-In #: 0120934453 
Korea (South) Dial-In #: 007986517393 
Latvia Dial-In #: 80003339 
Lithuania Dial-In #: 880030479 
Luxembourg Dial-In #: 80026595 
Malaysia Dial-In #: 1800814451 
Mexico Dial-In #: 0018664590915 
New Zealand Dial-In #: 0800888167 
Norway Dial-In #: 80012994 
Panama Dial-In #: 008002269184 
Philippines Dial-In #: 180011100991 
Poland Dial-In #: 008001210187 
Portugal Dial-In #: 800814625 
Russian Federation Dial-In #: 81080028341012 
Saint Kitts and Nevis Dial-In #: 18002059252 
Singapore Dial-In #: 8006162235 
Slovak Republic Dial-In #: 0800001441 
South Africa Dial-In #: 0800981148 
Spain Dial-In #: 800300524 
Sweden Dial-In #: 200896860 
Switzerland Dial-In #: 800650077 
Taiwan Dial-In #: 00801127141 
Thailand Dial-In #: 001800656966 
Trinidad and Tobago Dial-In #: 18002024615 
United Arab Emirates Dial-In #: 8000650591 
United Kingdom Dial-In #: 08006948057 
United States Dial-In #: 8004518679 
Uruguay Dial-In #: 00040190315 
Venezuela Dial-In #: 08001627182 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120102/f65f09ee/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: meeting.ics
Type: text/calendar
Size: 10959 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120102/f65f09ee/attachment.ics>

From masayag at redhat.com  Mon Jan  2 13:05:29 2012
From: masayag at redhat.com (Moti Asayag)
Date: Mon, 02 Jan 2012 15:05:29 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F018834.1090509@redhat.com>
References: <5f6415ff-f9e7-4f36-b8ef-b6b464505a90@zmail14.collab.prod.int.phx2.redhat.com>	<4F00D265.9070604@redhat.com>
	<4F00F42F.4080405@redhat.com>	<4F016272.5060807@redhat.com>
	<4F0166F8.3000804@redhat.com> <4F017350.5010301@redhat.com>
	<4F0175F1.6060703@redhat.com> <4F018834.1090509@redhat.com>
Message-ID: <4F01AB99.2000603@redhat.com>

On 01/02/2012 12:34 PM, Maor wrote:
> Hi Moti,
> I got a few questions regarding the Task Manager design:
> 
> Do we still need zombie tasks after this change, since I saw the user
> can now end the process from the GUI?
> 
> What should be the behaviour when user will try to end step deleteVolume
> from the GUI? How will it reflect the entire Job?
Task management is not in scope for 3.1 drop 1, only task monitoring,
therefore current behavior is maintained.

> 
> Referencing DB scope, Will step_name and external_system_type will be
> reflected by enums? if they are, maybe its better to use int instead of
> String in those fields.
On discussion held in [1], it was suggested to prefer varchar for enum
fields as early step before using the enum type (postgres 9.1).

I'll update the rest of enum value-based fields to type varchar.

[1] http://lists.ovirt.org/pipermail/engine-devel/2011-December/000258.html


> Would not it be better to declare correlation_id and external_id as UUID
> instead of String.
The correlation id id provided by the client and its structure is not
being enforced. From client PoV, they could set a value of
UUID_Connect-to-local-storage  as the correlation-id.

I'll update the external-id to Guid
> 
> Does the task name is calculated on create, or should it be persisted in
> the DB?
In order to skip the need for resolving the step description on clients,
and to refrain from saving the context of the Step (which entities a
specific step describes), I'll add a field named 'description' (UTF8 by
DB definition) which is evaluated on the creation of the Step and should
be the presentation name of the step (same as appeared in mockups).

> 
> Besides that, looks very good!
> 
> Regards,
> Maor Lipchuk
> 



From masayag at redhat.com  Mon Jan  2 13:40:13 2012
From: masayag at redhat.com (Moti Asayag)
Date: Mon, 02 Jan 2012 08:40:13 -0500 (EST)
Subject: [Engine-devel] Task Manager Design Review
Message-ID: <9da863f0-8de5-4458-9756-1939c2b1ba25@zmail14.collab.prod.int.phx2.redhat.com>

The following meeting has been modified:

Subject: Task Manager Design Review 
Organizer: "Moti Asayag" <masayag at redhat.com> 

Location: "Europe-tlv" <europe-tlv at redhat.com> 
Resources: "Europe-tlv" <europe-tlv at redhat.com> (Europe-tlv) 
Time: Monday, January 2, 2012, 3:30:00 PM - 4:30:00 PM GMT +02:00 Jerusalem
 
Required: engine-devel at ovirt.org; mgoldboi at redhat.com; ykaul at redhat.com; lpeer at redhat.com; ofrenkel at redhat.com; ovedo at redhat.com; gchaplik at redhat.com; mkolesni at redhat.com 
Optional: dfediuck at redhat.com 

*~*~*~*~*~*~*~*~*~*

Please connect to the updated bridge-id: 1814335863.

Sorry for the inconvenience


This is an invitation for a design review for the Task Manager feature. 
Task Manager Detailed design could be found here: 
http://ovirt.org/wiki/Features/TaskManagerDetailed 

You are invited to join. 

== Conference Call Info == 

* Toll Free Dial-In Number (US & Canada): (800) 451-8679 
* International dial-in listed below 
* Conference Code: 1814335863 

If you are not actively participating in the conversation 
please put your line on mute to make it easier for all 
participants to hear. 

*6 -- Mute your line 
#6 -- Unmute your line 

Global Access Numbers Local: 
Australia, Sydney Dial-In #: 0289852326 
Austria, Vienna Dial-In #: 012534978196 
Belgium, Brussels Dial-In #: 027920405 
China Dial-In #: 4006205013 
Denmark, Copenhagen Dial-In #: 32729215 
Finland, Helsinki Dial-In #: 0923194436 
France, Paris Dial-In #: 0170377140 
Germany, Berlin Dial-In #: 030300190579 
Ireland, Dublin Dial-In #: 014367793 
Italy, Milan Dial-In #: 0236269529 
Netherlands, Amsterdam Dial-In #: 0207975872 
Norway, Oslo Dial-In #: 21033188 
Singapore Dial-In #: 64840858 
Spain, Barcelona Dial-In #: 935452328 
Sweden, Stockholm Dial-In #: 0850513770 
Switzerland, Geneva Dial-In #: 0225927881 
United Kingdom Dial-In #: 02078970515 
United Kingdom Dial-In #: 08445790676 
United Kingdom, LocalCall Dial-In #: 08445790678 
United States Dial-In #: 2127295016 


Global Access Numbers Tollfree: 
Argentina Dial-In #: 8004441016 
Australia Dial-In #: 1800337169 
Austria Dial-In #: 0800005898 
Bahamas Dial-In #: 18002054776 
Bahrain Dial-In #: 80004377 
Belgium Dial-In #: 080048325 
Brazil Dial-In #: 08008921002 
Bulgaria Dial-In #: 008001100236 
Chile Dial-In #: 800370228 
Colombia Dial-In #: 018009134033 
Costa Rica Dial-In #: 08000131048 
Cyprus Dial-In #: 80095297 
Czech Republic Dial-In #: 800700318 
Denmark Dial-In #: 80887114 
Dominican Republic Dial-In #: 18887512313 
Estonia Dial-In #: 8000100232 
Finland Dial-In #: 0800117116 
France Dial-In #: 0805632867 
Germany Dial-In #: 8006647541 
Greece Dial-In #: 00800127562 
Hong Kong Dial-In #: 800930349 
Hungary Dial-In #: 0680016796 
Iceland Dial-In #: 8008967 
India Dial-In #: 0008006501533 
Indonesia Dial-In #: 0018030179162 
Ireland Dial-In #: 1800932401 
Israel Dial-In #: 1809462557 
Italy Dial-In #: 800985897 
Jamaica Dial-In #: 18002050328 
Japan Dial-In #: 0120934453 
Korea (South) Dial-In #: 007986517393 
Latvia Dial-In #: 80003339 
Lithuania Dial-In #: 880030479 
Luxembourg Dial-In #: 80026595 
Malaysia Dial-In #: 1800814451 
Mexico Dial-In #: 0018664590915 
New Zealand Dial-In #: 0800888167 
Norway Dial-In #: 80012994 
Panama Dial-In #: 008002269184 
Philippines Dial-In #: 180011100991 
Poland Dial-In #: 008001210187 
Portugal Dial-In #: 800814625 
Russian Federation Dial-In #: 81080028341012 
Saint Kitts and Nevis Dial-In #: 18002059252 
Singapore Dial-In #: 8006162235 
Slovak Republic Dial-In #: 0800001441 
South Africa Dial-In #: 0800981148 
Spain Dial-In #: 800300524 
Sweden Dial-In #: 200896860 
Switzerland Dial-In #: 800650077 
Taiwan Dial-In #: 00801127141 
Thailand Dial-In #: 001800656966 
Trinidad and Tobago Dial-In #: 18002024615 
United Arab Emirates Dial-In #: 8000650591 
United Kingdom Dial-In #: 08006948057 
United States Dial-In #: 8004518679 
Uruguay Dial-In #: 00040190315 
Venezuela Dial-In #: 08001627182 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120102/3cd03a3e/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: meeting.ics
Type: text/calendar
Size: 11179 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120102/3cd03a3e/attachment.ics>

From mkolesni at redhat.com  Mon Jan  2 15:02:22 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Mon, 02 Jan 2012 10:02:22 -0500 (EST)
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4F0175F1.6060703@redhat.com>
Message-ID: <4422359a-c027-4b01-98d1-3fc075954737@zmail14.collab.prod.int.phx2.redhat.com>

Original Message -----
> On 01/02/2012 11:05 AM, Yair Zaslavsky wrote:
> > On 01/02/2012 10:12 AM, Moti Asayag wrote:
> >> On 01/02/2012 09:53 AM, Itamar Heim wrote:
> >>> On 01/02/2012 02:02 AM, Moti Asayag wrote:
> >>> ...
> >>>
> >>>>>
> >>>>> 5. requirements: what about internal tasks originated by the
> >>>>> system (SPM
> >>>>> election, live migration due to load balancing, fencing taking
> >>>>> place,
> >>>>> refresh of users from a directory, etc.)?
> >>>> It was discussed on the meeting held today. It was agreed to
> >>>> report for
> >>>> specific internal action such as VM migration, Host fencing,...
> >>>> but not
> >>>> for the event itself (in order to prevent from flooding the
> >>>> Tasks view
> >>>> from frequent events).
> >>>
> >>> I think SPM election/status is an interesting enough task that if
> >>> it
> >>> happens it should be documented (as well as its process/results).
> >>>
> >>>>
> >>>>>
> >>>>>
> >>>>> 6. STEP table, start_time is "not null" - shouldn't it be
> >>>>> nullable?
> >>>> No. The Step is created in adjust to the execution unit it
> >>>> describes. In
> >>>> order to prevent from additional update, it should be created
> >>>> with
> >>>> status Started and start time.
> >>>
> >>> oh - I thought a job is pre-defined with its steps, then the
> >>> backend
> >>> runs them.
> >>> from your reply i understand the job/steps are just documentation
> >>> of
> >>> what already happened?
> > Actually, also of what is already happening, If I understand
> > correctly.
> Correct.
> See
> http://ovirt.org/wiki/Features/TaskManagerDetailed#Simple_Command_Invocation_Sequence_Diagram
> 

I think, as I said in the design review meeting, that having each job be broken down to 2-3 steps of: validation, execution, finalization is too verbose for the user of the UI.
I believe this can all be encapsulated in a status field for the job.

Of course, steps which represent a "child job" invocation are still logically perfect - the parent job will be in state "executing" and the child job will have it's own representation with the correct state.

> > 
> >> Correct. The Command implementation will determine by code when to
> >> add a
> >> step and how to decide it ended. A default implementation is
> >> provided
> >> for all commands, except those which specified on the requirements
> >> to be
> >> more detailed.
> >>
> >> Doing the other way (let the flow definition execute the steps)
> >> means
> >> implementing a business process manager (or using existing one -
> >> such as
> >> jBPM). I don't think the backend as implemented now is capable of
> >> adjusting to it and letting external service orchestrate the flow.
> >>
> 


Regards,
Mike


From ofrenkel at redhat.com  Mon Jan  2 16:25:44 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Mon, 02 Jan 2012 11:25:44 -0500 (EST)
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <4422359a-c027-4b01-98d1-3fc075954737@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <8cdc36ef-4343-4c1b-9f8e-056e76758c34@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Mike Kolesnik" <mkolesni at redhat.com>
> To: "Moti Asayag" <masayag at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Monday, January 2, 2012 5:02:22 PM
> Subject: Re: [Engine-devel] Task Manager Design Review
> 
> Original Message -----
> > On 01/02/2012 11:05 AM, Yair Zaslavsky wrote:
> > > On 01/02/2012 10:12 AM, Moti Asayag wrote:
> > >> On 01/02/2012 09:53 AM, Itamar Heim wrote:
> > >>> On 01/02/2012 02:02 AM, Moti Asayag wrote:
> > >>> ...
> > >>>
> > >>>>>
> > >>>>> 5. requirements: what about internal tasks originated by the
> > >>>>> system (SPM
> > >>>>> election, live migration due to load balancing, fencing
> > >>>>> taking
> > >>>>> place,
> > >>>>> refresh of users from a directory, etc.)?
> > >>>> It was discussed on the meeting held today. It was agreed to
> > >>>> report for
> > >>>> specific internal action such as VM migration, Host
> > >>>> fencing,...
> > >>>> but not
> > >>>> for the event itself (in order to prevent from flooding the
> > >>>> Tasks view
> > >>>> from frequent events).
> > >>>
> > >>> I think SPM election/status is an interesting enough task that
> > >>> if
> > >>> it
> > >>> happens it should be documented (as well as its
> > >>> process/results).
> > >>>
> > >>>>
> > >>>>>
> > >>>>>
> > >>>>> 6. STEP table, start_time is "not null" - shouldn't it be
> > >>>>> nullable?
> > >>>> No. The Step is created in adjust to the execution unit it
> > >>>> describes. In
> > >>>> order to prevent from additional update, it should be created
> > >>>> with
> > >>>> status Started and start time.
> > >>>
> > >>> oh - I thought a job is pre-defined with its steps, then the
> > >>> backend
> > >>> runs them.
> > >>> from your reply i understand the job/steps are just
> > >>> documentation
> > >>> of
> > >>> what already happened?
> > > Actually, also of what is already happening, If I understand
> > > correctly.
> > Correct.
> > See
> > http://ovirt.org/wiki/Features/TaskManagerDetailed#Simple_Command_Invocation_Sequence_Diagram
> > 
> 
> I think, as I said in the design review meeting, that having each job
> be broken down to 2-3 steps of: validation, execution, finalization
> is too verbose for the user of the UI.
> I believe this can all be encapsulated in a status field for the job.
> 
> Of course, steps which represent a "child job" invocation are still
> logically perfect - the parent job will be in state "executing" and
> the child job will have it's own representation with the correct
> state.
> 

i like this one less, because some of the state will be in steps and some in status,
it is more confusing for the user and for the developer 
(for example while implementing a new command, a question may rise if the next call should be step or status),
and i don't think it adds anything.

> > > 
> > >> Correct. The Command implementation will determine by code when
> > >> to
> > >> add a
> > >> step and how to decide it ended. A default implementation is
> > >> provided
> > >> for all commands, except those which specified on the
> > >> requirements
> > >> to be
> > >> more detailed.
> > >>
> > >> Doing the other way (let the flow definition execute the steps)
> > >> means
> > >> implementing a business process manager (or using existing one -
> > >> such as
> > >> jBPM). I don't think the backend as implemented now is capable
> > >> of
> > >> adjusting to it and letting external service orchestrate the
> > >> flow.
> > >>
> > 
> 
> 
> Regards,
> Mike
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From iheim at redhat.com  Mon Jan  2 18:58:04 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 02 Jan 2012 20:58:04 +0200
Subject: [Engine-devel] Task Manager Design Review
In-Reply-To: <8cdc36ef-4343-4c1b-9f8e-056e76758c34@zmail07.collab.prod.int.phx2.redhat.com>
References: <8cdc36ef-4343-4c1b-9f8e-056e76758c34@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F01FE3C.4090604@redhat.com>

On 01/02/2012 06:25 PM, Omer Frenkel wrote:
>
>
> ----- Original Message -----
>> From: "Mike Kolesnik"<mkolesni at redhat.com>
>> To: "Moti Asayag"<masayag at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Monday, January 2, 2012 5:02:22 PM
>> Subject: Re: [Engine-devel] Task Manager Design Review
>>
>> Original Message -----
>>> On 01/02/2012 11:05 AM, Yair Zaslavsky wrote:
>>>> On 01/02/2012 10:12 AM, Moti Asayag wrote:
>>>>> On 01/02/2012 09:53 AM, Itamar Heim wrote:
>>>>>> On 01/02/2012 02:02 AM, Moti Asayag wrote:
>>>>>> ...
>>>>>>
>>>>>>>>
>>>>>>>> 5. requirements: what about internal tasks originated by the
>>>>>>>> system (SPM
>>>>>>>> election, live migration due to load balancing, fencing
>>>>>>>> taking
>>>>>>>> place,
>>>>>>>> refresh of users from a directory, etc.)?
>>>>>>> It was discussed on the meeting held today. It was agreed to
>>>>>>> report for
>>>>>>> specific internal action such as VM migration, Host
>>>>>>> fencing,...
>>>>>>> but not
>>>>>>> for the event itself (in order to prevent from flooding the
>>>>>>> Tasks view
>>>>>>> from frequent events).
>>>>>>
>>>>>> I think SPM election/status is an interesting enough task that
>>>>>> if
>>>>>> it
>>>>>> happens it should be documented (as well as its
>>>>>> process/results).
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 6. STEP table, start_time is "not null" - shouldn't it be
>>>>>>>> nullable?
>>>>>>> No. The Step is created in adjust to the execution unit it
>>>>>>> describes. In
>>>>>>> order to prevent from additional update, it should be created
>>>>>>> with
>>>>>>> status Started and start time.
>>>>>>
>>>>>> oh - I thought a job is pre-defined with its steps, then the
>>>>>> backend
>>>>>> runs them.
>>>>>> from your reply i understand the job/steps are just
>>>>>> documentation
>>>>>> of
>>>>>> what already happened?
>>>> Actually, also of what is already happening, If I understand
>>>> correctly.
>>> Correct.
>>> See
>>> http://ovirt.org/wiki/Features/TaskManagerDetailed#Simple_Command_Invocation_Sequence_Diagram
>>>
>>
>> I think, as I said in the design review meeting, that having each job
>> be broken down to 2-3 steps of: validation, execution, finalization
>> is too verbose for the user of the UI.
>> I believe this can all be encapsulated in a status field for the job.
>>
>> Of course, steps which represent a "child job" invocation are still
>> logically perfect - the parent job will be in state "executing" and
>> the child job will have it's own representation with the correct
>> state.
>>
>
> i like this one less, because some of the state will be in steps and some in status,
> it is more confusing for the user and for the developer
> (for example while implementing a new command, a question may rise if the next call should be step or status),
> and i don't think it adds anything.

but every external step may have sub status changes (say, disk creation 
progress bar, or delete vs. wipe after delete steps, etc.)


From jhenner at redhat.com  Tue Jan  3 13:10:49 2012
From: jhenner at redhat.com (Jaroslav Henner)
Date: Tue, 03 Jan 2012 14:10:49 +0100
Subject: [Engine-devel] Removing class TimeLeaseVmPoolMapDAOHibernate
Message-ID: <4F02FE59.3040206@redhat.com>

Hi.

I'm a automation tester in RHEVM QE team. We want to raise code coverage 
and it seems there is plenty of dead code we cannot cover with blackbox 
tests. One example is everything around TimeLeaseVmPoolMapDAOHibernate.

So I prepared a patch removing these classes. It builds fine (I tried to 
build it with command:
mvn2 install -Pgwt-admin,gwt-user,dep -DskipTests=true 
-Dgwt.userAgent=gecko1_8


The stat of my patch shows that much of lines were removed so it will 
raise the coverage number relatively much. If no one has complains, 
comments or objections, I'll post this patch to gerrit.


commit 304c606cc9919d7add073dfdc8d0761ea2c99636
Author: Jaroslav Henner <jhenner at redhat.com>
Date:   Mon Jan 2 14:54:31 2012 +0100

     Remove TimeLeapseVmPool.

  backend/manager/dbscripts/create_tables.sql        |   10 -
  backend/manager/dbscripts/create_views.sql         |    3 +-
  backend/manager/dbscripts/vm_pools_sp.sql          |  194 ----------
  .../bll/AttachAdGroupTimeLeasedPoolCommand.java    |   39 --
  .../bll/AttachUserToTimeLeasedPoolCommand.java     |   42 ---
  .../DetachAdGroupFromTimeLeasedPoolCommand.java    |   85 -----
  .../bll/DetachUserFromTimeLeasedPoolCommand.java   |   81 ----
  ...GetAdGroupsAttachedToTimeLeasedVmPoolQuery.java |   17 -
  .../core/bll/GetAllVmPoolsAttachedToUserQuery.java |   21 -
  .../bll/GetTimeLeasedUsersByVmPoolIdQuery.java     |   18 -
  .../core/bll/InitBackendServicesOnStartupBean.java |    2 -
  .../engine/core/bll/RemoveVmFromPoolCommand.java   |    7 +-
  .../engine/core/bll/TimeLeasedVmPoolManager.java   |  390 
--------------------
  .../bll/UpdateAdGroupTimeLeasedPoolCommand.java    |   30 --
  .../bll/UpdateUserToTimeLeasedPoolCommand.java     |   27 --
  .../ADElementTimeLeasedVmPoolParametersBase.java   |   27 --
  ...tachAdGroupTimeLeasedPoolCommandParameters.java |   27 --
  .../AttachUserToTimeLeasedPoolParameters.java      |   30 --
  .../common/action/UpdateUserVmPoolParameters.java  |   27 --
  .../common/action/VdcActionParametersBase.java     |    8 +-
  .../businessentities/time_lease_vm_pool_map.java   |  190 ----------
  .../time_lease_vm_pool_map_id.java                 |   63 ----
  .../java/org/ovirt/engine/core/dao/AdGroupDAO.java |    9 -
  .../engine/core/dao/AdGroupDAODbFacadeImpl.java    |   11 -
  .../engine/core/dao/AdGroupDAOHibernateImpl.java   |    8 -
  .../java/org/ovirt/engine/core/dao/DbUserDAO.java  |    9 -
  .../engine/core/dao/DbUserDAODbFacadeImpl.java     |   11 -
  .../engine/core/dao/DbUserDAOWrapperImpl.java      |    6 -
  .../java/org/ovirt/engine/core/dao/VmPoolDAO.java  |   11 -
  .../engine/core/dao/VmPoolDAODbFacadeImpl.java     |   65 ----
  .../engine/core/dao/VmPoolDAOHibernateImpl.java    |   30 --
  .../TimeLeaseVmPoolMapDAOHibernateImpl.java        |   11 -
  .../org/ovirt/engine/core/dao/BaseDAOTestCase.java |    2 -
  .../org/ovirt/engine/core/dao/VmPoolDAOTest.java   |   61 ---
  .../modules/dal/src/test/resources/fixtures.xml    |   15 -
  .../main/java/org/ovirt/engine/SharedGwt.gwt.xml   |    3 -
  36 files changed, 5 insertions(+), 1585 deletions(-)


Note a bug about (not only) this
https://bugzilla.redhat.com/show_bug.cgi?id=735997
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Remove-TimeLeapseVmPool.patch
Type: text/x-patch
Size: 89292 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120103/7a04142a/attachment.bin>

From emesika at redhat.com  Tue Jan  3 21:16:39 2012
From: emesika at redhat.com (Eli Mesika)
Date: Tue, 03 Jan 2012 16:16:39 -0500 (EST)
Subject: [Engine-devel] Stable Device Addresses HLD/LLD for tomorrow
	meeting (15:00 GMT+1)
In-Reply-To: <4F00CB42.4070404@redhat.com>
Message-ID: <374700cb-eb47-48f3-afba-06bf306eb413@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Eli Mesika" <emesika at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Sunday, January 1, 2012 11:08:18 PM
> Subject: Re: [Engine-devel] Stable Device Addresses HLD/LLD for tomorrow meeting (15:00 GMT+1)
> 
> On 12/28/2011 06:52 PM, Eli Mesika wrote:
> > High level design :
> > http://www.ovirt.org/wiki/Features/Design/StableDeviceAddresses
> >
> > Low level design :
> > http://www.ovirt.org/wiki/Features/Design/DetailedStableDeviceAddresses
> > (probably will be modified until the meeting)
> 
> 
> 1. " hash -- holds the md5 like encryption indicating a change "
> 
> I don't think "encryption" is relevant here.
right , replaced with hash
> 
> 2. VMOldInfoManager
> 
> how will we know/flag when we expect it can be deleted?
> ("Old" is relative. what if you will need to create another OldOld
> class? maybe something more specific like 3_0, so you know you can
> delete it when you no longer have 3_0 clusters supported?)
Done
> 
> 3. "Addresses are kept in the database as a plain-text strings"
> at some point we will want to expose this to users. maybe a better
> representation would be in order (could be handled later as part of
> upgrade process i guess).
will ne handled later (when needed)
> 
> 4. video cards
> you mention only spice - what happens to VNC monitors?
Handled already, look on HLD for
 'qxl' - spice device,  'cirrus' - vnc device

> 
> 5. floppy/cd-rom
> expect there will be more than one cd-rom (looks as the most probably
> solution to user data injection afaiu)
sure, we assume that those can be multiple
> 
> 6. virtio-serial
> you mention it in the beginning, but not afterwards wrt
> implementation
> details. there may be multiple of these (qemu-ga, ovirt-guest-agent,
> spice-vd-agent?)

I think those will be handled as (un-managed) vm-device abd by definition can have multiple entries 

> 
> 
> 
> 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 


From iheim at redhat.com  Wed Jan  4 01:46:51 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 04 Jan 2012 03:46:51 +0200
Subject: [Engine-devel] Stable Device Addresses HLD/LLD for tomorrow
 meeting (15:00 GMT+1)
In-Reply-To: <374700cb-eb47-48f3-afba-06bf306eb413@zmail13.collab.prod.int.phx2.redhat.com>
References: <374700cb-eb47-48f3-afba-06bf306eb413@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F03AF8B.9010909@redhat.com>

Dor/Alon - a question below.
more comments inline.

On 01/03/2012 11:16 PM, Eli Mesika wrote:
>> From: "Itamar Heim"<iheim at redhat.com>
...

>> Subject: Re: [Engine-devel] Stable Device Addresses HLD/LLD for tomorrow meeting (15:00 GMT+1)
>>
>> On 12/28/2011 06:52 PM, Eli Mesika wrote:
>>> High level design :
>>> http://www.ovirt.org/wiki/Features/Design/StableDeviceAddresses
>>>
>>> Low level design :
>>> http://www.ovirt.org/wiki/Features/Design/DetailedStableDeviceAddresses
>>> (probably will be modified until the meeting)
...

>> 4. video cards
>> you mention only spice - what happens to VNC monitors?
> Handled already, look on HLD for
>   'qxl' - spice device,  'cirrus' - vnc device

iirc, there is supposed to be something around being able to use both 
vnc or spice (mutually exclusive) without rebooting the guest.
i.e., user would be able to connect with either one of these protocols
Dor/Alon?


another item (shouldn't affect design i hope, but just in case):
I just heard spice multi monitor support for linux guests may work 
differently than for windows.
in windows we expose 4 video cards.
for linux we may need to create one card with 4 times the memory.

...
>>
>> 6. virtio-serial
>> you mention it in the beginning, but not afterwards wrt
>> implementation
>> details. there may be multiple of these (qemu-ga, ovirt-guest-agent,
>> spice-vd-agent?)
>
> I think those will be handled as (un-managed) vm-device abd by definition can have multiple entries

but having/not having spice configured would affect the need for one of 
them (spice-vd-agent)
and cluster compatibility would affect the need for the qemu-ga one (not 
sure if that would be configurable or assumed as default for all guests)


From dfediuck at redhat.com  Wed Jan  4 08:13:00 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Wed, 04 Jan 2012 10:13 +0200
Subject: [Engine-devel] Agenda for bi-weekly oVirt engine core meeting (Wed
	Jan. 4th)
Message-ID: <2504045.jVhLVzBhks@doronf>

These are the topics that we are planning to discuss in the meeting: 

1. Yair Zaslavsky, presenting DB scheme verification (3.1 feature).
2. Muli Salem/Doron Fediuck, presenting pre-started VM's functionality (3.1 feature).
3. Muli Salem/Doron Fediuck, presenting SPM priority functionality (3.1 feature).

As time permits, feel free to come up with additional subjects.
Reminding you all we're using Bridge ID 1814335863.

-- 

/d

"2B | !2B = FF"



From dlaor at redhat.com  Wed Jan  4 14:28:29 2012
From: dlaor at redhat.com (Dor Laor)
Date: Wed, 04 Jan 2012 16:28:29 +0200
Subject: [Engine-devel] Stable Device Addresses HLD/LLD for tomorrow
 meeting (15:00 GMT+1)
In-Reply-To: <4F03AF8B.9010909@redhat.com>
References: <374700cb-eb47-48f3-afba-06bf306eb413@zmail13.collab.prod.int.phx2.redhat.com>
	<4F03AF8B.9010909@redhat.com>
Message-ID: <4F04620D.8040106@redhat.com>

On 01/04/2012 03:46 AM, Itamar Heim wrote:
> Dor/Alon - a question below.
> more comments inline.
>
> On 01/03/2012 11:16 PM, Eli Mesika wrote:
>>> From: "Itamar Heim"<iheim at redhat.com>
> ...
>
>>> Subject: Re: [Engine-devel] Stable Device Addresses HLD/LLD for
>>> tomorrow meeting (15:00 GMT+1)
>>>
>>> On 12/28/2011 06:52 PM, Eli Mesika wrote:
>>>> High level design :
>>>> http://www.ovirt.org/wiki/Features/Design/StableDeviceAddresses
>>>>
>>>> Low level design :
>>>> http://www.ovirt.org/wiki/Features/Design/DetailedStableDeviceAddresses
>>>> (probably will be modified until the meeting)
> ...
>
>>> 4. video cards
>>> you mention only spice - what happens to VNC monitors?
>> Handled already, look on HLD for
>> 'qxl' - spice device, 'cirrus' - vnc device
>
> iirc, there is supposed to be something around being able to use both
> vnc or spice (mutually exclusive) without rebooting the guest.
> i.e., user would be able to connect with either one of these protocols
> Dor/Alon?

There is a way to do that and it may be desirable on some situations. 
(why it is related to stable device addresses?)

>
>
> another item (shouldn't affect design i hope, but just in case):
> I just heard spice multi monitor support for linux guests may work
> differently than for windows.
> in windows we expose 4 video cards.
> for linux we may need to create one card with 4 times the memory.

While I don't know if that's possible, what's the relation to stable pci 
addresses? ovirt just need to support all mode and allow configurable 
number of pci cards, stable pci slots, multi-function devices and 
specify the amount of ram or any other properties specific devices will 
need.

>
> ...
>>>
>>> 6. virtio-serial
>>> you mention it in the beginning, but not afterwards wrt
>>> implementation
>>> details. there may be multiple of these (qemu-ga, ovirt-guest-agent,
>>> spice-vd-agent?)
>>
>> I think those will be handled as (un-managed) vm-device abd by
>> definition can have multiple entries
>
> but having/not having spice configured would affect the need for one of
> them (spice-vd-agent)
> and cluster compatibility would affect the need for the qemu-ga one (not
> sure if that would be configurable or assumed as default for all guests)



From dfediuck at redhat.com  Wed Jan  4 15:37:40 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Wed, 04 Jan 2012 17:37:40 +0200
Subject: [Engine-devel] MOM: oVirt engine core meeting (Wed Jan. 4th)
Message-ID: <36835923.dTYE9mBE0a@doronf>

1. DB scheme verification presented by Yair.
Notes: none.

2. Pre-started VM's functionality presented by Doron.
Notes:
- Consider adding the API the option to define pre-started VM's number
as a percentage of the pool size. This will make sure that when pool size
changes, the number of pre-started VM's is still relevant.

3. SPM priority functionality presented by Doron.
Notes:
- New related feature needed; SPM resignation. This will allow
the admin to make sure an SPM host can resign so another host
will be elected to become an SPM. This should be further discussed.

Thanks for everyone for showing up. Cya in 2 weeks!
-- 

/d

"Willyoupleasehelpmefixmykeyboard?Thespacebarisbroken!"



From alevy at redhat.com  Wed Jan  4 17:23:17 2012
From: alevy at redhat.com (Alon Levy)
Date: Wed, 4 Jan 2012 19:23:17 +0200
Subject: [Engine-devel] Stable Device Addresses HLD/LLD for tomorrow
 meeting (15:00 GMT+1)
In-Reply-To: <4F04620D.8040106@redhat.com>
References: <374700cb-eb47-48f3-afba-06bf306eb413@zmail13.collab.prod.int.phx2.redhat.com>
	<4F03AF8B.9010909@redhat.com> <4F04620D.8040106@redhat.com>
Message-ID: <20120104172317.GA14124@garlic.tlv.redhat.com>

On Wed, Jan 04, 2012 at 04:28:29PM +0200, Dor Laor wrote:
> On 01/04/2012 03:46 AM, Itamar Heim wrote:
> >Dor/Alon - a question below.
> >more comments inline.
> >
> >On 01/03/2012 11:16 PM, Eli Mesika wrote:
> >>>From: "Itamar Heim"<iheim at redhat.com>
> >...
> >
> >>>Subject: Re: [Engine-devel] Stable Device Addresses HLD/LLD for
> >>>tomorrow meeting (15:00 GMT+1)
> >>>
> >>>On 12/28/2011 06:52 PM, Eli Mesika wrote:
> >>>>High level design :
> >>>>http://www.ovirt.org/wiki/Features/Design/StableDeviceAddresses
> >>>>
> >>>>Low level design :
> >>>>http://www.ovirt.org/wiki/Features/Design/DetailedStableDeviceAddresses
> >>>>(probably will be modified until the meeting)
> >...
> >
> >>>4. video cards
> >>>you mention only spice - what happens to VNC monitors?
> >>Handled already, look on HLD for
> >>'qxl' - spice device, 'cirrus' - vnc device
> >
> >iirc, there is supposed to be something around being able to use both
> >vnc or spice (mutually exclusive) without rebooting the guest.
> >i.e., user would be able to connect with either one of these protocols
> >Dor/Alon?
> 
> There is a way to do that and it may be desirable on some
> situations. (why it is related to stable device addresses?)

(missed this email somehow). It is possible, Gerd made it so but it is
not good for performance - spice renders on the server all the time for
it to work (but I don't have any numbers).

> 
> >
> >
> >another item (shouldn't affect design i hope, but just in case):
> >I just heard spice multi monitor support for linux guests may work
> >differently than for windows.
> >in windows we expose 4 video cards.
> >for linux we may need to create one card with 4 times the memory.
> 
> While I don't know if that's possible, what's the relation to stable
> pci addresses? ovirt just need to support all mode and allow
> configurable number of pci cards, stable pci slots, multi-function
> devices and specify the amount of ram or any other properties
> specific devices will need.
> 
> >
> >...
> >>>
> >>>6. virtio-serial
> >>>you mention it in the beginning, but not afterwards wrt
> >>>implementation
> >>>details. there may be multiple of these (qemu-ga, ovirt-guest-agent,
> >>>spice-vd-agent?)
> >>
> >>I think those will be handled as (un-managed) vm-device abd by
> >>definition can have multiple entries
> >
> >but having/not having spice configured would affect the need for one of
> >them (spice-vd-agent)
> >and cluster compatibility would affect the need for the qemu-ga one (not
> >sure if that would be configurable or assumed as default for all guests)
> 


From iheim at redhat.com  Thu Jan  5 20:28:28 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 05 Jan 2012 22:28:28 +0200
Subject: [Engine-devel] FOSDEM sessions
Message-ID: <4F0607EC.3040902@redhat.com>

fyi we got the following sessions in the FOSDEM[1] Open Source 
Virtualization and Cloud devroom.

if you are planning to be in FOSDEM drop us an email if you would want 
to meet, discuss, catch up, etc.

1. Virtualization Management the oVirt way - Introducing oVirt (Itamar
    Heim)
2. VDSM - The oVirt Node Management Agent (Federico Simoncelli)
3. Open Virtualization ? Engine Core: Internals and Infrastructure
    (Omer Frenkel)

apart of it there are also some kvm sessions in the hypervisor main track.

Hope to meet you there,
    Itamar

[1] http://fosdem.org/2012/


From lhornyak at redhat.com  Fri Jan  6 13:58:01 2012
From: lhornyak at redhat.com (Laszlo Hornyak)
Date: Fri, 06 Jan 2012 08:58:01 -0500 (EST)
Subject: [Engine-devel] servlet api version
In-Reply-To: <ca07430b-2452-46b7-a6d7-518ece40547e@zmail01.collab.prod.int.phx2.redhat.com>
Message-ID: <da9e567a-9193-407b-8abe-925de5647d52@zmail01.collab.prod.int.phx2.redhat.com>

Hi,

What is the agreement on the servlet api version?

I ran into some problems this week and looked around in the code. I have found that the root pom.xml defines servlet 2.3. The backend is ok with this, but then the frontend uses it's own dependency. It actually uses gwt's library instead of the javax.servlet:servlet-api artifact and it is a servlet 2.4 api. It also builds on the setCharacterEncoding(String) (see http://tomcat.apache.org/tomcat-5.5-doc/servletapi/javax/servlet/ServletResponse.html#setCharacterEncoding(java.lang.String) ) method introduced in servlet 2.4. Therefore oVirt is not going to run with servlet 2.3 in runtime either.

my patch:
http://gerrit.ovirt.org/#change,877

Laszlo


From danken at redhat.com  Sun Jan  8 13:08:14 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Sun, 8 Jan 2012 15:08:14 +0200
Subject: [Engine-devel] getDeviceList should report partioned devices, too
Message-ID: <20120108130813.GC24375@redhat.com>

Hi Lists,

One cannot create a PV on a partitioned device, and therefor such
devices where not reported to Engine. This proved surprising to users
who woder where their LUN disappeared.

Vdsm should report all devices, and ovirt-engine should mark partitioned
devices as unworthy of a PV. In the future, Vdsm may allow to forcefully
remove a partition table from a device, to make it usable as a PV.

Douglas (CCed) would take resposibilty on the Vdsm side. Initial patch at
http://gerrit.ovirt.org/944 sports a backword-compatible API. Who's taking this
on Engine? This involves GUI, too, as partitioned devices should probably be
displayed greyed-out.

Dan.


From oschreib at redhat.com  Sun Jan  8 16:08:24 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Sun, 08 Jan 2012 11:08:24 -0500 (EST)
Subject: [Engine-devel] New oVirt-engine RPMs available
Message-ID: <031401ccce1f$c146c350$43d449f0$@redhat.com>

Hi all,

We've just finished uploading a new set of oVirt-engine rpms into
ovirt.org.
Beside the usual bug fixes, in this version we're introducing the brand
new ovirt-engine-cli and ovirt-engine-sdk rpms.

In order to install the RPMs, please follow the instructions described in
http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
For more info about the CLI and SDK please visit
http://www.ovirt.org/wiki/SDK and http://www.ovirt.org/wiki/CLI

Feedback, comments and bug reports are always welcome.

The oVirt-Engine team




From ykaul at redhat.com  Mon Jan  9 07:36:34 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Mon, 09 Jan 2012 09:36:34 +0200
Subject: [Engine-devel] Is there a 'Can run this VM?' query
Message-ID: <4F0A9902.6030905@redhat.com>

Beaker automated test system (http://fedoraproject.org/wiki/QA/Beaker) 
would like to integrate with the project, via the REST API.
Their main concern is that they'll provision VMs who would not be able 
to run on the ovirt setup.
I think there are two cases here:
1. Provisioning a VM with X vCPUs, while there isn't a host available 
with such number of pCPUs (or similar in memory) - this is easier, as it 
can be 'calculated' from the setup beforehand and we can avoid such 
provisioning.
2. More difficult - in runtime, the hosts are over-committed and we 
cannot run a VM - can we, before trying to run a VM, ask if it's 
feasible? I know it's a point in time, and may be wrong in the next 
second, but if we go 'no' as an answer, it already provides good 
information to Beaker to continue and look elsewhere.
Y.



From mkublin at redhat.com  Mon Jan  9 09:21:54 2012
From: mkublin at redhat.com (Michael Kublin)
Date: Mon, 09 Jan 2012 04:21:54 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <142ba04f-2c62-4e6e-9d01-e6de1ea52ea2@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <a326aab2-28ec-4147-9265-9554cc356077@zmail14.collab.prod.int.phx2.redhat.com>

Hi, the follow link is providing a low level design for HotPlug/HotUnplug feature : http://www.ovirt.org/wiki/Features/DetailedHotPlug .

The feature is simple and design is short

Regards Michael


From ykaul at redhat.com  Mon Jan  9 09:45:55 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Mon, 09 Jan 2012 11:45:55 +0200
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <a326aab2-28ec-4147-9265-9554cc356077@zmail14.collab.prod.int.phx2.redhat.com>
References: <a326aab2-28ec-4147-9265-9554cc356077@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F0AB753.80105@redhat.com>

On 01/09/2012 11:21 AM, Michael Kublin wrote:
> Hi, the follow link is providing a low level design for HotPlug/HotUnplug feature : http://www.ovirt.org/wiki/Features/DetailedHotPlug .
>
> The feature is simple and design is short
>
> Regards Michael
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

1. Corrected some typos, spelling, backend->engine, 'REST API' -> 
'API',  etc.  (didn't fix ' storage procedures' - but I think you've 
meant 'stored procedures' ?).
2. Missing/questions:
- Permissions? Quota?
- Explain how disk is created in the first place.
- Database (tables, etc.)
- Which cluster level is required?
- Is it an async or sync task?
- Can you plug a system disk?
- Can you unplug multiple at a time?
- What happens when you plug too many?
- How do you determine where (PCI bus-wise) to plug them?
- Any CanDoAction to allow/disallow plug/unplug from specific systems?
- I suspect we'd be happier with some agent cooperation before 
unplugging - is this done by QEMU? Not detailed anywhere.
- Link to KVM/QEMU feature description for it, if such exist, would be nice.
- Does it affect taking a snapshot? Live or not?
- Does it affect exporting a VM? I reckon you export with all disks, 
with their plug/unplug status?

Y.




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120109/e4985bab/attachment.html>

From dlaor at redhat.com  Mon Jan  9 10:17:39 2012
From: dlaor at redhat.com (Dor Laor)
Date: Mon, 09 Jan 2012 12:17:39 +0200
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <4F0AB753.80105@redhat.com>
References: <a326aab2-28ec-4147-9265-9554cc356077@zmail14.collab.prod.int.phx2.redhat.com>
	<4F0AB753.80105@redhat.com>
Message-ID: <4F0ABEC3.8060308@redhat.com>

On 01/09/2012 11:45 AM, Yaniv Kaul wrote:
> On 01/09/2012 11:21 AM, Michael Kublin wrote:
>> Hi, the follow link is providing a low level design for HotPlug/HotUnplug feature :http://www.ovirt.org/wiki/Features/DetailedHotPlug  .
>>
>> The feature is simple and design is short
>>
>> Regards Michael
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
> 1. Corrected some typos, spelling, backend->engine, 'REST API' -> 'API',
> etc. (didn't fix ' storage procedures' - but I think you've meant
> 'stored procedures' ?).
> 2. Missing/questions:
> - Permissions? Quota?
> - Explain how disk is created in the first place.
> - Database (tables, etc.)
> - Which cluster level is required?
> - Is it an async or sync task?
> - Can you plug a system disk?
> - Can you unplug multiple at a time?
> - What happens when you plug too many?
> - How do you determine where (PCI bus-wise) to plug them?
> - Any CanDoAction to allow/disallow plug/unplug from specific systems?
> - I suspect we'd be happier with some agent cooperation before
> unplugging - is this done by QEMU? Not detailed anywhere.
> - Link to KVM/QEMU feature description for it, if such exist, would be nice.
> - Does it affect taking a snapshot? Live or not?
> - Does it affect exporting a VM? I reckon you export with all disks,
> with their plug/unplug status?

In addition:
  - will you allow it during live migration (qemu allows it)?
  - Are you expecting events to pop?
  - You call it 'hotplug' but mention only disk device, what about
    memory/cpu/ any other pci card?
  - How do you define a system disk? Some VM may boot from
    pxe/nfsroot/etc
  - It should be a nice to have to check if the guest mounts any FS
    within the disk to be un hot pluged and warn the user

>
> Y.
>
>
>
>
>
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From iheim at redhat.com  Mon Jan  9 10:32:35 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 09 Jan 2012 12:32:35 +0200
Subject: [Engine-devel] Is there a 'Can run this VM?' query
In-Reply-To: <4F0A9902.6030905@redhat.com>
References: <4F0A9902.6030905@redhat.com>
Message-ID: <4F0AC243.2090800@redhat.com>

On 01/09/2012 09:36 AM, Yaniv Kaul wrote:
> Beaker automated test system (http://fedoraproject.org/wiki/QA/Beaker)
> would like to integrate with the project, via the REST API.
> Their main concern is that they'll provision VMs who would not be able
> to run on the ovirt setup.
> I think there are two cases here:
> 1. Provisioning a VM with X vCPUs, while there isn't a host available
> with such number of pCPUs (or similar in memory) - this is easier, as it
> can be 'calculated' from the setup beforehand and we can avoid such
> provisioning.
> 2. More difficult - in runtime, the hosts are over-committed and we
> cannot run a VM - can we, before trying to run a VM, ask if it's
> feasible? I know it's a point in time, and may be wrong in the next
> second, but if we go 'no' as an answer, it already provides good
> information to Beaker to continue and look elsewhere.

what about using the new quota feature to define a quota which will not 
overload the server and serve as a measure of SLA for them?


From ykaul at redhat.com  Mon Jan  9 10:34:31 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Mon, 09 Jan 2012 12:34:31 +0200
Subject: [Engine-devel] Is there a 'Can run this VM?' query
In-Reply-To: <4F0AC243.2090800@redhat.com>
References: <4F0A9902.6030905@redhat.com> <4F0AC243.2090800@redhat.com>
Message-ID: <4F0AC2B7.8020408@redhat.com>

On 01/09/2012 12:32 PM, Itamar Heim wrote:
> On 01/09/2012 09:36 AM, Yaniv Kaul wrote:
>> Beaker automated test system (http://fedoraproject.org/wiki/QA/Beaker)
>> would like to integrate with the project, via the REST API.
>> Their main concern is that they'll provision VMs who would not be able
>> to run on the ovirt setup.
>> I think there are two cases here:
>> 1. Provisioning a VM with X vCPUs, while there isn't a host available
>> with such number of pCPUs (or similar in memory) - this is easier, as it
>> can be 'calculated' from the setup beforehand and we can avoid such
>> provisioning.
>> 2. More difficult - in runtime, the hosts are over-committed and we
>> cannot run a VM - can we, before trying to run a VM, ask if it's
>> feasible? I know it's a point in time, and may be wrong in the next
>> second, but if we go 'no' as an answer, it already provides good
>> information to Beaker to continue and look elsewhere.
>
> what about using the new quota feature to define a quota which will 
> not overload the server and serve as a measure of SLA for them?

Yes, that would be helpful, when available. Already sent them the 
description for this feature.
Y.


From ofrenkel at redhat.com  Mon Jan  9 12:06:58 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Mon, 09 Jan 2012 07:06:58 -0500 (EST)
Subject: [Engine-devel] Is there a 'Can run this VM?' query
In-Reply-To: <4F0A9902.6030905@redhat.com>
Message-ID: <bb29413e-2164-4fb8-876e-692bbc3429c6@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Yaniv Kaul" <ykaul at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Monday, January 9, 2012 9:36:34 AM
> Subject: [Engine-devel] Is there a 'Can run this VM?' query
> 
> Beaker automated test system
> (http://fedoraproject.org/wiki/QA/Beaker)
> would like to integrate with the project, via the REST API.
> Their main concern is that they'll provision VMs who would not be
> able
> to run on the ovirt setup.
> I think there are two cases here:
> 1. Provisioning a VM with X vCPUs, while there isn't a host available
> with such number of pCPUs (or similar in memory) - this is easier, as
> it
> can be 'calculated' from the setup beforehand and we can avoid such
> provisioning.
> 2. More difficult - in runtime, the hosts are over-committed and we
> cannot run a VM - can we, before trying to run a VM, ask if it's
> feasible? I know it's a point in time, and may be wrong in the next
> second, but if we go 'no' as an answer, it already provides good
> information to Beaker to continue and look elsewhere.
> Y.
>

there is no such query currently,
the answer to this question actually is lots of other questions (status, storage, host availability and more)
BUT i think its rather easy creating such a query (though caller need to remember it might be a long one)
 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From rgolan at redhat.com  Mon Jan  9 12:21:29 2012
From: rgolan at redhat.com (Roy Golan)
Date: Mon, 09 Jan 2012 07:21:29 -0500 (EST)
Subject: [Engine-devel] Is there a 'Can run this VM?' query
In-Reply-To: <bb29413e-2164-4fb8-876e-692bbc3429c6@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <a2f32de8-928f-45aa-a0bd-6b1e3a34ce2d@zmail01.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Omer Frenkel" <ofrenkel at redhat.com>
> To: "Yaniv Kaul" <ykaul at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Monday, January 9, 2012 2:06:58 PM
> Subject: Re: [Engine-devel] Is there a 'Can run this VM?' query
> 
> 
> 
> ----- Original Message -----
> > From: "Yaniv Kaul" <ykaul at redhat.com>
> > To: engine-devel at ovirt.org
> > Sent: Monday, January 9, 2012 9:36:34 AM
> > Subject: [Engine-devel] Is there a 'Can run this VM?' query
> > 
> > Beaker automated test system
> > (http://fedoraproject.org/wiki/QA/Beaker)
> > would like to integrate with the project, via the REST API.
> > Their main concern is that they'll provision VMs who would not be
> > able
> > to run on the ovirt setup.
> > I think there are two cases here:
> > 1. Provisioning a VM with X vCPUs, while there isn't a host
> > available
> > with such number of pCPUs (or similar in memory) - this is easier,
> > as
> > it
> > can be 'calculated' from the setup beforehand and we can avoid such
> > provisioning.
> > 2. More difficult - in runtime, the hosts are over-committed and we
> > cannot run a VM - can we, before trying to run a VM, ask if it's
> > feasible? I know it's a point in time, and may be wrong in the next
> > second, but if we go 'no' as an answer, it already provides good
> > information to Beaker to continue and look elsewhere.
> > Y.
> >
> 
> there is no such query currently,
if we're going to supply one, we either encapsulate the canDoAction to a class and re-use or
supply a boolean argument on the parameter class, say "dryRun", to indicate command doesn't
perform the execute part realy.
> the answer to this question actually is lots of other questions
> (status, storage, host availability and more)
> BUT i think its rather easy creating such a query (though caller need
> to remember it might be a long one)
>  
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From ofrenkel at redhat.com  Mon Jan  9 12:44:55 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Mon, 09 Jan 2012 07:44:55 -0500 (EST)
Subject: [Engine-devel] Is there a 'Can run this VM?' query
In-Reply-To: <a2f32de8-928f-45aa-a0bd-6b1e3a34ce2d@zmail01.collab.prod.int.phx2.redhat.com>
Message-ID: <c7d18389-c1a7-4c89-8ceb-50451f799a4b@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Roy Golan" <rgolan at redhat.com>
> To: "Omer Frenkel" <ofrenkel at redhat.com>
> Cc: engine-devel at ovirt.org, "Yaniv Kaul" <ykaul at redhat.com>
> Sent: Monday, January 9, 2012 2:21:29 PM
> Subject: Re: [Engine-devel] Is there a 'Can run this VM?' query
> 
> 
> 
> ----- Original Message -----
> > From: "Omer Frenkel" <ofrenkel at redhat.com>
> > To: "Yaniv Kaul" <ykaul at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Monday, January 9, 2012 2:06:58 PM
> > Subject: Re: [Engine-devel] Is there a 'Can run this VM?' query
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Yaniv Kaul" <ykaul at redhat.com>
> > > To: engine-devel at ovirt.org
> > > Sent: Monday, January 9, 2012 9:36:34 AM
> > > Subject: [Engine-devel] Is there a 'Can run this VM?' query
> > > 
> > > Beaker automated test system
> > > (http://fedoraproject.org/wiki/QA/Beaker)
> > > would like to integrate with the project, via the REST API.
> > > Their main concern is that they'll provision VMs who would not be
> > > able
> > > to run on the ovirt setup.
> > > I think there are two cases here:
> > > 1. Provisioning a VM with X vCPUs, while there isn't a host
> > > available
> > > with such number of pCPUs (or similar in memory) - this is
> > > easier,
> > > as
> > > it
> > > can be 'calculated' from the setup beforehand and we can avoid
> > > such
> > > provisioning.
> > > 2. More difficult - in runtime, the hosts are over-committed and
> > > we
> > > cannot run a VM - can we, before trying to run a VM, ask if it's
> > > feasible? I know it's a point in time, and may be wrong in the
> > > next
> > > second, but if we go 'no' as an answer, it already provides good
> > > information to Beaker to continue and look elsewhere.
> > > Y.
> > >
> > 
> > there is no such query currently,
> if we're going to supply one, we either encapsulate the canDoAction
> to a class and re-use or
> supply a boolean argument on the parameter class, say "dryRun", to
> indicate command doesn't
> perform the execute part realy.

yep, RunVmCommand.CanRunVm() is public and static,
return boolean and even fill list messages.
this is why i said i think its easy to do it :)
just need to check it really covers anything and behave well

> > the answer to this question actually is lots of other questions
> > (status, storage, host availability and more)
> > BUT i think its rather easy creating such a query (though caller
> > need
> > to remember it might be a long one)
> >  
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 


From lpeer at redhat.com  Mon Jan  9 13:07:06 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Mon, 09 Jan 2012 15:07:06 +0200
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <4F0ABEC3.8060308@redhat.com>
References: <a326aab2-28ec-4147-9265-9554cc356077@zmail14.collab.prod.int.phx2.redhat.com>
	<4F0AB753.80105@redhat.com> <4F0ABEC3.8060308@redhat.com>
Message-ID: <4F0AE67A.9080302@redhat.com>

On 09/01/12 12:17, Dor Laor wrote:
> On 01/09/2012 11:45 AM, Yaniv Kaul wrote:
>> On 01/09/2012 11:21 AM, Michael Kublin wrote:
>>> Hi, the follow link is providing a low level design for
>>> HotPlug/HotUnplug feature
>>> :http://www.ovirt.org/wiki/Features/DetailedHotPlug  .
>>>
>>> The feature is simple and design is short
>>>
>>> Regards Michael
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>> 1. Corrected some typos, spelling, backend->engine, 'REST API' -> 'API',
>> etc. (didn't fix ' storage procedures' - but I think you've meant
>> 'stored procedures' ?).
>> 2. Missing/questions:
>> - Permissions? Quota?
>> - Explain how disk is created in the first place.
>> - Database (tables, etc.)
>> - Which cluster level is required?
>> - Is it an async or sync task?
>> - Can you plug a system disk?
>> - Can you unplug multiple at a time?
>> - What happens when you plug too many?
>> - How do you determine where (PCI bus-wise) to plug them?
>> - Any CanDoAction to allow/disallow plug/unplug from specific systems?
>> - I suspect we'd be happier with some agent cooperation before
>> unplugging - is this done by QEMU? Not detailed anywhere.
>> - Link to KVM/QEMU feature description for it, if such exist, would be
>> nice.
>> - Does it affect taking a snapshot? Live or not?
>> - Does it affect exporting a VM? I reckon you export with all disks,
>> with their plug/unplug status?
> 
> In addition:
>  - will you allow it during live migration (qemu allows it)?
>  - Are you expecting events to pop?
>  - You call it 'hotplug' but mention only disk device, what about
>    memory/cpu/ any other pci card?
>  - How do you define a system disk? Some VM may boot from
>    pxe/nfsroot/etc
>  - It should be a nice to have to check if the guest mounts any FS
>    within the disk to be un hot pluged and warn the user
> 

In addition - 2 :

- plugged/unplugged  disk is a VM property not disk's. (this of a shared
disk that can be plugged in one VM but unplugged in another)

- supporting the best effort flag is not related only to hot plug disk,
is it supported in the VDSM level on a per disk basis? again should not
be a disk property but a VM-Disk relation property.


>>
>> Y.
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From mkublin at redhat.com  Mon Jan  9 14:04:44 2012
From: mkublin at redhat.com (Michael Kublin)
Date: Mon, 09 Jan 2012 09:04:44 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <4F0AE67A.9080302@redhat.com>
Message-ID: <e7584929-dcee-443b-ab36-b27113782a5d@zmail14.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Michael Kublin" <mkublin at redhat.com>
> Cc: dlaor at redhat.com, "Yaniv Kaul" <ykaul at redhat.com>, engine-devel at ovirt.org
> Sent: Monday, January 9, 2012 3:07:06 PM
> Subject: Re: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
> 
> On 09/01/12 12:17, Dor Laor wrote:
> > On 01/09/2012 11:45 AM, Yaniv Kaul wrote:
> >> On 01/09/2012 11:21 AM, Michael Kublin wrote:
> >>> Hi, the follow link is providing a low level design for
> >>> HotPlug/HotUnplug feature
> >>> :http://www.ovirt.org/wiki/Features/DetailedHotPlug  .
> >>>
> >>> The feature is simple and design is short
> >>>
> >>> Regards Michael
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> >> 1. Corrected some typos, spelling, backend->engine, 'REST API' ->
> >> 'API',
> >> etc. (didn't fix ' storage procedures' - but I think you've meant
> >> 'stored procedures' ?).
Thanks
> >> 2. Missing/questions:
> >> - Permissions? Quota?
Regards Quota - has not influence on the feature
> >> - Explain how disk is created in the first place.
The disk is added to VM by AddDiskToVMCommand, by default is is plugged
> >> - Database (tables, etc.)
They are in design
> >> - Which cluster level is required?
Added to design , it is 3.1
> >> - Is it an async or sync task?
Sync
> >> - Can you plug a system disk?
No
> >> - Can you unplug multiple at a time?
Yes, we have not such limitation, the sane is to plug.
> >> - What happens when you plug too many?
We allow to plug disk which were attached to vm , by AddDiskToVmCommand, a max number of allowed disks is handled there
> >> - How do you determine where (PCI bus-wise) to plug them?
As I think, the PCI address will be added to most devices at Stable PCI device addresses feature, I will use it 
> >> - Any CanDoAction to allow/disallow plug/unplug from specific
> >> systems?
The operation will be allowed only for RHEL 5/6, Windows Server 2003 and Windows server 2008 operating systems, added to design
> >> - I suspect we'd be happier with some agent cooperation before
> >> unplugging - is this done by QEMU? Not detailed anywhere.
I will check this 
> >> - Link to KVM/QEMU feature description for it, if such exist,
> >> would be
> >> nice.
> >> - Does it affect taking a snapshot? Live or not?
No, all disk will be passed to snapshots as is
> >> - Does it affect exporting a VM? I reckon you export with all
> >> disks,
> >> with their plug/unplug status?
This one I need to check
> > In addition:
> >  - will you allow it during live migration (qemu allows it)?
I don't see any technical limitation from my side, but I will check
> >  - Are you expecting events to pop?
> >  - You call it 'hotplug' but mention only disk device, what about
> >    memory/cpu/ any other pci card?
The pci card it is another feature, in current version we have only hotplug / unplug for disks and pci card 
> >  - How do you define a system disk? Some VM may boot from
> >    pxe/nfsroot/etc
I had a property on the disk which means System, also I have a property that means bootable. I suppose that 
disk can not be plugged/unplugged with any of them 
> >  - It should be a nice to have to check if the guest mounts any FS
> >    within the disk to be un hot pluged and warn the user
Need to be check
> 
> In addition - 2 :
> 
> - plugged/unplugged  disk is a VM property not disk's. (this of a
> shared
> disk that can be plugged in one VM but unplugged in another)
Ok. I will update design
> - supporting the best effort flag is not related only to hot plug
> disk,
> is it supported in the VDSM level on a per disk basis? again should
> not
> be a disk property but a VM-Disk relation property.
> 
> 
> >>
> >> Y.
> >>
> >>
> >>
> >>
> >>
> >>
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 


From iheim at redhat.com  Mon Jan  9 14:35:53 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 09 Jan 2012 16:35:53 +0200
Subject: [Engine-devel] Is there a 'Can run this VM?' query
In-Reply-To: <c7d18389-c1a7-4c89-8ceb-50451f799a4b@zmail07.collab.prod.int.phx2.redhat.com>
References: <c7d18389-c1a7-4c89-8ceb-50451f799a4b@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F0AFB49.5050004@redhat.com>

On 01/09/2012 02:44 PM, Omer Frenkel wrote:
>
>
> ----- Original Message -----
>> From: "Roy Golan"<rgolan at redhat.com>
>> To: "Omer Frenkel"<ofrenkel at redhat.com>
>> Cc: engine-devel at ovirt.org, "Yaniv Kaul"<ykaul at redhat.com>
>> Sent: Monday, January 9, 2012 2:21:29 PM
>> Subject: Re: [Engine-devel] Is there a 'Can run this VM?' query
>>
>>
>>
>> ----- Original Message -----
>>> From: "Omer Frenkel"<ofrenkel at redhat.com>
>>> To: "Yaniv Kaul"<ykaul at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Monday, January 9, 2012 2:06:58 PM
>>> Subject: Re: [Engine-devel] Is there a 'Can run this VM?' query
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Yaniv Kaul"<ykaul at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Sent: Monday, January 9, 2012 9:36:34 AM
>>>> Subject: [Engine-devel] Is there a 'Can run this VM?' query
>>>>
>>>> Beaker automated test system
>>>> (http://fedoraproject.org/wiki/QA/Beaker)
>>>> would like to integrate with the project, via the REST API.
>>>> Their main concern is that they'll provision VMs who would not be
>>>> able
>>>> to run on the ovirt setup.
>>>> I think there are two cases here:
>>>> 1. Provisioning a VM with X vCPUs, while there isn't a host
>>>> available
>>>> with such number of pCPUs (or similar in memory) - this is
>>>> easier,
>>>> as
>>>> it
>>>> can be 'calculated' from the setup beforehand and we can avoid
>>>> such
>>>> provisioning.
>>>> 2. More difficult - in runtime, the hosts are over-committed and
>>>> we
>>>> cannot run a VM - can we, before trying to run a VM, ask if it's
>>>> feasible? I know it's a point in time, and may be wrong in the
>>>> next
>>>> second, but if we go 'no' as an answer, it already provides good
>>>> information to Beaker to continue and look elsewhere.
>>>> Y.
>>>>
>>>
>>> there is no such query currently,
>> if we're going to supply one, we either encapsulate the canDoAction
>> to a class and re-use or
>> supply a boolean argument on the parameter class, say "dryRun", to
>> indicate command doesn't
>> perform the execute part realy.
>
> yep, RunVmCommand.CanRunVm() is public and static,
> return boolean and even fill list messages.
> this is why i said i think its easy to do it :)
> just need to check it really covers anything and behave well

I'd start from trying to understand how you would model such a query in 
the REST API for users to consume...



From lpeer at redhat.com  Mon Jan  9 15:44:25 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Mon, 09 Jan 2012 17:44:25 +0200
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <e7584929-dcee-443b-ab36-b27113782a5d@zmail14.collab.prod.int.phx2.redhat.com>
References: <e7584929-dcee-443b-ab36-b27113782a5d@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F0B0B59.4050105@redhat.com>


<snip>

On 09/01/12 16:04, Michael Kublin wrote:
> 
> 
> ----- Original Message -----
>> From: "Livnat Peer" <lpeer at redhat.com>
>> To: "Michael Kublin" <mkublin at redhat.com>
>> Cc: dlaor at redhat.com, "Yaniv Kaul" <ykaul at redhat.com>, engine-devel at ovirt.org
>> Sent: Monday, January 9, 2012 3:07:06 PM
>> Subject: Re: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
>>
>> On 09/01/12 12:17, Dor Laor wrote:
>>> On 01/09/2012 11:45 AM, Yaniv Kaul wrote:
>>>> On 01/09/2012 11:21 AM, Michael Kublin wrote:
>>>>> Hi, the follow link is providing a low level design for
>>>>> HotPlug/HotUnplug feature
>>>>> :http://www.ovirt.org/wiki/Features/DetailedHotPlug  .
>>>>>
>>>>> The feature is simple and design is short
>>>>>
>>>>> Regards Michael
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>>> 1. Corrected some typos, spelling, backend->engine, 'REST API' ->
>>>> 'API',
>>>> etc. (didn't fix ' storage procedures' - but I think you've meant
>>>> 'stored procedures' ?).
> Thanks
>>>> 2. Missing/questions:
>>>> - Permissions? Quota?
> Regards Quota - has not influence on the feature

The storage quota enforcement is upon storage creation not disk plug.

>>>> - Explain how disk is created in the first place.
> The disk is added to VM by AddDiskToVMCommand, by default is is plugged

- The disk can be created as part of the VM or as a floating disk.
- The disk has a property if it is plugged to the VM or not (VM-DISK
relation property)
- If the disk is created as a VM disk and it is marked as plugged the
engine should try to hot plug the disk when the disk is created (if the
VM is running), if it fails the disk should be left unplugged.


>>>> - Database (tables, etc.)
> They are in design
>>>> - Which cluster level is required?
> Added to design , it is 3.1

3.1 cluster or DC?

Ayal - is there a storage limitation or can we activate this feature for
any 3.1 cluster?

>>>> - Is it an async or sync task?
> Sync
>>>> - Can you plug a system disk?
> No
>>>> - Can you unplug multiple at a time?
> Yes, we have not such limitation, the sane is to plug.

The engine does not have a single verb for plugging multiple disk in one
action.

>>>> - What happens when you plug too many?
> We allow to plug disk which were attached to vm , by AddDiskToVmCommand, a max number of allowed disks is handled there

>>>> - How do you determine where (PCI bus-wise) to plug them?
> As I think, the PCI address will be added to most devices at Stable PCI device addresses feature, I will use it 

The engine learns the device address from VDSM (given by libvirt), it is
not editable by the user or the engine at this phase.

>>>> - Any CanDoAction to allow/disallow plug/unplug from specific
>>>> systems?
> The operation will be allowed only for RHEL 5/6, Windows Server 2003 and Windows server 2008 operating systems, added to design
>>>> - I suspect we'd be happier with some agent cooperation before
>>>> unplugging - is this done by QEMU? Not detailed anywhere.
> I will check this 
>>>> - Link to KVM/QEMU feature description for it, if such exist,
>>>> would be
>>>> nice.
>>>> - Does it affect taking a snapshot? Live or not?
> No, all disk will be passed to snapshots as is

snapshot includes unplugged disks, part of the snapshot is the VM
configuration which includes for each disk if it is plugged or not.

>>>> - Does it affect exporting a VM? I reckon you export with all
>>>> disks,
>>>> with their plug/unplug status?
> This one I need to check

Export includes all (non-shared) disks and their status.

>>> In addition:
>>>  - will you allow it during live migration (qemu allows it)?
> I don't see any technical limitation from my side, but I will check
>>>  - Are you expecting events to pop?
>>>  - You call it 'hotplug' but mention only disk device, what about
>>>    memory/cpu/ any other pci card?
> The pci card it is another feature, in current version we have only hotplug / unplug for disks and pci card 
>>>  - How do you define a system disk? Some VM may boot from
>>>    pxe/nfsroot/etc
> I had a property on the disk which means System, also I have a property that means bootable. I suppose that 
> disk can not be plugged/unplugged with any of them 
>>>  - It should be a nice to have to check if the guest mounts any FS
>>>    within the disk to be un hot pluged and warn the user
> Need to be check
>>
>> In addition - 2 :
>>
>> - plugged/unplugged  disk is a VM property not disk's. (this of a
>> shared
>> disk that can be plugged in one VM but unplugged in another)
> Ok. I will update design
>> - supporting the best effort flag is not related only to hot plug
>> disk,
>> is it supported in the VDSM level on a per disk basis? again should
>> not
>> be a disk property but a VM-Disk relation property.
>>
>>
>>>>
>>>> Y.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>>



From mkenneth at redhat.com  Mon Jan  9 15:57:55 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Mon, 09 Jan 2012 10:57:55 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <4F0B0B59.4050105@redhat.com>
Message-ID: <5912219b-888e-4734-8fd5-a115697287a4@mkenneth.csb>

sorry for top posting (and maybe not directly related) but:
- is there a list of "free disks"? or a disk is always attached to a VM?

Miki

----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Michael Kublin" <mkublin at redhat.com>
> Cc: dlaor at redhat.com, engine-devel at ovirt.org
> Sent: Monday, January 9, 2012 5:44:25 PM
> Subject: Re: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
> 
> 
> <snip>
> 
> On 09/01/12 16:04, Michael Kublin wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Livnat Peer" <lpeer at redhat.com>
> >> To: "Michael Kublin" <mkublin at redhat.com>
> >> Cc: dlaor at redhat.com, "Yaniv Kaul" <ykaul at redhat.com>,
> >> engine-devel at ovirt.org
> >> Sent: Monday, January 9, 2012 3:07:06 PM
> >> Subject: Re: [Engine-devel] Low Level design for HotPlug/HotUnplug
> >> feature
> >>
> >> On 09/01/12 12:17, Dor Laor wrote:
> >>> On 01/09/2012 11:45 AM, Yaniv Kaul wrote:
> >>>> On 01/09/2012 11:21 AM, Michael Kublin wrote:
> >>>>> Hi, the follow link is providing a low level design for
> >>>>> HotPlug/HotUnplug feature
> >>>>> :http://www.ovirt.org/wiki/Features/DetailedHotPlug  .
> >>>>>
> >>>>> The feature is simple and design is short
> >>>>>
> >>>>> Regards Michael
> >>>>> _______________________________________________
> >>>>> Engine-devel mailing list
> >>>>> Engine-devel at ovirt.org
> >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>
> >>>> 1. Corrected some typos, spelling, backend->engine, 'REST API'
> >>>> ->
> >>>> 'API',
> >>>> etc. (didn't fix ' storage procedures' - but I think you've
> >>>> meant
> >>>> 'stored procedures' ?).
> > Thanks
> >>>> 2. Missing/questions:
> >>>> - Permissions? Quota?
> > Regards Quota - has not influence on the feature
> 
> The storage quota enforcement is upon storage creation not disk plug.
> 
> >>>> - Explain how disk is created in the first place.
> > The disk is added to VM by AddDiskToVMCommand, by default is is
> > plugged
> 
> - The disk can be created as part of the VM or as a floating disk.
> - The disk has a property if it is plugged to the VM or not (VM-DISK
> relation property)
> - If the disk is created as a VM disk and it is marked as plugged the
> engine should try to hot plug the disk when the disk is created (if
> the
> VM is running), if it fails the disk should be left unplugged.
> 
> 
> >>>> - Database (tables, etc.)
> > They are in design
> >>>> - Which cluster level is required?
> > Added to design , it is 3.1
> 
> 3.1 cluster or DC?
> 
> Ayal - is there a storage limitation or can we activate this feature
> for
> any 3.1 cluster?
> 
> >>>> - Is it an async or sync task?
> > Sync
> >>>> - Can you plug a system disk?
> > No
> >>>> - Can you unplug multiple at a time?
> > Yes, we have not such limitation, the sane is to plug.
> 
> The engine does not have a single verb for plugging multiple disk in
> one
> action.
> 
> >>>> - What happens when you plug too many?
> > We allow to plug disk which were attached to vm , by
> > AddDiskToVmCommand, a max number of allowed disks is handled there
> 
> >>>> - How do you determine where (PCI bus-wise) to plug them?
> > As I think, the PCI address will be added to most devices at Stable
> > PCI device addresses feature, I will use it
> 
> The engine learns the device address from VDSM (given by libvirt), it
> is
> not editable by the user or the engine at this phase.
> 
> >>>> - Any CanDoAction to allow/disallow plug/unplug from specific
> >>>> systems?
> > The operation will be allowed only for RHEL 5/6, Windows Server
> > 2003 and Windows server 2008 operating systems, added to design
> >>>> - I suspect we'd be happier with some agent cooperation before
> >>>> unplugging - is this done by QEMU? Not detailed anywhere.
> > I will check this
> >>>> - Link to KVM/QEMU feature description for it, if such exist,
> >>>> would be
> >>>> nice.
> >>>> - Does it affect taking a snapshot? Live or not?
> > No, all disk will be passed to snapshots as is
> 
> snapshot includes unplugged disks, part of the snapshot is the VM
> configuration which includes for each disk if it is plugged or not.
> 
> >>>> - Does it affect exporting a VM? I reckon you export with all
> >>>> disks,
> >>>> with their plug/unplug status?
> > This one I need to check
> 
> Export includes all (non-shared) disks and their status.
> 
> >>> In addition:
> >>>  - will you allow it during live migration (qemu allows it)?
> > I don't see any technical limitation from my side, but I will check
> >>>  - Are you expecting events to pop?
> >>>  - You call it 'hotplug' but mention only disk device, what about
> >>>    memory/cpu/ any other pci card?
> > The pci card it is another feature, in current version we have only
> > hotplug / unplug for disks and pci card
> >>>  - How do you define a system disk? Some VM may boot from
> >>>    pxe/nfsroot/etc
> > I had a property on the disk which means System, also I have a
> > property that means bootable. I suppose that
> > disk can not be plugged/unplugged with any of them
> >>>  - It should be a nice to have to check if the guest mounts any
> >>>  FS
> >>>    within the disk to be un hot pluged and warn the user
> > Need to be check
> >>
> >> In addition - 2 :
> >>
> >> - plugged/unplugged  disk is a VM property not disk's. (this of a
> >> shared
> >> disk that can be plugged in one VM but unplugged in another)
> > Ok. I will update design
> >> - supporting the best effort flag is not related only to hot plug
> >> disk,
> >> is it supported in the VDSM level on a per disk basis? again
> >> should
> >> not
> >> be a disk property but a VM-Disk relation property.
> >>
> >>
> >>>>
> >>>> Y.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Engine-devel mailing list
> >>>> Engine-devel at ovirt.org
> >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> >>
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkublin at redhat.com  Tue Jan 10 07:57:44 2012
From: mkublin at redhat.com (Michael Kublin)
Date: Tue, 10 Jan 2012 02:57:44 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <4F0B0B59.4050105@redhat.com>
Message-ID: <1c0e7728-1362-49cb-b541-81635a543c69@zmail14.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Michael Kublin" <mkublin at redhat.com>
> Cc: dlaor at redhat.com, engine-devel at ovirt.org, "Yaniv Kaul" <ykaul at redhat.com>
> Sent: Monday, January 9, 2012 5:44:25 PM
> Subject: Re: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
> 
> 
> <snip>
> 
> On 09/01/12 16:04, Michael Kublin wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Livnat Peer" <lpeer at redhat.com>
> >> To: "Michael Kublin" <mkublin at redhat.com>
> >> Cc: dlaor at redhat.com, "Yaniv Kaul" <ykaul at redhat.com>,
> >> engine-devel at ovirt.org
> >> Sent: Monday, January 9, 2012 3:07:06 PM
> >> Subject: Re: [Engine-devel] Low Level design for HotPlug/HotUnplug
> >> feature
> >>
> >> On 09/01/12 12:17, Dor Laor wrote:
> >>> On 01/09/2012 11:45 AM, Yaniv Kaul wrote:
> >>>> On 01/09/2012 11:21 AM, Michael Kublin wrote:
> >>>>> Hi, the follow link is providing a low level design for
> >>>>> HotPlug/HotUnplug feature
> >>>>> :http://www.ovirt.org/wiki/Features/DetailedHotPlug  .
> >>>>>
> >>>>> The feature is simple and design is short
> >>>>>
> >>>>> Regards Michael
> >>>>> _______________________________________________
> >>>>> Engine-devel mailing list
> >>>>> Engine-devel at ovirt.org
> >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>
> >>>> 1. Corrected some typos, spelling, backend->engine, 'REST API'
> >>>> ->
> >>>> 'API',
> >>>> etc. (didn't fix ' storage procedures' - but I think you've
> >>>> meant
> >>>> 'stored procedures' ?).
> > Thanks
> >>>> 2. Missing/questions:
> >>>> - Permissions? Quota?
> > Regards Quota - has not influence on the feature
> 
> The storage quota enforcement is upon storage creation not disk plug.
> 
> >>>> - Explain how disk is created in the first place.
> > The disk is added to VM by AddDiskToVMCommand, by default is is
> > plugged
> 
> - The disk can be created as part of the VM or as a floating disk.
> - The disk has a property if it is plugged to the VM or not (VM-DISK
> relation property)
> - If the disk is created as a VM disk and it is marked as plugged the
> engine should try to hot plug the disk when the disk is created (if
> the
> VM is running), if it fails the disk should be left unplugged.
> 
> 
> >>>> - Database (tables, etc.)
> > They are in design
> >>>> - Which cluster level is required?
> > Added to design , it is 3.1
> 
> 3.1 cluster or DC?
> 
> Ayal - is there a storage limitation or can we activate this feature
> for
> any 3.1 cluster?
> 
> >>>> - Is it an async or sync task?
> > Sync
> >>>> - Can you plug a system disk?
> > No
> >>>> - Can you unplug multiple at a time?
> > Yes, we have not such limitation, the sane is to plug.
> 
> The engine does not have a single verb for plugging multiple disk in
> one
> action.
I think that question was if it possible to plug/unplug couple of disk in parallel on the same vm, by running couple of hotplug/unplug
in the same time  
> >>>> - What happens when you plug too many?
> > We allow to plug disk which were attached to vm , by
> > AddDiskToVmCommand, a max number of allowed disks is handled there
> 
> >>>> - How do you determine where (PCI bus-wise) to plug them?
> > As I think, the PCI address will be added to most devices at Stable
> > PCI device addresses feature, I will use it
> 
> The engine learns the device address from VDSM (given by libvirt), it
> is
> not editable by the user or the engine at this phase.
> 
> >>>> - Any CanDoAction to allow/disallow plug/unplug from specific
> >>>> systems?
> > The operation will be allowed only for RHEL 5/6, Windows Server
> > 2003 and Windows server 2008 operating systems, added to design
> >>>> - I suspect we'd be happier with some agent cooperation before
> >>>> unplugging - is this done by QEMU? Not detailed anywhere.
> > I will check this
> >>>> - Link to KVM/QEMU feature description for it, if such exist,
> >>>> would be
> >>>> nice.
> >>>> - Does it affect taking a snapshot? Live or not?
> > No, all disk will be passed to snapshots as is
> 
> snapshot includes unplugged disks, part of the snapshot is the VM
> configuration which includes for each disk if it is plugged or not.
> 
> >>>> - Does it affect exporting a VM? I reckon you export with all
> >>>> disks,
> >>>> with their plug/unplug status?
> > This one I need to check
> 
> Export includes all (non-shared) disks and their status.
So, I need to add a new tag to ovf xml of export what name I should take <plugged> </plugged> or something else?

> >>> In addition:
> >>>  - will you allow it during live migration (qemu allows it)?
> > I don't see any technical limitation from my side, but I will check
> >>>  - Are you expecting events to pop?
> >>>  - You call it 'hotplug' but mention only disk device, what about
> >>>    memory/cpu/ any other pci card?
> > The pci card it is another feature, in current version we have only
> > hotplug / unplug for disks and pci card
> >>>  - How do you define a system disk? Some VM may boot from
> >>>    pxe/nfsroot/etc
> > I had a property on the disk which means System, also I have a
> > property that means bootable. I suppose that
> > disk can not be plugged/unplugged with any of them
> >>>  - It should be a nice to have to check if the guest mounts any
> >>>  FS
> >>>    within the disk to be un hot pluged and warn the user
> > Need to be check
> >>
> >> In addition - 2 :
> >>
> >> - plugged/unplugged  disk is a VM property not disk's. (this of a
> >> shared
> >> disk that can be plugged in one VM but unplugged in another)
> > Ok. I will update design
> >> - supporting the best effort flag is not related only to hot plug
> >> disk,
> >> is it supported in the VDSM level on a per disk basis? again
> >> should
> >> not
> >> be a disk property but a VM-Disk relation property.
> >>
> >>
> >>>>
> >>>> Y.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> Engine-devel mailing list
> >>>> Engine-devel at ovirt.org
> >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> >>
> 
> 


From ashakarc at redhat.com  Tue Jan 10 16:08:15 2012
From: ashakarc at redhat.com (Asaf Shakarchi)
Date: Tue, 10 Jan 2012 11:08:15 -0500 (EST)
Subject: [Engine-devel] Instructions (Wiki page) for adding error messages
	to Engine
In-Reply-To: <a824bfe0-8be0-415f-8d26-4e80d049079f@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <b718c858-d0c2-47f9-8add-d05d76c69354@zmail07.collab.prod.int.phx2.redhat.com>

Hey,

Below is a link which contains the details required to manage the Engine's messages.

http://www.ovirt.org/wiki/Engine_Adding_Messages




Thanks,

Regards, 
Asaf 





From zwu.kernel at gmail.com  Wed Jan 11 06:59:51 2012
From: zwu.kernel at gmail.com (Zhi Yong Wu)
Date: Wed, 11 Jan 2012 14:59:51 +0800
Subject: [Engine-devel] MOM: oVirt engine core meeting (Wed Jan. 4th)
In-Reply-To: <36835923.dTYE9mBE0a@doronf>
References: <36835923.dTYE9mBE0a@doronf>
Message-ID: <CAEH94LjHFfNGVmp1UorufN2TfpCt04kZ2KxfoDjAne5Y9Ym8Zw@mail.gmail.com>

HI, Is this a regular meeting every week? If yes, can you let me know
how to dial in or IRC?

On Wed, Jan 4, 2012 at 11:37 PM, Doron Fediuck <dfediuck at redhat.com> wrote:
> 1. DB scheme verification presented by Yair.
> Notes: none.
>
> 2. Pre-started VM's functionality presented by Doron.
> Notes:
> - Consider adding the API the option to define pre-started VM's number
> as a percentage of the pool size. This will make sure that when pool size
> changes, the number of pre-started VM's is still relevant.
>
> 3. SPM priority functionality presented by Doron.
> Notes:
> - New related feature needed; SPM resignation. This will allow
> the admin to make sure an SPM host can resign so another host
> will be elected to become an SPM. This should be further discussed.
>
> Thanks for everyone for showing up. Cya in 2 weeks!
> --
>
> /d
>
> "Willyoupleasehelpmefixmykeyboard?Thespacebarisbroken!"
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



-- 
Regards,

Zhi Yong Wu


From lpeer at redhat.com  Wed Jan 11 07:02:41 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 11 Jan 2012 09:02:41 +0200
Subject: [Engine-devel] MOM: oVirt engine core meeting (Wed Jan. 4th)
In-Reply-To: <CAEH94LjHFfNGVmp1UorufN2TfpCt04kZ2KxfoDjAne5Y9Ym8Zw@mail.gmail.com>
References: <36835923.dTYE9mBE0a@doronf>
	<CAEH94LjHFfNGVmp1UorufN2TfpCt04kZ2KxfoDjAne5Y9Ym8Zw@mail.gmail.com>
Message-ID: <4F0D3411.6080103@redhat.com>

On 11/01/12 08:59, Zhi Yong Wu wrote:
> HI, Is this a regular meeting every week? If yes, can you let me know
> how to dial in or IRC?

It is a bi weekly meeting (next meeting is Wednesday Jan 18th).
I will forward to the list the invite with all the relevant details
(dial in information).

Livnat

> 
> On Wed, Jan 4, 2012 at 11:37 PM, Doron Fediuck <dfediuck at redhat.com> wrote:
>> 1. DB scheme verification presented by Yair.
>> Notes: none.
>>
>> 2. Pre-started VM's functionality presented by Doron.
>> Notes:
>> - Consider adding the API the option to define pre-started VM's number
>> as a percentage of the pool size. This will make sure that when pool size
>> changes, the number of pre-started VM's is still relevant.
>>
>> 3. SPM priority functionality presented by Doron.
>> Notes:
>> - New related feature needed; SPM resignation. This will allow
>> the admin to make sure an SPM host can resign so another host
>> will be elected to become an SPM. This should be further discussed.
>>
>> Thanks for everyone for showing up. Cya in 2 weeks!
>> --
>>
>> /d
>>
>> "Willyoupleasehelpmefixmykeyboard?Thespacebarisbroken!"
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 
> 



From lpeer at redhat.com  Wed Jan 11 07:03:34 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 11 Jan 2012 02:03:34 -0500 (EST)
Subject: [Engine-devel] oVirt engine core
Message-ID: <347528aa-2445-4339-9f88-f6bd0769b77d@zmail14.collab.prod.int.phx2.redhat.com>

The following meeting has been modified:

Subject: oVirt engine core 
Organizer: "Livnat Peer" <lpeer at redhat.com> 

Time: 4:00:00 PM - 5:00:00 PM GMT +02:00 Jerusalem [MODIFIED]
 Recurrence : Every 2 week(s) on Wednesday No end date Effective Nov 23, 2011

Invitees: engine-devel at ovirt.org; wangbo_bupt at hotmail.com; mkolesni at redhat.com; ykaul at redhat.com; ofrenkel at redhat.com; lhornyak at redhat.com; smizrahi at redhat.com; oschreib at redhat.com; sgordon at redhat.com; dedutta at cisco.com; emesika at redhat.com ... 


*~*~*~*~*~*~*~*~*~*

This is a bi-weekly call about oVirt engine core development issues upstream. 
You are welcome to join. 

Bridge ID: 1814335863 

Dial-in information: 
Reservationless-Plus Toll Free Dial-In Number (US & Canada): (800) 451-8679 
Reservationless-Plus International Dial-In Number: (212) 729-5016 
Conference code: 1814335863 

Global Access Numbers Local: 
Australia, Sydney Dial-In #: 0289852326 
Austria, Vienna Dial-In #: 012534978196 
Belgium, Brussels Dial-In #: 027920405 
China Dial-In #: 4006205013 
Denmark, Copenhagen Dial-In #: 32729215 
Finland, Helsinki Dial-In #: 0923194436 
France, Paris Dial-In #: 0170377140 
Germany, Berlin Dial-In #: 030300190579 
Ireland, Dublin Dial-In #: 014367793 
Italy, Milan Dial-In #: 0236269529 
Netherlands, Amsterdam Dial-In #: 0207975872 
Norway, Oslo Dial-In #: 21033188 
Singapore Dial-In #: 64840858 
Spain, Barcelona Dial-In #: 935452328 
Sweden, Stockholm Dial-In #: 0850513770 
Switzerland, Geneva Dial-In #: 0225927881 
United Kingdom Dial-In #: 02078970515 
United Kingdom Dial-In #: 08445790676 
United Kingdom, LocalCall Dial-In #: 08445790678 
United States Dial-In #: 2127295016 


Global Access Numbers Tollfree: 
Argentina Dial-In #: 8004441016 
Australia Dial-In #: 1800337169 
Austria Dial-In #: 0800005898 
Bahamas Dial-In #: 18002054776 
Bahrain Dial-In #: 80004377 
Belgium Dial-In #: 080048325 
Brazil Dial-In #: 08008921002 
Bulgaria Dial-In #: 008001100236 
Chile Dial-In #: 800370228 
Colombia Dial-In #: 018009134033 
Costa Rica Dial-In #: 08000131048 
Cyprus Dial-In #: 80095297 
Czech Republic Dial-In #: 800700318 
Denmark Dial-In #: 80887114 
Dominican Republic Dial-In #: 18887512313 
Estonia Dial-In #: 8000100232 
Finland Dial-In #: 0800117116 
France Dial-In #: 0805632867 
Germany Dial-In #: 8006647541 
Greece Dial-In #: 00800127562 
Hong Kong Dial-In #: 800930349 
Hungary Dial-In #: 0680016796 
Iceland Dial-In #: 8008967 
India Dial-In #: 0008006501533 
Indonesia Dial-In #: 0018030179162 
Ireland Dial-In #: 1800932401 
Israel Dial-In #: 1809462557 
Italy Dial-In #: 800985897 
Jamaica Dial-In #: 18002050328 
Japan Dial-In #: 0120934453 
Korea (South) Dial-In #: 007986517393 
Latvia Dial-In #: 80003339 
Lithuania Dial-In #: 880030479 
Luxembourg Dial-In #: 80026595 
Malaysia Dial-In #: 1800814451 
Mexico Dial-In #: 0018664590915 
New Zealand Dial-In #: 0800888167 
Norway Dial-In #: 80012994 
Panama Dial-In #: 008002269184 
Philippines Dial-In #: 180011100991 
Poland Dial-In #: 008001210187 
Portugal Dial-In #: 800814625 
Russian Federation Dial-In #: 81080028341012 
Saint Kitts and Nevis Dial-In #: 18002059252 
Singapore Dial-In #: 8006162235 
Slovak Republic Dial-In #: 0800001441 
South Africa Dial-In #: 0800981148 
Spain Dial-In #: 800300524 
Sweden Dial-In #: 200896860 
Switzerland Dial-In #: 800650077 
Taiwan Dial-In #: 00801127141 
Thailand Dial-In #: 001800656966 
Trinidad and Tobago Dial-In #: 18002024615 
United Arab Emirates Dial-In #: 8000650591 
United Kingdom Dial-In #: 08006948057 
United States Dial-In #: 8004518679 
Uruguay Dial-In #: 00040190315 
Venezuela Dial-In #: 08001627182 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120111/f142af97/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: meeting.ics
Type: text/calendar
Size: 9870 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120111/f142af97/attachment.ics>

From ofrenkel at redhat.com  Wed Jan 11 07:48:49 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Wed, 11 Jan 2012 02:48:49 -0500 (EST)
Subject: [Engine-devel] Instructions (Wiki page) for adding error
	messages to Engine
In-Reply-To: <b718c858-d0c2-47f9-8add-d05d76c69354@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <cc0a5f4a-d229-479d-b540-8326d854a736@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Asaf Shakarchi" <ashakarc at redhat.com>
> To: engine-devel at ovirt.org
> Cc: "Livnat Peer" <lpeer at redhat.com>, "Einav Cohen" <ecohen at redhat.com>, "Doron Fediuck" <dfediuck at redhat.com>, "Omer
> Frenkel" <ofrenkel at redhat.com>
> Sent: Tuesday, January 10, 2012 6:08:15 PM
> Subject: Instructions (Wiki page) for adding error messages to Engine
> 
> Hey,
> 
> Below is a link which contains the details required to manage the
> Engine's messages.
> 
> http://www.ovirt.org/wiki/Engine_Adding_Messages
> 

added a little note that for each message/error a corresponding key should be added to the appropriate enum.

> 
> 
> 
> Thanks,
> 
> Regards,
> Asaf
> 
> 
> 
> 


From ovedo at redhat.com  Wed Jan 11 12:57:19 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Wed, 11 Jan 2012 07:57:19 -0500 (EST)
Subject: [Engine-devel] Moving to Jboss AS7
In-Reply-To: <3e60d81b-f4c2-419b-a061-294d6d95df7f@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <70a600f8-d58c-4e17-9c87-17378678a7e9@zmail02.collab.prod.int.phx2.redhat.com>

Hey all,

The code changes required to make the engine work on Jboss AS7 will soon be push
It will, of course, require you to install it, and start working with it :-)

A separate E-mail will be sent to notify you all once pushed, and then you'll have to perform the following steps:

1. Download jboss 7.1.0 Beta1b (http://download.jboss.org/jbossas/7.1/jboss-as-7.1.0.Beta1b/jboss-as-7.1.0.Beta1b.tar.gz) - There is a newer version, but it has issues in the REST-API, so we decided to work with the beta version until a proper fix will be released.
2. Fetch the latest changes from our git repository
3. Change the Jboss home to the new path, both in the JBOSS_HOME environment variable, and in maven settings file (~/.m2/settings.xml)
4. Build the engine, with profiles "dep" and "setup". This will put all the proper configuration files, postgresql driver and make all the other needed changes in Jboss in order to make it work properly
mvn clean install -Psetup,dep .......
5. In order to run Jboss go to JBOSS_HOME/bin and run ./standalone.sh
6. Look inside the JBOSS_HOME/bin/standalone.conf file in order to enable jboss debugging (just uncomment the line JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n")
7. If you have a krb5.conf file you are working with, put it in JBOSS_HOME/standalone/configuration directory
8. Run Jboss (and be impressed by the startup speed!)

The above will be also added to the wiki page for building the engine as soon as the patches will be pushed upstream.

Some facts about Jboss 7:
1. It supports both standalone deployment, and domain deployment. We use the standalone one.
2. Stuff related to the standalone mode can be found in the JBOSS_HOME/standalone directory:
 * configuration - contains the main configuration file called standalone.xml file, plus some other configuration files
 * deployments - that's where your deployments should be. When adding a new one, don't forget to create a file called "<deployment-name>.dodeploy" in order to make it deploy.
 * log - that's where the log files are written (unless stated differently in the standalone.xml file).
3. The different modules that come with Jboss 7 are located in JBOSS_HOME/modules directory
 * No more common/lib directory.
 * Every module has a module.xml file which contains it's dependencies in other modules, the jars that are part of the module, and etc.
 * In order to use a dependency from there you have to add "Dependencies" section to your manifest file (do git grep "Dependencies" to take a look at some examples done in the engine source code).
4. Useful links:
 * Documentation - https://docs.jboss.org/author/display/AS7/Documentation
 * Class loading changes - https://docs.jboss.org/author/display/AS7/Class+Loading+in+AS7
 * The Jboss community - http://community.jboss.org/wiki


Please send issues/feedback to this mailing list.

Thank you all,
Oved


From abaron at redhat.com  Wed Jan 11 13:14:27 2012
From: abaron at redhat.com (Ayal Baron)
Date: Wed, 11 Jan 2012 08:14:27 -0500 (EST)
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <4EF0E321.5090209@redhat.com>
Message-ID: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> Itamar,
> 
> The below, provided by David Lutterkort, is a good description
> of the requirements for Aeolus instance data injection.
> 
> Joe VLcek
> 
>          RHEV-M shall accept a small blob of data as part of the
>          'start
>          VM' action. That data has to be placed somewhere where the
>          VM
>          can easily and securely access it. The data must only be
>          visible
>          to the VM it is intended for.
> 
>          Possibilities for where to put the data include placing it
>          into
>          a file on a virtual floppy or CD-ROM that the instance can
>          mount, or posting it on a webserver that only the instance
>          has
>          access to (cf. EC2's handling of userData for the
>          RunInstances
>          call)
> 
>          The size limitation for the amount of data shouldn't be kept
>          artificially low, but if there are important reasons to make
>          it
>          this small 1k would certainly suffice.

For the above we want to create an iso containing the data (floppy seems too limiting) and pass as a cd to the guest

> 
>          In practical terms, the blob of data should be passed to the
>          'start VM' call base64 encoded, and RHEV-M should decode it
>          just
>          before putting it into its proper place.

This requirement is not clear, why not just pass data as 'binary' to 'start vm' and vdsm would write as is into the file, REST already knows how to encode/decode.


> 
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From yzaslavs at redhat.com  Wed Jan 11 13:20:43 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Wed, 11 Jan 2012 15:20:43 +0200
Subject: [Engine-devel] Moving to Jboss AS7
In-Reply-To: <70a600f8-d58c-4e17-9c87-17378678a7e9@zmail02.collab.prod.int.phx2.redhat.com>
References: <70a600f8-d58c-4e17-9c87-17378678a7e9@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <4F0D8CAB.7080302@redhat.com>

On 01/11/2012 02:57 PM, Oved Ourfalli wrote:
> Hey all,
> 
> The code changes required to make the engine work on Jboss AS7 will soon be push
> It will, of course, require you to install it, and start working with it :-)
> 
> A separate E-mail will be sent to notify you all once pushed, and then you'll have to perform the following steps:
> 
> 1. Download jboss 7.1.0 Beta1b (http://download.jboss.org/jbossas/7.1/jboss-as-7.1.0.Beta1b/jboss-as-7.1.0.Beta1b.tar.gz) - There is a newer version, but it has issues in the REST-API, so we decided to work with the beta version until a proper fix will be released.
> 2. Fetch the latest changes from our git repository
> 3. Change the Jboss home to the new path, both in the JBOSS_HOME environment variable, and in maven settings file (~/.m2/settings.xml)
> 4. Build the engine, with profiles "dep" and "setup". This will put all the proper configuration files, postgresql driver and make all the other needed changes in Jboss in order to make it work properly
> mvn clean install -Psetup,dep .......
> 5. In order to run Jboss go to JBOSS_HOME/bin and run ./standalone.sh
> 6. Look inside the JBOSS_HOME/bin/standalone.conf file in order to enable jboss debugging (just uncomment the line JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n")
> 7. If you have a krb5.conf file you are working with, put it in JBOSS_HOME/standalone/configuration directory
> 8. Run Jboss (and be impressed by the startup speed!)
> 
> The above will be also added to the wiki page for building the engine as soon as the patches will be pushed upstream.
> 
> Some facts about Jboss 7:
> 1. It supports both standalone deployment, and domain deployment. We use the standalone one.
> 2. Stuff related to the standalone mode can be found in the JBOSS_HOME/standalone directory:
>  * configuration - contains the main configuration file called standalone.xml file, plus some other configuration files
>  * deployments - that's where your deployments should be. When adding a new one, don't forget to create a file called "<deployment-name>.dodeploy" in order to make it deploy.
>  * log - that's where the log files are written (unless stated differently in the standalone.xml file).
> 3. The different modules that come with Jboss 7 are located in JBOSS_HOME/modules directory
>  * No more common/lib directory.
>  * Every module has a module.xml file which contains it's dependencies in other modules, the jars that are part of the module, and etc.
>  * In order to use a dependency from there you have to add "Dependencies" section to your manifest file (do git grep "Dependencies" to take a look at some examples done in the engine source code).
> 4. Useful links:
>  * Documentation - https://docs.jboss.org/author/display/AS7/Documentation
>  * Class loading changes - https://docs.jboss.org/author/display/AS7/Class+Loading+in+AS7
>  * The Jboss community - http://community.jboss.org/wiki
> 

Can you explain what should be changed in its script files (or
configuration files?) so one can enable remote debugging?
Thanks,
Yair

> 
> Please send issues/feedback to this mailing list.
> 
> Thank you all,
> Oved
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From yzaslavs at redhat.com  Wed Jan 11 13:28:21 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Wed, 11 Jan 2012 15:28:21 +0200
Subject: [Engine-devel] Moving to Jboss AS7
In-Reply-To: <4F0D8CAB.7080302@redhat.com>
References: <70a600f8-d58c-4e17-9c87-17378678a7e9@zmail02.collab.prod.int.phx2.redhat.com>
	<4F0D8CAB.7080302@redhat.com>
Message-ID: <4F0D8E75.3040306@redhat.com>

On 01/11/2012 03:20 PM, Yair Zaslavsky wrote:
> On 01/11/2012 02:57 PM, Oved Ourfalli wrote:
>> Hey all,
>>
>> The code changes required to make the engine work on Jboss AS7 will soon be push
>> It will, of course, require you to install it, and start working with it :-)
>>
>> A separate E-mail will be sent to notify you all once pushed, and then you'll have to perform the following steps:
>>
>> 1. Download jboss 7.1.0 Beta1b (http://download.jboss.org/jbossas/7.1/jboss-as-7.1.0.Beta1b/jboss-as-7.1.0.Beta1b.tar.gz) - There is a newer version, but it has issues in the REST-API, so we decided to work with the beta version until a proper fix will be released.
>> 2. Fetch the latest changes from our git repository
>> 3. Change the Jboss home to the new path, both in the JBOSS_HOME environment variable, and in maven settings file (~/.m2/settings.xml)
>> 4. Build the engine, with profiles "dep" and "setup". This will put all the proper configuration files, postgresql driver and make all the other needed changes in Jboss in order to make it work properly
>> mvn clean install -Psetup,dep .......
>> 5. In order to run Jboss go to JBOSS_HOME/bin and run ./standalone.sh
>> 6. Look inside the JBOSS_HOME/bin/standalone.conf file in order to enable jboss debugging (just uncomment the line JAVA_OPTS="$JAVA_OPTS -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n")

>> 7. If you have a krb5.conf file you are working with, put it in JBOSS_HOME/standalone/configuration directory
>> 8. Run Jboss (and be impressed by the startup speed!)
>>
>> The above will be also added to the wiki page for building the engine as soon as the patches will be pushed upstream.
>>
>> Some facts about Jboss 7:
>> 1. It supports both standalone deployment, and domain deployment. We use the standalone one.
>> 2. Stuff related to the standalone mode can be found in the JBOSS_HOME/standalone directory:
>>  * configuration - contains the main configuration file called standalone.xml file, plus some other configuration files
>>  * deployments - that's where your deployments should be. When adding a new one, don't forget to create a file called "<deployment-name>.dodeploy" in order to make it deploy.
>>  * log - that's where the log files are written (unless stated differently in the standalone.xml file).
>> 3. The different modules that come with Jboss 7 are located in JBOSS_HOME/modules directory
>>  * No more common/lib directory.
>>  * Every module has a module.xml file which contains it's dependencies in other modules, the jars that are part of the module, and etc.
>>  * In order to use a dependency from there you have to add "Dependencies" section to your manifest file (do git grep "Dependencies" to take a look at some examples done in the engine source code).
>> 4. Useful links:
>>  * Documentation - https://docs.jboss.org/author/display/AS7/Documentation
>>  * Class loading changes - https://docs.jboss.org/author/display/AS7/Class+Loading+in+AS7
>>  * The Jboss community - http://community.jboss.org/wiki
>>
> 
> Can you explain what should be changed in its script files (or
> configuration files?) so one can enable remote debugging?
> Thanks,
> Yair

Actually mentioned at 6 - I guess CTRL+F with "debug" didnt work for me
for some reason. I see it's similar to the way we did it in 5.1.0 GA.
Thanks

Yair

> 
>>
>> Please send issues/feedback to this mailing list.
>>
>> Thank you all,
>> Oved
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From lutter at redhat.com  Wed Jan 11 20:10:04 2012
From: lutter at redhat.com (David Lutterkort)
Date: Wed, 11 Jan 2012 12:10:04 -0800
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>
References: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <1326312604.12462.17.camel@melon.watzmann.net>

On Wed, 2012-01-11 at 08:14 -0500, Ayal Baron wrote:
> 
> ----- Original Message -----
> > Itamar,
> > 
> > The below, provided by David Lutterkort, is a good description
> > of the requirements for Aeolus instance data injection.
> > 
> > Joe VLcek
> > 
> >          RHEV-M shall accept a small blob of data as part of the
> >          'start
> >          VM' action. That data has to be placed somewhere where the
> >          VM
> >          can easily and securely access it. The data must only be
> >          visible
> >          to the VM it is intended for.
> > 
> >          Possibilities for where to put the data include placing it
> >          into
> >          a file on a virtual floppy or CD-ROM that the instance can
> >          mount, or posting it on a webserver that only the instance
> >          has
> >          access to (cf. EC2's handling of userData for the
> >          RunInstances
> >          call)
> > 
> >          The size limitation for the amount of data shouldn't be kept
> >          artificially low, but if there are important reasons to make
> >          it
> >          this small 1k would certainly suffice.
> 
> For the above we want to create an iso containing the data (floppy
> seems too limiting) and pass as a cd to the guest

Just to provide some more background info: for DMTF CIMI, I will be
pusing to standardize the EC2 approach, since it is the only one that
properly decouples the VM from the infrastructure; IOW, the standard
will hopefully mandate that each instance can access a web server at
http://169.254.169.254/ from which it can get the user data.

One of the advantages of this aproach is that it makes it possible to
expose dynamic data to the VM, whereas the floppy injection approach is
limited to static data put together at instance launch.

OpenStack has code for this already (setting up iptables rules,
presenting the VM-specific data in responses, etc.) and it doesn't seem
like a huge undertaking.

I am not saying that oVirt has to adopt that approach, but it would
certainly simplify things for oVirt users, and might well be necessary
at some point when CIMI becomes an official standard.

> > 
> >          In practical terms, the blob of data should be passed to the
> >          'start VM' call base64 encoded, and RHEV-M should decode it
> >          just
> >          before putting it into its proper place.
> 
> This requirement is not clear, why not just pass data as 'binary' to
> 'start vm' and vdsm would write as is into the file, REST already
> knows how to encode/decode.

I mostly mentioned base64 because that's how other API's (EC2, RAX)
encode this data; to be pedantic, it's not REST that determines the
encoding, but how the request payload is serialized (XML, JSON,
application/x-www-form-urlencoded ..) The nice thing about base64 is
that it works with all of them, without requiring add'l encoding, but
ultimately anything works that allows passing arbitrary binary blobs as
user_data.

David




From iheim at redhat.com  Wed Jan 11 20:21:59 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 11 Jan 2012 22:21:59 +0200
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <1326312604.12462.17.camel@melon.watzmann.net>
References: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>
	<1326312604.12462.17.camel@melon.watzmann.net>
Message-ID: <4F0DEF67.5040000@redhat.com>

On 01/11/2012 10:10 PM, David Lutterkort wrote:
> On Wed, 2012-01-11 at 08:14 -0500, Ayal Baron wrote:
>>
>> ----- Original Message -----
>>> Itamar,
>>>
>>> The below, provided by David Lutterkort, is a good description
>>> of the requirements for Aeolus instance data injection.
>>>
>>> Joe VLcek
>>>
>>>           RHEV-M shall accept a small blob of data as part of the
>>>           'start
>>>           VM' action. That data has to be placed somewhere where the
>>>           VM
>>>           can easily and securely access it. The data must only be
>>>           visible
>>>           to the VM it is intended for.
>>>
>>>           Possibilities for where to put the data include placing it
>>>           into
>>>           a file on a virtual floppy or CD-ROM that the instance can
>>>           mount, or posting it on a webserver that only the instance
>>>           has
>>>           access to (cf. EC2's handling of userData for the
>>>           RunInstances
>>>           call)
>>>
>>>           The size limitation for the amount of data shouldn't be kept
>>>           artificially low, but if there are important reasons to make
>>>           it
>>>           this small 1k would certainly suffice.
>>
>> For the above we want to create an iso containing the data (floppy
>> seems too limiting) and pass as a cd to the guest
>
> Just to provide some more background info: for DMTF CIMI, I will be
> pusing to standardize the EC2 approach, since it is the only one that
> properly decouples the VM from the infrastructure; IOW, the standard
> will hopefully mandate that each instance can access a web server at
> http://169.254.169.254/ from which it can get the user data.

so you would provide the real IP of that web server, and oVirt would set 
up a re-route/nat/whatever from 169.254.169.254 to the requested IP?
any details on how this is done at the technical level wrt iptables, etc.?
what parameter are you passing for this?

>
> One of the advantages of this aproach is that it makes it possible to
> expose dynamic data to the VM, whereas the floppy injection approach is
> limited to static data put together at instance launch.
>
> OpenStack has code for this already (setting up iptables rules,
> presenting the VM-specific data in responses, etc.) and it doesn't seem
> like a huge undertaking.
>
> I am not saying that oVirt has to adopt that approach, but it would
> certainly simplify things for oVirt users, and might well be necessary
> at some point when CIMI becomes an official standard.
>
>>>
>>>           In practical terms, the blob of data should be passed to the
>>>           'start VM' call base64 encoded, and RHEV-M should decode it
>>>           just
>>>           before putting it into its proper place.
>>
>> This requirement is not clear, why not just pass data as 'binary' to
>> 'start vm' and vdsm would write as is into the file, REST already
>> knows how to encode/decode.
>
> I mostly mentioned base64 because that's how other API's (EC2, RAX)
> encode this data; to be pedantic, it's not REST that determines the
> encoding, but how the request payload is serialized (XML, JSON,
> application/x-www-form-urlencoded ..) The nice thing about base64 is
> that it works with all of them, without requiring add'l encoding, but
> ultimately anything works that allows passing arbitrary binary blobs as
> user_data.
>
> David
>
>



From berrange at redhat.com  Wed Jan 11 20:25:11 2012
From: berrange at redhat.com (Daniel P. Berrange)
Date: Wed, 11 Jan 2012 20:25:11 +0000
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <1326312604.12462.17.camel@melon.watzmann.net>
References: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>
	<1326312604.12462.17.camel@melon.watzmann.net>
Message-ID: <20120111202511.GB8929@redhat.com>

On Wed, Jan 11, 2012 at 12:10:04PM -0800, David Lutterkort wrote:
> Just to provide some more background info: for DMTF CIMI, I will be
> pusing to standardize the EC2 approach, since it is the only one that
> properly decouples the VM from the infrastructure; IOW, the standard
> will hopefully mandate that each instance can access a web server at
> http://169.254.169.254/ from which it can get the user data.

I trust you will also push to get an IPv6 address standarized,
probably something from the unique local address block:

  https://en.wikipedia.org/wiki/Unique_local_address

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|


From danken at redhat.com  Wed Jan 11 20:35:54 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Wed, 11 Jan 2012 22:35:54 +0200
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <1326312604.12462.17.camel@melon.watzmann.net>
References: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>
	<1326312604.12462.17.camel@melon.watzmann.net>
Message-ID: <20120111203553.GF6723@redhat.com>

On Wed, Jan 11, 2012 at 12:10:04PM -0800, David Lutterkort wrote:
> > > 
> > >          In practical terms, the blob of data should be passed to the
> > >          'start VM' call base64 encoded, and RHEV-M should decode it
> > >          just
> > >          before putting it into its proper place.
> > 
> > This requirement is not clear, why not just pass data as 'binary' to
> > 'start vm' and vdsm would write as is into the file, REST already
> > knows how to encode/decode.
> 
> I mostly mentioned base64 because that's how other API's (EC2, RAX)
> encode6 this data; to be pedantic, it's not REST that determines the
> encoding, but how the request payload is serialized (XML, JSON,
> application/x-www-form-urlencoded ..) The nice thing about base64 is
> that it works with all of them, without requiring add'l encoding, but
> ultimately anything works that allows passing arbitrary binary blobs as
> user_data.

But base64 is the additional encoding... I'm generally reluctant to intorduce
an application-level encoding if the transport already has one. Does any REST
serializer have a problem with natively chewing binary data (I really do not
know)?

Dan.


From lutter at redhat.com  Wed Jan 11 20:48:44 2012
From: lutter at redhat.com (David Lutterkort)
Date: Wed, 11 Jan 2012 12:48:44 -0800
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <4F0DEF67.5040000@redhat.com>
References: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>
	<1326312604.12462.17.camel@melon.watzmann.net>
	<4F0DEF67.5040000@redhat.com>
Message-ID: <1326314924.12462.33.camel@melon.watzmann.net>

On Wed, 2012-01-11 at 22:21 +0200, Itamar Heim wrote:
> On 01/11/2012 10:10 PM, David Lutterkort wrote:
> > On Wed, 2012-01-11 at 08:14 -0500, Ayal Baron wrote:
> >>
> >> ----- Original Message -----
> >>> Itamar,
> >>>
> >>> The below, provided by David Lutterkort, is a good description
> >>> of the requirements for Aeolus instance data injection.
> >>>
> >>> Joe VLcek
> >>>
> >>>           RHEV-M shall accept a small blob of data as part of the
> >>>           'start
> >>>           VM' action. That data has to be placed somewhere where the
> >>>           VM
> >>>           can easily and securely access it. The data must only be
> >>>           visible
> >>>           to the VM it is intended for.
> >>>
> >>>           Possibilities for where to put the data include placing it
> >>>           into
> >>>           a file on a virtual floppy or CD-ROM that the instance can
> >>>           mount, or posting it on a webserver that only the instance
> >>>           has
> >>>           access to (cf. EC2's handling of userData for the
> >>>           RunInstances
> >>>           call)
> >>>
> >>>           The size limitation for the amount of data shouldn't be kept
> >>>           artificially low, but if there are important reasons to make
> >>>           it
> >>>           this small 1k would certainly suffice.
> >>
> >> For the above we want to create an iso containing the data (floppy
> >> seems too limiting) and pass as a cd to the guest
> >
> > Just to provide some more background info: for DMTF CIMI, I will be
> > pusing to standardize the EC2 approach, since it is the only one that
> > properly decouples the VM from the infrastructure; IOW, the standard
> > will hopefully mandate that each instance can access a web server at
> > http://169.254.169.254/ from which it can get the user data.
> 
> so you would provide the real IP of that web server, and oVirt would set 
> up a re-route/nat/whatever from 169.254.169.254 to the requested IP?
> any details on how this is done at the technical level wrt iptables, etc.?
> what parameter are you passing for this?

No, the web server is provided by the cloud/virt infrastructure; the
user_data that comes in from the 'start VM' request is put onto that web
server by the infrastructure, and the infrastructure sets up a route for
the VM so that 169.254.169.254[1] goes to the infrastructure's web
server.

As an example how this can be done (quoting apevec):

        ...
        
        remote IP is used to figure out which instance requested its metadata:
        https://github.com/openstack/nova/blob/master/nova/api/metadata/handler.py#L243
        
        ...
        
        openstack sets iptables rules to forward metadata requests from compute node to the node running openstack-api service:
        https://github.com/openstack/nova/blob/master/nova/network/linux_net.py#L384
        
David


[1] link local address, see http://www.rfc-editor.org/rfc/rfc5735.txt




From jchoate at redhat.com  Thu Jan 12 12:19:48 2012
From: jchoate at redhat.com (Jon Choate)
Date: Thu, 12 Jan 2012 07:19:48 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <cc3faeb9-1f76-49b3-8549-bf5d9dea1d65@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <ca1f6800-a08f-445a-893d-ad63f0832dfb@zmail15.collab.prod.int.phx2.redhat.com>

We are going to be able to store the disks for a template on different storage domains due to the multiple storage domain feature.  Cloning a template will still be possible, but will it provide any value?  Thoughts?


From iheim at redhat.com  Thu Jan 12 13:44:56 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 12 Jan 2012 15:44:56 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <ca1f6800-a08f-445a-893d-ad63f0832dfb@zmail15.collab.prod.int.phx2.redhat.com>
References: <ca1f6800-a08f-445a-893d-ad63f0832dfb@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <4F0EE3D8.30101@redhat.com>

On 01/12/2012 02:19 PM, Jon Choate wrote:
> We are going to be able to store the disks for a template on different storage domains due to the multiple storage domain feature.  Cloning a template will still be possible, but will it provide any value?  Thoughts?

you would want to clone the disks to the storage domains you would want 
to create thinly provisioned disks from for this template.

so a template could have disk1 and disk2 on SD1, then disk2 cloned to 
SD2 to allow creating a VM with thin COW for disk2 on SD2 as well.

> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From jchoate at redhat.com  Thu Jan 12 13:50:09 2012
From: jchoate at redhat.com (Jon Choate)
Date: Thu, 12 Jan 2012 08:50:09 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F0EE3D8.30101@redhat.com>
Message-ID: <9fb105bd-5b8e-49a0-a1ed-0a0d75c0f18d@zmail15.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Jon Choate" <jchoate at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Thursday, January 12, 2012 8:44:56 AM
> Subject: Re: [Engine-devel] the future of template cloning
> 
> On 01/12/2012 02:19 PM, Jon Choate wrote:
> > We are going to be able to store the disks for a template on
> > different storage domains due to the multiple storage domain
> > feature.  Cloning a template will still be possible, but will it
> > provide any value?  Thoughts?
> 
> you would want to clone the disks to the storage domains you would
> want
> to create thinly provisioned disks from for this template.
> 
> so a template could have disk1 and disk2 on SD1, then disk2 cloned to
> SD2 to allow creating a VM with thin COW for disk2 on SD2 as well.
> 

But with the ability to create a template with disk 1 on SD1 and disk2 on SD2 is that still a compelling feature?


> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 


From iheim at redhat.com  Thu Jan 12 13:56:05 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 12 Jan 2012 15:56:05 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <9fb105bd-5b8e-49a0-a1ed-0a0d75c0f18d@zmail15.collab.prod.int.phx2.redhat.com>
References: <9fb105bd-5b8e-49a0-a1ed-0a0d75c0f18d@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <4F0EE675.9020707@redhat.com>

On 01/12/2012 03:50 PM, Jon Choate wrote:
>
>
> ----- Original Message -----
>> From: "Itamar Heim"<iheim at redhat.com>
>> To: "Jon Choate"<jchoate at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Thursday, January 12, 2012 8:44:56 AM
>> Subject: Re: [Engine-devel] the future of template cloning
>>
>> On 01/12/2012 02:19 PM, Jon Choate wrote:
>>> We are going to be able to store the disks for a template on
>>> different storage domains due to the multiple storage domain
>>> feature.  Cloning a template will still be possible, but will it
>>> provide any value?  Thoughts?
>>
>> you would want to clone the disks to the storage domains you would
>> want
>> to create thinly provisioned disks from for this template.
>>
>> so a template could have disk1 and disk2 on SD1, then disk2 cloned to
>> SD2 to allow creating a VM with thin COW for disk2 on SD2 as well.
>>
>
> But with the ability to create a template with disk 1 on SD1 and disk2 on SD2 is that still a compelling feature?

yes, since I may want to create virtual machines from this template 
where the 2nd disk is sometimes on SD1 and sometime on SD2 (just like 
today, where i can instantiate a VM from the template on multiple 
storage domains)
The new feature will just allow me to not clone all the disks of the 
template to all storage domains, rather a subset.


From ewoud+ovirt at kohlvanwijngaarden.nl  Thu Jan 12 14:04:29 2012
From: ewoud+ovirt at kohlvanwijngaarden.nl (Ewoud Kohl van Wijngaarden)
Date: Thu, 12 Jan 2012 15:04:29 +0100
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F0EE675.9020707@redhat.com>
References: <9fb105bd-5b8e-49a0-a1ed-0a0d75c0f18d@zmail15.collab.prod.int.phx2.redhat.com>
	<4F0EE675.9020707@redhat.com>
Message-ID: <20120112140428.GA19086@bogey.xentower.nl>

On Thu, Jan 12, 2012 at 03:56:05PM +0200, Itamar Heim wrote:
> On 01/12/2012 03:50 PM, Jon Choate wrote:
> >> On Thu, Jan 12, 2012 at 8:44:56PM, Itamar Heim wrote:
> >> On 01/12/2012 02:19 PM, Jon Choate wrote:
> >>> We are going to be able to store the disks for a template on
> >>> different storage domains due to the multiple storage domain
> >>> feature.  Cloning a template will still be possible, but will it
> >>> provide any value?  Thoughts?
> >>
> >> you would want to clone the disks to the storage domains you would
> >> want to create thinly provisioned disks from for this template.
> >>
> >> so a template could have disk1 and disk2 on SD1, then disk2 cloned to
> >> SD2 to allow creating a VM with thin COW for disk2 on SD2 as well.
> >>
> >
> > But with the ability to create a template with disk 1 on SD1 and
> > disk2 on SD2 is that still a compelling feature?
>
> yes, since I may want to create virtual machines from this template
> where the 2nd disk is sometimes on SD1 and sometime on SD2 (just
> like today, where i can instantiate a VM from the template on
> multiple storage domains)
> The new feature will just allow me to not clone all the disks of the
> template to all storage domains, rather a subset.
But would this be interesting to the user or would it be better to do
this transparently so the user doesn't have to care? The user just wants
to use template X and store it on storage domain Y with COW. The system
could deduce it has to be cloned first and not bother the user.


From iheim at redhat.com  Thu Jan 12 15:50:56 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 12 Jan 2012 17:50:56 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <20120112140428.GA19086@bogey.xentower.nl>
References: <9fb105bd-5b8e-49a0-a1ed-0a0d75c0f18d@zmail15.collab.prod.int.phx2.redhat.com>
	<4F0EE675.9020707@redhat.com>
	<20120112140428.GA19086@bogey.xentower.nl>
Message-ID: <4F0F0160.1070200@redhat.com>

On 01/12/2012 04:04 PM, Ewoud Kohl van Wijngaarden wrote:
> On Thu, Jan 12, 2012 at 03:56:05PM +0200, Itamar Heim wrote:
>> On 01/12/2012 03:50 PM, Jon Choate wrote:
>>>> On Thu, Jan 12, 2012 at 8:44:56PM, Itamar Heim wrote:
>>>> On 01/12/2012 02:19 PM, Jon Choate wrote:
>>>>> We are going to be able to store the disks for a template on
>>>>> different storage domains due to the multiple storage domain
>>>>> feature.  Cloning a template will still be possible, but will it
>>>>> provide any value?  Thoughts?
>>>>
>>>> you would want to clone the disks to the storage domains you would
>>>> want to create thinly provisioned disks from for this template.
>>>>
>>>> so a template could have disk1 and disk2 on SD1, then disk2 cloned to
>>>> SD2 to allow creating a VM with thin COW for disk2 on SD2 as well.
>>>>
>>>
>>> But with the ability to create a template with disk 1 on SD1 and
>>> disk2 on SD2 is that still a compelling feature?
>>
>> yes, since I may want to create virtual machines from this template
>> where the 2nd disk is sometimes on SD1 and sometime on SD2 (just
>> like today, where i can instantiate a VM from the template on
>> multiple storage domains)
>> The new feature will just allow me to not clone all the disks of the
>> template to all storage domains, rather a subset.
> But would this be interesting to the user or would it be better to do
> this transparently so the user doesn't have to care? The user just wants
> to use template X and store it on storage domain Y with COW. The system
> could deduce it has to be cloned first and not bother the user.

that means a user would be able to affect where the template resides. 
user may not have permissions to do so, would require to configure which 
quota this will happen from, etc.
so I'm not sure doing it implicitly based on a user trying to create a 
disk on a domain.


From ewoud+ovirt at kohlvanwijngaarden.nl  Thu Jan 12 16:47:41 2012
From: ewoud+ovirt at kohlvanwijngaarden.nl (Ewoud Kohl van Wijngaarden)
Date: Thu, 12 Jan 2012 17:47:41 +0100
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F0F0160.1070200@redhat.com>
References: <9fb105bd-5b8e-49a0-a1ed-0a0d75c0f18d@zmail15.collab.prod.int.phx2.redhat.com>
	<4F0EE675.9020707@redhat.com>
	<20120112140428.GA19086@bogey.xentower.nl>
	<4F0F0160.1070200@redhat.com>
Message-ID: <20120112164741.GB19086@bogey.xentower.nl>

On Thu, Jan 12, 2012 at 05:50:56PM +0200, Itamar Heim wrote:
> On 01/12/2012 04:04 PM, Ewoud Kohl van Wijngaarden wrote:
> >On Thu, Jan 12, 2012 at 03:56:05PM +0200, Itamar Heim wrote:
> >>On 01/12/2012 03:50 PM, Jon Choate wrote:
> >>>>On Thu, Jan 12, 2012 at 8:44:56PM, Itamar Heim wrote:
> >>>>On 01/12/2012 02:19 PM, Jon Choate wrote:
> >>>>>We are going to be able to store the disks for a template on
> >>>>>different storage domains due to the multiple storage domain
> >>>>>feature.  Cloning a template will still be possible, but will it
> >>>>>provide any value?  Thoughts?
> >>>>
> >>>>you would want to clone the disks to the storage domains you would
> >>>>want to create thinly provisioned disks from for this template.
> >>>>
> >>>>so a template could have disk1 and disk2 on SD1, then disk2 cloned to
> >>>>SD2 to allow creating a VM with thin COW for disk2 on SD2 as well.
> >>>>
> >>>
> >>>But with the ability to create a template with disk 1 on SD1 and
> >>>disk2 on SD2 is that still a compelling feature?
> >>
> >>yes, since I may want to create virtual machines from this template
> >>where the 2nd disk is sometimes on SD1 and sometime on SD2 (just
> >>like today, where i can instantiate a VM from the template on
> >>multiple storage domains)
> >>The new feature will just allow me to not clone all the disks of the
> >>template to all storage domains, rather a subset.
> >But would this be interesting to the user or would it be better to do
> >this transparently so the user doesn't have to care? The user just wants
> >to use template X and store it on storage domain Y with COW. The system
> >could deduce it has to be cloned first and not bother the user.
>
> that means a user would be able to affect where the template
> resides. user may not have permissions to do so, would require to
> configure which quota this will happen from, etc.
> so I'm not sure doing it implicitly based on a user trying to create
> a disk on a domain.
For my point of view I assume templates are immutable so a copy only
wastes space but I don't see how that matters in this case. If the user
has no permission to copy a template, [s]he has to make full copy of the
template to create the VM anyway. Making a copy and then create a thin
COW VM consumes the same amount of space with as benefit that additional
VMs based on the template can also use COW.

Side note: I haven't at oVirt and my experience is based on RHEV 2.2, so
maybe I've overlooking several new features that would limit this.


From iheim at redhat.com  Thu Jan 12 18:43:56 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 12 Jan 2012 20:43:56 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <20120112164741.GB19086@bogey.xentower.nl>
References: <9fb105bd-5b8e-49a0-a1ed-0a0d75c0f18d@zmail15.collab.prod.int.phx2.redhat.com>
	<4F0EE675.9020707@redhat.com>
	<20120112140428.GA19086@bogey.xentower.nl>
	<4F0F0160.1070200@redhat.com>
	<20120112164741.GB19086@bogey.xentower.nl>
Message-ID: <4F0F29EC.2050400@redhat.com>

On 01/12/2012 06:47 PM, Ewoud Kohl van Wijngaarden wrote:
> On Thu, Jan 12, 2012 at 05:50:56PM +0200, Itamar Heim wrote:
>> On 01/12/2012 04:04 PM, Ewoud Kohl van Wijngaarden wrote:
>>> On Thu, Jan 12, 2012 at 03:56:05PM +0200, Itamar Heim wrote:
>>>> On 01/12/2012 03:50 PM, Jon Choate wrote:
>>>>>> On Thu, Jan 12, 2012 at 8:44:56PM, Itamar Heim wrote:
>>>>>> On 01/12/2012 02:19 PM, Jon Choate wrote:
>>>>>>> We are going to be able to store the disks for a template on
>>>>>>> different storage domains due to the multiple storage domain
>>>>>>> feature.  Cloning a template will still be possible, but will it
>>>>>>> provide any value?  Thoughts?
>>>>>>
>>>>>> you would want to clone the disks to the storage domains you would
>>>>>> want to create thinly provisioned disks from for this template.
>>>>>>
>>>>>> so a template could have disk1 and disk2 on SD1, then disk2 cloned to
>>>>>> SD2 to allow creating a VM with thin COW for disk2 on SD2 as well.
>>>>>>
>>>>>
>>>>> But with the ability to create a template with disk 1 on SD1 and
>>>>> disk2 on SD2 is that still a compelling feature?
>>>>
>>>> yes, since I may want to create virtual machines from this template
>>>> where the 2nd disk is sometimes on SD1 and sometime on SD2 (just
>>>> like today, where i can instantiate a VM from the template on
>>>> multiple storage domains)
>>>> The new feature will just allow me to not clone all the disks of the
>>>> template to all storage domains, rather a subset.
>>> But would this be interesting to the user or would it be better to do
>>> this transparently so the user doesn't have to care? The user just wants
>>> to use template X and store it on storage domain Y with COW. The system
>>> could deduce it has to be cloned first and not bother the user.
>>
>> that means a user would be able to affect where the template
>> resides. user may not have permissions to do so, would require to
>> configure which quota this will happen from, etc.
>> so I'm not sure doing it implicitly based on a user trying to create
>> a disk on a domain.
> For my point of view I assume templates are immutable so a copy only
> wastes space but I don't see how that matters in this case. If the user
> has no permission to copy a template, [s]he has to make full copy of the
> template to create the VM anyway. Making a copy and then create a thin
> COW VM consumes the same amount of space with as benefit that additional
> VMs based on the template can also use COW.

my point is there would be a difference between the admin pre-populating 
the template on the storage domains they wish to allow thinly 
provisioned VMs from it, from the admin quota.

>
> Side note: I haven't at oVirt and my experience is based on RHEV 2.2, so
> maybe I've overlooking several new features that would limit this.



From lutter at redhat.com  Thu Jan 12 18:52:29 2012
From: lutter at redhat.com (David Lutterkort)
Date: Thu, 12 Jan 2012 10:52:29 -0800
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <20120111203553.GF6723@redhat.com>
References: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>
	<1326312604.12462.17.camel@melon.watzmann.net>
	<20120111203553.GF6723@redhat.com>
Message-ID: <1326394349.21474.38.camel@melon.watzmann.net>

On Wed, 2012-01-11 at 22:35 +0200, Dan Kenigsberg wrote:
> On Wed, Jan 11, 2012 at 12:10:04PM -0800, David Lutterkort wrote:
> > > > 
> > > >          In practical terms, the blob of data should be passed to the
> > > >          'start VM' call base64 encoded, and RHEV-M should decode it
> > > >          just
> > > >          before putting it into its proper place.
> > > 
> > > This requirement is not clear, why not just pass data as 'binary' to
> > > 'start vm' and vdsm would write as is into the file, REST already
> > > knows how to encode/decode.
> > 
> > I mostly mentioned base64 because that's how other API's (EC2, RAX)
> > encode6 this data; to be pedantic, it's not REST that determines the
> > encoding, but how the request payload is serialized (XML, JSON,
> > application/x-www-form-urlencoded ..) The nice thing about base64 is
> > that it works with all of them, without requiring add'l encoding, but
> > ultimately anything works that allows passing arbitrary binary blobs as
> > user_data.
> 
> But base64 is the additional encoding... I'm generally reluctant to intorduce
> an application-level encoding if the transport already has one. Does any REST
> serializer have a problem with natively chewing binary data (I really do not
> know)?

There's no such thing as a REST serializer; serialization is handled by
whatever you use to process your payloads (XML, JSON, ...) and the
libraries people use for that have to handle serialization-specific
encodings; base64 is a little friendlier for really underpowered
clients, like shell scripts.

Don't underestimate how much people automate armed with little more than
bash and curl - making that easy is one of the big attractions of REST.

David



From abaron at redhat.com  Thu Jan 12 20:45:04 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 12 Jan 2012 15:45:04 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <ca1f6800-a08f-445a-893d-ad63f0832dfb@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <9486ec23-5155-4a1e-85d0-1a5aefad3098@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> We are going to be able to store the disks for a template on
> different storage domains due to the multiple storage domain
> feature.  Cloning a template will still be possible, but will it
> provide any value?  Thoughts?

I see no relation between the two options.

Scenario 1: I can create a VM with a single disk and create a template from it.
I would still want to be able to clone the template in order to provision VMs from it on different domains.

Scenario 2: same thing with multiple disks on same domain.

Scenario 3: I have a template with 2 disks on 2 different domains (domain A and domain B) and I want to have another copy of the template on domain C and domain D

> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From abaron at redhat.com  Thu Jan 12 21:45:17 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 12 Jan 2012 16:45:17 -0500 (EST)
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <1326394349.21474.38.camel@melon.watzmann.net>
Message-ID: <8077d619-9d9c-4b9b-b135-89435fb63dd2@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On Wed, 2012-01-11 at 22:35 +0200, Dan Kenigsberg wrote:
> > On Wed, Jan 11, 2012 at 12:10:04PM -0800, David Lutterkort wrote:
> > > > > 
> > > > >          In practical terms, the blob of data should be
> > > > >          passed to the
> > > > >          'start VM' call base64 encoded, and RHEV-M should
> > > > >          decode it
> > > > >          just
> > > > >          before putting it into its proper place.
> > > > 
> > > > This requirement is not clear, why not just pass data as
> > > > 'binary' to
> > > > 'start vm' and vdsm would write as is into the file, REST
> > > > already
> > > > knows how to encode/decode.
> > > 
> > > I mostly mentioned base64 because that's how other API's (EC2,
> > > RAX)
> > > encode6 this data; to be pedantic, it's not REST that determines
> > > the
> > > encoding, but how the request payload is serialized (XML, JSON,
> > > application/x-www-form-urlencoded ..) The nice thing about base64
> > > is
> > > that it works with all of them, without requiring add'l encoding,
> > > but
> > > ultimately anything works that allows passing arbitrary binary
> > > blobs as
> > > user_data.
> > 
> > But base64 is the additional encoding... I'm generally reluctant to
> > intorduce
> > an application-level encoding if the transport already has one.
> > Does any REST
> > serializer have a problem with natively chewing binary data (I
> > really do not
> > know)?
> 
> There's no such thing as a REST serializer; serialization is handled
> by
> whatever you use to process your payloads (XML, JSON, ...) and the
> libraries people use for that have to handle serialization-specific
> encodings; base64 is a little friendlier for really underpowered
> clients, like shell scripts.
> 
> Don't underestimate how much people automate armed with little more
> than
> bash and curl - making that easy is one of the big attractions of
> REST.

Still, if the data is already passed encoded to 'start vm' then the guest itself should decode it (encoding is done outside of the system and so should the decoding).


> 
> David
> 
> 


From lutter at redhat.com  Thu Jan 12 22:32:44 2012
From: lutter at redhat.com (David Lutterkort)
Date: Thu, 12 Jan 2012 14:32:44 -0800
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <8077d619-9d9c-4b9b-b135-89435fb63dd2@zmail13.collab.prod.int.phx2.redhat.com>
References: <8077d619-9d9c-4b9b-b135-89435fb63dd2@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <1326407564.21474.53.camel@melon.watzmann.net>

On Thu, 2012-01-12 at 16:45 -0500, Ayal Baron wrote:
> 
> ----- Original Message -----
> > On Wed, 2012-01-11 at 22:35 +0200, Dan Kenigsberg wrote:
> > > On Wed, Jan 11, 2012 at 12:10:04PM -0800, David Lutterkort wrote:
> > > > > > 
> > > > > >          In practical terms, the blob of data should be
> > > > > >          passed to the
> > > > > >          'start VM' call base64 encoded, and RHEV-M should
> > > > > >          decode it
> > > > > >          just
> > > > > >          before putting it into its proper place.
> > > > > 
> > > > > This requirement is not clear, why not just pass data as
> > > > > 'binary' to
> > > > > 'start vm' and vdsm would write as is into the file, REST
> > > > > already
> > > > > knows how to encode/decode.
> > > > 
> > > > I mostly mentioned base64 because that's how other API's (EC2,
> > > > RAX)
> > > > encode6 this data; to be pedantic, it's not REST that determines
> > > > the
> > > > encoding, but how the request payload is serialized (XML, JSON,
> > > > application/x-www-form-urlencoded ..) The nice thing about base64
> > > > is
> > > > that it works with all of them, without requiring add'l encoding,
> > > > but
> > > > ultimately anything works that allows passing arbitrary binary
> > > > blobs as
> > > > user_data.
> > > 
> > > But base64 is the additional encoding... I'm generally reluctant to
> > > intorduce
> > > an application-level encoding if the transport already has one.
> > > Does any REST
> > > serializer have a problem with natively chewing binary data (I
> > > really do not
> > > know)?
> > 
> > There's no such thing as a REST serializer; serialization is handled
> > by
> > whatever you use to process your payloads (XML, JSON, ...) and the
> > libraries people use for that have to handle serialization-specific
> > encodings; base64 is a little friendlier for really underpowered
> > clients, like shell scripts.
> > 
> > Don't underestimate how much people automate armed with little more
> > than
> > bash and curl - making that easy is one of the big attractions of
> > REST.
> 
> Still, if the data is already passed encoded to 'start vm' then the
> guest itself should decode it (encoding is done outside of the system
> and so should the decoding).

Both EC2 and RAX (and Deltacloud) ask for base64 encoded data, and
decode it before presenting it to the instance. Why introduce a
difference for such a minor point ?

David




From lutter at redhat.com  Thu Jan 12 22:42:39 2012
From: lutter at redhat.com (David Lutterkort)
Date: Thu, 12 Jan 2012 14:42:39 -0800
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <1326312604.12462.17.camel@melon.watzmann.net>
References: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>
	<1326312604.12462.17.camel@melon.watzmann.net>
Message-ID: <1326408159.21474.56.camel@melon.watzmann.net>

On Wed, 2012-01-11 at 12:10 -0800, David Lutterkort wrote:
> On Wed, 2012-01-11 at 08:14 -0500, Ayal Baron wrote:
> > 
> Just to provide some more background info: for DMTF CIMI, I will be
> pusing to standardize the EC2 approach, since it is the only one that
> properly decouples the VM from the infrastructure; IOW, the standard
> will hopefully mandate that each instance can access a web server at
> http://169.254.169.254/ from which it can get the user data.

To add more background^2:  OVF has the notion of an 'activation engine',
which includes passing some XML to the VM via a CD-ROM. Depending on
where OVF support is on the ovirt roadmap, we could support user data
injection that way.

David




From abaron at redhat.com  Fri Jan 13 09:07:14 2012
From: abaron at redhat.com (Ayal Baron)
Date: Fri, 13 Jan 2012 04:07:14 -0500 (EST)
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <1326408159.21474.56.camel@melon.watzmann.net>
Message-ID: <3b584b5c-040e-47e0-8e4f-df3aeb721361@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On Wed, 2012-01-11 at 12:10 -0800, David Lutterkort wrote:
> > On Wed, 2012-01-11 at 08:14 -0500, Ayal Baron wrote:
> > > 
> > Just to provide some more background info: for DMTF CIMI, I will be
> > pusing to standardize the EC2 approach, since it is the only one
> > that
> > properly decouples the VM from the infrastructure; IOW, the
> > standard
> > will hopefully mandate that each instance can access a web server
> > at
> > http://169.254.169.254/ from which it can get the user data.
> 
> To add more background^2:  OVF has the notion of an 'activation
> engine',
> which includes passing some XML to the VM via a CD-ROM. Depending on
> where OVF support is on the ovirt roadmap, we could support user data
> injection that way.

Unfortunately oVirt modified ovirt in ways that make it not really usable as an API for passing to the hypervisor.

> 
> David
> 
> 
> 


From lutter at redhat.com  Fri Jan 13 16:23:17 2012
From: lutter at redhat.com (David Lutterkort)
Date: Fri, 13 Jan 2012 08:23:17 -0800
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <3b584b5c-040e-47e0-8e4f-df3aeb721361@zmail13.collab.prod.int.phx2.redhat.com>
References: <3b584b5c-040e-47e0-8e4f-df3aeb721361@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <1326471797.12216.0.camel@avon.watzmann.net>

On Fri, 2012-01-13 at 04:07 -0500, Ayal Baron wrote:
> 
> ----- Original Message -----
> > On Wed, 2012-01-11 at 12:10 -0800, David Lutterkort wrote:
> > > On Wed, 2012-01-11 at 08:14 -0500, Ayal Baron wrote:
> > > > 
> > > Just to provide some more background info: for DMTF CIMI, I will be
> > > pusing to standardize the EC2 approach, since it is the only one
> > > that
> > > properly decouples the VM from the infrastructure; IOW, the
> > > standard
> > > will hopefully mandate that each instance can access a web server
> > > at
> > > http://169.254.169.254/ from which it can get the user data.
> > 
> > To add more background^2:  OVF has the notion of an 'activation
> > engine',
> > which includes passing some XML to the VM via a CD-ROM. Depending on
> > where OVF support is on the ovirt roadmap, we could support user data
> > injection that way.
> 
> Unfortunately oVirt modified ovirt in ways that make it not really
> usable as an API for passing to the hypervisor.

I don't understand.

David




From iheim at redhat.com  Sat Jan 14 12:22:46 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sat, 14 Jan 2012 07:22:46 -0500 (EST)
Subject: [Engine-devel] Requirements for Aeolus instance data injection.
In-Reply-To: <1326408159.21474.56.camel@melon.watzmann.net>
References: <f10c1959-4bd1-482f-b180-fcfb6fa8401b@zmail13.collab.prod.int.phx2.redhat.com>	
	<1326312604.12462.17.camel@melon.watzmann.net>
	<1326408159.21474.56.camel@melon.watzmann.net>
Message-ID: <0fb201ccd2b6$d7c78600$87569200$@com>



> -----Original Message-----
> From: David Lutterkort [mailto:lutter at redhat.com]
> Sent: Friday, January 13, 2012 0:43 AM
> To: Ayal Baron
> Cc: jvlcek at redhat.com; Michal Fojtik; engine-devel at ovirt.org; Itamar
Heim; Shahar Havivi; Dan Kenigsberg
> Subject: Re: [Engine-devel] Requirements for Aeolus instance data
injection.
> 
> On Wed, 2012-01-11 at 12:10 -0800, David Lutterkort wrote:
> > On Wed, 2012-01-11 at 08:14 -0500, Ayal Baron wrote:
> > >
> > Just to provide some more background info: for DMTF CIMI, I will be
> > pusing to standardize the EC2 approach, since it is the only one that
> > properly decouples the VM from the infrastructure; IOW, the standard
> > will hopefully mandate that each instance can access a web server at
> > http://169.254.169.254/ from which it can get the user data.
> 
> To add more background^2:  OVF has the notion of an 'activation engine',
> which includes passing some XML to the VM via a CD-ROM. Depending on
> where OVF support is on the ovirt roadmap, we could support user data
> injection that way.

So iiuc, there are two common methods:
1. accept data via the OVF 'activation engine' field to pass XML via a
CD-ROM (also via a specific field of the VM or Run Once verb I guess, so
no real need to pass it via the OVF). Though I must say I think it is not
the best name for such a field?
2. configure a known IP that the guest can communicate with the engine to
get the activation data from (engine will need to identify the calling
guest uniquely, etc.).

Obviously, #1 is easier to implement and has less security considerations.
I assume it is satisfactory as well?



From lpeer at redhat.com  Sun Jan 15 08:10:41 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 15 Jan 2012 10:10:41 +0200
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <1c0e7728-1362-49cb-b541-81635a543c69@zmail14.collab.prod.int.phx2.redhat.com>
References: <1c0e7728-1362-49cb-b541-81635a543c69@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F128A01.2000005@redhat.com>


>> The engine does not have a single verb for plugging multiple disk in
>> one
>> action.
> I think that question was if it possible to plug/unplug couple of disk in parallel on the same vm, by running couple of hotplug/unplug
> in the same time  

Can you make sure the storage supports that?




From ovedo at redhat.com  Sun Jan 15 09:05:33 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Sun, 15 Jan 2012 04:05:33 -0500 (EST)
Subject: [Engine-devel] Moving to Jboss AS7
In-Reply-To: <70a600f8-d58c-4e17-9c87-17378678a7e9@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <803c568c-626c-4bd9-8287-60c4601e815a@zmail02.collab.prod.int.phx2.redhat.com>

Hey all,
The patches are now pushed!

So please, if you fetch from now on you'll have to perform the actions below.
The wiki was updated as well.

Short list of steps:
1. Download jboss 7.1.0 Beta1b (wget http://download.jboss.org/jbossas/7.1/jboss-as-7.1.0.Beta1b/jboss-as-7.1.0.Beta1b.tar.gz)
2. Fetch the latest changes from our git repository
3. Change the Jboss home to the new path, both in the JBOSS_HOME
environment variable, and in maven settings file
(~/.m2/settings.xml)
4. Build the engine, with profiles "dep" and "setup". This will put
all the proper configuration files, postgresql driver and make all
the other needed changes in Jboss in order to make it work properly
mvn clean install -Psetup,dep .......
5. In order to run Jboss go to JBOSS_HOME/bin and run ./standalone.sh

A more descriptive set of steps and notes can be found in my previous E-mail below.

I'm here if you need any help.

Thank you,
Oved

----- Original Message -----
> From: "Oved Ourfalli" <ovedo at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Wednesday, January 11, 2012 2:57:19 PM
> Subject: Moving to Jboss AS7
> 
> Hey all,
> 
> The code changes required to make the engine work on Jboss AS7 will
> soon be push
> It will, of course, require you to install it, and start working with
> it :-)
> 
> A separate E-mail will be sent to notify you all once pushed, and
> then you'll have to perform the following steps:
> 
> 1. Download jboss 7.1.0 Beta1b
> (http://download.jboss.org/jbossas/7.1/jboss-as-7.1.0.Beta1b/jboss-as-7.1.0.Beta1b.tar.gz)
> - There is a newer version, but it has issues in the REST-API, so we
> decided to work with the beta version until a proper fix will be
> released.
> 2. Fetch the latest changes from our git repository
> 3. Change the Jboss home to the new path, both in the JBOSS_HOME
> environment variable, and in maven settings file
> (~/.m2/settings.xml)
> 4. Build the engine, with profiles "dep" and "setup". This will put
> all the proper configuration files, postgresql driver and make all
> the other needed changes in Jboss in order to make it work properly
> mvn clean install -Psetup,dep .......
> 5. In order to run Jboss go to JBOSS_HOME/bin and run ./standalone.sh
> 6. Look inside the JBOSS_HOME/bin/standalone.conf file in order to
> enable jboss debugging (just uncomment the line
> JAVA_OPTS="$JAVA_OPTS
> -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n")
> 7. If you have a krb5.conf file you are working with, put it in
> JBOSS_HOME/standalone/configuration directory
> 8. Run Jboss (and be impressed by the startup speed!)
> 
> The above will be also added to the wiki page for building the engine
> as soon as the patches will be pushed upstream.
> 
> Some facts about Jboss 7:
> 1. It supports both standalone deployment, and domain deployment. We
> use the standalone one.
> 2. Stuff related to the standalone mode can be found in the
> JBOSS_HOME/standalone directory:
>  * configuration - contains the main configuration file called
>  standalone.xml file, plus some other configuration files
>  * deployments - that's where your deployments should be. When adding
>  a new one, don't forget to create a file called
>  "<deployment-name>.dodeploy" in order to make it deploy.
>  * log - that's where the log files are written (unless stated
>  differently in the standalone.xml file).
> 3. The different modules that come with Jboss 7 are located in
> JBOSS_HOME/modules directory
>  * No more common/lib directory.
>  * Every module has a module.xml file which contains it's
>  dependencies in other modules, the jars that are part of the
>  module, and etc.
>  * In order to use a dependency from there you have to add
>  "Dependencies" section to your manifest file (do git grep
>  "Dependencies" to take a look at some examples done in the engine
>  source code).
> 4. Useful links:
>  * Documentation -
>  https://docs.jboss.org/author/display/AS7/Documentation
>  * Class loading changes -
>  https://docs.jboss.org/author/display/AS7/Class+Loading+in+AS7
>  * The Jboss community - http://community.jboss.org/wiki
> 
> 
> Please send issues/feedback to this mailing list.
> 
> Thank you all,
> Oved


From abaron at redhat.com  Sun Jan 15 09:50:51 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 15 Jan 2012 04:50:51 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <4F0B0B59.4050105@redhat.com>
Message-ID: <bfaf6002-667a-47c7-a3ee-fa80896be36f@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> <snip>
> 
> >>>> - Database (tables, etc.)
> > They are in design
> >>>> - Which cluster level is required?
> > Added to design , it is 3.1
> 
> 3.1 cluster or DC?
> 
> Ayal - is there a storage limitation or can we activate this feature
> for
> any 3.1 cluster?

3.1 cluster

> 
> >>>> - Does it affect exporting a VM? I reckon you export with all
> >>>> disks,
> >>>> with their plug/unplug status?
> > This one I need to check
> 
> Export includes all (non-shared) disks and their status.

How would one export a shared disk?

[snip]

> >>>  - How do you define a system disk? Some VM may boot from
> >>>    pxe/nfsroot/etc
> > I had a property on the disk which means System, also I have a
> > property that means bootable. I suppose that
> > disk can not be plugged/unplugged with any of them

I thought we're making 'system disk' devoid of meaning and not adding any logic to such annotations...

[snip]


From abaron at redhat.com  Sun Jan 15 10:35:26 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 15 Jan 2012 05:35:26 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <1c0e7728-1362-49cb-b541-81635a543c69@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <12a24462-26b8-4b46-a3d4-07a18d2f03ca@zmail13.collab.prod.int.phx2.redhat.com>

[SNIP]

> > >>>> - Any CanDoAction to allow/disallow plug/unplug from specific
> > >>>> systems?
> > > The operation will be allowed only for RHEL 5/6, Windows Server
> > > 2003 and Windows server 2008 operating systems, added to design

IMO this behaviour is wrong.  We should not prevent user from running hot plug on an unsupported system, just warn that this operation might fail and can crash the VM (downstream versions can warn that it is unsupported).
E.g. if user installed a service pack on a windows not listed that adds support for hotplug, why should we prevent running the action?
or worse, a new windows version is released and users of the existing system would not be able to hot plug.

Alternatively, the list of OSs on which this works can be customizable by user in which case a knowledgeable user would modify at her own risk.

IIRC there is an 'other' os type when creating a VM, why block on this? again, either warn or allow customizing behaviour.

> > >>>> - I suspect we'd be happier with some agent cooperation before
> > >>>> unplugging - is this done by QEMU? Not detailed anywhere.

This is not currently planned, but should definitely be on the road map (need to add to the feature page and state that it will not be covered in 3.1)


From emesika at redhat.com  Sun Jan 15 11:46:33 2012
From: emesika at redhat.com (Eli Mesika)
Date: Sun, 15 Jan 2012 06:46:33 -0500 (EST)
Subject: [Engine-devel] please run refreshStoredProcedures after taking
	latest
In-Reply-To: <7945edb3-2477-4924-ad2d-3c951918f08c@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <ae5ce519-625f-4aed-8812-62e042853c3d@zmail13.collab.prod.int.phx2.redhat.com>

Hi

If you take the latest version and run the upgrade script you will have to run the following script also

> refreshStoredProcedures.sh -u <user> -d <database>

This will be fixed ASAP (generally the upgrade script detects such a change and run this script implicitly)


From ilvovsky at redhat.com  Sun Jan 15 12:50:22 2012
From: ilvovsky at redhat.com (Igor Lvovsky)
Date: Sun, 15 Jan 2012 07:50:22 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <7f0387f1-acae-42db-9338-7c92d4f33897@zmail13.collab.prod.int.phx2.redhat.com>
References: <4F128A01.2000005@redhat.com>
	<7f0387f1-acae-42db-9338-7c92d4f33897@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <01b801ccd384$d578a860$8069f920$@com>



> -----Original Message-----
> From: Ayal Baron [mailto:abaron at redhat.com]
> Sent: Sunday, January 15, 2012 11:52 AM
> To: Igor Lvovsky; Eduardo Warszawski
> Cc: Dan Kenigsberg
> Subject: Fwd: [Engine-devel] Low Level design for HotPlug/HotUnplug 
> feature
>
> One of you please reply upstream.
>
> ----- Forwarded Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Michael Kublin" <mkublin at redhat.com>
> Cc: "Ayal Baron" <abaron at redhat.com>, engine-devel at ovirt.org
> Sent: Sunday, January 15, 2012 10:10:41 AM
> Subject: Re: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
>
>
> >> The engine does not have a single verb for plugging multiple disk in
> >> one
> >> action.
> > I think that question was if it possible to plug/unplug couple of disk 
> > in
> parallel on the same vm, by running couple of hotplug/unplug
> > in the same time
>
> Can you make sure the storage supports that?
>

VDSM has hotPlug/hotUnplug verbs for single disk. I don't know about any 
restrictions to run these verbs
simultaneously for several disks.


From emesika at redhat.com  Sun Jan 15 14:30:08 2012
From: emesika at redhat.com (Eli Mesika)
Date: Sun, 15 Jan 2012 09:30:08 -0500 (EST)
Subject: [Engine-devel] please run refreshStoredProcedures after
	taking	latest
In-Reply-To: <ae5ce519-625f-4aed-8812-62e042853c3d@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <34e7177a-0299-4fab-87c4-fe698601399b@zmail13.collab.prod.int.phx2.redhat.com>

After checking that, this was a result of pushing an upgrade script with a syntax error (missing semi-colon at end of line)
This was fixed and pushed upstream
getting latest and running the upgrade script should do the work.
Thanks
 

----- Original Message -----
> From: "Eli Mesika" <emesika at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Sunday, January 15, 2012 1:46:33 PM
> Subject: [Engine-devel] please run refreshStoredProcedures after taking	latest
> 
> Hi
> 
> If you take the latest version and run the upgrade script you will
> have to run the following script also
> 
> > refreshStoredProcedures.sh -u <user> -d <database>
> 
> This will be fixed ASAP (generally the upgrade script detects such a
> change and run this script implicitly)
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From oschreib at redhat.com  Sun Jan 15 16:36:19 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Sun, 15 Jan 2012 11:36:19 -0500 (EST)
Subject: [Engine-devel] New engine_3.0 branch
Message-ID: <05a701ccd3a3$d086b1c0$71941540$@redhat.com>

Hey all,

Heading towards our first release, I've just created a new git branch,
named engine_3.0 on gerrit.ovirt.org.
This branch is based on commit hash 7af27 and will be used for producing
our first release, scheduled for the 31st of January.

If you're fixing an existing bug in the system, and you feel it's
important enough to get into the 3.0 release, please contact your
component owner and push your fix into the release branch (just rebase
against engine_3.0, and push your commit for review using `git push
gerrit.ovirt.org:ovirt-engine-HEAD:refs/for/engine_3.0`).

Regards, 

--
Ofer Schreiber 
Red Hat Israel, Ra'anana

+972-9-7692347
www.redhat.com




From shuming at linux.vnet.ibm.com  Mon Jan 16 14:45:29 2012
From: shuming at linux.vnet.ibm.com (Shu Ming)
Date: Mon, 16 Jan 2012 22:45:29 +0800
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <a326aab2-28ec-4147-9265-9554cc356077@zmail14.collab.prod.int.phx2.redhat.com>
References: <a326aab2-28ec-4147-9265-9554cc356077@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F143809.3060603@linux.vnet.ibm.com>

Suppose, we unplug and then re-plug a disk, will VDSM expect the disk be 
in the same device path as before?  More generic, will the unplugging 
and plugging keep the PCI device path as the same as before?

On 2012-1-9 17:21, Michael Kublin wrote:
> Hi, the follow link is providing a low level design for HotPlug/HotUnplug feature : http://www.ovirt.org/wiki/Features/DetailedHotPlug .
>
> The feature is simple and design is short
>
> Regards Michael
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>


-- 
Shu Ming<shuming at linux.vnet.ibm.com>
IBM China Systems and Technology Laboratory




From lpeer at redhat.com  Mon Jan 16 14:46:40 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Mon, 16 Jan 2012 16:46:40 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <9486ec23-5155-4a1e-85d0-1a5aefad3098@zmail13.collab.prod.int.phx2.redhat.com>
References: <9486ec23-5155-4a1e-85d0-1a5aefad3098@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F143850.7090508@redhat.com>

On 12/01/12 22:45, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> We are going to be able to store the disks for a template on
>> different storage domains due to the multiple storage domain
>> feature.  Cloning a template will still be possible, but will it
>> provide any value?  Thoughts?
> 
> I see no relation between the two options.
> 
> Scenario 1: I can create a VM with a single disk and create a template from it.
> I would still want to be able to clone the template in order to provision VMs from it on different domains.
> 
> Scenario 2: same thing with multiple disks on same domain.
> 
> Scenario 3: I have a template with 2 disks on 2 different domains (domain A and domain B) and I want to have another copy of the template on domain C and domain D
> 

Hi Jon,

After talking to Michael Pasternak it seems that we did not implemented
copyTemplate in the REST API, it seems to be a gap that we have.

This gap is playing in our favor, we can remove the copyTemplate verb
and introduce copyDisk verb.

The template disks can be copied to another SD.
When creating a VM from template the user can choose per disk the
destination SD (only SD with the disks are eligible candidates).

Livnat


From jchoate at redhat.com  Mon Jan 16 15:26:58 2012
From: jchoate at redhat.com (Jon Choate)
Date: Mon, 16 Jan 2012 10:26:58 -0500 (EST)
Subject: [Engine-devel] move disk command
In-Reply-To: <5ba93729-52fc-4b6e-9372-599caaf2221c@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <6682702d-55ce-40af-8ec4-26ea7efb378e@zmail15.collab.prod.int.phx2.redhat.com>

As part of the multiple storage domains feature there will be new functionality added to allow users to move individual disks. 

What are the prerequisites for moving a disk?

1) the disk must exist
2) the associated VM must be down
3) the associated VM must not be locked
4) the source storage domain must exist
5) the source storage domain must be available
6) the target domain must exist
7) the target domain must be available
8) the target domain must have adequate disk space
9) the target domain cannot be an ISO or export domain
10) the source domain cannot be an ISO or export domain

What am I missing?

Also, should we allow the moving of a template disk that has VM disks based on it? Unless I'm wrong this would require all of the disks based on the template to be moved as well.

thoughts?


From jchoate at redhat.com  Mon Jan 16 15:46:39 2012
From: jchoate at redhat.com (Jon Choate)
Date: Mon, 16 Jan 2012 10:46:39 -0500
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F143850.7090508@redhat.com>
References: <9486ec23-5155-4a1e-85d0-1a5aefad3098@zmail13.collab.prod.int.phx2.redhat.com>
	<4F143850.7090508@redhat.com>
Message-ID: <4F14465F.4000407@redhat.com>

On 01/16/2012 09:46 AM, Livnat Peer wrote:
> On 12/01/12 22:45, Ayal Baron wrote:
>>
>> ----- Original Message -----
>>> We are going to be able to store the disks for a template on
>>> different storage domains due to the multiple storage domain
>>> feature.  Cloning a template will still be possible, but will it
>>> provide any value?  Thoughts?
>> I see no relation between the two options.
>>
>> Scenario 1: I can create a VM with a single disk and create a template from it.
>> I would still want to be able to clone the template in order to provision VMs from it on different domains.
>>
>> Scenario 2: same thing with multiple disks on same domain.
>>
>> Scenario 3: I have a template with 2 disks on 2 different domains (domain A and domain B) and I want to have another copy of the template on domain C and domain D
>>
> Hi Jon,
>
> After talking to Michael Pasternak it seems that we did not implemented
> copyTemplate in the REST API, it seems to be a gap that we have.
>
> This gap is playing in our favor, we can remove the copyTemplate verb
> and introduce copyDisk verb.
>
> The template disks can be copied to another SD.
> When creating a VM from template the user can choose per disk the
> destination SD (only SD with the disks are eligible candidates).
wait, when creating a VM from a template, the user won't get a choice 
will they? Won't the VM disks have to go on the same storage domain as 
the template disks they were created from?
>
> Livnat



From jchoate at redhat.com  Mon Jan 16 15:48:45 2012
From: jchoate at redhat.com (Jon Choate)
Date: Mon, 16 Jan 2012 10:48:45 -0500
Subject: [Engine-devel] move disk command
In-Reply-To: <6682702d-55ce-40af-8ec4-26ea7efb378e@zmail15.collab.prod.int.phx2.redhat.com>
References: <6682702d-55ce-40af-8ec4-26ea7efb378e@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1446DD.8070907@redhat.com>

On 01/16/2012 10:26 AM, Jon Choate wrote:
> As part of the multiple storage domains feature there will be new functionality added to allow users to move individual disks.
>
> What are the prerequisites for moving a disk?
>
> 1) the disk must exist
> 2) the associated VM must be down
> 3) the associated VM must not be locked
> 4) the source storage domain must exist
> 5) the source storage domain must be available
> 6) the target domain must exist
> 7) the target domain must be available
> 8) the target domain must have adequate disk space
> 9) the target domain cannot be an ISO or export domain
> 10) the source domain cannot be an ISO or export domain
11) the source and destination domains have to be different!
>
> What am I missing?
>
> Also, should we allow the moving of a template disk that has VM disks based on it? Unless I'm wrong this would require all of the disks based on the template to be moved as well.
>
> thoughts?
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From iheim at redhat.com  Mon Jan 16 15:58:40 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 16 Jan 2012 17:58:40 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F14465F.4000407@redhat.com>
References: <9486ec23-5155-4a1e-85d0-1a5aefad3098@zmail13.collab.prod.int.phx2.redhat.com>
	<4F143850.7090508@redhat.com> <4F14465F.4000407@redhat.com>
Message-ID: <4F144930.4060009@redhat.com>

On 01/16/2012 05:46 PM, Jon Choate wrote:
> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>> On 12/01/12 22:45, Ayal Baron wrote:
>>>
>>> ----- Original Message -----
>>>> We are going to be able to store the disks for a template on
>>>> different storage domains due to the multiple storage domain
>>>> feature. Cloning a template will still be possible, but will it
>>>> provide any value? Thoughts?
>>> I see no relation between the two options.
>>>
>>> Scenario 1: I can create a VM with a single disk and create a
>>> template from it.
>>> I would still want to be able to clone the template in order to
>>> provision VMs from it on different domains.
>>>
>>> Scenario 2: same thing with multiple disks on same domain.
>>>
>>> Scenario 3: I have a template with 2 disks on 2 different domains
>>> (domain A and domain B) and I want to have another copy of the
>>> template on domain C and domain D
>>>
>> Hi Jon,
>>
>> After talking to Michael Pasternak it seems that we did not implemented
>> copyTemplate in the REST API, it seems to be a gap that we have.
>>
>> This gap is playing in our favor, we can remove the copyTemplate verb
>> and introduce copyDisk verb.
>>
>> The template disks can be copied to another SD.
>> When creating a VM from template the user can choose per disk the
>> destination SD (only SD with the disks are eligible candidates).
> wait, when creating a VM from a template, the user won't get a choice
> will they? Won't the VM disks have to go on the same storage domain as
> the template disks they were created from?

yes, but the template disks can be copied to multiple storage domains, 
so the user can choose for the VM/disk which storage domain to create 
them from (per storage domains that have copies of that disk)


From iheim at redhat.com  Mon Jan 16 16:05:46 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 16 Jan 2012 18:05:46 +0200
Subject: [Engine-devel] move disk command
In-Reply-To: <6682702d-55ce-40af-8ec4-26ea7efb378e@zmail15.collab.prod.int.phx2.redhat.com>
References: <6682702d-55ce-40af-8ec4-26ea7efb378e@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <4F144ADA.9080602@redhat.com>

On 01/16/2012 05:26 PM, Jon Choate wrote:
> As part of the multiple storage domains feature there will be new functionality added to allow users to move individual disks.
>
> What are the prerequisites for moving a disk?
>
> 1) the disk must exist
> 2) the associated VM must be down

this can't just be a CanDoAction check - the lock has to be real to 
prevent a race from starting the VM after the validation.

> 3) the associated VM must not be locked
> 4) the source storage domain must exist
> 5) the source storage domain must be available
> 6) the target domain must exist
> 7) the target domain must be available
> 8) the target domain must have adequate disk space
> 9) the target domain cannot be an ISO or export domain
> 10) the source domain cannot be an ISO or export domain

user must provide same/other quota for the target domain which has 
enough quota left for the requested size.

>
> What am I missing?
>
> Also, should we allow the moving of a template disk that has VM disks based on it? Unless I'm wrong this would require all of the disks based on the template to be moved as well.

I'd say no. you can only move a template disk if it is not used.
I'd go further and say one should copy the template disk and delete, 
rather than support move for it at all (not relevant for VM disk, since 
we don't have the same concept of multiple copies for it).

>
> thoughts?
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From jchoate at redhat.com  Mon Jan 16 16:16:50 2012
From: jchoate at redhat.com (Jon Choate)
Date: Mon, 16 Jan 2012 11:16:50 -0500
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F144930.4060009@redhat.com>
References: <9486ec23-5155-4a1e-85d0-1a5aefad3098@zmail13.collab.prod.int.phx2.redhat.com>
	<4F143850.7090508@redhat.com> <4F14465F.4000407@redhat.com>
	<4F144930.4060009@redhat.com>
Message-ID: <4F144D72.6010609@redhat.com>

On 01/16/2012 10:58 AM, Itamar Heim wrote:
> On 01/16/2012 05:46 PM, Jon Choate wrote:
>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>
>>>> ----- Original Message -----
>>>>> We are going to be able to store the disks for a template on
>>>>> different storage domains due to the multiple storage domain
>>>>> feature. Cloning a template will still be possible, but will it
>>>>> provide any value? Thoughts?
>>>> I see no relation between the two options.
>>>>
>>>> Scenario 1: I can create a VM with a single disk and create a
>>>> template from it.
>>>> I would still want to be able to clone the template in order to
>>>> provision VMs from it on different domains.
>>>>
>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>
>>>> Scenario 3: I have a template with 2 disks on 2 different domains
>>>> (domain A and domain B) and I want to have another copy of the
>>>> template on domain C and domain D
>>>>
>>> Hi Jon,
>>>
>>> After talking to Michael Pasternak it seems that we did not implemented
>>> copyTemplate in the REST API, it seems to be a gap that we have.
>>>
>>> This gap is playing in our favor, we can remove the copyTemplate verb
>>> and introduce copyDisk verb.
>>>
>>> The template disks can be copied to another SD.
>>> When creating a VM from template the user can choose per disk the
>>> destination SD (only SD with the disks are eligible candidates).
>> wait, when creating a VM from a template, the user won't get a choice
>> will they? Won't the VM disks have to go on the same storage domain as
>> the template disks they were created from?
>
> yes, but the template disks can be copied to multiple storage domains, 
> so the user can choose for the VM/disk which storage domain to create 
> them from (per storage domains that have copies of that disk)
OH! I totally misunderstood.  So what you are saying is that a template 
can have N number of copies of the same disk each on a different storage 
domain. I had thought that if you wanted that type of situation you 
would have multiple copies of the template itself too.

Just to be clear, does this mean that the plan is to phase out the 
current clone template command and instead implementing a clone disk 
command so that a template can duplicate its disks individually?


From abaron at redhat.com  Mon Jan 16 18:58:36 2012
From: abaron at redhat.com (Ayal Baron)
Date: Mon, 16 Jan 2012 13:58:36 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F145DC0.7080807@redhat.com>
Message-ID: <349ff6ef-c17d-4df7-957a-7baae67e699f@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/16/2012 06:16 PM, Jon Choate wrote:
> > On 01/16/2012 10:58 AM, Itamar Heim wrote:
> >> On 01/16/2012 05:46 PM, Jon Choate wrote:
> >>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
> >>>> On 12/01/12 22:45, Ayal Baron wrote:
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> We are going to be able to store the disks for a template on
> >>>>>> different storage domains due to the multiple storage domain
> >>>>>> feature. Cloning a template will still be possible, but will
> >>>>>> it
> >>>>>> provide any value? Thoughts?
> >>>>> I see no relation between the two options.
> >>>>>
> >>>>> Scenario 1: I can create a VM with a single disk and create a
> >>>>> template from it.
> >>>>> I would still want to be able to clone the template in order to
> >>>>> provision VMs from it on different domains.
> >>>>>
> >>>>> Scenario 2: same thing with multiple disks on same domain.
> >>>>>
> >>>>> Scenario 3: I have a template with 2 disks on 2 different
> >>>>> domains
> >>>>> (domain A and domain B) and I want to have another copy of the
> >>>>> template on domain C and domain D
> >>>>>
> >>>> Hi Jon,
> >>>>
> >>>> After talking to Michael Pasternak it seems that we did not
> >>>> implemented
> >>>> copyTemplate in the REST API, it seems to be a gap that we have.
> >>>>
> >>>> This gap is playing in our favor, we can remove the copyTemplate
> >>>> verb
> >>>> and introduce copyDisk verb.
> >>>>
> >>>> The template disks can be copied to another SD.
> >>>> When creating a VM from template the user can choose per disk
> >>>> the
> >>>> destination SD (only SD with the disks are eligible candidates).
> >>> wait, when creating a VM from a template, the user won't get a
> >>> choice
> >>> will they? Won't the VM disks have to go on the same storage
> >>> domain as
> >>> the template disks they were created from?
> >>
> >> yes, but the template disks can be copied to multiple storage
> >> domains,
> >> so the user can choose for the VM/disk which storage domain to
> >> create
> >> them from (per storage domains that have copies of that disk)
> > OH! I totally misunderstood. So what you are saying is that a
> > template
> > can have N number of copies of the same disk each on a different
> > storage
> > domain. I had thought that if you wanted that type of situation you
> > would have multiple copies of the template itself too.
> >
> > Just to be clear, does this mean that the plan is to phase out the
> > current clone template command and instead implementing a clone
> > disk
> > command so that a template can duplicate its disks individually?
> 
> pretty much, yes.
> though i'd imagine 'clone template' would still be useful to have for
> the user. not sure if it implies core should expose it as well to
> allow
> easier usage at UI level for such a task.
> 

Just bear in mind that user should still be able to export an entire template (should be single action to choose export domain and that's it).

When importing a template, user should be able to specify per disk the destination domain.
(default should keep all disks in same domain)

When creating a VM from template there should still be a default for each disk, preferably user configurable default (it would be very annoying to deploy 20 VMs from the template and changing the default every time).

Clone VM from template should allow user to choose per disk *any* storage domain even if it doesn't have a copy of the template disk.

I'm assuming template versioning would require user to recopy all the disks, I would add a checkbox to initiate copy to the same domains as the parent template.  If child template has additional disks - no special treatment should be given (user can later on copy those disks wherever she chooses)


From jchoate at redhat.com  Mon Jan 16 19:21:49 2012
From: jchoate at redhat.com (Jon Choate)
Date: Mon, 16 Jan 2012 14:21:49 -0500
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <349ff6ef-c17d-4df7-957a-7baae67e699f@zmail13.collab.prod.int.phx2.redhat.com>
References: <349ff6ef-c17d-4df7-957a-7baae67e699f@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1478CD.1070200@redhat.com>

On 01/16/2012 01:58 PM, Ayal Baron wrote:
>
> ----- Original Message -----
>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>> ----- Original Message -----
>>>>>>>> We are going to be able to store the disks for a template on
>>>>>>>> different storage domains due to the multiple storage domain
>>>>>>>> feature. Cloning a template will still be possible, but will
>>>>>>>> it
>>>>>>>> provide any value? Thoughts?
>>>>>>> I see no relation between the two options.
>>>>>>>
>>>>>>> Scenario 1: I can create a VM with a single disk and create a
>>>>>>> template from it.
>>>>>>> I would still want to be able to clone the template in order to
>>>>>>> provision VMs from it on different domains.
>>>>>>>
>>>>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>>>>
>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>> domains
>>>>>>> (domain A and domain B) and I want to have another copy of the
>>>>>>> template on domain C and domain D
>>>>>>>
>>>>>> Hi Jon,
>>>>>>
>>>>>> After talking to Michael Pasternak it seems that we did not
>>>>>> implemented
>>>>>> copyTemplate in the REST API, it seems to be a gap that we have.
>>>>>>
>>>>>> This gap is playing in our favor, we can remove the copyTemplate
>>>>>> verb
>>>>>> and introduce copyDisk verb.
>>>>>>
>>>>>> The template disks can be copied to another SD.
>>>>>> When creating a VM from template the user can choose per disk
>>>>>> the
>>>>>> destination SD (only SD with the disks are eligible candidates).
>>>>> wait, when creating a VM from a template, the user won't get a
>>>>> choice
>>>>> will they? Won't the VM disks have to go on the same storage
>>>>> domain as
>>>>> the template disks they were created from?
>>>> yes, but the template disks can be copied to multiple storage
>>>> domains,
>>>> so the user can choose for the VM/disk which storage domain to
>>>> create
>>>> them from (per storage domains that have copies of that disk)
>>> OH! I totally misunderstood. So what you are saying is that a
>>> template
>>> can have N number of copies of the same disk each on a different
>>> storage
>>> domain. I had thought that if you wanted that type of situation you
>>> would have multiple copies of the template itself too.
>>>
>>> Just to be clear, does this mean that the plan is to phase out the
>>> current clone template command and instead implementing a clone
>>> disk
>>> command so that a template can duplicate its disks individually?
>> pretty much, yes.
>> though i'd imagine 'clone template' would still be useful to have for
>> the user. not sure if it implies core should expose it as well to
>> allow
>> easier usage at UI level for such a task.
>>
> Just bear in mind that user should still be able to export an entire template (should be single action to choose export domain and that's it).
>
> When importing a template, user should be able to specify per disk the destination domain.
> (default should keep all disks in same domain)
>
> When creating a VM from template there should still be a default for each disk, preferably user configurable default (it would be very annoying to deploy 20 VMs from the template and changing the default every time).
>
> Clone VM from template should allow user to choose per disk *any* storage domain even if it doesn't have a copy of the template disk.
Does this imply that we would copy the template behind the scenes for 
them or does it mean we are going to let them put the disk in a place 
where it can't work?

> I'm assuming template versioning would require user to recopy all the disks, I would add a checkbox to initiate copy to the same domains as the parent template.  If child template has additional disks - no special treatment should be given (user can later on copy those disks wherever she chooses)



From abaron at redhat.com  Mon Jan 16 19:26:48 2012
From: abaron at redhat.com (Ayal Baron)
Date: Mon, 16 Jan 2012 14:26:48 -0500 (EST)
Subject: [Engine-devel] move disk command
In-Reply-To: <4F144ADA.9080602@redhat.com>
Message-ID: <0f84cc5c-5e7b-43d7-92c8-664cb0d7de4e@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/16/2012 05:26 PM, Jon Choate wrote:
> > As part of the multiple storage domains feature there will be new
> > functionality added to allow users to move individual disks.
> >
> > What are the prerequisites for moving a disk?
> >
> > 1) the disk must exist
> > 2) the associated VM must be down
> 
> this can't just be a CanDoAction check - the lock has to be real to
> prevent a race from starting the VM after the validation.
> 

Either down or disk is unplugged.

> > 3) the associated VM must not be locked
> > 4) the source storage domain must exist
> > 5) the source storage domain must be available
> > 6) the target domain must exist
> > 7) the target domain must be available
> > 8) the target domain must have adequate disk space
> > 9) the target domain cannot be an ISO or export domain
> > 10) the source domain cannot be an ISO or export domain

This may be unrelated, but user would be allowed to export and import a floating disk, right?
I would like the ability to import *any* disk in the export domain as a floating disk, but in the least, export and import disks not associated with a VM.

> 
> user must provide same/other quota for the target domain which has
> enough quota left for the requested size.
> 
> >
> > What am I missing?
> >
> > Also, should we allow the moving of a template disk that has VM
> > disks based on it? Unless I'm wrong this would require all of the
> > disks based on the template to be moved as well.
> 
> I'd say no. you can only move a template disk if it is not used.
> I'd go further and say one should copy the template disk and delete,
> rather than support move for it at all (not relevant for VM disk,
> since
> we don't have the same concept of multiple copies for it).

As long as you can delete a copy of the disk from a domain where there are no VM disks derived from it.

> 
> >
> > thoughts?
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From jchoate at redhat.com  Mon Jan 16 19:39:55 2012
From: jchoate at redhat.com (Jon Choate)
Date: Mon, 16 Jan 2012 14:39:55 -0500
Subject: [Engine-devel] move disk command
In-Reply-To: <0f84cc5c-5e7b-43d7-92c8-664cb0d7de4e@zmail13.collab.prod.int.phx2.redhat.com>
References: <0f84cc5c-5e7b-43d7-92c8-664cb0d7de4e@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F147D0B.2000504@redhat.com>

On 01/16/2012 02:26 PM, Ayal Baron wrote:
>
> ----- Original Message -----
>> On 01/16/2012 05:26 PM, Jon Choate wrote:
>>> As part of the multiple storage domains feature there will be new
>>> functionality added to allow users to move individual disks.
>>>
>>> What are the prerequisites for moving a disk?
>>>
>>> 1) the disk must exist
>>> 2) the associated VM must be down
>> this can't just be a CanDoAction check - the lock has to be real to
>> prevent a race from starting the VM after the validation.
>>
> Either down or disk is unplugged.
>
>>> 3) the associated VM must not be locked
>>> 4) the source storage domain must exist
>>> 5) the source storage domain must be available
>>> 6) the target domain must exist
>>> 7) the target domain must be available
>>> 8) the target domain must have adequate disk space
>>> 9) the target domain cannot be an ISO or export domain
>>> 10) the source domain cannot be an ISO or export domain
> This may be unrelated, but user would be allowed to export and import a floating disk, right?
> I would like the ability to import *any* disk in the export domain as a floating disk, but in the least, export and import disks not associated with a VM.
This was not in scope for the work I am currently doing.  If this is 
something desirable I think it needs to be prioritized and worked in at 
a later time.  If it does need to happen now then we are going to need 
to be able to do full crud for a floating disk I would think.

>> user must provide same/other quota for the target domain which has
>> enough quota left for the requested size.
>>
>>> What am I missing?
>>>
>>> Also, should we allow the moving of a template disk that has VM
>>> disks based on it? Unless I'm wrong this would require all of the
>>> disks based on the template to be moved as well.
>> I'd say no. you can only move a template disk if it is not used.
>> I'd go further and say one should copy the template disk and delete,
>> rather than support move for it at all (not relevant for VM disk,
>> since
>> we don't have the same concept of multiple copies for it).
> As long as you can delete a copy of the disk from a domain where there are no VM disks derived from it.
>
>>> thoughts?
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From abaron at redhat.com  Mon Jan 16 20:06:35 2012
From: abaron at redhat.com (Ayal Baron)
Date: Mon, 16 Jan 2012 15:06:35 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F1478CD.1070200@redhat.com>
Message-ID: <e40ad5a4-2790-4dda-b971-c0179b507dc9@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/16/2012 01:58 PM, Ayal Baron wrote:
> >
> > ----- Original Message -----
> >> On 01/16/2012 06:16 PM, Jon Choate wrote:
> >>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
> >>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
> >>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
> >>>>>> On 12/01/12 22:45, Ayal Baron wrote:
> >>>>>>> ----- Original Message -----
> >>>>>>>> We are going to be able to store the disks for a template on
> >>>>>>>> different storage domains due to the multiple storage domain
> >>>>>>>> feature. Cloning a template will still be possible, but will
> >>>>>>>> it
> >>>>>>>> provide any value? Thoughts?
> >>>>>>> I see no relation between the two options.
> >>>>>>>
> >>>>>>> Scenario 1: I can create a VM with a single disk and create a
> >>>>>>> template from it.
> >>>>>>> I would still want to be able to clone the template in order
> >>>>>>> to
> >>>>>>> provision VMs from it on different domains.
> >>>>>>>
> >>>>>>> Scenario 2: same thing with multiple disks on same domain.
> >>>>>>>
> >>>>>>> Scenario 3: I have a template with 2 disks on 2 different
> >>>>>>> domains
> >>>>>>> (domain A and domain B) and I want to have another copy of
> >>>>>>> the
> >>>>>>> template on domain C and domain D
> >>>>>>>
> >>>>>> Hi Jon,
> >>>>>>
> >>>>>> After talking to Michael Pasternak it seems that we did not
> >>>>>> implemented
> >>>>>> copyTemplate in the REST API, it seems to be a gap that we
> >>>>>> have.
> >>>>>>
> >>>>>> This gap is playing in our favor, we can remove the
> >>>>>> copyTemplate
> >>>>>> verb
> >>>>>> and introduce copyDisk verb.
> >>>>>>
> >>>>>> The template disks can be copied to another SD.
> >>>>>> When creating a VM from template the user can choose per disk
> >>>>>> the
> >>>>>> destination SD (only SD with the disks are eligible
> >>>>>> candidates).
> >>>>> wait, when creating a VM from a template, the user won't get a
> >>>>> choice
> >>>>> will they? Won't the VM disks have to go on the same storage
> >>>>> domain as
> >>>>> the template disks they were created from?
> >>>> yes, but the template disks can be copied to multiple storage
> >>>> domains,
> >>>> so the user can choose for the VM/disk which storage domain to
> >>>> create
> >>>> them from (per storage domains that have copies of that disk)
> >>> OH! I totally misunderstood. So what you are saying is that a
> >>> template
> >>> can have N number of copies of the same disk each on a different
> >>> storage
> >>> domain. I had thought that if you wanted that type of situation
> >>> you
> >>> would have multiple copies of the template itself too.
> >>>
> >>> Just to be clear, does this mean that the plan is to phase out
> >>> the
> >>> current clone template command and instead implementing a clone
> >>> disk
> >>> command so that a template can duplicate its disks individually?
> >> pretty much, yes.
> >> though i'd imagine 'clone template' would still be useful to have
> >> for
> >> the user. not sure if it implies core should expose it as well to
> >> allow
> >> easier usage at UI level for such a task.
> >>
> > Just bear in mind that user should still be able to export an
> > entire template (should be single action to choose export domain
> > and that's it).
> >
> > When importing a template, user should be able to specify per disk
> > the destination domain.
> > (default should keep all disks in same domain)
> >
> > When creating a VM from template there should still be a default
> > for each disk, preferably user configurable default (it would be
> > very annoying to deploy 20 VMs from the template and changing the
> > default every time).
> >
> > Clone VM from template should allow user to choose per disk *any*
> > storage domain even if it doesn't have a copy of the template
> > disk.
> Does this imply that we would copy the template behind the scenes for
> them or does it mean we are going to let them put the disk in a place
> where it can't work?

Please empty lines to separate your responses, it is really difficult to find them.

Anyway, clone VM copies all the bits and does not require the template.
Create VM from template without clone would only allow specifying per disk domains that have a copy of the template disk.

> 
> > I'm assuming template versioning would require user to recopy all
> > the disks, I would add a checkbox to initiate copy to the same
> > domains as the parent template.  If child template has additional
> > disks - no special treatment should be given (user can later on
> > copy those disks wherever she chooses)
> 
> 


From abaron at redhat.com  Mon Jan 16 20:10:46 2012
From: abaron at redhat.com (Ayal Baron)
Date: Mon, 16 Jan 2012 15:10:46 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <4F143809.3060603@linux.vnet.ibm.com>
Message-ID: <093917f4-0eaa-40a5-a808-d6387e53094b@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> Suppose, we unplug and then re-plug a disk, will VDSM expect the disk
> be
> in the same device path as before?  More generic, will the unplugging
> and plugging keep the PCI device path as the same as before?

No guarantee for that.
For example, suppose you unplug a disk, plug another disk then re-plug the first one.
Unplug is just like removing a physical disk from your machine, the disk does not have any property stating what PCI address it had.

It would be nice though if the address were kept and when plugging, if the address is not taken then use the same one and if taken then remove address and let the system choose where to place it...


> 
> On 2012-1-9 17:21, Michael Kublin wrote:
> > Hi, the follow link is providing a low level design for
> > HotPlug/HotUnplug feature :
> > http://www.ovirt.org/wiki/Features/DetailedHotPlug .
> >
> > The feature is simple and design is short
> >
> > Regards Michael
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> >
> 
> 
> --
> Shu Ming<shuming at linux.vnet.ibm.com>
> IBM China Systems and Technology Laboratory
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From iheim at redhat.com  Mon Jan 16 17:26:24 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 16 Jan 2012 19:26:24 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F144D72.6010609@redhat.com>
References: <9486ec23-5155-4a1e-85d0-1a5aefad3098@zmail13.collab.prod.int.phx2.redhat.com>
	<4F143850.7090508@redhat.com> <4F14465F.4000407@redhat.com>
	<4F144930.4060009@redhat.com> <4F144D72.6010609@redhat.com>
Message-ID: <4F145DC0.7080807@redhat.com>

On 01/16/2012 06:16 PM, Jon Choate wrote:
> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>
>>>>> ----- Original Message -----
>>>>>> We are going to be able to store the disks for a template on
>>>>>> different storage domains due to the multiple storage domain
>>>>>> feature. Cloning a template will still be possible, but will it
>>>>>> provide any value? Thoughts?
>>>>> I see no relation between the two options.
>>>>>
>>>>> Scenario 1: I can create a VM with a single disk and create a
>>>>> template from it.
>>>>> I would still want to be able to clone the template in order to
>>>>> provision VMs from it on different domains.
>>>>>
>>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>>
>>>>> Scenario 3: I have a template with 2 disks on 2 different domains
>>>>> (domain A and domain B) and I want to have another copy of the
>>>>> template on domain C and domain D
>>>>>
>>>> Hi Jon,
>>>>
>>>> After talking to Michael Pasternak it seems that we did not implemented
>>>> copyTemplate in the REST API, it seems to be a gap that we have.
>>>>
>>>> This gap is playing in our favor, we can remove the copyTemplate verb
>>>> and introduce copyDisk verb.
>>>>
>>>> The template disks can be copied to another SD.
>>>> When creating a VM from template the user can choose per disk the
>>>> destination SD (only SD with the disks are eligible candidates).
>>> wait, when creating a VM from a template, the user won't get a choice
>>> will they? Won't the VM disks have to go on the same storage domain as
>>> the template disks they were created from?
>>
>> yes, but the template disks can be copied to multiple storage domains,
>> so the user can choose for the VM/disk which storage domain to create
>> them from (per storage domains that have copies of that disk)
> OH! I totally misunderstood. So what you are saying is that a template
> can have N number of copies of the same disk each on a different storage
> domain. I had thought that if you wanted that type of situation you
> would have multiple copies of the template itself too.
>
> Just to be clear, does this mean that the plan is to phase out the
> current clone template command and instead implementing a clone disk
> command so that a template can duplicate its disks individually?

pretty much, yes.
though i'd imagine 'clone template' would still be useful to have for 
the user. not sure if it implies core should expose it as well to allow 
easier usage at UI level for such a task.


From jchoate at redhat.com  Tue Jan 17 02:58:24 2012
From: jchoate at redhat.com (Jon Choate)
Date: Mon, 16 Jan 2012 21:58:24 -0500
Subject: [Engine-devel] a different approach to the command classes
Message-ID: <4F14E3D0.7050509@redhat.com>

The way the command classes are written has bothered me for a while.  
While implementing the multiple storage domain features I am presented 
with the opportunity to create a new command from scratch.  I gave some 
thought to what I would like the command classes to look like while 
balancing that the class must still fit in with the existing structure.  
So here is what I came up with. I'd appreciate any feedback.

The Command class encompasses only the rules of what needs to be done.  
It relies upon Validator classes to determine if the canDoAction 
conditions have been met.

     @Override
     public boolean canDoAction() {
       ...
         checkTargetDomainHasSpace();
         checkTargetDomainIsValidTarget();
         checkSourceDomainIsValidSource();
         checkSourceAndTargetAreDifferent();
      ...
}

...

   private void checkTargetDomainHasSpace() {
         if(!actionAllowed) return;
         
if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId())) {
           
addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
           actionAllowed = false;
         }
     }


Each of the checks follows a similar pattern of
     - guard clause to see of we already failed and bail if we did
     - test for failure of the condition
     - add failure message if needed
     - set variable to failed if needed

Storing the success flag in a variable allows us to keep the canDoAction 
method readable as a series of commands and to allow it to be accessed 
by all the private methods without them having to pass it around.

The execution of the command will follow a similar pattern where the 
command class will only know how to describe what needs to be done and 
to rely on supporting objects to handle the implementation of these 
steps.  Getting the implementation out of the command classes will allow 
the commands to share validation and implementation details and remove a 
lot of the duplication that currently exists within the commands.


How do people feel about this approach?


From lpeer at redhat.com  Tue Jan 17 07:05:16 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 17 Jan 2012 09:05:16 +0200
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <4F14E3D0.7050509@redhat.com>
References: <4F14E3D0.7050509@redhat.com>
Message-ID: <4F151DAC.5000008@redhat.com>

On 17/01/12 04:58, Jon Choate wrote:
> The way the command classes are written has bothered me for a while. 
> While implementing the multiple storage domain features I am presented
> with the opportunity to create a new command from scratch.  I gave some
> thought to what I would like the command classes to look like while
> balancing that the class must still fit in with the existing structure. 
> So here is what I came up with. I'd appreciate any feedback.
> 
> The Command class encompasses only the rules of what needs to be done. 
> It relies upon Validator classes to determine if the canDoAction
> conditions have been met.
> 
>     @Override
>     public boolean canDoAction() {
>       ...
>         checkTargetDomainHasSpace();
>         checkTargetDomainIsValidTarget();
>         checkSourceDomainIsValidSource();
>         checkSourceAndTargetAreDifferent();
>      ...
> }
> 
> ...
> 
>   private void checkTargetDomainHasSpace() {
>         if(!actionAllowed) return;
>        
> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId())) {
>          
> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
>           actionAllowed = false;
>         }
>     }
> 
> 
> Each of the checks follows a similar pattern of
>     - guard clause to see of we already failed and bail if we did
>     - test for failure of the condition
>     - add failure message if needed
>     - set variable to failed if needed
> 
> Storing the success flag in a variable allows us to keep the canDoAction
> method readable as a series of commands and to allow it to be accessed
> by all the private methods without them having to pass it around.
> 
> The execution of the command will follow a similar pattern where the
> command class will only know how to describe what needs to be done and
> to rely on supporting objects to handle the implementation of these
> steps.  Getting the implementation out of the command classes will allow
> the commands to share validation and implementation details and remove a
> lot of the duplication that currently exists within the commands.
> 
> 
> How do people feel about this approach?


Hi Jon,

The scope of your proposal is changing the implementation of the
canDoAction method, I think that the title of this mail is a bit misleading.

Basically what you are suggesting is to split the canDoAction
implementation into methods and then extract them from the command class
to a shared class so they can be reused.

In many cases we can use (are using) inheritance for reusing code, there
are cases where inheritance does not do the work and we can extract to
external classes.

I think such a change is welcomed but on a needed basis, I think it is
overkill for the general use case and will make the code more cumbersome
(if the original canDoAction was 4-5 lines long...).

One thing I don't like in the above suggestion is the way you validate
that the previous condition succeeded/failed. Having this condition at
the beginning of each validation method is not a good approach IMO.


Livnat




From abaron at redhat.com  Tue Jan 17 07:24:41 2012
From: abaron at redhat.com (Ayal Baron)
Date: Tue, 17 Jan 2012 02:24:41 -0500 (EST)
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <4F151DAC.5000008@redhat.com>
Message-ID: <f2df4565-dcbe-409c-abae-df9364fa4e6f@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 17/01/12 04:58, Jon Choate wrote:
> > The way the command classes are written has bothered me for a
> > while.
> > While implementing the multiple storage domain features I am
> > presented
> > with the opportunity to create a new command from scratch.  I gave
> > some
> > thought to what I would like the command classes to look like while
> > balancing that the class must still fit in with the existing
> > structure.
> > So here is what I came up with. I'd appreciate any feedback.
> > 
> > The Command class encompasses only the rules of what needs to be
> > done.
> > It relies upon Validator classes to determine if the canDoAction
> > conditions have been met.
> > 
> >     @Override
> >     public boolean canDoAction() {
> >       ...
> >         checkTargetDomainHasSpace();
> >         checkTargetDomainIsValidTarget();
> >         checkSourceDomainIsValidSource();
> >         checkSourceAndTargetAreDifferent();
> >      ...
> > }
> > 
> > ...
> > 
> >   private void checkTargetDomainHasSpace() {
> >         if(!actionAllowed) return;
> >        
> > if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
> > {
> >          
> > addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
> >           actionAllowed = false;
> >         }
> >     }
> > 
> > 
> > Each of the checks follows a similar pattern of
> >     - guard clause to see of we already failed and bail if we did
> >     - test for failure of the condition
> >     - add failure message if needed
> >     - set variable to failed if needed
> > 
> > Storing the success flag in a variable allows us to keep the
> > canDoAction
> > method readable as a series of commands and to allow it to be
> > accessed
> > by all the private methods without them having to pass it around.
> > 
> > The execution of the command will follow a similar pattern where
> > the
> > command class will only know how to describe what needs to be done
> > and
> > to rely on supporting objects to handle the implementation of these
> > steps.  Getting the implementation out of the command classes will
> > allow
> > the commands to share validation and implementation details and
> > remove a
> > lot of the duplication that currently exists within the commands.
> > 
> > 
> > How do people feel about this approach?
> 
> 
> Hi Jon,
> 
> The scope of your proposal is changing the implementation of the
> canDoAction method, I think that the title of this mail is a bit
> misleading.

Actually I think Jon was suggesting to do the same in the command itself.

> 
> Basically what you are suggesting is to split the canDoAction
> implementation into methods and then extract them from the command
> class
> to a shared class so they can be reused.
> 
> In many cases we can use (are using) inheritance for reusing code,
> there
> are cases where inheritance does not do the work and we can extract
> to
> external classes.
> 
> I think such a change is welcomed but on a needed basis, I think it
> is
> overkill for the general use case and will make the code more
> cumbersome
> (if the original canDoAction was 4-5 lines long...).

Jon, I think the burden of proof is on you here to show a real example and how it makes the code clearer (e.g. change 2 commands which share similar checks).
Without 'seeing' it I don't think we would be able to appreciate the advantage of your approach.

> 
> One thing I don't like in the above suggestion is the way you
> validate
> that the previous condition succeeded/failed. Having this condition
> at
> the beginning of each validation method is not a good approach IMO.

+1, if the previous check failed it should raise an exception, not rely on the next check to bail.
It would be easier (less code, clearer) to wrap all the calls with a try catch clause (1 for all the calls), catch the specific exception that says the check failed and return whatever you want to return.

> 
> 
> Livnat
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From lpeer at redhat.com  Tue Jan 17 07:28:22 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 17 Jan 2012 09:28:22 +0200
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <093917f4-0eaa-40a5-a808-d6387e53094b@zmail13.collab.prod.int.phx2.redhat.com>
References: <093917f4-0eaa-40a5-a808-d6387e53094b@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F152316.6020408@redhat.com>

On 16/01/12 22:10, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> Suppose, we unplug and then re-plug a disk, will VDSM expect the disk
>> be
>> in the same device path as before?  More generic, will the unplugging
>> and plugging keep the PCI device path as the same as before?
> 
> No guarantee for that.
> For example, suppose you unplug a disk, plug another disk then re-plug the first one.
> Unplug is just like removing a physical disk from your machine, the disk does not have any property stating what PCI address it had.
> 
> It would be nice though if the address were kept and when plugging, if the address is not taken then use the same one and if taken then remove address and let the system choose where to place it...
> 
> 
ack.

Basically when you unplug the disk the management clears the device
address and the device is left with no address. next time you plug the
device and run the VM the management will learn the new address and
persist it.

Livnat

>>
>> On 2012-1-9 17:21, Michael Kublin wrote:
>>> Hi, the follow link is providing a low level design for
>>> HotPlug/HotUnplug feature :
>>> http://www.ovirt.org/wiki/Features/DetailedHotPlug .
>>>
>>> The feature is simple and design is short
>>>
>>> Regards Michael
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>
>>
>> --
>> Shu Ming<shuming at linux.vnet.ibm.com>
>> IBM China Systems and Technology Laboratory
>>
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From masayag at redhat.com  Tue Jan 17 07:47:48 2012
From: masayag at redhat.com (Moti Asayag)
Date: Tue, 17 Jan 2012 09:47:48 +0200
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <4F151DAC.5000008@redhat.com>
References: <4F14E3D0.7050509@redhat.com> <4F151DAC.5000008@redhat.com>
Message-ID: <4F1527A4.3090808@redhat.com>

On 01/17/2012 09:05 AM, Livnat Peer wrote:
> On 17/01/12 04:58, Jon Choate wrote:
>> The way the command classes are written has bothered me for a while. 
>> While implementing the multiple storage domain features I am presented
>> with the opportunity to create a new command from scratch.  I gave some
>> thought to what I would like the command classes to look like while
>> balancing that the class must still fit in with the existing structure. 
>> So here is what I came up with. I'd appreciate any feedback.
>>
>> The Command class encompasses only the rules of what needs to be done. 
>> It relies upon Validator classes to determine if the canDoAction
>> conditions have been met.
>>
>>     @Override
>>     public boolean canDoAction() {
>>       ...
>>         checkTargetDomainHasSpace();
>>         checkTargetDomainIsValidTarget();
>>         checkSourceDomainIsValidSource();
>>         checkSourceAndTargetAreDifferent();
>>      ...
>> }
>>
>> ...
>>
>>   private void checkTargetDomainHasSpace() {
>>         if(!actionAllowed) return;
>>        
>> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId())) {
>>          
>> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
>>           actionAllowed = false;
>>         }
>>     }
>>
>>
>> Each of the checks follows a similar pattern of
>>     - guard clause to see of we already failed and bail if we did
>>     - test for failure of the condition
>>     - add failure message if needed
>>     - set variable to failed if needed
>>
>> Storing the success flag in a variable allows us to keep the canDoAction
>> method readable as a series of commands and to allow it to be accessed
>> by all the private methods without them having to pass it around.
>>
>> The execution of the command will follow a similar pattern where the
>> command class will only know how to describe what needs to be done and
>> to rely on supporting objects to handle the implementation of these
>> steps.  Getting the implementation out of the command classes will allow
>> the commands to share validation and implementation details and remove a
>> lot of the duplication that currently exists within the commands.
>>
>>
>> How do people feel about this approach?
> 
> 
> Hi Jon,
> 
> The scope of your proposal is changing the implementation of the
> canDoAction method, I think that the title of this mail is a bit misleading.
> 
> Basically what you are suggesting is to split the canDoAction
> implementation into methods and then extract them from the command class
> to a shared class so they can be reused.
> 
> In many cases we can use (are using) inheritance for reusing code, there
> are cases where inheritance does not do the work and we can extract to
> external classes.
> 
> I think such a change is welcomed but on a needed basis, I think it is
> overkill for the general use case and will make the code more cumbersome
> (if the original canDoAction was 4-5 lines long...).
> 
> One thing I don't like in the above suggestion is the way you validate
> that the previous condition succeeded/failed. Having this condition at
> the beginning of each validation method is not a good approach IMO.
> 

In addition, it prevents from independent validations (inside the same
canDoAction) to report on several validation failures in a single
execution instead of getting a single failure, rerun the command again,
get another failure and on...
This is the reason why the type of canDoActionMessages is a List.

> 
> Livnat
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From ofrenkel at redhat.com  Tue Jan 17 08:05:15 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Tue, 17 Jan 2012 03:05:15 -0500 (EST)
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <4F1527A4.3090808@redhat.com>
Message-ID: <d4ed5faf-667e-4474-acb8-29867fceff4f@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Moti Asayag" <masayag at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Tuesday, January 17, 2012 9:47:48 AM
> Subject: Re: [Engine-devel] a different approach to the command classes
> 
> On 01/17/2012 09:05 AM, Livnat Peer wrote:
> > On 17/01/12 04:58, Jon Choate wrote:
> >> The way the command classes are written has bothered me for a
> >> while.
> >> While implementing the multiple storage domain features I am
> >> presented
> >> with the opportunity to create a new command from scratch.  I gave
> >> some
> >> thought to what I would like the command classes to look like
> >> while
> >> balancing that the class must still fit in with the existing
> >> structure.
> >> So here is what I came up with. I'd appreciate any feedback.
> >>
> >> The Command class encompasses only the rules of what needs to be
> >> done.
> >> It relies upon Validator classes to determine if the canDoAction
> >> conditions have been met.
> >>
> >>     @Override
> >>     public boolean canDoAction() {
> >>       ...
> >>         checkTargetDomainHasSpace();
> >>         checkTargetDomainIsValidTarget();
> >>         checkSourceDomainIsValidSource();
> >>         checkSourceAndTargetAreDifferent();
> >>      ...
> >> }
> >>
> >> ...
> >>
> >>   private void checkTargetDomainHasSpace() {
> >>         if(!actionAllowed) return;
> >>        
> >> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
> >> {
> >>          
> >> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
> >>           actionAllowed = false;
> >>         }
> >>     }
> >>
> >>
> >> Each of the checks follows a similar pattern of
> >>     - guard clause to see of we already failed and bail if we did
> >>     - test for failure of the condition
> >>     - add failure message if needed
> >>     - set variable to failed if needed
> >>
> >> Storing the success flag in a variable allows us to keep the
> >> canDoAction
> >> method readable as a series of commands and to allow it to be
> >> accessed
> >> by all the private methods without them having to pass it around.
> >>
> >> The execution of the command will follow a similar pattern where
> >> the
> >> command class will only know how to describe what needs to be done
> >> and
> >> to rely on supporting objects to handle the implementation of
> >> these
> >> steps.  Getting the implementation out of the command classes will
> >> allow
> >> the commands to share validation and implementation details and
> >> remove a
> >> lot of the duplication that currently exists within the commands.
> >>
> >>
> >> How do people feel about this approach?
> > 
> > 
> > Hi Jon,
> > 
> > The scope of your proposal is changing the implementation of the
> > canDoAction method, I think that the title of this mail is a bit
> > misleading.
> > 
> > Basically what you are suggesting is to split the canDoAction
> > implementation into methods and then extract them from the command
> > class
> > to a shared class so they can be reused.
> > 
> > In many cases we can use (are using) inheritance for reusing code,
> > there
> > are cases where inheritance does not do the work and we can extract
> > to
> > external classes.
> > 
> > I think such a change is welcomed but on a needed basis, I think it
> > is
> > overkill for the general use case and will make the code more
> > cumbersome
> > (if the original canDoAction was 4-5 lines long...).
> > 

i agree as well

> > One thing I don't like in the above suggestion is the way you
> > validate
> > that the previous condition succeeded/failed. Having this condition
> > at
> > the beginning of each validation method is not a good approach IMO.
> > 
> 
> In addition, it prevents from independent validations (inside the
> same
> canDoAction) to report on several validation failures in a single
> execution instead of getting a single failure, rerun the command
> again,
> get another failure and on...
> This is the reason why the type of canDoActionMessages is a List.
> 

actually this is how it works today in most cases, i think its not bad, 
some checks relay on the fact that if this check is executed,
then it's safe to do things, for example - first check validates vm is not null, 
second check use the vm, assuming its not null.


i would go further and make the validation methods return boolean and call them in that way:

    public boolean canDoAction() {
      ...return 
        checkTargetDomainHasSpace() &&
        checkTargetDomainIsValidTarget() &&
        checkSourceDomainIsValidSource() &&
        checkSourceAndTargetAreDifferent();
     ...
}

private void checkTargetDomainHasSpace() {
 if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
 {
   addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
   reutrn false;
 }
 return true;
}

> > 
> > Livnat
> > 
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From ofrenkel at redhat.com  Tue Jan 17 08:32:34 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Tue, 17 Jan 2012 03:32:34 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F145DC0.7080807@redhat.com>
Message-ID: <98c0137e-5d8c-48db-ae39-1936ec9c5a67@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Jon Choate" <jchoate at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Monday, January 16, 2012 7:26:24 PM
> Subject: Re: [Engine-devel] the future of template cloning
> 
> On 01/16/2012 06:16 PM, Jon Choate wrote:
> > On 01/16/2012 10:58 AM, Itamar Heim wrote:
> >> On 01/16/2012 05:46 PM, Jon Choate wrote:
> >>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
> >>>> On 12/01/12 22:45, Ayal Baron wrote:
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> We are going to be able to store the disks for a template on
> >>>>>> different storage domains due to the multiple storage domain
> >>>>>> feature. Cloning a template will still be possible, but will
> >>>>>> it
> >>>>>> provide any value? Thoughts?
> >>>>> I see no relation between the two options.
> >>>>>
> >>>>> Scenario 1: I can create a VM with a single disk and create a
> >>>>> template from it.
> >>>>> I would still want to be able to clone the template in order to
> >>>>> provision VMs from it on different domains.
> >>>>>
> >>>>> Scenario 2: same thing with multiple disks on same domain.
> >>>>>
> >>>>> Scenario 3: I have a template with 2 disks on 2 different
> >>>>> domains
> >>>>> (domain A and domain B) and I want to have another copy of the
> >>>>> template on domain C and domain D
> >>>>>
> >>>> Hi Jon,
> >>>>
> >>>> After talking to Michael Pasternak it seems that we did not
> >>>> implemented
> >>>> copyTemplate in the REST API, it seems to be a gap that we have.
> >>>>
> >>>> This gap is playing in our favor, we can remove the copyTemplate
> >>>> verb
> >>>> and introduce copyDisk verb.
> >>>>
> >>>> The template disks can be copied to another SD.
> >>>> When creating a VM from template the user can choose per disk
> >>>> the
> >>>> destination SD (only SD with the disks are eligible candidates).
> >>> wait, when creating a VM from a template, the user won't get a
> >>> choice
> >>> will they? Won't the VM disks have to go on the same storage
> >>> domain as
> >>> the template disks they were created from?
> >>
> >> yes, but the template disks can be copied to multiple storage
> >> domains,
> >> so the user can choose for the VM/disk which storage domain to
> >> create
> >> them from (per storage domains that have copies of that disk)
> > OH! I totally misunderstood. So what you are saying is that a
> > template
> > can have N number of copies of the same disk each on a different
> > storage
> > domain. I had thought that if you wanted that type of situation you
> > would have multiple copies of the template itself too.

yes, one copy of disk per domain though.

> >
> > Just to be clear, does this mean that the plan is to phase out the
> > current clone template command and instead implementing a clone
> > disk
> > command so that a template can duplicate its disks individually?
> 
> pretty much, yes.
> though i'd imagine 'clone template' would still be useful to have for
> the user. not sure if it implies core should expose it as well to
> allow
> easier usage at UI level for such a task.

we can leave it untouched - means copyTemplate get 1 destination domain, and copies all disks to it,
but i think it will be unusable (and problematic - what if one of the disks already exists on the destination?), 
what the user really wants is to specify which disks to copy
and destination per disk, and i don't see a reason to create a backend command to do it

> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From ofrenkel at redhat.com  Tue Jan 17 08:41:43 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Tue, 17 Jan 2012 03:41:43 -0500 (EST)
Subject: [Engine-devel] move disk command
In-Reply-To: <4F147D0B.2000504@redhat.com>
Message-ID: <af7e51b5-b30d-42ea-80c2-ca015e844729@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Jon Choate" <jchoate at redhat.com>
> To: "Ayal Baron" <abaron at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Monday, January 16, 2012 9:39:55 PM
> Subject: Re: [Engine-devel] move disk command
> 
> On 01/16/2012 02:26 PM, Ayal Baron wrote:
> >
> > ----- Original Message -----
> >> On 01/16/2012 05:26 PM, Jon Choate wrote:
> >>> As part of the multiple storage domains feature there will be new
> >>> functionality added to allow users to move individual disks.
> >>>
> >>> What are the prerequisites for moving a disk?
> >>>
> >>> 1) the disk must exist
> >>> 2) the associated VM must be down
> >> this can't just be a CanDoAction check - the lock has to be real
> >> to
> >> prevent a race from starting the VM after the validation.
> >>
> > Either down or disk is unplugged.
> >
> >>> 3) the associated VM must not be locked
> >>> 4) the source storage domain must exist
> >>> 5) the source storage domain must be available
> >>> 6) the target domain must exist
> >>> 7) the target domain must be available
> >>> 8) the target domain must have adequate disk space
> >>> 9) the target domain cannot be an ISO or export domain
> >>> 10) the source domain cannot be an ISO or export domain
> > This may be unrelated, but user would be allowed to export and
> > import a floating disk, right?
> > I would like the ability to import *any* disk in the export domain
> > as a floating disk, but in the least, export and import disks not
> > associated with a VM.

you are right, it is unrelated, this thread is about move disk of a vm between SDs,
export and import is copy, and floating disks is part of the shared disk feature,
this indeed need to be discussed in that scope.

> This was not in scope for the work I am currently doing.  If this is
> something desirable I think it needs to be prioritized and worked in
> at
> a later time.  If it does need to happen now then we are going to
> need
> to be able to do full crud for a floating disk I would think.
> 
> >> user must provide same/other quota for the target domain which has
> >> enough quota left for the requested size.
> >>
> >>> What am I missing?
> >>>
> >>> Also, should we allow the moving of a template disk that has VM
> >>> disks based on it? Unless I'm wrong this would require all of the
> >>> disks based on the template to be moved as well.
> >> I'd say no. you can only move a template disk if it is not used.
> >> I'd go further and say one should copy the template disk and
> >> delete,
> >> rather than support move for it at all (not relevant for VM disk,
> >> since
> >> we don't have the same concept of multiple copies for it).
> > As long as you can delete a copy of the disk from a domain where
> > there are no VM disks derived from it.
> >
> >>> thoughts?
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From abaron at redhat.com  Tue Jan 17 08:45:53 2012
From: abaron at redhat.com (Ayal Baron)
Date: Tue, 17 Jan 2012 03:45:53 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export (was: Re:
	move disk command)
In-Reply-To: <af7e51b5-b30d-42ea-80c2-ca015e844729@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <917fe2c1-5166-4fdd-8007-691630f55ae7@zmail13.collab.prod.int.phx2.redhat.com>


[SNIP]

> > > This may be unrelated, but user would be allowed to export and
> > > import a floating disk, right?
> > > I would like the ability to import *any* disk in the export
> > > domain
> > > as a floating disk, but in the least, export and import disks not
> > > associated with a VM.
> 
> you are right, it is unrelated, this thread is about move disk of a
> vm between SDs,
> export and import is copy, and floating disks is part of the shared
> disk feature,
> this indeed need to be discussed in that scope.

Ok, so I've changed the subject, now let's discuss it.


From iheim at redhat.com  Tue Jan 17 08:46:27 2012
From: iheim at redhat.com (Itamar Heim)
Date: Tue, 17 Jan 2012 10:46:27 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <98c0137e-5d8c-48db-ae39-1936ec9c5a67@zmail07.collab.prod.int.phx2.redhat.com>
References: <98c0137e-5d8c-48db-ae39-1936ec9c5a67@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F153563.4030104@redhat.com>

On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>
>
> ----- Original Message -----
>> From: "Itamar Heim"<iheim at redhat.com>
>> To: "Jon Choate"<jchoate at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Monday, January 16, 2012 7:26:24 PM
>> Subject: Re: [Engine-devel] the future of template cloning
>>
>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> We are going to be able to store the disks for a template on
>>>>>>>> different storage domains due to the multiple storage domain
>>>>>>>> feature. Cloning a template will still be possible, but will
>>>>>>>> it
>>>>>>>> provide any value? Thoughts?
>>>>>>> I see no relation between the two options.
>>>>>>>
>>>>>>> Scenario 1: I can create a VM with a single disk and create a
>>>>>>> template from it.
>>>>>>> I would still want to be able to clone the template in order to
>>>>>>> provision VMs from it on different domains.
>>>>>>>
>>>>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>>>>
>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>> domains
>>>>>>> (domain A and domain B) and I want to have another copy of the
>>>>>>> template on domain C and domain D
>>>>>>>
>>>>>> Hi Jon,
>>>>>>
>>>>>> After talking to Michael Pasternak it seems that we did not
>>>>>> implemented
>>>>>> copyTemplate in the REST API, it seems to be a gap that we have.
>>>>>>
>>>>>> This gap is playing in our favor, we can remove the copyTemplate
>>>>>> verb
>>>>>> and introduce copyDisk verb.
>>>>>>
>>>>>> The template disks can be copied to another SD.
>>>>>> When creating a VM from template the user can choose per disk
>>>>>> the
>>>>>> destination SD (only SD with the disks are eligible candidates).
>>>>> wait, when creating a VM from a template, the user won't get a
>>>>> choice
>>>>> will they? Won't the VM disks have to go on the same storage
>>>>> domain as
>>>>> the template disks they were created from?
>>>>
>>>> yes, but the template disks can be copied to multiple storage
>>>> domains,
>>>> so the user can choose for the VM/disk which storage domain to
>>>> create
>>>> them from (per storage domains that have copies of that disk)
>>> OH! I totally misunderstood. So what you are saying is that a
>>> template
>>> can have N number of copies of the same disk each on a different
>>> storage
>>> domain. I had thought that if you wanted that type of situation you
>>> would have multiple copies of the template itself too.
>
> yes, one copy of disk per domain though.
>
>>>
>>> Just to be clear, does this mean that the plan is to phase out the
>>> current clone template command and instead implementing a clone
>>> disk
>>> command so that a template can duplicate its disks individually?
>>
>> pretty much, yes.
>> though i'd imagine 'clone template' would still be useful to have for
>> the user. not sure if it implies core should expose it as well to
>> allow
>> easier usage at UI level for such a task.
>
> we can leave it untouched - means copyTemplate get 1 destination domain, and copies all disks to it,
> but i think it will be unusable (and problematic - what if one of the disks already exists on the destination?),

then don't copy it, it is already there

> what the user really wants is to specify which disks to copy
> and destination per disk, and i don't see a reason to create a backend command to do it
>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From ykaul at redhat.com  Tue Jan 17 08:46:55 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Tue, 17 Jan 2012 10:46:55 +0200
Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
Message-ID: <4F15357F.9070607@redhat.com>

After successful compilation and deployment, on a host that ran 
previously fine, I'm getting some error on executing.
Not sure what happened to postgres access - it ran fine previously 
(previous JBoss):

[ykaul at ykaul ear]$ /usr/local/jboss-as-7.1.0.Beta1b/bin/standalone.sh
=========================================================================

   JBoss Bootstrap Environment

   JBOSS_HOME: /usr/local/jboss-as-7.1.0.Beta1b

   JAVA: java

   JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MaxPermSize=256m 
-Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true 
-Dsun.rmi.dgc.client.gcInterval=3600000 
-Dsun.rmi.dgc.server.gcInterval=3600000 
-Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

=========================================================================

10:42:57,373 INFO  [org.jboss.modules] JBoss Modules version 1.1.0.CR4
10:42:57,608 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA
10:42:57,665 INFO  [org.jboss.as] JBoss AS 7.1.0.Beta1b "Tesla" starting
10:42:58,429 INFO  [org.jboss.as] Creating http management service 
using  socket-binding (management-http)
10:42:58,434 INFO  [org.xnio] XNIO Version 3.0.0.CR5
10:42:58,461 INFO  [org.xnio.nio] XNIO NIO Implementation Version 3.0.0.CR5
10:42:58,471 INFO  [org.jboss.remoting] JBoss Remoting version 3.2.0.CR6
10:42:58,477 INFO  [org.jboss.as.logging] JBAS011502: Removing bootstrap 
log handlers
2012-01-17 10:42:58,502 INFO  [org.jboss.as.clustering] (ServerService 
Thread Pool -- 28) JBAS010300: Activating Infinispan subsystem.
2012-01-17 10:42:58,551 INFO  [org.jboss.as.connector] (MSC service 
thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar 
1.0.5.Final)
2012-01-17 10:42:58,564 INFO  [org.jboss.as.naming] (ServerService 
Thread Pool -- 35) JBAS011800: Activating Naming Subsystem
2012-01-17 10:42:58,592 INFO  [org.jboss.as.osgi] (ServerService Thread 
Pool -- 36) JBAS011910: Activating OSGi Subsystem
2012-01-17 10:42:58,602 INFO  [org.jboss.as.naming] (MSC service thread 
1-1) JBAS011802: Starting Naming Service
2012-01-17 10:42:58,605 INFO  [org.jboss.as.security] (ServerService 
Thread Pool -- 41) Activating Security Subsystem
2012-01-17 10:42:58,614 INFO  [org.jboss.as.mail.extension] (MSC service 
thread 1-1) JBAS015400: Bound mail session [java:jboss/mail/Default]
2012-01-17 10:42:58,615 INFO  [org.jboss.as.security] (MSC service 
thread 1-1) Picketbox version=4.0.6.Beta1
2012-01-17 10:42:58,623 INFO  
[org.jboss.as.connector.subsystems.datasources] (ServerService Thread 
Pool -- 24) JBAS010404: Deploying non-JDBC-compliant driver class 
org.postgresql.Driver (version 8.4)
2012-01-17 10:42:58,694 INFO  [org.jboss.as.remoting] (MSC service 
thread 1-1) Listening on /127.0.0.1:9999
2012-01-17 10:42:58,697 INFO  [org.jboss.as.remoting] (MSC service 
thread 1-2) Listening on /0.0.0.0:4447
2012-01-17 10:42:58,785 INFO  
[org.apache.catalina.core.AprLifecycleListener] (MSC service thread 1-4) 
The Apache Tomcat Native library which allows optimal performance in 
production environments was not found on the java.library.path: 
/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2012-01-17 10:42:58,864 INFO  [org.apache.coyote.http11.Http11Protocol] 
(MSC service thread 1-4) Starting Coyote HTTP/1.1 on http--0.0.0.0-8080
2012-01-17 10:42:59,312 INFO  [org.jboss.as.server.deployment.scanner] 
(MSC service thread 1-4) JBAS015012: Started FileSystemDeploymentService 
for directory /usr/local/jboss-as-7.1.0.Beta1b/standalone/deployments
2012-01-17 10:42:59,341 INFO  
[org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-4) 
JBAS010400: Bound data source [java:/ENGINEDataSource]
2012-01-17 10:42:59,355 INFO  [org.jboss.as] (Controller Boot Thread) 
JBoss AS 7.1.0.Beta1b "Tesla" started in 2128ms - Started 129 of 190 
services (60 services are passive or on-demand)
2012-01-17 10:42:59,420 INFO  [org.jboss.as.server.deployment.scanner] 
(DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed 
deployment engine.ear
2012-01-17 10:42:59,459 WARN  
[org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (JCA 
PoolFiller) IJ000610: Unable to fill pool: 
javax.resource.ResourceException: Could not create connection
at 
org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277)
     at 
org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235)
     at 
org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:736) 
[ironjacamar-core-impl-1.0.5.Final.jar:]
     at 
org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.fillToMin(SemaphoreArrayListManagedConnectionPool.java:683) 
[ironjacamar-core-impl-1.0.5.Final.jar:]
     at 
org.jboss.jca.core.connectionmanager.pool.mcp.PoolFiller.run(PoolFiller.java:97) 
[ironjacamar-core-impl-1.0.5.Final.jar:]
     at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
Caused by: org.postgresql.util.PSQLException: FATAL: password 
authentication failed for user "postgres"
     at 
org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
     at 
org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
     at 
org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
     at 
org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
     at 
org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
     at 
org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
     at 
org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
     at org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
     at org.postgresql.Driver.makeConnection(Driver.java:393)
     at org.postgresql.Driver.connect(Driver.java:267)
     at 
org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249)
     ... 5 more

2012-01-17 10:42:59,491 INFO  [org.jboss.as.server.deployment.scanner] 
(DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in 
deployment directory. To trigger deployment create a file called 
engine.ear.dodeploy
2012-01-17 10:42:59,502 ERROR [org.jboss.as.controller] 
(DeploymentScanner-threads - 2) JBAS014612: Operation ("add") failed - 
address: ([("deployment" => "engine.ear")]): 
java.lang.IllegalStateException: JBAS014666: Duplicate resource 
[("deployment" => "engine.ear")]
     at 
org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
     at 
org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
     at 
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
[:1.6.0_22]
     at java.util.concurrent.FutureTask.run(FutureTask.java:166) [:1.6.0_22]
     at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) 
[:1.6.0_22]
     at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) 
[:1.6.0_22]
     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
[:1.6.0_22]
     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
[:1.6.0_22]
     at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
     at org.jboss.threads.JBossThread.run(JBossThread.java:122) 
[jboss-threads-2.0.0.GA.jar:]

2012-01-17 10:42:59,505 ERROR [org.jboss.as.server.deployment.scanner] 
(DeploymentScanner-threads - 1) JBAS014654: Composite operation was 
rolled back
2012-01-17 10:42:59,506 ERROR [org.jboss.as.server.deployment.scanner] 
(DeploymentScanner-threads - 1) {"JBAS014653: Composite operation failed 
and was rolled back. Steps that failed:" => {"Operation step-1" => 
"JBAS014749: Operation handler failed: JBAS014666: Duplicate resource 
[(\"deployment\" => \"engine.ear\")]"}}


From ovedo at redhat.com  Tue Jan 17 08:57:36 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Tue, 17 Jan 2012 03:57:36 -0500 (EST)
Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
In-Reply-To: <4F15357F.9070607@redhat.com>
Message-ID: <93f9564e-d92b-4f43-8360-818b55480681@zmail02.collab.prod.int.phx2.redhat.com>

Did you use a username+password in the previous datasource (in AS5) or was a trust defined for the postgres instance?

If needed, the username+password must be defined in $JBOSS_HOME/standalone/configuration/standalone.xml file:
1. If encrypted then in the EncryptDBPassword section, and reference it in the data source, by using the following instead of username:
                        <security-domain>
                            EncryptDBPassword
                        </security-domain>
2. If not encrypted then just put the password in the datasource definition.

I can connect to your machine and help if you want.

Oved


----- Original Message -----
> From: "Yaniv Kaul" <ykaul at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Tuesday, January 17, 2012 10:46:55 AM
> Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
> 
> After successful compilation and deployment, on a host that ran
> previously fine, I'm getting some error on executing.
> Not sure what happened to postgres access - it ran fine previously
> (previous JBoss):
> 
> [ykaul at ykaul ear]$ /usr/local/jboss-as-7.1.0.Beta1b/bin/standalone.sh
> =========================================================================
> 
>    JBoss Bootstrap Environment
> 
>    JBOSS_HOME: /usr/local/jboss-as-7.1.0.Beta1b
> 
>    JAVA: java
> 
>    JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MaxPermSize=256m
> -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true
> -Dsun.rmi.dgc.client.gcInterval=3600000
> -Dsun.rmi.dgc.server.gcInterval=3600000
> -Djboss.modules.system.pkgs=org.jboss.byteman
> -Djava.awt.headless=true
> 
> =========================================================================
> 
> 10:42:57,373 INFO  [org.jboss.modules] JBoss Modules version
> 1.1.0.CR4
> 10:42:57,608 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA
> 10:42:57,665 INFO  [org.jboss.as] JBoss AS 7.1.0.Beta1b "Tesla"
> starting
> 10:42:58,429 INFO  [org.jboss.as] Creating http management service
> using  socket-binding (management-http)
> 10:42:58,434 INFO  [org.xnio] XNIO Version 3.0.0.CR5
> 10:42:58,461 INFO  [org.xnio.nio] XNIO NIO Implementation Version
> 3.0.0.CR5
> 10:42:58,471 INFO  [org.jboss.remoting] JBoss Remoting version
> 3.2.0.CR6
> 10:42:58,477 INFO  [org.jboss.as.logging] JBAS011502: Removing
> bootstrap
> log handlers
> 2012-01-17 10:42:58,502 INFO  [org.jboss.as.clustering]
> (ServerService
> Thread Pool -- 28) JBAS010300: Activating Infinispan subsystem.
> 2012-01-17 10:42:58,551 INFO  [org.jboss.as.connector] (MSC service
> thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar
> 1.0.5.Final)
> 2012-01-17 10:42:58,564 INFO  [org.jboss.as.naming] (ServerService
> Thread Pool -- 35) JBAS011800: Activating Naming Subsystem
> 2012-01-17 10:42:58,592 INFO  [org.jboss.as.osgi] (ServerService
> Thread
> Pool -- 36) JBAS011910: Activating OSGi Subsystem
> 2012-01-17 10:42:58,602 INFO  [org.jboss.as.naming] (MSC service
> thread
> 1-1) JBAS011802: Starting Naming Service
> 2012-01-17 10:42:58,605 INFO  [org.jboss.as.security] (ServerService
> Thread Pool -- 41) Activating Security Subsystem
> 2012-01-17 10:42:58,614 INFO  [org.jboss.as.mail.extension] (MSC
> service
> thread 1-1) JBAS015400: Bound mail session [java:jboss/mail/Default]
> 2012-01-17 10:42:58,615 INFO  [org.jboss.as.security] (MSC service
> thread 1-1) Picketbox version=4.0.6.Beta1
> 2012-01-17 10:42:58,623 INFO
> [org.jboss.as.connector.subsystems.datasources] (ServerService Thread
> Pool -- 24) JBAS010404: Deploying non-JDBC-compliant driver class
> org.postgresql.Driver (version 8.4)
> 2012-01-17 10:42:58,694 INFO  [org.jboss.as.remoting] (MSC service
> thread 1-1) Listening on /127.0.0.1:9999
> 2012-01-17 10:42:58,697 INFO  [org.jboss.as.remoting] (MSC service
> thread 1-2) Listening on /0.0.0.0:4447
> 2012-01-17 10:42:58,785 INFO
> [org.apache.catalina.core.AprLifecycleListener] (MSC service thread
> 1-4)
> The Apache Tomcat Native library which allows optimal performance in
> production environments was not found on the java.library.path:
> /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> 2012-01-17 10:42:58,864 INFO
>  [org.apache.coyote.http11.Http11Protocol]
> (MSC service thread 1-4) Starting Coyote HTTP/1.1 on
> http--0.0.0.0-8080
> 2012-01-17 10:42:59,312 INFO
>  [org.jboss.as.server.deployment.scanner]
> (MSC service thread 1-4) JBAS015012: Started
> FileSystemDeploymentService
> for directory /usr/local/jboss-as-7.1.0.Beta1b/standalone/deployments
> 2012-01-17 10:42:59,341 INFO
> [org.jboss.as.connector.subsystems.datasources] (MSC service thread
> 1-4)
> JBAS010400: Bound data source [java:/ENGINEDataSource]
> 2012-01-17 10:42:59,355 INFO  [org.jboss.as] (Controller Boot Thread)
> JBoss AS 7.1.0.Beta1b "Tesla" started in 2128ms - Started 129 of 190
> services (60 services are passive or on-demand)
> 2012-01-17 10:42:59,420 INFO
>  [org.jboss.as.server.deployment.scanner]
> (DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed
> deployment engine.ear
> 2012-01-17 10:42:59,459 WARN
> [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (JCA
> PoolFiller) IJ000610: Unable to fill pool:
> javax.resource.ResourceException: Could not create connection
> at
> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277)
>      at
> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235)
>      at
> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:736)
> [ironjacamar-core-impl-1.0.5.Final.jar:]
>      at
> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.fillToMin(SemaphoreArrayListManagedConnectionPool.java:683)
> [ironjacamar-core-impl-1.0.5.Final.jar:]
>      at
> org.jboss.jca.core.connectionmanager.pool.mcp.PoolFiller.run(PoolFiller.java:97)
> [ironjacamar-core-impl-1.0.5.Final.jar:]
>      at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
> Caused by: org.postgresql.util.PSQLException: FATAL: password
> authentication failed for user "postgres"
>      at
> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
>      at
> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
>      at
> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
>      at
> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
>      at
> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
>      at
> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
>      at
> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
>      at
>      org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
>      at org.postgresql.Driver.makeConnection(Driver.java:393)
>      at org.postgresql.Driver.connect(Driver.java:267)
>      at
> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249)
>      ... 5 more
> 
> 2012-01-17 10:42:59,491 INFO
>  [org.jboss.as.server.deployment.scanner]
> (DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in
> deployment directory. To trigger deployment create a file called
> engine.ear.dodeploy
> 2012-01-17 10:42:59,502 ERROR [org.jboss.as.controller]
> (DeploymentScanner-threads - 2) JBAS014612: Operation ("add") failed
> -
> address: ([("deployment" => "engine.ear")]):
> java.lang.IllegalStateException: JBAS014666: Duplicate resource
> [("deployment" => "engine.ear")]
>      at
> org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
>      at
> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
>      at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> [:1.6.0_22]
>      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>      [:1.6.0_22]
>      at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> [:1.6.0_22]
>      at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> [:1.6.0_22]
>      at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> [:1.6.0_22]
>      at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [:1.6.0_22]
>      at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>      at org.jboss.threads.JBossThread.run(JBossThread.java:122)
> [jboss-threads-2.0.0.GA.jar:]
> 
> 2012-01-17 10:42:59,505 ERROR
> [org.jboss.as.server.deployment.scanner]
> (DeploymentScanner-threads - 1) JBAS014654: Composite operation was
> rolled back
> 2012-01-17 10:42:59,506 ERROR
> [org.jboss.as.server.deployment.scanner]
> (DeploymentScanner-threads - 1) {"JBAS014653: Composite operation
> failed
> and was rolled back. Steps that failed:" => {"Operation step-1" =>
> "JBAS014749: Operation handler failed: JBAS014666: Duplicate resource
> [(\"deployment\" => \"engine.ear\")]"}}
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From lpeer at redhat.com  Tue Jan 17 09:26:18 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 17 Jan 2012 11:26:18 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F153563.4030104@redhat.com>
References: <98c0137e-5d8c-48db-ae39-1936ec9c5a67@zmail07.collab.prod.int.phx2.redhat.com>
	<4F153563.4030104@redhat.com>
Message-ID: <4F153EBA.8060504@redhat.com>

On 17/01/12 10:46, Itamar Heim wrote:
> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>>
>>
>> ----- Original Message -----
>>> From: "Itamar Heim"<iheim at redhat.com>
>>> To: "Jon Choate"<jchoate at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Monday, January 16, 2012 7:26:24 PM
>>> Subject: Re: [Engine-devel] the future of template cloning
>>>
>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>>
>>>>>>>> ----- Original Message -----
>>>>>>>>> We are going to be able to store the disks for a template on
>>>>>>>>> different storage domains due to the multiple storage domain
>>>>>>>>> feature. Cloning a template will still be possible, but will
>>>>>>>>> it
>>>>>>>>> provide any value? Thoughts?
>>>>>>>> I see no relation between the two options.
>>>>>>>>
>>>>>>>> Scenario 1: I can create a VM with a single disk and create a
>>>>>>>> template from it.
>>>>>>>> I would still want to be able to clone the template in order to
>>>>>>>> provision VMs from it on different domains.
>>>>>>>>
>>>>>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>>>>>
>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>>> domains
>>>>>>>> (domain A and domain B) and I want to have another copy of the
>>>>>>>> template on domain C and domain D
>>>>>>>>
>>>>>>> Hi Jon,
>>>>>>>
>>>>>>> After talking to Michael Pasternak it seems that we did not
>>>>>>> implemented
>>>>>>> copyTemplate in the REST API, it seems to be a gap that we have.
>>>>>>>
>>>>>>> This gap is playing in our favor, we can remove the copyTemplate
>>>>>>> verb
>>>>>>> and introduce copyDisk verb.
>>>>>>>
>>>>>>> The template disks can be copied to another SD.
>>>>>>> When creating a VM from template the user can choose per disk
>>>>>>> the
>>>>>>> destination SD (only SD with the disks are eligible candidates).
>>>>>> wait, when creating a VM from a template, the user won't get a
>>>>>> choice
>>>>>> will they? Won't the VM disks have to go on the same storage
>>>>>> domain as
>>>>>> the template disks they were created from?
>>>>>
>>>>> yes, but the template disks can be copied to multiple storage
>>>>> domains,
>>>>> so the user can choose for the VM/disk which storage domain to
>>>>> create
>>>>> them from (per storage domains that have copies of that disk)
>>>> OH! I totally misunderstood. So what you are saying is that a
>>>> template
>>>> can have N number of copies of the same disk each on a different
>>>> storage
>>>> domain. I had thought that if you wanted that type of situation you
>>>> would have multiple copies of the template itself too.
>>
>> yes, one copy of disk per domain though.
>>
>>>>
>>>> Just to be clear, does this mean that the plan is to phase out the
>>>> current clone template command and instead implementing a clone
>>>> disk
>>>> command so that a template can duplicate its disks individually?
>>>
>>> pretty much, yes.
>>> though i'd imagine 'clone template' would still be useful to have for
>>> the user. not sure if it implies core should expose it as well to
>>> allow
>>> easier usage at UI level for such a task.
>>
>> we can leave it untouched - means copyTemplate get 1 destination
>> domain, and copies all disks to it,
>> but i think it will be unusable (and problematic - what if one of the
>> disks already exists on the destination?),
> 
> then don't copy it, it is already there
> 

I agree with Omer, there is no reason to support copy template, if the
user wants to clone all the disks he can use multiple actions, we don't
need a specific verb for this.
If the UI chooses to expose such operation it will use the
multipleRunAction API which makes it easier to expose to the user
partial success, we could clone disk A and Disk B but Disk C failed etc.



>> what the user really wants is to specify which disks to copy
>> and destination per disk, and i don't see a reason to create a backend
>> command to do it
>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From mlipchuk at redhat.com  Tue Jan 17 09:33:19 2012
From: mlipchuk at redhat.com (Maor)
Date: Tue, 17 Jan 2012 11:33:19 +0200
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <f2df4565-dcbe-409c-abae-df9364fa4e6f@zmail13.collab.prod.int.phx2.redhat.com>
References: <f2df4565-dcbe-409c-abae-df9364fa4e6f@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F15405F.3000508@redhat.com>

On 01/17/2012 09:24 AM, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> On 17/01/12 04:58, Jon Choate wrote:
>>> The way the command classes are written has bothered me for a
>>> while.
>>> While implementing the multiple storage domain features I am
>>> presented
>>> with the opportunity to create a new command from scratch.  I gave
>>> some
>>> thought to what I would like the command classes to look like while
>>> balancing that the class must still fit in with the existing
>>> structure.
>>> So here is what I came up with. I'd appreciate any feedback.
>>>
>>> The Command class encompasses only the rules of what needs to be
>>> done.
>>> It relies upon Validator classes to determine if the canDoAction
>>> conditions have been met.
>>>
>>>     @Override
>>>     public boolean canDoAction() {
>>>       ...
>>>         checkTargetDomainHasSpace();
>>>         checkTargetDomainIsValidTarget();
>>>         checkSourceDomainIsValidSource();
>>>         checkSourceAndTargetAreDifferent();
>>>      ...
>>> }
>>>
>>> ...
>>>
>>>   private void checkTargetDomainHasSpace() {
>>>         if(!actionAllowed) return;
>>>        
>>> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
>>> {
>>>          
>>> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
>>>           actionAllowed = false;
>>>         }
>>>     }
>>>
>>>
>>> Each of the checks follows a similar pattern of
>>>     - guard clause to see of we already failed and bail if we did
>>>     - test for failure of the condition
>>>     - add failure message if needed
>>>     - set variable to failed if needed
>>>
>>> Storing the success flag in a variable allows us to keep the
>>> canDoAction
>>> method readable as a series of commands and to allow it to be
>>> accessed
>>> by all the private methods without them having to pass it around.
>>>
>>> The execution of the command will follow a similar pattern where
>>> the
>>> command class will only know how to describe what needs to be done
>>> and
>>> to rely on supporting objects to handle the implementation of these
>>> steps.  Getting the implementation out of the command classes will
>>> allow
>>> the commands to share validation and implementation details and
>>> remove a
>>> lot of the duplication that currently exists within the commands.
>>>
>>>
>>> How do people feel about this approach?
>>
>>
>> Hi Jon,
>>
>> The scope of your proposal is changing the implementation of the
>> canDoAction method, I think that the title of this mail is a bit
>> misleading.
> 
> Actually I think Jon was suggesting to do the same in the command itself.

I think, using shared canDoAction validation between commands might be a
good idea also, it can be seen in operations such as add/update commands.
In most cases, the update validation, is already based on the add
command validation, sometime it is implemented with inheritance/external
class, in other cases it is just implemented with multiple code.
> 
>>
>> Basically what you are suggesting is to split the canDoAction
>> implementation into methods and then extract them from the command
>> class
>> to a shared class so they can be reused.
>>
>> In many cases we can use (are using) inheritance for reusing code,
>> there
>> are cases where inheritance does not do the work and we can extract
>> to
>> external classes.
>>
>> I think such a change is welcomed but on a needed basis, I think it
>> is
>> overkill for the general use case and will make the code more
>> cumbersome
>> (if the original canDoAction was 4-5 lines long...).
> 
> Jon, I think the burden of proof is on you here to show a real example and how it makes the code clearer (e.g. change 2 commands which share similar checks).
> Without 'seeing' it I don't think we would be able to appreciate the advantage of your approach.
> 
>>
>> One thing I don't like in the above suggestion is the way you
>> validate
>> that the previous condition succeeded/failed. Having this condition
>> at
>> the beginning of each validation method is not a good approach IMO.
> 
> +1, if the previous check failed it should raise an exception, not rely on the next check to bail.
> It would be easier (less code, clearer) to wrap all the calls with a try catch clause (1 for all the calls), catch the specific exception that says the check failed and return whatever you want to return.
> 
>>
>>
>> Livnat
>>
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From abaron at redhat.com  Tue Jan 17 09:45:33 2012
From: abaron at redhat.com (Ayal Baron)
Date: Tue, 17 Jan 2012 04:45:33 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F153EBA.8060504@redhat.com>
Message-ID: <a00e9a8b-2754-421c-a874-1f340bfdafe9@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 17/01/12 10:46, Itamar Heim wrote:
> > On 01/17/2012 10:32 AM, Omer Frenkel wrote:
> >>
> >>
> >> ----- Original Message -----
> >>> From: "Itamar Heim"<iheim at redhat.com>
> >>> To: "Jon Choate"<jchoate at redhat.com>
> >>> Cc: engine-devel at ovirt.org
> >>> Sent: Monday, January 16, 2012 7:26:24 PM
> >>> Subject: Re: [Engine-devel] the future of template cloning
> >>>
> >>> On 01/16/2012 06:16 PM, Jon Choate wrote:
> >>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
> >>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
> >>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
> >>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
> >>>>>>>>
> >>>>>>>> ----- Original Message -----
> >>>>>>>>> We are going to be able to store the disks for a template
> >>>>>>>>> on
> >>>>>>>>> different storage domains due to the multiple storage
> >>>>>>>>> domain
> >>>>>>>>> feature. Cloning a template will still be possible, but
> >>>>>>>>> will
> >>>>>>>>> it
> >>>>>>>>> provide any value? Thoughts?
> >>>>>>>> I see no relation between the two options.
> >>>>>>>>
> >>>>>>>> Scenario 1: I can create a VM with a single disk and create
> >>>>>>>> a
> >>>>>>>> template from it.
> >>>>>>>> I would still want to be able to clone the template in order
> >>>>>>>> to
> >>>>>>>> provision VMs from it on different domains.
> >>>>>>>>
> >>>>>>>> Scenario 2: same thing with multiple disks on same domain.
> >>>>>>>>
> >>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
> >>>>>>>> domains
> >>>>>>>> (domain A and domain B) and I want to have another copy of
> >>>>>>>> the
> >>>>>>>> template on domain C and domain D
> >>>>>>>>
> >>>>>>> Hi Jon,
> >>>>>>>
> >>>>>>> After talking to Michael Pasternak it seems that we did not
> >>>>>>> implemented
> >>>>>>> copyTemplate in the REST API, it seems to be a gap that we
> >>>>>>> have.
> >>>>>>>
> >>>>>>> This gap is playing in our favor, we can remove the
> >>>>>>> copyTemplate
> >>>>>>> verb
> >>>>>>> and introduce copyDisk verb.
> >>>>>>>
> >>>>>>> The template disks can be copied to another SD.
> >>>>>>> When creating a VM from template the user can choose per disk
> >>>>>>> the
> >>>>>>> destination SD (only SD with the disks are eligible
> >>>>>>> candidates).
> >>>>>> wait, when creating a VM from a template, the user won't get a
> >>>>>> choice
> >>>>>> will they? Won't the VM disks have to go on the same storage
> >>>>>> domain as
> >>>>>> the template disks they were created from?
> >>>>>
> >>>>> yes, but the template disks can be copied to multiple storage
> >>>>> domains,
> >>>>> so the user can choose for the VM/disk which storage domain to
> >>>>> create
> >>>>> them from (per storage domains that have copies of that disk)
> >>>> OH! I totally misunderstood. So what you are saying is that a
> >>>> template
> >>>> can have N number of copies of the same disk each on a different
> >>>> storage
> >>>> domain. I had thought that if you wanted that type of situation
> >>>> you
> >>>> would have multiple copies of the template itself too.
> >>
> >> yes, one copy of disk per domain though.
> >>
> >>>>
> >>>> Just to be clear, does this mean that the plan is to phase out
> >>>> the
> >>>> current clone template command and instead implementing a clone
> >>>> disk
> >>>> command so that a template can duplicate its disks individually?
> >>>
> >>> pretty much, yes.
> >>> though i'd imagine 'clone template' would still be useful to have
> >>> for
> >>> the user. not sure if it implies core should expose it as well to
> >>> allow
> >>> easier usage at UI level for such a task.
> >>
> >> we can leave it untouched - means copyTemplate get 1 destination
> >> domain, and copies all disks to it,
> >> but i think it will be unusable (and problematic - what if one of
> >> the
> >> disks already exists on the destination?),
> > 
> > then don't copy it, it is already there
> > 
> 
> I agree with Omer, there is no reason to support copy template, if
> the
> user wants to clone all the disks he can use multiple actions, we
> don't
> need a specific verb for this.

Reason or lack of depends on the common usage.
If we assume that normally all disks of a template would reside on the same domain then it makes sense to have a verb to copy the template in its entirety and not burden the user.
The general recommendation should be to use a single storage domain, so I think there is room for such a verb.

> If the UI chooses to expose such operation it will use the
> multipleRunAction API which makes it easier to expose to the user
> partial success, we could clone disk A and Disk B but Disk C failed
> etc.

The multipleRunAction is for user marking multiple objects in GUI and running an action on all of these objects.
Here however, the action the user wants is to copy 1 object (the template) which has sub objects and it should run as a single action.
For example, if there is enough space on the target domain for 2/4 disks then using multipleRunAction would result in 2 disks being copied and 2 failing.
If however it is a single action then the free space test would fail the entire action and user would be able to choose if he wants to copy just 2.
Note that in this case, actually performing the copy of the 2 disks is detrimental as it can negatively affect VMs on this domain.

> 
> 
> 
> >> what the user really wants is to specify which disks to copy
> >> and destination per disk, and i don't see a reason to create a
> >> backend
> >> command to do it
> >>
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From ofrenkel at redhat.com  Tue Jan 17 09:57:47 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Tue, 17 Jan 2012 04:57:47 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export (was:
	Re: move disk command)
In-Reply-To: <917fe2c1-5166-4fdd-8007-691630f55ae7@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <d73d764a-a001-43c5-85e7-c7e66b7aef1b@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Omer Frenkel" <ofrenkel at redhat.com>
> Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>
> Sent: Tuesday, January 17, 2012 10:45:53 AM
> Subject: Shared disk / Floating disk import/export (was: Re: [Engine-devel] move disk command)
> 
> 
> [SNIP]
> 
> > > > This may be unrelated, but user would be allowed to export and
> > > > import a floating disk, right?
> > > > I would like the ability to import *any* disk in the export
> > > > domain
> > > > as a floating disk, but in the least, export and import disks
> > > > not
> > > > associated with a VM.
> > 
> > you are right, it is unrelated, this thread is about move disk of a
> > vm between SDs,
> > export and import is copy, and floating disks is part of the shared
> > disk feature,
> > this indeed need to be discussed in that scope.
> 
> Ok, so I've changed the subject, now let's discuss it.
> 

Adding Maor,
not sure if we have any plan regard export/import of floating disk,
or in general, exporting disk without it's vm (if I understood Ayal correctly)

Maor, any comments?


From mkolesni at redhat.com  Tue Jan 17 10:07:50 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Tue, 17 Jan 2012 05:07:50 -0500 (EST)
Subject: [Engine-devel] move disk command
In-Reply-To: <4F144ADA.9080602@redhat.com>
Message-ID: <819437d3-cc56-4e74-bafe-3054e41da353@zmail14.collab.prod.int.phx2.redhat.com>


----- Original Message -----
> On 01/16/2012 05:26 PM, Jon Choate wrote:
> > As part of the multiple storage domains feature there will be new
> > functionality added to allow users to move individual disks.
> >
> > What are the prerequisites for moving a disk?
> >
> > 1) the disk must exist

1.5) None of the disk images is locked.

> > 2) the associated VM must be down
> 
> this can't just be a CanDoAction check - the lock has to be real to
> prevent a race from starting the VM after the validation.
> 
> > 3) the associated VM must not be locked
> > 4) the source storage domain must exist
> > 5) the source storage domain must be available
> > 6) the target domain must exist
> > 7) the target domain must be available
> > 8) the target domain must have adequate disk space
> > 9) the target domain cannot be an ISO or export domain
> > 10) the source domain cannot be an ISO or export domain
> 
> user must provide same/other quota for the target domain which has
> enough quota left for the requested size.
> 
> >
> > What am I missing?
> >
> > Also, should we allow the moving of a template disk that has VM
> > disks based on it? Unless I'm wrong this would require all of the
> > disks based on the template to be moved as well.
> 
> I'd say no. you can only move a template disk if it is not used.
> I'd go further and say one should copy the template disk and delete,
> rather than support move for it at all (not relevant for VM disk,
> since
> we don't have the same concept of multiple copies for it).

Talking of templates, is it ok to move the disk which is based off of a template disk to a domain that the template disk doesn't exist on?

> 
> >
> > thoughts?
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From lpeer at redhat.com  Tue Jan 17 12:29:43 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 17 Jan 2012 14:29:43 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <a00e9a8b-2754-421c-a874-1f340bfdafe9@zmail13.collab.prod.int.phx2.redhat.com>
References: <a00e9a8b-2754-421c-a874-1f340bfdafe9@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1569B7.9050100@redhat.com>

On 17/01/12 11:45, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> On 17/01/12 10:46, Itamar Heim wrote:
>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Itamar Heim"<iheim at redhat.com>
>>>>> To: "Jon Choate"<jchoate at redhat.com>
>>>>> Cc: engine-devel at ovirt.org
>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
>>>>> Subject: Re: [Engine-devel] the future of template cloning
>>>>>
>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>>>>
>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>> We are going to be able to store the disks for a template
>>>>>>>>>>> on
>>>>>>>>>>> different storage domains due to the multiple storage
>>>>>>>>>>> domain
>>>>>>>>>>> feature. Cloning a template will still be possible, but
>>>>>>>>>>> will
>>>>>>>>>>> it
>>>>>>>>>>> provide any value? Thoughts?
>>>>>>>>>> I see no relation between the two options.
>>>>>>>>>>
>>>>>>>>>> Scenario 1: I can create a VM with a single disk and create
>>>>>>>>>> a
>>>>>>>>>> template from it.
>>>>>>>>>> I would still want to be able to clone the template in order
>>>>>>>>>> to
>>>>>>>>>> provision VMs from it on different domains.
>>>>>>>>>>
>>>>>>>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>>>>>>>
>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>>>>> domains
>>>>>>>>>> (domain A and domain B) and I want to have another copy of
>>>>>>>>>> the
>>>>>>>>>> template on domain C and domain D
>>>>>>>>>>
>>>>>>>>> Hi Jon,
>>>>>>>>>
>>>>>>>>> After talking to Michael Pasternak it seems that we did not
>>>>>>>>> implemented
>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
>>>>>>>>> have.
>>>>>>>>>
>>>>>>>>> This gap is playing in our favor, we can remove the
>>>>>>>>> copyTemplate
>>>>>>>>> verb
>>>>>>>>> and introduce copyDisk verb.
>>>>>>>>>
>>>>>>>>> The template disks can be copied to another SD.
>>>>>>>>> When creating a VM from template the user can choose per disk
>>>>>>>>> the
>>>>>>>>> destination SD (only SD with the disks are eligible
>>>>>>>>> candidates).
>>>>>>>> wait, when creating a VM from a template, the user won't get a
>>>>>>>> choice
>>>>>>>> will they? Won't the VM disks have to go on the same storage
>>>>>>>> domain as
>>>>>>>> the template disks they were created from?
>>>>>>>
>>>>>>> yes, but the template disks can be copied to multiple storage
>>>>>>> domains,
>>>>>>> so the user can choose for the VM/disk which storage domain to
>>>>>>> create
>>>>>>> them from (per storage domains that have copies of that disk)
>>>>>> OH! I totally misunderstood. So what you are saying is that a
>>>>>> template
>>>>>> can have N number of copies of the same disk each on a different
>>>>>> storage
>>>>>> domain. I had thought that if you wanted that type of situation
>>>>>> you
>>>>>> would have multiple copies of the template itself too.
>>>>
>>>> yes, one copy of disk per domain though.
>>>>
>>>>>>
>>>>>> Just to be clear, does this mean that the plan is to phase out
>>>>>> the
>>>>>> current clone template command and instead implementing a clone
>>>>>> disk
>>>>>> command so that a template can duplicate its disks individually?
>>>>>
>>>>> pretty much, yes.
>>>>> though i'd imagine 'clone template' would still be useful to have
>>>>> for
>>>>> the user. not sure if it implies core should expose it as well to
>>>>> allow
>>>>> easier usage at UI level for such a task.
>>>>
>>>> we can leave it untouched - means copyTemplate get 1 destination
>>>> domain, and copies all disks to it,
>>>> but i think it will be unusable (and problematic - what if one of
>>>> the
>>>> disks already exists on the destination?),
>>>
>>> then don't copy it, it is already there
>>>
>>
>> I agree with Omer, there is no reason to support copy template, if
>> the
>> user wants to clone all the disks he can use multiple actions, we
>> don't
>> need a specific verb for this.
> 
> Reason or lack of depends on the common usage.
> If we assume that normally all disks of a template would reside on the same domain then it makes sense to have a verb to copy the template in its entirety and not burden the user.
> The general recommendation should be to use a single storage domain, so I think there is room for such a verb.
> 


>> If the UI chooses to expose such operation it will use the
>> multipleRunAction API which makes it easier to expose to the user
>> partial success, we could clone disk A and Disk B but Disk C failed
>> etc.
> 
> The multipleRunAction is for user marking multiple objects in GUI and running an action on all of these objects.

Exactly, choosing the disks to copy to the storage domain.

> Here however, the action the user wants is to copy 1 object (the template) which has sub objects and it should run as a single action.

We are not cloning the template itself we are only cloning the template
disks.


> For example, if there is enough space on the target domain for 2/4 disks then using multipleRunAction would result in 2 disks being copied and 2 failing.

> If however it is a single action then the free space test would fail the entire action and user would be able to choose if he wants to copy just 2.
> Note that in this case, actually performing the copy of the 2 disks is detrimental as it can negatively affect VMs on this domain.
> 
>>
>>
>>
>>>> what the user really wants is to specify which disks to copy
>>>> and destination per disk, and i don't see a reason to create a
>>>> backend
>>>> command to do it
>>>>
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From ykaul at redhat.com  Tue Jan 17 12:43:24 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Tue, 17 Jan 2012 14:43:24 +0200
Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
In-Reply-To: <93f9564e-d92b-4f43-8360-818b55480681@zmail02.collab.prod.int.phx2.redhat.com>
References: <93f9564e-d92b-4f43-8360-818b55480681@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <4F156CEC.8020906@redhat.com>

On 01/17/2012 10:57 AM, Oved Ourfalli wrote:
> Did you use a username+password in the previous datasource (in AS5) or was a trust defined for the postgres instance?
>
> If needed, the username+password must be defined in $JBOSS_HOME/standalone/configuration/standalone.xml file:
> 1. If encrypted then in the EncryptDBPassword section, and reference it in the data source, by using the following instead of username:
>                          <security-domain>
>                              EncryptDBPassword
>                          </security-domain>
> 2. If not encrypted then just put the password in the datasource definition.

This is what I have:
<security>
<user-name>
                             postgres
</user-name>
<!-- IF NEEDED CHANGE THIS TO THE CORRECT PASSWORD/SECURITY DOMAIN -->
<!--password>
                             password
</password-->
<!--security-domain>
                             EncryptDBPassword
</security-domain-->
</security>

So as far as I understand, both options are actually commented out.
Y.

>
> I can connect to your machine and help if you want.
>
> Oved
>
>
> ----- Original Message -----
>> From: "Yaniv Kaul"<ykaul at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Tuesday, January 17, 2012 10:46:55 AM
>> Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
>>
>> After successful compilation and deployment, on a host that ran
>> previously fine, I'm getting some error on executing.
>> Not sure what happened to postgres access - it ran fine previously
>> (previous JBoss):
>>
>> [ykaul at ykaul ear]$ /usr/local/jboss-as-7.1.0.Beta1b/bin/standalone.sh
>> =========================================================================
>>
>>     JBoss Bootstrap Environment
>>
>>     JBOSS_HOME: /usr/local/jboss-as-7.1.0.Beta1b
>>
>>     JAVA: java
>>
>>     JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MaxPermSize=256m
>> -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true
>> -Dsun.rmi.dgc.client.gcInterval=3600000
>> -Dsun.rmi.dgc.server.gcInterval=3600000
>> -Djboss.modules.system.pkgs=org.jboss.byteman
>> -Djava.awt.headless=true
>>
>> =========================================================================
>>
>> 10:42:57,373 INFO  [org.jboss.modules] JBoss Modules version
>> 1.1.0.CR4
>> 10:42:57,608 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA
>> 10:42:57,665 INFO  [org.jboss.as] JBoss AS 7.1.0.Beta1b "Tesla"
>> starting
>> 10:42:58,429 INFO  [org.jboss.as] Creating http management service
>> using  socket-binding (management-http)
>> 10:42:58,434 INFO  [org.xnio] XNIO Version 3.0.0.CR5
>> 10:42:58,461 INFO  [org.xnio.nio] XNIO NIO Implementation Version
>> 3.0.0.CR5
>> 10:42:58,471 INFO  [org.jboss.remoting] JBoss Remoting version
>> 3.2.0.CR6
>> 10:42:58,477 INFO  [org.jboss.as.logging] JBAS011502: Removing
>> bootstrap
>> log handlers
>> 2012-01-17 10:42:58,502 INFO  [org.jboss.as.clustering]
>> (ServerService
>> Thread Pool -- 28) JBAS010300: Activating Infinispan subsystem.
>> 2012-01-17 10:42:58,551 INFO  [org.jboss.as.connector] (MSC service
>> thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar
>> 1.0.5.Final)
>> 2012-01-17 10:42:58,564 INFO  [org.jboss.as.naming] (ServerService
>> Thread Pool -- 35) JBAS011800: Activating Naming Subsystem
>> 2012-01-17 10:42:58,592 INFO  [org.jboss.as.osgi] (ServerService
>> Thread
>> Pool -- 36) JBAS011910: Activating OSGi Subsystem
>> 2012-01-17 10:42:58,602 INFO  [org.jboss.as.naming] (MSC service
>> thread
>> 1-1) JBAS011802: Starting Naming Service
>> 2012-01-17 10:42:58,605 INFO  [org.jboss.as.security] (ServerService
>> Thread Pool -- 41) Activating Security Subsystem
>> 2012-01-17 10:42:58,614 INFO  [org.jboss.as.mail.extension] (MSC
>> service
>> thread 1-1) JBAS015400: Bound mail session [java:jboss/mail/Default]
>> 2012-01-17 10:42:58,615 INFO  [org.jboss.as.security] (MSC service
>> thread 1-1) Picketbox version=4.0.6.Beta1
>> 2012-01-17 10:42:58,623 INFO
>> [org.jboss.as.connector.subsystems.datasources] (ServerService Thread
>> Pool -- 24) JBAS010404: Deploying non-JDBC-compliant driver class
>> org.postgresql.Driver (version 8.4)
>> 2012-01-17 10:42:58,694 INFO  [org.jboss.as.remoting] (MSC service
>> thread 1-1) Listening on /127.0.0.1:9999
>> 2012-01-17 10:42:58,697 INFO  [org.jboss.as.remoting] (MSC service
>> thread 1-2) Listening on /0.0.0.0:4447
>> 2012-01-17 10:42:58,785 INFO
>> [org.apache.catalina.core.AprLifecycleListener] (MSC service thread
>> 1-4)
>> The Apache Tomcat Native library which allows optimal performance in
>> production environments was not found on the java.library.path:
>> /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>> 2012-01-17 10:42:58,864 INFO
>>   [org.apache.coyote.http11.Http11Protocol]
>> (MSC service thread 1-4) Starting Coyote HTTP/1.1 on
>> http--0.0.0.0-8080
>> 2012-01-17 10:42:59,312 INFO
>>   [org.jboss.as.server.deployment.scanner]
>> (MSC service thread 1-4) JBAS015012: Started
>> FileSystemDeploymentService
>> for directory /usr/local/jboss-as-7.1.0.Beta1b/standalone/deployments
>> 2012-01-17 10:42:59,341 INFO
>> [org.jboss.as.connector.subsystems.datasources] (MSC service thread
>> 1-4)
>> JBAS010400: Bound data source [java:/ENGINEDataSource]
>> 2012-01-17 10:42:59,355 INFO  [org.jboss.as] (Controller Boot Thread)
>> JBoss AS 7.1.0.Beta1b "Tesla" started in 2128ms - Started 129 of 190
>> services (60 services are passive or on-demand)
>> 2012-01-17 10:42:59,420 INFO
>>   [org.jboss.as.server.deployment.scanner]
>> (DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed
>> deployment engine.ear
>> 2012-01-17 10:42:59,459 WARN
>> [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (JCA
>> PoolFiller) IJ000610: Unable to fill pool:
>> javax.resource.ResourceException: Could not create connection
>> at
>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277)
>>       at
>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235)
>>       at
>> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:736)
>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>       at
>> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.fillToMin(SemaphoreArrayListManagedConnectionPool.java:683)
>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>       at
>> org.jboss.jca.core.connectionmanager.pool.mcp.PoolFiller.run(PoolFiller.java:97)
>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>       at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>> Caused by: org.postgresql.util.PSQLException: FATAL: password
>> authentication failed for user "postgres"
>>       at
>> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
>>       at
>> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
>>       at
>> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
>>       at
>> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
>>       at
>> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
>>       at
>> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
>>       at
>> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
>>       at
>>       org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
>>       at org.postgresql.Driver.makeConnection(Driver.java:393)
>>       at org.postgresql.Driver.connect(Driver.java:267)
>>       at
>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249)
>>       ... 5 more
>>
>> 2012-01-17 10:42:59,491 INFO
>>   [org.jboss.as.server.deployment.scanner]
>> (DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in
>> deployment directory. To trigger deployment create a file called
>> engine.ear.dodeploy
>> 2012-01-17 10:42:59,502 ERROR [org.jboss.as.controller]
>> (DeploymentScanner-threads - 2) JBAS014612: Operation ("add") failed
>> -
>> address: ([("deployment" =>  "engine.ear")]):
>> java.lang.IllegalStateException: JBAS014666: Duplicate resource
>> [("deployment" =>  "engine.ear")]
>>       at
>> org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
>>       at
>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
>>       at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> [:1.6.0_22]
>>       at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>       [:1.6.0_22]
>>       at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>> [:1.6.0_22]
>>       at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>> [:1.6.0_22]
>>       at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> [:1.6.0_22]
>>       at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> [:1.6.0_22]
>>       at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>>       at org.jboss.threads.JBossThread.run(JBossThread.java:122)
>> [jboss-threads-2.0.0.GA.jar:]
>>
>> 2012-01-17 10:42:59,505 ERROR
>> [org.jboss.as.server.deployment.scanner]
>> (DeploymentScanner-threads - 1) JBAS014654: Composite operation was
>> rolled back
>> 2012-01-17 10:42:59,506 ERROR
>> [org.jboss.as.server.deployment.scanner]
>> (DeploymentScanner-threads - 1) {"JBAS014653: Composite operation
>> failed
>> and was rolled back. Steps that failed:" =>  {"Operation step-1" =>
>> "JBAS014749: Operation handler failed: JBAS014666: Duplicate resource
>> [(\"deployment\" =>  \"engine.ear\")]"}}
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From ovedo at redhat.com  Tue Jan 17 13:05:02 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Tue, 17 Jan 2012 08:05:02 -0500 (EST)
Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
In-Reply-To: <4F156CEC.8020906@redhat.com>
Message-ID: <d203a1b9-960a-4b73-acfe-91ffc97434eb@zmail02.collab.prod.int.phx2.redhat.com>

The question is what did you have in the previous environment (as5) in postgres-ds.xml file.


----- Original Message -----
> From: "Yaniv Kaul" <ykaul at redhat.com>
> To: "Oved Ourfalli" <ovedo at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Tuesday, January 17, 2012 2:43:24 PM
> Subject: Re: [Engine-devel] Failure to deploy (with JBoss 7.1)
> 
> On 01/17/2012 10:57 AM, Oved Ourfalli wrote:
> > Did you use a username+password in the previous datasource (in AS5)
> > or was a trust defined for the postgres instance?
> >
> > If needed, the username+password must be defined in
> > $JBOSS_HOME/standalone/configuration/standalone.xml file:
> > 1. If encrypted then in the EncryptDBPassword section, and
> > reference it in the data source, by using the following instead of
> > username:
> >                          <security-domain>
> >                              EncryptDBPassword
> >                          </security-domain>
> > 2. If not encrypted then just put the password in the datasource
> > definition.
> 
> This is what I have:
> <security>
> <user-name>
>                              postgres
> </user-name>
> <!-- IF NEEDED CHANGE THIS TO THE CORRECT PASSWORD/SECURITY DOMAIN
> -->
> <!--password>
>                              password
> </password-->
> <!--security-domain>
>                              EncryptDBPassword
> </security-domain-->
> </security>
> 
> So as far as I understand, both options are actually commented out.
> Y.
> 
> >
> > I can connect to your machine and help if you want.
> >
> > Oved
> >
> >
> > ----- Original Message -----
> >> From: "Yaniv Kaul"<ykaul at redhat.com>
> >> To: engine-devel at ovirt.org
> >> Sent: Tuesday, January 17, 2012 10:46:55 AM
> >> Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
> >>
> >> After successful compilation and deployment, on a host that ran
> >> previously fine, I'm getting some error on executing.
> >> Not sure what happened to postgres access - it ran fine previously
> >> (previous JBoss):
> >>
> >> [ykaul at ykaul ear]$
> >> /usr/local/jboss-as-7.1.0.Beta1b/bin/standalone.sh
> >> =========================================================================
> >>
> >>     JBoss Bootstrap Environment
> >>
> >>     JBOSS_HOME: /usr/local/jboss-as-7.1.0.Beta1b
> >>
> >>     JAVA: java
> >>
> >>     JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MaxPermSize=256m
> >> -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true
> >> -Dsun.rmi.dgc.client.gcInterval=3600000
> >> -Dsun.rmi.dgc.server.gcInterval=3600000
> >> -Djboss.modules.system.pkgs=org.jboss.byteman
> >> -Djava.awt.headless=true
> >>
> >> =========================================================================
> >>
> >> 10:42:57,373 INFO  [org.jboss.modules] JBoss Modules version
> >> 1.1.0.CR4
> >> 10:42:57,608 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA
> >> 10:42:57,665 INFO  [org.jboss.as] JBoss AS 7.1.0.Beta1b "Tesla"
> >> starting
> >> 10:42:58,429 INFO  [org.jboss.as] Creating http management service
> >> using  socket-binding (management-http)
> >> 10:42:58,434 INFO  [org.xnio] XNIO Version 3.0.0.CR5
> >> 10:42:58,461 INFO  [org.xnio.nio] XNIO NIO Implementation Version
> >> 3.0.0.CR5
> >> 10:42:58,471 INFO  [org.jboss.remoting] JBoss Remoting version
> >> 3.2.0.CR6
> >> 10:42:58,477 INFO  [org.jboss.as.logging] JBAS011502: Removing
> >> bootstrap
> >> log handlers
> >> 2012-01-17 10:42:58,502 INFO  [org.jboss.as.clustering]
> >> (ServerService
> >> Thread Pool -- 28) JBAS010300: Activating Infinispan subsystem.
> >> 2012-01-17 10:42:58,551 INFO  [org.jboss.as.connector] (MSC
> >> service
> >> thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar
> >> 1.0.5.Final)
> >> 2012-01-17 10:42:58,564 INFO  [org.jboss.as.naming] (ServerService
> >> Thread Pool -- 35) JBAS011800: Activating Naming Subsystem
> >> 2012-01-17 10:42:58,592 INFO  [org.jboss.as.osgi] (ServerService
> >> Thread
> >> Pool -- 36) JBAS011910: Activating OSGi Subsystem
> >> 2012-01-17 10:42:58,602 INFO  [org.jboss.as.naming] (MSC service
> >> thread
> >> 1-1) JBAS011802: Starting Naming Service
> >> 2012-01-17 10:42:58,605 INFO  [org.jboss.as.security]
> >> (ServerService
> >> Thread Pool -- 41) Activating Security Subsystem
> >> 2012-01-17 10:42:58,614 INFO  [org.jboss.as.mail.extension] (MSC
> >> service
> >> thread 1-1) JBAS015400: Bound mail session
> >> [java:jboss/mail/Default]
> >> 2012-01-17 10:42:58,615 INFO  [org.jboss.as.security] (MSC service
> >> thread 1-1) Picketbox version=4.0.6.Beta1
> >> 2012-01-17 10:42:58,623 INFO
> >> [org.jboss.as.connector.subsystems.datasources] (ServerService
> >> Thread
> >> Pool -- 24) JBAS010404: Deploying non-JDBC-compliant driver class
> >> org.postgresql.Driver (version 8.4)
> >> 2012-01-17 10:42:58,694 INFO  [org.jboss.as.remoting] (MSC service
> >> thread 1-1) Listening on /127.0.0.1:9999
> >> 2012-01-17 10:42:58,697 INFO  [org.jboss.as.remoting] (MSC service
> >> thread 1-2) Listening on /0.0.0.0:4447
> >> 2012-01-17 10:42:58,785 INFO
> >> [org.apache.catalina.core.AprLifecycleListener] (MSC service
> >> thread
> >> 1-4)
> >> The Apache Tomcat Native library which allows optimal performance
> >> in
> >> production environments was not found on the java.library.path:
> >> /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >> 2012-01-17 10:42:58,864 INFO
> >>   [org.apache.coyote.http11.Http11Protocol]
> >> (MSC service thread 1-4) Starting Coyote HTTP/1.1 on
> >> http--0.0.0.0-8080
> >> 2012-01-17 10:42:59,312 INFO
> >>   [org.jboss.as.server.deployment.scanner]
> >> (MSC service thread 1-4) JBAS015012: Started
> >> FileSystemDeploymentService
> >> for directory
> >> /usr/local/jboss-as-7.1.0.Beta1b/standalone/deployments
> >> 2012-01-17 10:42:59,341 INFO
> >> [org.jboss.as.connector.subsystems.datasources] (MSC service
> >> thread
> >> 1-4)
> >> JBAS010400: Bound data source [java:/ENGINEDataSource]
> >> 2012-01-17 10:42:59,355 INFO  [org.jboss.as] (Controller Boot
> >> Thread)
> >> JBoss AS 7.1.0.Beta1b "Tesla" started in 2128ms - Started 129 of
> >> 190
> >> services (60 services are passive or on-demand)
> >> 2012-01-17 10:42:59,420 INFO
> >>   [org.jboss.as.server.deployment.scanner]
> >> (DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed
> >> deployment engine.ear
> >> 2012-01-17 10:42:59,459 WARN
> >> [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (JCA
> >> PoolFiller) IJ000610: Unable to fill pool:
> >> javax.resource.ResourceException: Could not create connection
> >> at
> >> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277)
> >>       at
> >> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235)
> >>       at
> >> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:736)
> >> [ironjacamar-core-impl-1.0.5.Final.jar:]
> >>       at
> >> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.fillToMin(SemaphoreArrayListManagedConnectionPool.java:683)
> >> [ironjacamar-core-impl-1.0.5.Final.jar:]
> >>       at
> >> org.jboss.jca.core.connectionmanager.pool.mcp.PoolFiller.run(PoolFiller.java:97)
> >> [ironjacamar-core-impl-1.0.5.Final.jar:]
> >>       at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
> >> Caused by: org.postgresql.util.PSQLException: FATAL: password
> >> authentication failed for user "postgres"
> >>       at
> >> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
> >>       at
> >> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
> >>       at
> >> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
> >>       at
> >> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
> >>       at
> >> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
> >>       at
> >> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
> >>       at
> >> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
> >>       at
> >>       org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
> >>       at org.postgresql.Driver.makeConnection(Driver.java:393)
> >>       at org.postgresql.Driver.connect(Driver.java:267)
> >>       at
> >> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249)
> >>       ... 5 more
> >>
> >> 2012-01-17 10:42:59,491 INFO
> >>   [org.jboss.as.server.deployment.scanner]
> >> (DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in
> >> deployment directory. To trigger deployment create a file called
> >> engine.ear.dodeploy
> >> 2012-01-17 10:42:59,502 ERROR [org.jboss.as.controller]
> >> (DeploymentScanner-threads - 2) JBAS014612: Operation ("add")
> >> failed
> >> -
> >> address: ([("deployment" =>  "engine.ear")]):
> >> java.lang.IllegalStateException: JBAS014666: Duplicate resource
> >> [("deployment" =>  "engine.ear")]
> >>       at
> >> org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286)
> >> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>       at
> >> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
> >>       at
> >> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
> >>       at
> >> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >> [:1.6.0_22]
> >>       at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >>       [:1.6.0_22]
> >>       at
> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> >> [:1.6.0_22]
> >>       at
> >> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> >> [:1.6.0_22]
> >>       at
> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >> [:1.6.0_22]
> >>       at
> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >> [:1.6.0_22]
> >>       at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
> >>       at org.jboss.threads.JBossThread.run(JBossThread.java:122)
> >> [jboss-threads-2.0.0.GA.jar:]
> >>
> >> 2012-01-17 10:42:59,505 ERROR
> >> [org.jboss.as.server.deployment.scanner]
> >> (DeploymentScanner-threads - 1) JBAS014654: Composite operation
> >> was
> >> rolled back
> >> 2012-01-17 10:42:59,506 ERROR
> >> [org.jboss.as.server.deployment.scanner]
> >> (DeploymentScanner-threads - 1) {"JBAS014653: Composite operation
> >> failed
> >> and was rolled back. Steps that failed:" =>  {"Operation step-1"
> >> =>
> >> "JBAS014749: Operation handler failed: JBAS014666: Duplicate
> >> resource
> >> [(\"deployment\" =>  \"engine.ear\")]"}}
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mlipchuk at redhat.com  Tue Jan 17 13:18:24 2012
From: mlipchuk at redhat.com (Maor)
Date: Tue, 17 Jan 2012 15:18:24 +0200
Subject: [Engine-devel] Shared disk / Floating disk import/export (was:
 Re: move disk command)
In-Reply-To: <d73d764a-a001-43c5-85e7-c7e66b7aef1b@zmail07.collab.prod.int.phx2.redhat.com>
References: <d73d764a-a001-43c5-85e7-c7e66b7aef1b@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F157520.9090906@redhat.com>

On 01/17/2012 11:57 AM, Omer Frenkel wrote:
> 
> 
> ----- Original Message -----
>> From: "Ayal Baron" <abaron at redhat.com>
>> To: "Omer Frenkel" <ofrenkel at redhat.com>
>> Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>
>> Sent: Tuesday, January 17, 2012 10:45:53 AM
>> Subject: Shared disk / Floating disk import/export (was: Re: [Engine-devel] move disk command)
>>
>>
>> [SNIP]
>>
>>>>> This may be unrelated, but user would be allowed to export and
>>>>> import a floating disk, right?
>>>>> I would like the ability to import *any* disk in the export
>>>>> domain
>>>>> as a floating disk, but in the least, export and import disks
>>>>> not
>>>>> associated with a VM.
>>>
>>> you are right, it is unrelated, this thread is about move disk of a
>>> vm between SDs,
>>> export and import is copy, and floating disks is part of the shared
>>> disk feature,
>>> this indeed need to be discussed in that scope.
>>
>> Ok, so I've changed the subject, now let's discuss it.
>>
> 
> Adding Maor,
> not sure if we have any plan regard export/import of floating disk,
> or in general, exporting disk without it's vm (if I understood Ayal correctly)
> 
> Maor, any comments?
I remember, we mentioned export/import issue in our last meeting with
Andrew in the context of shared raw disk.
It was decided then, that export/import will not be supported by shared
raw disk, I'm not sure if we also decided that for floating disk,
but I think its a PM decision.

Support import/export domain might evolve to a big change, since if we
want to export disk, we might also want to reflect all its configuration
and functionality there, also reflect disks which are attached to VMs
and templates as well.


From rgolan at redhat.com  Tue Jan 17 13:25:49 2012
From: rgolan at redhat.com (Roy Golan)
Date: Tue, 17 Jan 2012 15:25:49 +0200
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <d4ed5faf-667e-4474-acb8-29867fceff4f@zmail07.collab.prod.int.phx2.redhat.com>
References: <4F1527A4.3090808@redhat.com>
	<d4ed5faf-667e-4474-acb8-29867fceff4f@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1576DD.2070001@redhat.com>

On Tue 17 Jan 2012 10:05:15 AM IST, Omer Frenkel wrote:
>
>
> ----- Original Message -----
>> From: "Moti Asayag"<masayag at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Tuesday, January 17, 2012 9:47:48 AM
>> Subject: Re: [Engine-devel] a different approach to the command classes
>>
>> On 01/17/2012 09:05 AM, Livnat Peer wrote:
>>> On 17/01/12 04:58, Jon Choate wrote:
>>>> The way the command classes are written has bothered me for a
>>>> while.
>>>> While implementing the multiple storage domain features I am
>>>> presented
>>>> with the opportunity to create a new command from scratch.  I gave
>>>> some
>>>> thought to what I would like the command classes to look like
>>>> while
>>>> balancing that the class must still fit in with the existing
>>>> structure.
>>>> So here is what I came up with. I'd appreciate any feedback.
>>>>
>>>> The Command class encompasses only the rules of what needs to be
>>>> done.
>>>> It relies upon Validator classes to determine if the canDoAction
>>>> conditions have been met.
>>>>
>>>>      @Override
>>>>      public boolean canDoAction() {
>>>>        ...
>>>>          checkTargetDomainHasSpace();
>>>>          checkTargetDomainIsValidTarget();
>>>>          checkSourceDomainIsValidSource();
>>>>          checkSourceAndTargetAreDifferent();
>>>>       ...
>>>> }
>>>>
>>>> ...
>>>>
>>>>    private void checkTargetDomainHasSpace() {
>>>>          if(!actionAllowed) return;
>>>>
>>>> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
>>>> {
>>>>
>>>> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
>>>>            actionAllowed = false;
>>>>          }
>>>>      }
>>>>
>>>>
>>>> Each of the checks follows a similar pattern of
>>>>      - guard clause to see of we already failed and bail if we did
>>>>      - test for failure of the condition
>>>>      - add failure message if needed
>>>>      - set variable to failed if needed
>>>>
>>>> Storing the success flag in a variable allows us to keep the
>>>> canDoAction
>>>> method readable as a series of commands and to allow it to be
>>>> accessed
>>>> by all the private methods without them having to pass it around.
>>>>
>>>> The execution of the command will follow a similar pattern where
>>>> the
>>>> command class will only know how to describe what needs to be done
>>>> and
>>>> to rely on supporting objects to handle the implementation of
>>>> these
>>>> steps.  Getting the implementation out of the command classes will
>>>> allow
>>>> the commands to share validation and implementation details and
>>>> remove a
>>>> lot of the duplication that currently exists within the commands.
>>>>
>>>>
>>>> How do people feel about this approach?
>>>
>>>
>>> Hi Jon,
>>>
>>> The scope of your proposal is changing the implementation of the
>>> canDoAction method, I think that the title of this mail is a bit
>>> misleading.
>>>
>>> Basically what you are suggesting is to split the canDoAction
>>> implementation into methods and then extract them from the command
>>> class
>>> to a shared class so they can be reused.
>>>
>>> In many cases we can use (are using) inheritance for reusing code,
>>> there
>>> are cases where inheritance does not do the work and we can extract
>>> to
>>> external classes.
>>>
>>> I think such a change is welcomed but on a needed basis, I think it
>>> is
>>> overkill for the general use case and will make the code more
>>> cumbersome
>>> (if the original canDoAction was 4-5 lines long...).
>>>
>
> i agree as well
>
>>> One thing I don't like in the above suggestion is the way you
>>> validate
>>> that the previous condition succeeded/failed. Having this condition
>>> at
>>> the beginning of each validation method is not a good approach IMO.
>>>
>>
>> In addition, it prevents from independent validations (inside the
>> same
>> canDoAction) to report on several validation failures in a single
>> execution instead of getting a single failure, rerun the command
>> again,
>> get another failure and on...
>> This is the reason why the type of canDoActionMessages is a List.
>>
>
> actually this is how it works today in most cases, i think its not bad,
> some checks relay on the fact that if this check is executed,
> then it's safe to do things, for example - first check validates vm is not null,
> second check use the vm, assuming its not null.
>
>
> i would go further and make the validation methods return boolean and call them in that way:
>
>      public boolean canDoAction() {
>        ...return
>          checkTargetDomainHasSpace()&&
>          checkTargetDomainIsValidTarget()&&
>          checkSourceDomainIsValidSource()&&
>          checkSourceAndTargetAreDifferent();
>       ...
> }
>
> private void checkTargetDomainHasSpace() {
>   if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
>   {
>     addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
>     reutrn false;
>   }
>   return true;
> }
>
>>>
>>> Livnat
>>>
>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel


I agree that it works better mainly in long and complex checks (did 
that in setupNetworks). Its better to encapsulate shared validation 
code in validator classes instead of using inheritance for that.
It is also easier to test the validator than constructing a command to 
test  the canDoAction part.

I'll take advantage and a rule of thumb - to make the canDoAction clean 
and simple make sure all parameter validation is done
via the validation framework. 
code  like if (getParameters().getSomeParam != null ) { 
addCanDoMsg(msg) } shouldn't be a part of your canDoAction.
Instead use:

class SomeBaseParameters

@NotNull
SomeParam someParam


From ykaul at redhat.com  Tue Jan 17 13:27:50 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Tue, 17 Jan 2012 15:27:50 +0200
Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
In-Reply-To: <d203a1b9-960a-4b73-acfe-91ffc97434eb@zmail02.collab.prod.int.phx2.redhat.com>
References: <d203a1b9-960a-4b73-acfe-91ffc97434eb@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <4F157756.3020406@redhat.com>

On 01/17/2012 03:05 PM, Oved Ourfalli wrote:
> The question is what did you have in the previous environment (as5) in postgres-ds.xml file.

I've had the 'password' field enabled there. I suspect it's because I 
did 'setup' in my deployment with Jboss 7.
I've edited standalone.xml, and I'm now into my next issue:
2012-01-17 15:24:34,678 INFO  [org.jboss.as.server.deployment.scanner] 
(DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed 
deployment engine.ear
2012-01-17 15:24:34,747 INFO  [org.jboss.as.server.deployment.scanner] 
(DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in 
deployment directory. To trigger deployment create a file called 
engine.ear.dodeploy
2012-01-17 15:24:34,758 ERROR [org.jboss.as.controller] 
(DeploymentScanner-threads - 2) JBAS014612: Operation ("add") failed - 
address: ([("deployment" => "engine.ear")]): 
java.lang.IllegalStateException: JBAS014666: Duplicate resource 
[("deployment" => "engine.ear")]
     at 
org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286) 
[jboss-as-controller-7.1.0.Beta1b.jar:]
     at 
org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
     at 
org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
     at 
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) 
[:1.6.0_22]
     at java.util.concurrent.FutureTask.run(FutureTask.java:166) [:1.6.0_22]
     at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) 
[:1.6.0_22]
     at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) 
[:1.6.0_22]
     at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) 
[:1.6.0_22]
     at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) 
[:1.6.0_22]
     at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
     at org.jboss.threads.JBossThread.run(JBossThread.java:122) 
[jboss-threads-2.0.0.GA.jar:]

2012-01-17 15:24:34,761 ERROR [org.jboss.as.server.deployment.scanner] 
(DeploymentScanner-threads - 1) JBAS014654: Composite operation was 
rolled back
2012-01-17 15:24:34,762 ERROR [org.jboss.as.server.deployment.scanner] 
(DeploymentScanner-threads - 1) {"JBAS014653: Composite operation failed 
and was rolled back. Steps that failed:" => {"Operation step-1" => 
"JBAS014749: Operation handler failed: JBAS014666: Duplicate resource 
[(\"deployment\" => \"engine.ear\")]"}}

>
>
> ----- Original Message -----
>> From: "Yaniv Kaul"<ykaul at redhat.com>
>> To: "Oved Ourfalli"<ovedo at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Tuesday, January 17, 2012 2:43:24 PM
>> Subject: Re: [Engine-devel] Failure to deploy (with JBoss 7.1)
>>
>> On 01/17/2012 10:57 AM, Oved Ourfalli wrote:
>>> Did you use a username+password in the previous datasource (in AS5)
>>> or was a trust defined for the postgres instance?
>>>
>>> If needed, the username+password must be defined in
>>> $JBOSS_HOME/standalone/configuration/standalone.xml file:
>>> 1. If encrypted then in the EncryptDBPassword section, and
>>> reference it in the data source, by using the following instead of
>>> username:
>>>                           <security-domain>
>>>                               EncryptDBPassword
>>>                           </security-domain>
>>> 2. If not encrypted then just put the password in the datasource
>>> definition.
>> This is what I have:
>> <security>
>> <user-name>
>>                               postgres
>> </user-name>
>> <!-- IF NEEDED CHANGE THIS TO THE CORRECT PASSWORD/SECURITY DOMAIN
>> -->
>> <!--password>
>>                               password
>> </password-->
>> <!--security-domain>
>>                               EncryptDBPassword
>> </security-domain-->
>> </security>
>>
>> So as far as I understand, both options are actually commented out.
>> Y.
>>
>>> I can connect to your machine and help if you want.
>>>
>>> Oved
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Yaniv Kaul"<ykaul at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Sent: Tuesday, January 17, 2012 10:46:55 AM
>>>> Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
>>>>
>>>> After successful compilation and deployment, on a host that ran
>>>> previously fine, I'm getting some error on executing.
>>>> Not sure what happened to postgres access - it ran fine previously
>>>> (previous JBoss):
>>>>
>>>> [ykaul at ykaul ear]$
>>>> /usr/local/jboss-as-7.1.0.Beta1b/bin/standalone.sh
>>>> =========================================================================
>>>>
>>>>      JBoss Bootstrap Environment
>>>>
>>>>      JBOSS_HOME: /usr/local/jboss-as-7.1.0.Beta1b
>>>>
>>>>      JAVA: java
>>>>
>>>>      JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MaxPermSize=256m
>>>> -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true
>>>> -Dsun.rmi.dgc.client.gcInterval=3600000
>>>> -Dsun.rmi.dgc.server.gcInterval=3600000
>>>> -Djboss.modules.system.pkgs=org.jboss.byteman
>>>> -Djava.awt.headless=true
>>>>
>>>> =========================================================================
>>>>
>>>> 10:42:57,373 INFO  [org.jboss.modules] JBoss Modules version
>>>> 1.1.0.CR4
>>>> 10:42:57,608 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA
>>>> 10:42:57,665 INFO  [org.jboss.as] JBoss AS 7.1.0.Beta1b "Tesla"
>>>> starting
>>>> 10:42:58,429 INFO  [org.jboss.as] Creating http management service
>>>> using  socket-binding (management-http)
>>>> 10:42:58,434 INFO  [org.xnio] XNIO Version 3.0.0.CR5
>>>> 10:42:58,461 INFO  [org.xnio.nio] XNIO NIO Implementation Version
>>>> 3.0.0.CR5
>>>> 10:42:58,471 INFO  [org.jboss.remoting] JBoss Remoting version
>>>> 3.2.0.CR6
>>>> 10:42:58,477 INFO  [org.jboss.as.logging] JBAS011502: Removing
>>>> bootstrap
>>>> log handlers
>>>> 2012-01-17 10:42:58,502 INFO  [org.jboss.as.clustering]
>>>> (ServerService
>>>> Thread Pool -- 28) JBAS010300: Activating Infinispan subsystem.
>>>> 2012-01-17 10:42:58,551 INFO  [org.jboss.as.connector] (MSC
>>>> service
>>>> thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss IronJacamar
>>>> 1.0.5.Final)
>>>> 2012-01-17 10:42:58,564 INFO  [org.jboss.as.naming] (ServerService
>>>> Thread Pool -- 35) JBAS011800: Activating Naming Subsystem
>>>> 2012-01-17 10:42:58,592 INFO  [org.jboss.as.osgi] (ServerService
>>>> Thread
>>>> Pool -- 36) JBAS011910: Activating OSGi Subsystem
>>>> 2012-01-17 10:42:58,602 INFO  [org.jboss.as.naming] (MSC service
>>>> thread
>>>> 1-1) JBAS011802: Starting Naming Service
>>>> 2012-01-17 10:42:58,605 INFO  [org.jboss.as.security]
>>>> (ServerService
>>>> Thread Pool -- 41) Activating Security Subsystem
>>>> 2012-01-17 10:42:58,614 INFO  [org.jboss.as.mail.extension] (MSC
>>>> service
>>>> thread 1-1) JBAS015400: Bound mail session
>>>> [java:jboss/mail/Default]
>>>> 2012-01-17 10:42:58,615 INFO  [org.jboss.as.security] (MSC service
>>>> thread 1-1) Picketbox version=4.0.6.Beta1
>>>> 2012-01-17 10:42:58,623 INFO
>>>> [org.jboss.as.connector.subsystems.datasources] (ServerService
>>>> Thread
>>>> Pool -- 24) JBAS010404: Deploying non-JDBC-compliant driver class
>>>> org.postgresql.Driver (version 8.4)
>>>> 2012-01-17 10:42:58,694 INFO  [org.jboss.as.remoting] (MSC service
>>>> thread 1-1) Listening on /127.0.0.1:9999
>>>> 2012-01-17 10:42:58,697 INFO  [org.jboss.as.remoting] (MSC service
>>>> thread 1-2) Listening on /0.0.0.0:4447
>>>> 2012-01-17 10:42:58,785 INFO
>>>> [org.apache.catalina.core.AprLifecycleListener] (MSC service
>>>> thread
>>>> 1-4)
>>>> The Apache Tomcat Native library which allows optimal performance
>>>> in
>>>> production environments was not found on the java.library.path:
>>>> /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>> 2012-01-17 10:42:58,864 INFO
>>>>    [org.apache.coyote.http11.Http11Protocol]
>>>> (MSC service thread 1-4) Starting Coyote HTTP/1.1 on
>>>> http--0.0.0.0-8080
>>>> 2012-01-17 10:42:59,312 INFO
>>>>    [org.jboss.as.server.deployment.scanner]
>>>> (MSC service thread 1-4) JBAS015012: Started
>>>> FileSystemDeploymentService
>>>> for directory
>>>> /usr/local/jboss-as-7.1.0.Beta1b/standalone/deployments
>>>> 2012-01-17 10:42:59,341 INFO
>>>> [org.jboss.as.connector.subsystems.datasources] (MSC service
>>>> thread
>>>> 1-4)
>>>> JBAS010400: Bound data source [java:/ENGINEDataSource]
>>>> 2012-01-17 10:42:59,355 INFO  [org.jboss.as] (Controller Boot
>>>> Thread)
>>>> JBoss AS 7.1.0.Beta1b "Tesla" started in 2128ms - Started 129 of
>>>> 190
>>>> services (60 services are passive or on-demand)
>>>> 2012-01-17 10:42:59,420 INFO
>>>>    [org.jboss.as.server.deployment.scanner]
>>>> (DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed
>>>> deployment engine.ear
>>>> 2012-01-17 10:42:59,459 WARN
>>>> [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (JCA
>>>> PoolFiller) IJ000610: Unable to fill pool:
>>>> javax.resource.ResourceException: Could not create connection
>>>> at
>>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277)
>>>>        at
>>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235)
>>>>        at
>>>> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:736)
>>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>>>        at
>>>> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.fillToMin(SemaphoreArrayListManagedConnectionPool.java:683)
>>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>>>        at
>>>> org.jboss.jca.core.connectionmanager.pool.mcp.PoolFiller.run(PoolFiller.java:97)
>>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>>>        at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>>>> Caused by: org.postgresql.util.PSQLException: FATAL: password
>>>> authentication failed for user "postgres"
>>>>        at
>>>> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
>>>>        at
>>>> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
>>>>        at
>>>> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
>>>>        at
>>>> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
>>>>        at
>>>> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
>>>>        at
>>>> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
>>>>        at
>>>> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
>>>>        at
>>>>        org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
>>>>        at org.postgresql.Driver.makeConnection(Driver.java:393)
>>>>        at org.postgresql.Driver.connect(Driver.java:267)
>>>>        at
>>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249)
>>>>        ... 5 more
>>>>
>>>> 2012-01-17 10:42:59,491 INFO
>>>>    [org.jboss.as.server.deployment.scanner]
>>>> (DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in
>>>> deployment directory. To trigger deployment create a file called
>>>> engine.ear.dodeploy
>>>> 2012-01-17 10:42:59,502 ERROR [org.jboss.as.controller]
>>>> (DeploymentScanner-threads - 2) JBAS014612: Operation ("add")
>>>> failed
>>>> -
>>>> address: ([("deployment" =>   "engine.ear")]):
>>>> java.lang.IllegalStateException: JBAS014666: Duplicate resource
>>>> [("deployment" =>   "engine.ear")]
>>>>        at
>>>> org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286)
>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>        at
>>>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
>>>>        at
>>>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
>>>>        at
>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>>>> [:1.6.0_22]
>>>>        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>>>        [:1.6.0_22]
>>>>        at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>>>> [:1.6.0_22]
>>>>        at
>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>>>> [:1.6.0_22]
>>>>        at
>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>> [:1.6.0_22]
>>>>        at
>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>> [:1.6.0_22]
>>>>        at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>>>>        at org.jboss.threads.JBossThread.run(JBossThread.java:122)
>>>> [jboss-threads-2.0.0.GA.jar:]
>>>>
>>>> 2012-01-17 10:42:59,505 ERROR
>>>> [org.jboss.as.server.deployment.scanner]
>>>> (DeploymentScanner-threads - 1) JBAS014654: Composite operation
>>>> was
>>>> rolled back
>>>> 2012-01-17 10:42:59,506 ERROR
>>>> [org.jboss.as.server.deployment.scanner]
>>>> (DeploymentScanner-threads - 1) {"JBAS014653: Composite operation
>>>> failed
>>>> and was rolled back. Steps that failed:" =>   {"Operation step-1"
>>>> =>
>>>> "JBAS014749: Operation handler failed: JBAS014666: Duplicate
>>>> resource
>>>> [(\"deployment\" =>   \"engine.ear\")]"}}
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From jchoate at redhat.com  Tue Jan 17 13:37:54 2012
From: jchoate at redhat.com (Jon Choate)
Date: Tue, 17 Jan 2012 08:37:54 -0500 (EST)
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <4F15405F.3000508@redhat.com>
Message-ID: <39727574-be45-443d-8dfe-f9b7df1ae52b@zmail15.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Maor" <mlipchuk at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Tuesday, January 17, 2012 4:33:19 AM
> Subject: Re: [Engine-devel] a different approach to the command classes
> 
> On 01/17/2012 09:24 AM, Ayal Baron wrote:
> > 
> > 
> > ----- Original Message -----
> >> On 17/01/12 04:58, Jon Choate wrote:
> >>> The way the command classes are written has bothered me for a
> >>> while.
> >>> While implementing the multiple storage domain features I am
> >>> presented
> >>> with the opportunity to create a new command from scratch.  I
> >>> gave
> >>> some
> >>> thought to what I would like the command classes to look like
> >>> while
> >>> balancing that the class must still fit in with the existing
> >>> structure.
> >>> So here is what I came up with. I'd appreciate any feedback.
> >>>
> >>> The Command class encompasses only the rules of what needs to be
> >>> done.
> >>> It relies upon Validator classes to determine if the canDoAction
> >>> conditions have been met.
> >>>
> >>>     @Override
> >>>     public boolean canDoAction() {
> >>>       ...
> >>>         checkTargetDomainHasSpace();
> >>>         checkTargetDomainIsValidTarget();
> >>>         checkSourceDomainIsValidSource();
> >>>         checkSourceAndTargetAreDifferent();
> >>>      ...
> >>> }
> >>>
> >>> ...
> >>>
> >>>   private void checkTargetDomainHasSpace() {
> >>>         if(!actionAllowed) return;
> >>>        
> >>> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
> >>> {
> >>>          
> >>> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
> >>>           actionAllowed = false;
> >>>         }
> >>>     }
> >>>
> >>>
> >>> Each of the checks follows a similar pattern of
> >>>     - guard clause to see of we already failed and bail if we did
> >>>     - test for failure of the condition
> >>>     - add failure message if needed
> >>>     - set variable to failed if needed
> >>>
> >>> Storing the success flag in a variable allows us to keep the
> >>> canDoAction
> >>> method readable as a series of commands and to allow it to be
> >>> accessed
> >>> by all the private methods without them having to pass it around.
> >>>
> >>> The execution of the command will follow a similar pattern where
> >>> the
> >>> command class will only know how to describe what needs to be
> >>> done
> >>> and
> >>> to rely on supporting objects to handle the implementation of
> >>> these
> >>> steps.  Getting the implementation out of the command classes
> >>> will
> >>> allow
> >>> the commands to share validation and implementation details and
> >>> remove a
> >>> lot of the duplication that currently exists within the commands.
> >>>
> >>>
> >>> How do people feel about this approach?
> >>
> >>
> >> Hi Jon,
> >>
> >> The scope of your proposal is changing the implementation of the
> >> canDoAction method, I think that the title of this mail is a bit
> >> misleading.
> > 
> > Actually I think Jon was suggesting to do the same in the command
> > itself.


Yes I am.  I just haven't written that code yet!


> 
> I think, using shared canDoAction validation between commands might
> be a
> good idea also, it can be seen in operations such as add/update
> commands.
> In most cases, the update validation, is already based on the add
> command validation, sometime it is implemented with
> inheritance/external
> class, in other cases it is just implemented with multiple code.
> > 
> >>
> >> Basically what you are suggesting is to split the canDoAction
> >> implementation into methods and then extract them from the command
> >> class
> >> to a shared class so they can be reused.
> >>
> >> In many cases we can use (are using) inheritance for reusing code,
> >> there
> >> are cases where inheritance does not do the work and we can
> >> extract
> >> to
> >> external classes.
> >>

Our overuse of inheritance is one of the things I'm trying to rectify.  Inheritance wasn't added 
to the language to facilitate reuse. It is there to represent object hierarchies.  In some cases
we abuse this to leverage code reuse.  This is often a code smell that the code is in the wrong
place to begin with.  If both classes need the code and they are not logically related in an object
hierarchy, the code really needs to be outside the classes.

We have cases where the use of inheritance for reuse have gone too far. For instance:

public class RemoveVdsSpmIdCommand<T extends VdsActionParameters> extends AddVdsSpmIdCommand<T>

So this says that a RemoveVdsSmpIdCommand is a type of AddVdsSpmIdCommand?  That implies that I can
use a RemoveVdsSmpIdCommand anywhere that an AddVdsSpmIdCommand is expected. Otherwise we have broken
one of the fundamental contracts of object oriented programming.

> >> I think such a change is welcomed but on a needed basis, I think
> >> it
> >> is
> >> overkill for the general use case and will make the code more
> >> cumbersome
> >> (if the original canDoAction was 4-5 lines long...).

>From what I have seen those canDoActions are in the minority.  The overall goals are to increase
the readability and maintainability of the code so I would suggest being pragmatic about any approach.
If doing it doesn't help achieve the goal, then don't do it.


One of the ideas I'm trying to convey here is that the command classes should be fairly ignorant.
They should be responsible for knowing the list of steps involved in a workflow, but not the details
of how those steps are carried out. Imo knowing both the steps and their implementation violates the SRP.


> > 
> > Jon, I think the burden of proof is on you here to show a real
> > example and how it makes the code clearer (e.g. change 2 commands
> > which share similar checks).
> > Without 'seeing' it I don't think we would be able to appreciate
> > the advantage of your approach.
> > 

I need to get the multiple storage domains feature put away and then I will
definitely do some refactoring to illustrate the value. I wholeheartedly agree
that having actual code to kick around is better. I just can't do it right now! 


> >>
> >> One thing I don't like in the above suggestion is the way you
> >> validate
> >> that the previous condition succeeded/failed. Having this
> >> condition
> >> at
> >> the beginning of each validation method is not a good approach
> >> IMO.

Can you elaborate on your opposition to the use of guard clauses? They have
been considered a best practice for quite some time.

> > 
> > +1, if the previous check failed it should raise an exception, not
> > rely on the next check to bail.
> > It would be easier (less code, clearer) to wrap all the calls with
> > a try catch clause (1 for all the calls), catch the specific
> > exception that says the check failed and return whatever you want
> > to return.

I'm not really a fan of using exceptions for controlling flow of execution.  Failing one of these
checks is not an exceptional event. Its an expected part of the workflow and should
be handled as a normal occurrence.

I would argue that wrapping the entire canDoAction method in a try/catch would not make the code clearer:


  public boolean canDoAction() {
      boolean succeeded = true;
      try {
         do1();
         do2();
         do3();
      catch(ValidationException e) {
         succeeded = false;
      }
      return succeeded;
  }


has a lot more noise than:

  private boolean canDoAction = true;

  public boolean canDoAction() {
    do1();
    do2();
    do3();
    return canDoAction;
  }

In the second form, there is no extra indentation or curly braces.  The validation steps
look just like a list of steps and imo it makes the code really easy to understand by a maintainer.

> > 
> >>
> >>
> >> Livnat
> >>
> >>
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From ovedo at redhat.com  Tue Jan 17 13:41:25 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Tue, 17 Jan 2012 08:41:25 -0500 (EST)
Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
In-Reply-To: <4F157756.3020406@redhat.com>
Message-ID: <483fd37c-e875-448f-9d0a-922efdb0132c@zmail02.collab.prod.int.phx2.redhat.com>

I saw that error before, and running mvn clean install -Psetup,dep fixed this issue.
But, I would like to examine it a bit before I tell you to do that.
Can you send me the details to your environment offline, and I'll do some testing there?

----- Original Message -----
> From: "Yaniv Kaul" <ykaul at redhat.com>
> To: "Oved Ourfalli" <ovedo at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Tuesday, January 17, 2012 3:27:50 PM
> Subject: Re: [Engine-devel] Failure to deploy (with JBoss 7.1)
> 
> On 01/17/2012 03:05 PM, Oved Ourfalli wrote:
> > The question is what did you have in the previous environment (as5)
> > in postgres-ds.xml file.
> 
> I've had the 'password' field enabled there. I suspect it's because I
> did 'setup' in my deployment with Jboss 7.
> I've edited standalone.xml, and I'm now into my next issue:
> 2012-01-17 15:24:34,678 INFO
>  [org.jboss.as.server.deployment.scanner]
> (DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed
> deployment engine.ear
> 2012-01-17 15:24:34,747 INFO
>  [org.jboss.as.server.deployment.scanner]
> (DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in
> deployment directory. To trigger deployment create a file called
> engine.ear.dodeploy
> 2012-01-17 15:24:34,758 ERROR [org.jboss.as.controller]
> (DeploymentScanner-threads - 2) JBAS014612: Operation ("add") failed
> -
> address: ([("deployment" => "engine.ear")]):
> java.lang.IllegalStateException: JBAS014666: Duplicate resource
> [("deployment" => "engine.ear")]
>      at
> org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286)
> [jboss-as-controller-7.1.0.Beta1b.jar:]
>      at
> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
>      at
> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
>      at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> [:1.6.0_22]
>      at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>      [:1.6.0_22]
>      at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> [:1.6.0_22]
>      at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> [:1.6.0_22]
>      at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> [:1.6.0_22]
>      at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> [:1.6.0_22]
>      at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>      at org.jboss.threads.JBossThread.run(JBossThread.java:122)
> [jboss-threads-2.0.0.GA.jar:]
> 
> 2012-01-17 15:24:34,761 ERROR
> [org.jboss.as.server.deployment.scanner]
> (DeploymentScanner-threads - 1) JBAS014654: Composite operation was
> rolled back
> 2012-01-17 15:24:34,762 ERROR
> [org.jboss.as.server.deployment.scanner]
> (DeploymentScanner-threads - 1) {"JBAS014653: Composite operation
> failed
> and was rolled back. Steps that failed:" => {"Operation step-1" =>
> "JBAS014749: Operation handler failed: JBAS014666: Duplicate resource
> [(\"deployment\" => \"engine.ear\")]"}}
> 
> >
> >
> > ----- Original Message -----
> >> From: "Yaniv Kaul"<ykaul at redhat.com>
> >> To: "Oved Ourfalli"<ovedo at redhat.com>
> >> Cc: engine-devel at ovirt.org
> >> Sent: Tuesday, January 17, 2012 2:43:24 PM
> >> Subject: Re: [Engine-devel] Failure to deploy (with JBoss 7.1)
> >>
> >> On 01/17/2012 10:57 AM, Oved Ourfalli wrote:
> >>> Did you use a username+password in the previous datasource (in
> >>> AS5)
> >>> or was a trust defined for the postgres instance?
> >>>
> >>> If needed, the username+password must be defined in
> >>> $JBOSS_HOME/standalone/configuration/standalone.xml file:
> >>> 1. If encrypted then in the EncryptDBPassword section, and
> >>> reference it in the data source, by using the following instead
> >>> of
> >>> username:
> >>>                           <security-domain>
> >>>                               EncryptDBPassword
> >>>                           </security-domain>
> >>> 2. If not encrypted then just put the password in the datasource
> >>> definition.
> >> This is what I have:
> >> <security>
> >> <user-name>
> >>                               postgres
> >> </user-name>
> >> <!-- IF NEEDED CHANGE THIS TO THE CORRECT PASSWORD/SECURITY DOMAIN
> >> -->
> >> <!--password>
> >>                               password
> >> </password-->
> >> <!--security-domain>
> >>                               EncryptDBPassword
> >> </security-domain-->
> >> </security>
> >>
> >> So as far as I understand, both options are actually commented
> >> out.
> >> Y.
> >>
> >>> I can connect to your machine and help if you want.
> >>>
> >>> Oved
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Yaniv Kaul"<ykaul at redhat.com>
> >>>> To: engine-devel at ovirt.org
> >>>> Sent: Tuesday, January 17, 2012 10:46:55 AM
> >>>> Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
> >>>>
> >>>> After successful compilation and deployment, on a host that ran
> >>>> previously fine, I'm getting some error on executing.
> >>>> Not sure what happened to postgres access - it ran fine
> >>>> previously
> >>>> (previous JBoss):
> >>>>
> >>>> [ykaul at ykaul ear]$
> >>>> /usr/local/jboss-as-7.1.0.Beta1b/bin/standalone.sh
> >>>> =========================================================================
> >>>>
> >>>>      JBoss Bootstrap Environment
> >>>>
> >>>>      JBOSS_HOME: /usr/local/jboss-as-7.1.0.Beta1b
> >>>>
> >>>>      JAVA: java
> >>>>
> >>>>      JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MaxPermSize=256m
> >>>> -Djava.net.preferIPv4Stack=true
> >>>> -Dorg.jboss.resolver.warning=true
> >>>> -Dsun.rmi.dgc.client.gcInterval=3600000
> >>>> -Dsun.rmi.dgc.server.gcInterval=3600000
> >>>> -Djboss.modules.system.pkgs=org.jboss.byteman
> >>>> -Djava.awt.headless=true
> >>>>
> >>>> =========================================================================
> >>>>
> >>>> 10:42:57,373 INFO  [org.jboss.modules] JBoss Modules version
> >>>> 1.1.0.CR4
> >>>> 10:42:57,608 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA
> >>>> 10:42:57,665 INFO  [org.jboss.as] JBoss AS 7.1.0.Beta1b "Tesla"
> >>>> starting
> >>>> 10:42:58,429 INFO  [org.jboss.as] Creating http management
> >>>> service
> >>>> using  socket-binding (management-http)
> >>>> 10:42:58,434 INFO  [org.xnio] XNIO Version 3.0.0.CR5
> >>>> 10:42:58,461 INFO  [org.xnio.nio] XNIO NIO Implementation
> >>>> Version
> >>>> 3.0.0.CR5
> >>>> 10:42:58,471 INFO  [org.jboss.remoting] JBoss Remoting version
> >>>> 3.2.0.CR6
> >>>> 10:42:58,477 INFO  [org.jboss.as.logging] JBAS011502: Removing
> >>>> bootstrap
> >>>> log handlers
> >>>> 2012-01-17 10:42:58,502 INFO  [org.jboss.as.clustering]
> >>>> (ServerService
> >>>> Thread Pool -- 28) JBAS010300: Activating Infinispan subsystem.
> >>>> 2012-01-17 10:42:58,551 INFO  [org.jboss.as.connector] (MSC
> >>>> service
> >>>> thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss
> >>>> IronJacamar
> >>>> 1.0.5.Final)
> >>>> 2012-01-17 10:42:58,564 INFO  [org.jboss.as.naming]
> >>>> (ServerService
> >>>> Thread Pool -- 35) JBAS011800: Activating Naming Subsystem
> >>>> 2012-01-17 10:42:58,592 INFO  [org.jboss.as.osgi] (ServerService
> >>>> Thread
> >>>> Pool -- 36) JBAS011910: Activating OSGi Subsystem
> >>>> 2012-01-17 10:42:58,602 INFO  [org.jboss.as.naming] (MSC service
> >>>> thread
> >>>> 1-1) JBAS011802: Starting Naming Service
> >>>> 2012-01-17 10:42:58,605 INFO  [org.jboss.as.security]
> >>>> (ServerService
> >>>> Thread Pool -- 41) Activating Security Subsystem
> >>>> 2012-01-17 10:42:58,614 INFO  [org.jboss.as.mail.extension] (MSC
> >>>> service
> >>>> thread 1-1) JBAS015400: Bound mail session
> >>>> [java:jboss/mail/Default]
> >>>> 2012-01-17 10:42:58,615 INFO  [org.jboss.as.security] (MSC
> >>>> service
> >>>> thread 1-1) Picketbox version=4.0.6.Beta1
> >>>> 2012-01-17 10:42:58,623 INFO
> >>>> [org.jboss.as.connector.subsystems.datasources] (ServerService
> >>>> Thread
> >>>> Pool -- 24) JBAS010404: Deploying non-JDBC-compliant driver
> >>>> class
> >>>> org.postgresql.Driver (version 8.4)
> >>>> 2012-01-17 10:42:58,694 INFO  [org.jboss.as.remoting] (MSC
> >>>> service
> >>>> thread 1-1) Listening on /127.0.0.1:9999
> >>>> 2012-01-17 10:42:58,697 INFO  [org.jboss.as.remoting] (MSC
> >>>> service
> >>>> thread 1-2) Listening on /0.0.0.0:4447
> >>>> 2012-01-17 10:42:58,785 INFO
> >>>> [org.apache.catalina.core.AprLifecycleListener] (MSC service
> >>>> thread
> >>>> 1-4)
> >>>> The Apache Tomcat Native library which allows optimal
> >>>> performance
> >>>> in
> >>>> production environments was not found on the java.library.path:
> >>>> /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
> >>>> 2012-01-17 10:42:58,864 INFO
> >>>>    [org.apache.coyote.http11.Http11Protocol]
> >>>> (MSC service thread 1-4) Starting Coyote HTTP/1.1 on
> >>>> http--0.0.0.0-8080
> >>>> 2012-01-17 10:42:59,312 INFO
> >>>>    [org.jboss.as.server.deployment.scanner]
> >>>> (MSC service thread 1-4) JBAS015012: Started
> >>>> FileSystemDeploymentService
> >>>> for directory
> >>>> /usr/local/jboss-as-7.1.0.Beta1b/standalone/deployments
> >>>> 2012-01-17 10:42:59,341 INFO
> >>>> [org.jboss.as.connector.subsystems.datasources] (MSC service
> >>>> thread
> >>>> 1-4)
> >>>> JBAS010400: Bound data source [java:/ENGINEDataSource]
> >>>> 2012-01-17 10:42:59,355 INFO  [org.jboss.as] (Controller Boot
> >>>> Thread)
> >>>> JBoss AS 7.1.0.Beta1b "Tesla" started in 2128ms - Started 129 of
> >>>> 190
> >>>> services (60 services are passive or on-demand)
> >>>> 2012-01-17 10:42:59,420 INFO
> >>>>    [org.jboss.as.server.deployment.scanner]
> >>>> (DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed
> >>>> deployment engine.ear
> >>>> 2012-01-17 10:42:59,459 WARN
> >>>> [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool]
> >>>> (JCA
> >>>> PoolFiller) IJ000610: Unable to fill pool:
> >>>> javax.resource.ResourceException: Could not create connection
> >>>> at
> >>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277)
> >>>>        at
> >>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235)
> >>>>        at
> >>>> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:736)
> >>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
> >>>>        at
> >>>> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.fillToMin(SemaphoreArrayListManagedConnectionPool.java:683)
> >>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
> >>>>        at
> >>>> org.jboss.jca.core.connectionmanager.pool.mcp.PoolFiller.run(PoolFiller.java:97)
> >>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
> >>>>        at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
> >>>> Caused by: org.postgresql.util.PSQLException: FATAL: password
> >>>> authentication failed for user "postgres"
> >>>>        at
> >>>> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
> >>>>        at
> >>>> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
> >>>>        at
> >>>> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
> >>>>        at
> >>>> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
> >>>>        at
> >>>> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
> >>>>        at
> >>>> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
> >>>>        at
> >>>> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
> >>>>        at
> >>>>        org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
> >>>>        at org.postgresql.Driver.makeConnection(Driver.java:393)
> >>>>        at org.postgresql.Driver.connect(Driver.java:267)
> >>>>        at
> >>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249)
> >>>>        ... 5 more
> >>>>
> >>>> 2012-01-17 10:42:59,491 INFO
> >>>>    [org.jboss.as.server.deployment.scanner]
> >>>> (DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in
> >>>> deployment directory. To trigger deployment create a file called
> >>>> engine.ear.dodeploy
> >>>> 2012-01-17 10:42:59,502 ERROR [org.jboss.as.controller]
> >>>> (DeploymentScanner-threads - 2) JBAS014612: Operation ("add")
> >>>> failed
> >>>> -
> >>>> address: ([("deployment" =>   "engine.ear")]):
> >>>> java.lang.IllegalStateException: JBAS014666: Duplicate resource
> >>>> [("deployment" =>   "engine.ear")]
> >>>>        at
> >>>> org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286)
> >>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
> >>>>        at
> >>>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
> >>>>        at
> >>>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
> >>>>        at
> >>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> >>>> [:1.6.0_22]
> >>>>        at
> >>>>        java.util.concurrent.FutureTask.run(FutureTask.java:166)
> >>>>        [:1.6.0_22]
> >>>>        at
> >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> >>>> [:1.6.0_22]
> >>>>        at
> >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> >>>> [:1.6.0_22]
> >>>>        at
> >>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >>>> [:1.6.0_22]
> >>>>        at
> >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >>>> [:1.6.0_22]
> >>>>        at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
> >>>>        at
> >>>>        org.jboss.threads.JBossThread.run(JBossThread.java:122)
> >>>> [jboss-threads-2.0.0.GA.jar:]
> >>>>
> >>>> 2012-01-17 10:42:59,505 ERROR
> >>>> [org.jboss.as.server.deployment.scanner]
> >>>> (DeploymentScanner-threads - 1) JBAS014654: Composite operation
> >>>> was
> >>>> rolled back
> >>>> 2012-01-17 10:42:59,506 ERROR
> >>>> [org.jboss.as.server.deployment.scanner]
> >>>> (DeploymentScanner-threads - 1) {"JBAS014653: Composite
> >>>> operation
> >>>> failed
> >>>> and was rolled back. Steps that failed:" =>   {"Operation
> >>>> step-1"
> >>>> =>
> >>>> "JBAS014749: Operation handler failed: JBAS014666: Duplicate
> >>>> resource
> >>>> [(\"deployment\" =>   \"engine.ear\")]"}}
> >>>> _______________________________________________
> >>>> Engine-devel mailing list
> >>>> Engine-devel at ovirt.org
> >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> 
> 


From jchoate at redhat.com  Tue Jan 17 13:52:34 2012
From: jchoate at redhat.com (Jon Choate)
Date: Tue, 17 Jan 2012 08:52:34 -0500
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F1569B7.9050100@redhat.com>
References: <a00e9a8b-2754-421c-a874-1f340bfdafe9@zmail13.collab.prod.int.phx2.redhat.com>
	<4F1569B7.9050100@redhat.com>
Message-ID: <4F157D22.5040106@redhat.com>

On 01/17/2012 07:29 AM, Livnat Peer wrote:
> On 17/01/12 11:45, Ayal Baron wrote:
>>
>> ----- Original Message -----
>>> On 17/01/12 10:46, Itamar Heim wrote:
>>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Itamar Heim"<iheim at redhat.com>
>>>>>> To: "Jon Choate"<jchoate at redhat.com>
>>>>>> Cc: engine-devel at ovirt.org
>>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
>>>>>> Subject: Re: [Engine-devel] the future of template cloning
>>>>>>
>>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>> We are going to be able to store the disks for a template
>>>>>>>>>>>> on
>>>>>>>>>>>> different storage domains due to the multiple storage
>>>>>>>>>>>> domain
>>>>>>>>>>>> feature. Cloning a template will still be possible, but
>>>>>>>>>>>> will
>>>>>>>>>>>> it
>>>>>>>>>>>> provide any value? Thoughts?
>>>>>>>>>>> I see no relation between the two options.
>>>>>>>>>>>
>>>>>>>>>>> Scenario 1: I can create a VM with a single disk and create
>>>>>>>>>>> a
>>>>>>>>>>> template from it.
>>>>>>>>>>> I would still want to be able to clone the template in order
>>>>>>>>>>> to
>>>>>>>>>>> provision VMs from it on different domains.
>>>>>>>>>>>
>>>>>>>>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>>>>>>>>
>>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>>>>>> domains
>>>>>>>>>>> (domain A and domain B) and I want to have another copy of
>>>>>>>>>>> the
>>>>>>>>>>> template on domain C and domain D
>>>>>>>>>>>
>>>>>>>>>> Hi Jon,
>>>>>>>>>>
>>>>>>>>>> After talking to Michael Pasternak it seems that we did not
>>>>>>>>>> implemented
>>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
>>>>>>>>>> have.
>>>>>>>>>>
>>>>>>>>>> This gap is playing in our favor, we can remove the
>>>>>>>>>> copyTemplate
>>>>>>>>>> verb
>>>>>>>>>> and introduce copyDisk verb.
>>>>>>>>>>
>>>>>>>>>> The template disks can be copied to another SD.
>>>>>>>>>> When creating a VM from template the user can choose per disk
>>>>>>>>>> the
>>>>>>>>>> destination SD (only SD with the disks are eligible
>>>>>>>>>> candidates).
>>>>>>>>> wait, when creating a VM from a template, the user won't get a
>>>>>>>>> choice
>>>>>>>>> will they? Won't the VM disks have to go on the same storage
>>>>>>>>> domain as
>>>>>>>>> the template disks they were created from?
>>>>>>>> yes, but the template disks can be copied to multiple storage
>>>>>>>> domains,
>>>>>>>> so the user can choose for the VM/disk which storage domain to
>>>>>>>> create
>>>>>>>> them from (per storage domains that have copies of that disk)
>>>>>>> OH! I totally misunderstood. So what you are saying is that a
>>>>>>> template
>>>>>>> can have N number of copies of the same disk each on a different
>>>>>>> storage
>>>>>>> domain. I had thought that if you wanted that type of situation
>>>>>>> you
>>>>>>> would have multiple copies of the template itself too.
>>>>> yes, one copy of disk per domain though.
>>>>>
>>>>>>> Just to be clear, does this mean that the plan is to phase out
>>>>>>> the
>>>>>>> current clone template command and instead implementing a clone
>>>>>>> disk
>>>>>>> command so that a template can duplicate its disks individually?
>>>>>> pretty much, yes.
>>>>>> though i'd imagine 'clone template' would still be useful to have
>>>>>> for
>>>>>> the user. not sure if it implies core should expose it as well to
>>>>>> allow
>>>>>> easier usage at UI level for such a task.
>>>>> we can leave it untouched - means copyTemplate get 1 destination
>>>>> domain, and copies all disks to it,
>>>>> but i think it will be unusable (and problematic - what if one of
>>>>> the
>>>>> disks already exists on the destination?),
>>>> then don't copy it, it is already there
>>>>
>>> I agree with Omer, there is no reason to support copy template, if
>>> the
>>> user wants to clone all the disks he can use multiple actions, we
>>> don't
>>> need a specific verb for this.
>> Reason or lack of depends on the common usage.
>> If we assume that normally all disks of a template would reside on the same domain then it makes sense to have a verb to copy the template in its entirety and not burden the user.
>> The general recommendation should be to use a single storage domain, so I think there is room for such a verb.
>>
>
>>> If the UI chooses to expose such operation it will use the
>>> multipleRunAction API which makes it easier to expose to the user
>>> partial success, we could clone disk A and Disk B but Disk C failed
>>> etc.
>> The multipleRunAction is for user marking multiple objects in GUI and running an action on all of these objects.
> Exactly, choosing the disks to copy to the storage domain.
>
>> Here however, the action the user wants is to copy 1 object (the template) which has sub objects and it should run as a single action.
> We are not cloning the template itself we are only cloning the template
> disks.
>
>
>> For example, if there is enough space on the target domain for 2/4 disks then using multipleRunAction would result in 2 disks being copied and 2 failing.
>> If however it is a single action then the free space test would fail the entire action and user would be able to choose if he wants to copy just 2.
>> Note that in this case, actually performing the copy of the 2 disks is detrimental as it can negatively affect VMs on this domain.
>>
>>>
>>>
>>>>> what the user really wants is to specify which disks to copy
>>>>> and destination per disk, and i don't see a reason to create a
>>>>> backend
>>>>> command to do it
>>>>>
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
I would like this discussion to continue but I need some short-term 
closure since I only have two days to get something into code. He is 
what I am proposing to do:

1) leave clone template as it is. It will try to copy all of the disks 
to the same storage domain.
2) Expose a copy disk command so that the user can select a single disk 
and create a copy of it on another storage domain.

Is everyone ok with starting there and then refining once we can reach 
more of a consensus on what the end product should be?


From danken at redhat.com  Tue Jan 17 14:08:24 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Tue, 17 Jan 2012 16:08:24 +0200
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <20120108130813.GC24375@redhat.com>
References: <20120108130813.GC24375@redhat.com>
Message-ID: <20120117140823.GE20867@redhat.com>

On Sun, Jan 08, 2012 at 03:08:13PM +0200, Dan Kenigsberg wrote:
> Hi Lists,
> 
> One cannot create a PV on a partitioned device, and therefor such
> devices where not reported to Engine. This proved surprising to users
> who woder where their LUN disappeared.
> 
> Vdsm should report all devices, and ovirt-engine should mark partitioned
> devices as unworthy of a PV. In the future, Vdsm may allow to forcefully
> remove a partition table from a device, to make it usable as a PV.
> 
> Douglas (CCed) would take resposibilty on the Vdsm side. Initial patch at
> http://gerrit.ovirt.org/944 sports a backword-compatible API. Who's taking this
> on Engine? This involves GUI, too, as partitioned devices should probably be
> displayed greyed-out.

Livnat, is there any taker for this on your side?

Since the change is backward-compatible, I would push it if there are no
objections.

Dan.


From mkolesni at redhat.com  Tue Jan 17 14:55:03 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Tue, 17 Jan 2012 09:55:03 -0500 (EST)
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <39727574-be45-443d-8dfe-f9b7df1ae52b@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <271fd9c9-a2cd-4363-a3ea-8cf86cb647b8@zmail14.collab.prod.int.phx2.redhat.com>


----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Maor" <mlipchuk at redhat.com>
> > To: engine-devel at ovirt.org
> > Sent: Tuesday, January 17, 2012 4:33:19 AM
> > Subject: Re: [Engine-devel] a different approach to the command
> > classes
> > 
> > On 01/17/2012 09:24 AM, Ayal Baron wrote:
> > > 
> > > 
> > > ----- Original Message -----
> > >> On 17/01/12 04:58, Jon Choate wrote:
> > >>> The way the command classes are written has bothered me for a
> > >>> while.
> > >>> While implementing the multiple storage domain features I am
> > >>> presented
> > >>> with the opportunity to create a new command from scratch.  I
> > >>> gave
> > >>> some
> > >>> thought to what I would like the command classes to look like
> > >>> while
> > >>> balancing that the class must still fit in with the existing
> > >>> structure.
> > >>> So here is what I came up with. I'd appreciate any feedback.
> > >>>
> > >>> The Command class encompasses only the rules of what needs to
> > >>> be
> > >>> done.
> > >>> It relies upon Validator classes to determine if the
> > >>> canDoAction
> > >>> conditions have been met.
> > >>>
> > >>>     @Override
> > >>>     public boolean canDoAction() {
> > >>>       ...
> > >>>         checkTargetDomainHasSpace();
> > >>>         checkTargetDomainIsValidTarget();
> > >>>         checkSourceDomainIsValidSource();
> > >>>         checkSourceAndTargetAreDifferent();

I suggest that we can use a POJO - ValidatorResult - to pass back the (obviously) result of validation.
This will contain a status and (default) message for the failure.

Then, can do action check can either interpret it themselves, or use a common method which will do it for them (by adding the message to the messages list, in case the validation had failed).
This will look something like this:
  boolean isValid = validate(SomeValidator.validateSomething(param1, param2, ...));

This will allow the validator class to be unaware of how the command actually handles the message, while at the same time keep the code clean & DRY.

> > >>>      ...
> > >>> }
> > >>>
> > >>> ...
> > >>>
> > >>>   private void checkTargetDomainHasSpace() {
> > >>>         if(!actionAllowed) return;
> > >>>        
> > >>> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
> > >>> {
> > >>>          
> > >>> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
> > >>>           actionAllowed = false;
> > >>>         }
> > >>>     }
> > >>>
> > >>>
> > >>> Each of the checks follows a similar pattern of
> > >>>     - guard clause to see of we already failed and bail if we
> > >>>     did
> > >>>     - test for failure of the condition
> > >>>     - add failure message if needed
> > >>>     - set variable to failed if needed
> > >>>
> > >>> Storing the success flag in a variable allows us to keep the
> > >>> canDoAction
> > >>> method readable as a series of commands and to allow it to be
> > >>> accessed
> > >>> by all the private methods without them having to pass it
> > >>> around.
> > >>>
> > >>> The execution of the command will follow a similar pattern
> > >>> where
> > >>> the
> > >>> command class will only know how to describe what needs to be
> > >>> done
> > >>> and
> > >>> to rely on supporting objects to handle the implementation of
> > >>> these
> > >>> steps.  Getting the implementation out of the command classes
> > >>> will
> > >>> allow
> > >>> the commands to share validation and implementation details and
> > >>> remove a
> > >>> lot of the duplication that currently exists within the
> > >>> commands.
> > >>>
> > >>>
> > >>> How do people feel about this approach?
> > >>
> > >>
> > >> Hi Jon,
> > >>
> > >> The scope of your proposal is changing the implementation of the
> > >> canDoAction method, I think that the title of this mail is a bit
> > >> misleading.
> > > 
> > > Actually I think Jon was suggesting to do the same in the command
> > > itself.
> 
> 
> Yes I am.  I just haven't written that code yet!
> 
> 
> > 
> > I think, using shared canDoAction validation between commands might
> > be a
> > good idea also, it can be seen in operations such as add/update
> > commands.
> > In most cases, the update validation, is already based on the add
> > command validation, sometime it is implemented with
> > inheritance/external
> > class, in other cases it is just implemented with multiple code.
> > > 
> > >>
> > >> Basically what you are suggesting is to split the canDoAction
> > >> implementation into methods and then extract them from the
> > >> command
> > >> class
> > >> to a shared class so they can be reused.
> > >>
> > >> In many cases we can use (are using) inheritance for reusing
> > >> code,
> > >> there
> > >> are cases where inheritance does not do the work and we can
> > >> extract
> > >> to
> > >> external classes.
> > >>
> 
> Our overuse of inheritance is one of the things I'm trying to
> rectify.  Inheritance wasn't added
> to the language to facilitate reuse. It is there to represent object
> hierarchies.  In some cases
> we abuse this to leverage code reuse.  This is often a code smell
> that the code is in the wrong
> place to begin with.  If both classes need the code and they are not
> logically related in an object
> hierarchy, the code really needs to be outside the classes.
> 
> We have cases where the use of inheritance for reuse have gone too
> far. For instance:
> 
> public class RemoveVdsSpmIdCommand<T extends VdsActionParameters>
> extends AddVdsSpmIdCommand<T>
> 
> So this says that a RemoveVdsSmpIdCommand is a type of
> AddVdsSpmIdCommand?  That implies that I can
> use a RemoveVdsSmpIdCommand anywhere that an AddVdsSpmIdCommand is
> expected. Otherwise we have broken
> one of the fundamental contracts of object oriented programming.

I agree that this is a problematic use of inheritance, and we should think of another way to model command classes internally that is not an abuse of OOP.

> 
> > >> I think such a change is welcomed but on a needed basis, I think
> > >> it
> > >> is
> > >> overkill for the general use case and will make the code more
> > >> cumbersome
> > >> (if the original canDoAction was 4-5 lines long...).
> 
> From what I have seen those canDoActions are in the minority.  The
> overall goals are to increase
> the readability and maintainability of the code so I would suggest
> being pragmatic about any approach.
> If doing it doesn't help achieve the goal, then don't do it.
> 
> 
> One of the ideas I'm trying to convey here is that the command
> classes should be fairly ignorant.
> They should be responsible for knowing the list of steps involved in
> a workflow, but not the details
> of how those steps are carried out. Imo knowing both the steps and
> their implementation violates the SRP.
> 
> 
> > > 
> > > Jon, I think the burden of proof is on you here to show a real
> > > example and how it makes the code clearer (e.g. change 2 commands
> > > which share similar checks).
> > > Without 'seeing' it I don't think we would be able to appreciate
> > > the advantage of your approach.
> > > 
> 
> I need to get the multiple storage domains feature put away and then
> I will
> definitely do some refactoring to illustrate the value. I
> wholeheartedly agree
> that having actual code to kick around is better. I just can't do it
> right now!
> 
> 
> > >>
> > >> One thing I don't like in the above suggestion is the way you
> > >> validate
> > >> that the previous condition succeeded/failed. Having this
> > >> condition
> > >> at
> > >> the beginning of each validation method is not a good approach
> > >> IMO.
> 
> Can you elaborate on your opposition to the use of guard clauses?
> They have
> been considered a best practice for quite some time.
> 
> > > 
> > > +1, if the previous check failed it should raise an exception,
> > > not
> > > rely on the next check to bail.
> > > It would be easier (less code, clearer) to wrap all the calls
> > > with
> > > a try catch clause (1 for all the calls), catch the specific
> > > exception that says the check failed and return whatever you want
> > > to return.
> 
> I'm not really a fan of using exceptions for controlling flow of
> execution.  Failing one of these
> checks is not an exceptional event. Its an expected part of the
> workflow and should
> be handled as a normal occurrence.

+1 Exceptions are for unpreditable cases where you really son't have much choice what to do and should probably stop dead in your tacks, while validation is actually just asking "hey, is this thing ok or not?".

> 
> I would argue that wrapping the entire canDoAction method in a
> try/catch would not make the code clearer:
> 
> 
>   public boolean canDoAction() {
>       boolean succeeded = true;
>       try {
>          do1();
>          do2();
>          do3();
>       catch(ValidationException e) {
>          succeeded = false;
>       }
>       return succeeded;
>   }
> 
> 
> has a lot more noise than:
> 
>   private boolean canDoAction = true;
> 
>   public boolean canDoAction() {
>     do1();
>     do2();
>     do3();
>     return canDoAction;
>   }
> 
> In the second form, there is no extra indentation or curly braces.
>  The validation steps
> look just like a list of steps and imo it makes the code really easy
> to understand by a maintainer.
> 


From ofrenkel at redhat.com  Tue Jan 17 15:04:06 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Tue, 17 Jan 2012 10:04:06 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F157D22.5040106@redhat.com>
Message-ID: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Jon Choate" <jchoate at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Tuesday, January 17, 2012 3:52:34 PM
> Subject: Re: [Engine-devel] the future of template cloning
> 
> On 01/17/2012 07:29 AM, Livnat Peer wrote:
> > On 17/01/12 11:45, Ayal Baron wrote:
> >>
> >> ----- Original Message -----
> >>> On 17/01/12 10:46, Itamar Heim wrote:
> >>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "Itamar Heim"<iheim at redhat.com>
> >>>>>> To: "Jon Choate"<jchoate at redhat.com>
> >>>>>> Cc: engine-devel at ovirt.org
> >>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
> >>>>>> Subject: Re: [Engine-devel] the future of template cloning
> >>>>>>
> >>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
> >>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
> >>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
> >>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
> >>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
> >>>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>>> We are going to be able to store the disks for a
> >>>>>>>>>>>> template
> >>>>>>>>>>>> on
> >>>>>>>>>>>> different storage domains due to the multiple storage
> >>>>>>>>>>>> domain
> >>>>>>>>>>>> feature. Cloning a template will still be possible, but
> >>>>>>>>>>>> will
> >>>>>>>>>>>> it
> >>>>>>>>>>>> provide any value? Thoughts?
> >>>>>>>>>>> I see no relation between the two options.
> >>>>>>>>>>>
> >>>>>>>>>>> Scenario 1: I can create a VM with a single disk and
> >>>>>>>>>>> create
> >>>>>>>>>>> a
> >>>>>>>>>>> template from it.
> >>>>>>>>>>> I would still want to be able to clone the template in
> >>>>>>>>>>> order
> >>>>>>>>>>> to
> >>>>>>>>>>> provision VMs from it on different domains.
> >>>>>>>>>>>
> >>>>>>>>>>> Scenario 2: same thing with multiple disks on same
> >>>>>>>>>>> domain.
> >>>>>>>>>>>
> >>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
> >>>>>>>>>>> domains
> >>>>>>>>>>> (domain A and domain B) and I want to have another copy
> >>>>>>>>>>> of
> >>>>>>>>>>> the
> >>>>>>>>>>> template on domain C and domain D
> >>>>>>>>>>>
> >>>>>>>>>> Hi Jon,
> >>>>>>>>>>
> >>>>>>>>>> After talking to Michael Pasternak it seems that we did
> >>>>>>>>>> not
> >>>>>>>>>> implemented
> >>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
> >>>>>>>>>> have.
> >>>>>>>>>>
> >>>>>>>>>> This gap is playing in our favor, we can remove the
> >>>>>>>>>> copyTemplate
> >>>>>>>>>> verb
> >>>>>>>>>> and introduce copyDisk verb.
> >>>>>>>>>>
> >>>>>>>>>> The template disks can be copied to another SD.
> >>>>>>>>>> When creating a VM from template the user can choose per
> >>>>>>>>>> disk
> >>>>>>>>>> the
> >>>>>>>>>> destination SD (only SD with the disks are eligible
> >>>>>>>>>> candidates).
> >>>>>>>>> wait, when creating a VM from a template, the user won't
> >>>>>>>>> get a
> >>>>>>>>> choice
> >>>>>>>>> will they? Won't the VM disks have to go on the same
> >>>>>>>>> storage
> >>>>>>>>> domain as
> >>>>>>>>> the template disks they were created from?
> >>>>>>>> yes, but the template disks can be copied to multiple
> >>>>>>>> storage
> >>>>>>>> domains,
> >>>>>>>> so the user can choose for the VM/disk which storage domain
> >>>>>>>> to
> >>>>>>>> create
> >>>>>>>> them from (per storage domains that have copies of that
> >>>>>>>> disk)
> >>>>>>> OH! I totally misunderstood. So what you are saying is that a
> >>>>>>> template
> >>>>>>> can have N number of copies of the same disk each on a
> >>>>>>> different
> >>>>>>> storage
> >>>>>>> domain. I had thought that if you wanted that type of
> >>>>>>> situation
> >>>>>>> you
> >>>>>>> would have multiple copies of the template itself too.
> >>>>> yes, one copy of disk per domain though.
> >>>>>
> >>>>>>> Just to be clear, does this mean that the plan is to phase
> >>>>>>> out
> >>>>>>> the
> >>>>>>> current clone template command and instead implementing a
> >>>>>>> clone
> >>>>>>> disk
> >>>>>>> command so that a template can duplicate its disks
> >>>>>>> individually?
> >>>>>> pretty much, yes.
> >>>>>> though i'd imagine 'clone template' would still be useful to
> >>>>>> have
> >>>>>> for
> >>>>>> the user. not sure if it implies core should expose it as well
> >>>>>> to
> >>>>>> allow
> >>>>>> easier usage at UI level for such a task.
> >>>>> we can leave it untouched - means copyTemplate get 1
> >>>>> destination
> >>>>> domain, and copies all disks to it,
> >>>>> but i think it will be unusable (and problematic - what if one
> >>>>> of
> >>>>> the
> >>>>> disks already exists on the destination?),
> >>>> then don't copy it, it is already there
> >>>>
> >>> I agree with Omer, there is no reason to support copy template,
> >>> if
> >>> the
> >>> user wants to clone all the disks he can use multiple actions, we
> >>> don't
> >>> need a specific verb for this.
> >> Reason or lack of depends on the common usage.
> >> If we assume that normally all disks of a template would reside on
> >> the same domain then it makes sense to have a verb to copy the
> >> template in its entirety and not burden the user.
> >> The general recommendation should be to use a single storage
> >> domain, so I think there is room for such a verb.
> >>
> >
> >>> If the UI chooses to expose such operation it will use the
> >>> multipleRunAction API which makes it easier to expose to the user
> >>> partial success, we could clone disk A and Disk B but Disk C
> >>> failed
> >>> etc.
> >> The multipleRunAction is for user marking multiple objects in GUI
> >> and running an action on all of these objects.
> > Exactly, choosing the disks to copy to the storage domain.
> >
> >> Here however, the action the user wants is to copy 1 object (the
> >> template) which has sub objects and it should run as a single
> >> action.
> > We are not cloning the template itself we are only cloning the
> > template
> > disks.
> >
> >
> >> For example, if there is enough space on the target domain for 2/4
> >> disks then using multipleRunAction would result in 2 disks being
> >> copied and 2 failing.
> >> If however it is a single action then the free space test would
> >> fail the entire action and user would be able to choose if he
> >> wants to copy just 2.
> >> Note that in this case, actually performing the copy of the 2
> >> disks is detrimental as it can negatively affect VMs on this
> >> domain.
> >>
> >>>
> >>>
> >>>>> what the user really wants is to specify which disks to copy
> >>>>> and destination per disk, and i don't see a reason to create a
> >>>>> backend
> >>>>> command to do it
> >>>>>
> >>>>>> _______________________________________________
> >>>>>> Engine-devel mailing list
> >>>>>> Engine-devel at ovirt.org
> >>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>>>
> >>>> _______________________________________________
> >>>> Engine-devel mailing list
> >>>> Engine-devel at ovirt.org
> >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> I would like this discussion to continue but I need some short-term
> closure since I only have two days to get something into code. He is
> what I am proposing to do:
> 
> 1) leave clone template as it is. It will try to copy all of the
> disks
> to the same storage domain.
> 2) Expose a copy disk command so that the user can select a single
> disk
> and create a copy of it on another storage domain.
> 
> Is everyone ok with starting there and then refining once we can
> reach
> more of a consensus on what the end product should be?

fine by me, although i think we should remove the clone template (actually called copy template),
as i think the copy disk gives a way to do it, with more granularity, 
and the user will know exactly what to expect.

unfortunately, there is another issue,
currently, since template can be fully on a domain or not,
remove template can get list of storage domains to remove the template from,
and if the list contains all the domain the template is on, the template is fully deleted from the db,
if its partial, then only the copies of the disks are removed from the domain.

what i suggest is, making it simple:
remove template will remove the template, and all of the disks copies from all the domains.
we will need new removeTemplateDisk command that will remove a disk from a domain, 
as long there is another copy of the disk. 
if its the last copy it should fail, as removing the disk from all the domains will change the template,
which is not a supported flow.

thoughts?

> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkolesni at redhat.com  Tue Jan 17 15:37:12 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Tue, 17 Jan 2012 10:37:12 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <a1537d18-9a6a-42ce-aa36-fa0e3651cc47@zmail14.collab.prod.int.phx2.redhat.com>


----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Jon Choate" <jchoate at redhat.com>
> > To: engine-devel at ovirt.org
> > Sent: Tuesday, January 17, 2012 3:52:34 PM
> > Subject: Re: [Engine-devel] the future of template cloning
> > 
> > On 01/17/2012 07:29 AM, Livnat Peer wrote:
> > > On 17/01/12 11:45, Ayal Baron wrote:
> > >>
> > >> ----- Original Message -----
> > >>> On 17/01/12 10:46, Itamar Heim wrote:
> > >>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
> > >>>>>
> > >>>>> ----- Original Message -----
> > >>>>>> From: "Itamar Heim"<iheim at redhat.com>
> > >>>>>> To: "Jon Choate"<jchoate at redhat.com>
> > >>>>>> Cc: engine-devel at ovirt.org
> > >>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
> > >>>>>> Subject: Re: [Engine-devel] the future of template cloning
> > >>>>>>
> > >>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
> > >>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
> > >>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
> > >>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
> > >>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
> > >>>>>>>>>>> ----- Original Message -----
> > >>>>>>>>>>>> We are going to be able to store the disks for a
> > >>>>>>>>>>>> template
> > >>>>>>>>>>>> on
> > >>>>>>>>>>>> different storage domains due to the multiple storage
> > >>>>>>>>>>>> domain
> > >>>>>>>>>>>> feature. Cloning a template will still be possible,
> > >>>>>>>>>>>> but
> > >>>>>>>>>>>> will
> > >>>>>>>>>>>> it
> > >>>>>>>>>>>> provide any value? Thoughts?
> > >>>>>>>>>>> I see no relation between the two options.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Scenario 1: I can create a VM with a single disk and
> > >>>>>>>>>>> create
> > >>>>>>>>>>> a
> > >>>>>>>>>>> template from it.
> > >>>>>>>>>>> I would still want to be able to clone the template in
> > >>>>>>>>>>> order
> > >>>>>>>>>>> to
> > >>>>>>>>>>> provision VMs from it on different domains.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Scenario 2: same thing with multiple disks on same
> > >>>>>>>>>>> domain.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2
> > >>>>>>>>>>> different
> > >>>>>>>>>>> domains
> > >>>>>>>>>>> (domain A and domain B) and I want to have another copy
> > >>>>>>>>>>> of
> > >>>>>>>>>>> the
> > >>>>>>>>>>> template on domain C and domain D
> > >>>>>>>>>>>
> > >>>>>>>>>> Hi Jon,
> > >>>>>>>>>>
> > >>>>>>>>>> After talking to Michael Pasternak it seems that we did
> > >>>>>>>>>> not
> > >>>>>>>>>> implemented
> > >>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that
> > >>>>>>>>>> we
> > >>>>>>>>>> have.
> > >>>>>>>>>>
> > >>>>>>>>>> This gap is playing in our favor, we can remove the
> > >>>>>>>>>> copyTemplate
> > >>>>>>>>>> verb
> > >>>>>>>>>> and introduce copyDisk verb.
> > >>>>>>>>>>
> > >>>>>>>>>> The template disks can be copied to another SD.
> > >>>>>>>>>> When creating a VM from template the user can choose per
> > >>>>>>>>>> disk
> > >>>>>>>>>> the
> > >>>>>>>>>> destination SD (only SD with the disks are eligible
> > >>>>>>>>>> candidates).
> > >>>>>>>>> wait, when creating a VM from a template, the user won't
> > >>>>>>>>> get a
> > >>>>>>>>> choice
> > >>>>>>>>> will they? Won't the VM disks have to go on the same
> > >>>>>>>>> storage
> > >>>>>>>>> domain as
> > >>>>>>>>> the template disks they were created from?
> > >>>>>>>> yes, but the template disks can be copied to multiple
> > >>>>>>>> storage
> > >>>>>>>> domains,
> > >>>>>>>> so the user can choose for the VM/disk which storage
> > >>>>>>>> domain
> > >>>>>>>> to
> > >>>>>>>> create
> > >>>>>>>> them from (per storage domains that have copies of that
> > >>>>>>>> disk)
> > >>>>>>> OH! I totally misunderstood. So what you are saying is that
> > >>>>>>> a
> > >>>>>>> template
> > >>>>>>> can have N number of copies of the same disk each on a
> > >>>>>>> different
> > >>>>>>> storage
> > >>>>>>> domain. I had thought that if you wanted that type of
> > >>>>>>> situation
> > >>>>>>> you
> > >>>>>>> would have multiple copies of the template itself too.
> > >>>>> yes, one copy of disk per domain though.
> > >>>>>
> > >>>>>>> Just to be clear, does this mean that the plan is to phase
> > >>>>>>> out
> > >>>>>>> the
> > >>>>>>> current clone template command and instead implementing a
> > >>>>>>> clone
> > >>>>>>> disk
> > >>>>>>> command so that a template can duplicate its disks
> > >>>>>>> individually?
> > >>>>>> pretty much, yes.
> > >>>>>> though i'd imagine 'clone template' would still be useful to
> > >>>>>> have
> > >>>>>> for
> > >>>>>> the user. not sure if it implies core should expose it as
> > >>>>>> well
> > >>>>>> to
> > >>>>>> allow
> > >>>>>> easier usage at UI level for such a task.
> > >>>>> we can leave it untouched - means copyTemplate get 1
> > >>>>> destination
> > >>>>> domain, and copies all disks to it,
> > >>>>> but i think it will be unusable (and problematic - what if
> > >>>>> one
> > >>>>> of
> > >>>>> the
> > >>>>> disks already exists on the destination?),
> > >>>> then don't copy it, it is already there
> > >>>>
> > >>> I agree with Omer, there is no reason to support copy template,
> > >>> if
> > >>> the
> > >>> user wants to clone all the disks he can use multiple actions,
> > >>> we
> > >>> don't
> > >>> need a specific verb for this.
> > >> Reason or lack of depends on the common usage.
> > >> If we assume that normally all disks of a template would reside
> > >> on
> > >> the same domain then it makes sense to have a verb to copy the
> > >> template in its entirety and not burden the user.
> > >> The general recommendation should be to use a single storage
> > >> domain, so I think there is room for such a verb.
> > >>
> > >
> > >>> If the UI chooses to expose such operation it will use the
> > >>> multipleRunAction API which makes it easier to expose to the
> > >>> user
> > >>> partial success, we could clone disk A and Disk B but Disk C
> > >>> failed
> > >>> etc.
> > >> The multipleRunAction is for user marking multiple objects in
> > >> GUI
> > >> and running an action on all of these objects.
> > > Exactly, choosing the disks to copy to the storage domain.
> > >
> > >> Here however, the action the user wants is to copy 1 object (the
> > >> template) which has sub objects and it should run as a single
> > >> action.
> > > We are not cloning the template itself we are only cloning the
> > > template
> > > disks.
> > >
> > >
> > >> For example, if there is enough space on the target domain for
> > >> 2/4
> > >> disks then using multipleRunAction would result in 2 disks being
> > >> copied and 2 failing.
> > >> If however it is a single action then the free space test would
> > >> fail the entire action and user would be able to choose if he
> > >> wants to copy just 2.
> > >> Note that in this case, actually performing the copy of the 2
> > >> disks is detrimental as it can negatively affect VMs on this
> > >> domain.
> > >>
> > >>>
> > >>>
> > >>>>> what the user really wants is to specify which disks to copy
> > >>>>> and destination per disk, and i don't see a reason to create
> > >>>>> a
> > >>>>> backend
> > >>>>> command to do it
> > >>>>>
> > >>>>>> _______________________________________________
> > >>>>>> Engine-devel mailing list
> > >>>>>> Engine-devel at ovirt.org
> > >>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>>>>>
> > >>>> _______________________________________________
> > >>>> Engine-devel mailing list
> > >>>> Engine-devel at ovirt.org
> > >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>> _______________________________________________
> > >>> Engine-devel mailing list
> > >>> Engine-devel at ovirt.org
> > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>>
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > I would like this discussion to continue but I need some short-term
> > closure since I only have two days to get something into code. He
> > is
> > what I am proposing to do:
> > 
> > 1) leave clone template as it is. It will try to copy all of the
> > disks
> > to the same storage domain.
> > 2) Expose a copy disk command so that the user can select a single
> > disk
> > and create a copy of it on another storage domain.
> > 
> > Is everyone ok with starting there and then refining once we can
> > reach
> > more of a consensus on what the end product should be?
> 
> fine by me, although i think we should remove the clone template
> (actually called copy template),
> as i think the copy disk gives a way to do it, with more granularity,
> and the user will know exactly what to expect.
> 
> unfortunately, there is another issue,
> currently, since template can be fully on a domain or not,
> remove template can get list of storage domains to remove the
> template from,
> and if the list contains all the domain the template is on, the
> template is fully deleted from the db,
> if its partial, then only the copies of the disks are removed from
> the domain.
> 
> what i suggest is, making it simple:
> remove template will remove the template, and all of the disks copies
> from all the domains.
> we will need new removeTemplateDisk command that will remove a disk
> from a domain,
> as long there is another copy of the disk.
> if its the last copy it should fail, as removing the disk from all
> the domains will change the template,
> which is not a supported flow.
> 
> thoughts?

This of course needs to be supported in clients, but how exactly will the user keep track of where each disk of the template resides?

> 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From lpeer at redhat.com  Tue Jan 17 15:49:12 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 17 Jan 2012 17:49:12 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>
References: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F159878.7000501@redhat.com>

On 17/01/12 17:04, Omer Frenkel wrote:
> 
> 
> ----- Original Message -----
>> From: "Jon Choate" <jchoate at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Tuesday, January 17, 2012 3:52:34 PM
>> Subject: Re: [Engine-devel] the future of template cloning
>>
>> On 01/17/2012 07:29 AM, Livnat Peer wrote:
>>> On 17/01/12 11:45, Ayal Baron wrote:
>>>>
>>>> ----- Original Message -----
>>>>> On 17/01/12 10:46, Itamar Heim wrote:
>>>>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Itamar Heim"<iheim at redhat.com>
>>>>>>>> To: "Jon Choate"<jchoate at redhat.com>
>>>>>>>> Cc: engine-devel at ovirt.org
>>>>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
>>>>>>>> Subject: Re: [Engine-devel] the future of template cloning
>>>>>>>>
>>>>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>>>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>> We are going to be able to store the disks for a
>>>>>>>>>>>>>> template
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> different storage domains due to the multiple storage
>>>>>>>>>>>>>> domain
>>>>>>>>>>>>>> feature. Cloning a template will still be possible, but
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>> provide any value? Thoughts?
>>>>>>>>>>>>> I see no relation between the two options.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Scenario 1: I can create a VM with a single disk and
>>>>>>>>>>>>> create
>>>>>>>>>>>>> a
>>>>>>>>>>>>> template from it.
>>>>>>>>>>>>> I would still want to be able to clone the template in
>>>>>>>>>>>>> order
>>>>>>>>>>>>> to
>>>>>>>>>>>>> provision VMs from it on different domains.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Scenario 2: same thing with multiple disks on same
>>>>>>>>>>>>> domain.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>>>>>>>> domains
>>>>>>>>>>>>> (domain A and domain B) and I want to have another copy
>>>>>>>>>>>>> of
>>>>>>>>>>>>> the
>>>>>>>>>>>>> template on domain C and domain D
>>>>>>>>>>>>>
>>>>>>>>>>>> Hi Jon,
>>>>>>>>>>>>
>>>>>>>>>>>> After talking to Michael Pasternak it seems that we did
>>>>>>>>>>>> not
>>>>>>>>>>>> implemented
>>>>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
>>>>>>>>>>>> have.
>>>>>>>>>>>>
>>>>>>>>>>>> This gap is playing in our favor, we can remove the
>>>>>>>>>>>> copyTemplate
>>>>>>>>>>>> verb
>>>>>>>>>>>> and introduce copyDisk verb.
>>>>>>>>>>>>
>>>>>>>>>>>> The template disks can be copied to another SD.
>>>>>>>>>>>> When creating a VM from template the user can choose per
>>>>>>>>>>>> disk
>>>>>>>>>>>> the
>>>>>>>>>>>> destination SD (only SD with the disks are eligible
>>>>>>>>>>>> candidates).
>>>>>>>>>>> wait, when creating a VM from a template, the user won't
>>>>>>>>>>> get a
>>>>>>>>>>> choice
>>>>>>>>>>> will they? Won't the VM disks have to go on the same
>>>>>>>>>>> storage
>>>>>>>>>>> domain as
>>>>>>>>>>> the template disks they were created from?
>>>>>>>>>> yes, but the template disks can be copied to multiple
>>>>>>>>>> storage
>>>>>>>>>> domains,
>>>>>>>>>> so the user can choose for the VM/disk which storage domain
>>>>>>>>>> to
>>>>>>>>>> create
>>>>>>>>>> them from (per storage domains that have copies of that
>>>>>>>>>> disk)
>>>>>>>>> OH! I totally misunderstood. So what you are saying is that a
>>>>>>>>> template
>>>>>>>>> can have N number of copies of the same disk each on a
>>>>>>>>> different
>>>>>>>>> storage
>>>>>>>>> domain. I had thought that if you wanted that type of
>>>>>>>>> situation
>>>>>>>>> you
>>>>>>>>> would have multiple copies of the template itself too.
>>>>>>> yes, one copy of disk per domain though.
>>>>>>>
>>>>>>>>> Just to be clear, does this mean that the plan is to phase
>>>>>>>>> out
>>>>>>>>> the
>>>>>>>>> current clone template command and instead implementing a
>>>>>>>>> clone
>>>>>>>>> disk
>>>>>>>>> command so that a template can duplicate its disks
>>>>>>>>> individually?
>>>>>>>> pretty much, yes.
>>>>>>>> though i'd imagine 'clone template' would still be useful to
>>>>>>>> have
>>>>>>>> for
>>>>>>>> the user. not sure if it implies core should expose it as well
>>>>>>>> to
>>>>>>>> allow
>>>>>>>> easier usage at UI level for such a task.
>>>>>>> we can leave it untouched - means copyTemplate get 1
>>>>>>> destination
>>>>>>> domain, and copies all disks to it,
>>>>>>> but i think it will be unusable (and problematic - what if one
>>>>>>> of
>>>>>>> the
>>>>>>> disks already exists on the destination?),
>>>>>> then don't copy it, it is already there
>>>>>>
>>>>> I agree with Omer, there is no reason to support copy template,
>>>>> if
>>>>> the
>>>>> user wants to clone all the disks he can use multiple actions, we
>>>>> don't
>>>>> need a specific verb for this.
>>>> Reason or lack of depends on the common usage.
>>>> If we assume that normally all disks of a template would reside on
>>>> the same domain then it makes sense to have a verb to copy the
>>>> template in its entirety and not burden the user.
>>>> The general recommendation should be to use a single storage
>>>> domain, so I think there is room for such a verb.
>>>>
>>>
>>>>> If the UI chooses to expose such operation it will use the
>>>>> multipleRunAction API which makes it easier to expose to the user
>>>>> partial success, we could clone disk A and Disk B but Disk C
>>>>> failed
>>>>> etc.
>>>> The multipleRunAction is for user marking multiple objects in GUI
>>>> and running an action on all of these objects.
>>> Exactly, choosing the disks to copy to the storage domain.
>>>
>>>> Here however, the action the user wants is to copy 1 object (the
>>>> template) which has sub objects and it should run as a single
>>>> action.
>>> We are not cloning the template itself we are only cloning the
>>> template
>>> disks.
>>>
>>>
>>>> For example, if there is enough space on the target domain for 2/4
>>>> disks then using multipleRunAction would result in 2 disks being
>>>> copied and 2 failing.
>>>> If however it is a single action then the free space test would
>>>> fail the entire action and user would be able to choose if he
>>>> wants to copy just 2.
>>>> Note that in this case, actually performing the copy of the 2
>>>> disks is detrimental as it can negatively affect VMs on this
>>>> domain.
>>>>
>>>>>
>>>>>
>>>>>>> what the user really wants is to specify which disks to copy
>>>>>>> and destination per disk, and i don't see a reason to create a
>>>>>>> backend
>>>>>>> command to do it
>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Engine-devel mailing list
>>>>>>>> Engine-devel at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>> I would like this discussion to continue but I need some short-term
>> closure since I only have two days to get something into code. He is
>> what I am proposing to do:
>>
>> 1) leave clone template as it is. It will try to copy all of the
>> disks
>> to the same storage domain.
>> 2) Expose a copy disk command so that the user can select a single
>> disk
>> and create a copy of it on another storage domain.
>>
>> Is everyone ok with starting there and then refining once we can
>> reach
>> more of a consensus on what the end product should be?
> 
> fine by me, although i think we should remove the clone template (actually called copy template),
> as i think the copy disk gives a way to do it, with more granularity, 
> and the user will know exactly what to expect.
> 


We are removing the cloneTemplate and introducing cloneTemplatedisk,
following the above discussion does anyone have strong objection for
doing this?


> unfortunately, there is another issue,
> currently, since template can be fully on a domain or not,
> remove template can get list of storage domains to remove the template from,
> and if the list contains all the domain the template is on, the template is fully deleted from the db,
> if its partial, then only the copies of the disks are removed from the domain.
> 

Worth Noting that we have another REST API gap that we can use in our
favor, the delete template from storage domain is not modeled yet and we
don't need to support it.

> what i suggest is, making it simple:
> remove template will remove the template, and all of the disks copies from all the domains.

+1 for that approach.

> we will need new removeTemplateDisk command that will remove a disk from a domain, 
> as long there is another copy of the disk. 
> if its the last copy it should fail, as removing the disk from all the domains will change the template,
> which is not a supported flow.
> 
> thoughts?

I would implement a single verb - removeDisk, it has optional parameter
storage domain.
If the storage domain is not specified it is simply removing a disk (for
template, VM or floating) if a SD is specified in case there is only one
image that represent this disk (the only use case for VMs) we remove the
disk (same as no passing no SD except for extra validation) if more than
one image is associated with this disk (the image-SD map has more than
one entry) then remove the relevant mapping.

Note: in case of deleting the disk and there is only one image
associated with the disk we should remove the device from the vm_devices
table as well.

Implementing a single generic verb makes it easier to keep backwards
compatibility when the model changes.

Livnat



> 
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From jchoate at redhat.com  Tue Jan 17 16:04:42 2012
From: jchoate at redhat.com (Jon Choate)
Date: Tue, 17 Jan 2012 11:04:42 -0500
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F159878.7000501@redhat.com>
References: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>
	<4F159878.7000501@redhat.com>
Message-ID: <4F159C1A.20209@redhat.com>

On 01/17/2012 10:49 AM, Livnat Peer wrote:
> On 17/01/12 17:04, Omer Frenkel wrote:
>>
>> ----- Original Message -----
>>> From: "Jon Choate"<jchoate at redhat.com>
>>> To: engine-devel at ovirt.org
>>> Sent: Tuesday, January 17, 2012 3:52:34 PM
>>> Subject: Re: [Engine-devel] the future of template cloning
>>>
>>> On 01/17/2012 07:29 AM, Livnat Peer wrote:
>>>> On 17/01/12 11:45, Ayal Baron wrote:
>>>>> ----- Original Message -----
>>>>>> On 17/01/12 10:46, Itamar Heim wrote:
>>>>>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>>>>>>>> ----- Original Message -----
>>>>>>>>> From: "Itamar Heim"<iheim at redhat.com>
>>>>>>>>> To: "Jon Choate"<jchoate at redhat.com>
>>>>>>>>> Cc: engine-devel at ovirt.org
>>>>>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
>>>>>>>>> Subject: Re: [Engine-devel] the future of template cloning
>>>>>>>>>
>>>>>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>>>>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>>>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>> We are going to be able to store the disks for a
>>>>>>>>>>>>>>> template
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> different storage domains due to the multiple storage
>>>>>>>>>>>>>>> domain
>>>>>>>>>>>>>>> feature. Cloning a template will still be possible, but
>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> provide any value? Thoughts?
>>>>>>>>>>>>>> I see no relation between the two options.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Scenario 1: I can create a VM with a single disk and
>>>>>>>>>>>>>> create
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> template from it.
>>>>>>>>>>>>>> I would still want to be able to clone the template in
>>>>>>>>>>>>>> order
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> provision VMs from it on different domains.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Scenario 2: same thing with multiple disks on same
>>>>>>>>>>>>>> domain.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>>>>>>>>> domains
>>>>>>>>>>>>>> (domain A and domain B) and I want to have another copy
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> template on domain C and domain D
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Jon,
>>>>>>>>>>>>>
>>>>>>>>>>>>> After talking to Michael Pasternak it seems that we did
>>>>>>>>>>>>> not
>>>>>>>>>>>>> implemented
>>>>>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
>>>>>>>>>>>>> have.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This gap is playing in our favor, we can remove the
>>>>>>>>>>>>> copyTemplate
>>>>>>>>>>>>> verb
>>>>>>>>>>>>> and introduce copyDisk verb.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The template disks can be copied to another SD.
>>>>>>>>>>>>> When creating a VM from template the user can choose per
>>>>>>>>>>>>> disk
>>>>>>>>>>>>> the
>>>>>>>>>>>>> destination SD (only SD with the disks are eligible
>>>>>>>>>>>>> candidates).
>>>>>>>>>>>> wait, when creating a VM from a template, the user won't
>>>>>>>>>>>> get a
>>>>>>>>>>>> choice
>>>>>>>>>>>> will they? Won't the VM disks have to go on the same
>>>>>>>>>>>> storage
>>>>>>>>>>>> domain as
>>>>>>>>>>>> the template disks they were created from?
>>>>>>>>>>> yes, but the template disks can be copied to multiple
>>>>>>>>>>> storage
>>>>>>>>>>> domains,
>>>>>>>>>>> so the user can choose for the VM/disk which storage domain
>>>>>>>>>>> to
>>>>>>>>>>> create
>>>>>>>>>>> them from (per storage domains that have copies of that
>>>>>>>>>>> disk)
>>>>>>>>>> OH! I totally misunderstood. So what you are saying is that a
>>>>>>>>>> template
>>>>>>>>>> can have N number of copies of the same disk each on a
>>>>>>>>>> different
>>>>>>>>>> storage
>>>>>>>>>> domain. I had thought that if you wanted that type of
>>>>>>>>>> situation
>>>>>>>>>> you
>>>>>>>>>> would have multiple copies of the template itself too.
>>>>>>>> yes, one copy of disk per domain though.
>>>>>>>>
>>>>>>>>>> Just to be clear, does this mean that the plan is to phase
>>>>>>>>>> out
>>>>>>>>>> the
>>>>>>>>>> current clone template command and instead implementing a
>>>>>>>>>> clone
>>>>>>>>>> disk
>>>>>>>>>> command so that a template can duplicate its disks
>>>>>>>>>> individually?
>>>>>>>>> pretty much, yes.
>>>>>>>>> though i'd imagine 'clone template' would still be useful to
>>>>>>>>> have
>>>>>>>>> for
>>>>>>>>> the user. not sure if it implies core should expose it as well
>>>>>>>>> to
>>>>>>>>> allow
>>>>>>>>> easier usage at UI level for such a task.
>>>>>>>> we can leave it untouched - means copyTemplate get 1
>>>>>>>> destination
>>>>>>>> domain, and copies all disks to it,
>>>>>>>> but i think it will be unusable (and problematic - what if one
>>>>>>>> of
>>>>>>>> the
>>>>>>>> disks already exists on the destination?),
>>>>>>> then don't copy it, it is already there
>>>>>>>
>>>>>> I agree with Omer, there is no reason to support copy template,
>>>>>> if
>>>>>> the
>>>>>> user wants to clone all the disks he can use multiple actions, we
>>>>>> don't
>>>>>> need a specific verb for this.
>>>>> Reason or lack of depends on the common usage.
>>>>> If we assume that normally all disks of a template would reside on
>>>>> the same domain then it makes sense to have a verb to copy the
>>>>> template in its entirety and not burden the user.
>>>>> The general recommendation should be to use a single storage
>>>>> domain, so I think there is room for such a verb.
>>>>>
>>>>>> If the UI chooses to expose such operation it will use the
>>>>>> multipleRunAction API which makes it easier to expose to the user
>>>>>> partial success, we could clone disk A and Disk B but Disk C
>>>>>> failed
>>>>>> etc.
>>>>> The multipleRunAction is for user marking multiple objects in GUI
>>>>> and running an action on all of these objects.
>>>> Exactly, choosing the disks to copy to the storage domain.
>>>>
>>>>> Here however, the action the user wants is to copy 1 object (the
>>>>> template) which has sub objects and it should run as a single
>>>>> action.
>>>> We are not cloning the template itself we are only cloning the
>>>> template
>>>> disks.
>>>>
>>>>
>>>>> For example, if there is enough space on the target domain for 2/4
>>>>> disks then using multipleRunAction would result in 2 disks being
>>>>> copied and 2 failing.
>>>>> If however it is a single action then the free space test would
>>>>> fail the entire action and user would be able to choose if he
>>>>> wants to copy just 2.
>>>>> Note that in this case, actually performing the copy of the 2
>>>>> disks is detrimental as it can negatively affect VMs on this
>>>>> domain.
>>>>>
>>>>>>
>>>>>>>> what the user really wants is to specify which disks to copy
>>>>>>>> and destination per disk, and i don't see a reason to create a
>>>>>>>> backend
>>>>>>>> command to do it
>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Engine-devel mailing list
>>>>>>>>> Engine-devel at ovirt.org
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Engine-devel mailing list
>>>>>>> Engine-devel at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>> I would like this discussion to continue but I need some short-term
>>> closure since I only have two days to get something into code. He is
>>> what I am proposing to do:
>>>
>>> 1) leave clone template as it is. It will try to copy all of the
>>> disks
>>> to the same storage domain.
>>> 2) Expose a copy disk command so that the user can select a single
>>> disk
>>> and create a copy of it on another storage domain.
>>>
>>> Is everyone ok with starting there and then refining once we can
>>> reach
>>> more of a consensus on what the end product should be?
>> fine by me, although i think we should remove the clone template (actually called copy template),
>> as i think the copy disk gives a way to do it, with more granularity,
>> and the user will know exactly what to expect.
>>
>
> We are removing the cloneTemplate and introducing cloneTemplatedisk,
> following the above discussion does anyone have strong objection for
> doing this?
>

We can't remove cloneTemplate until it is removed from the UI or else we 
will break functionality.  For now we are just going to ensure that it 
works as it always has until the UI is ready to remove it.

>> unfortunately, there is another issue,
>> currently, since template can be fully on a domain or not,
>> remove template can get list of storage domains to remove the template from,
>> and if the list contains all the domain the template is on, the template is fully deleted from the db,
>> if its partial, then only the copies of the disks are removed from the domain.
>>
> Worth Noting that we have another REST API gap that we can use in our
> favor, the delete template from storage domain is not modeled yet and we
> don't need to support it.
>
>> what i suggest is, making it simple:
>> remove template will remove the template, and all of the disks copies from all the domains.
> +1 for that approach.
>
>> we will need new removeTemplateDisk command that will remove a disk from a domain,
>> as long there is another copy of the disk.
>> if its the last copy it should fail, as removing the disk from all the domains will change the template,
>> which is not a supported flow.
>>
>> thoughts?
> I would implement a single verb - removeDisk, it has optional parameter
> storage domain.
> If the storage domain is not specified it is simply removing a disk (for
> template, VM or floating) if a SD is specified in case there is only one
> image that represent this disk (the only use case for VMs) we remove the
> disk (same as no passing no SD except for extra validation) if more than
> one image is associated with this disk (the image-SD map has more than
> one entry) then remove the relevant mapping.
>
> Note: in case of deleting the disk and there is only one image
> associated with the disk we should remove the device from the vm_devices
> table as well.
>
> Implementing a single generic verb makes it easier to keep backwards
> compatibility when the model changes.
>
> Livnat
>
>
>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel



From lpeer at redhat.com  Tue Jan 17 16:35:14 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 17 Jan 2012 18:35:14 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F159C1A.20209@redhat.com>
References: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>
	<4F159878.7000501@redhat.com> <4F159C1A.20209@redhat.com>
Message-ID: <4F15A342.20802@redhat.com>

On 17/01/12 18:04, Jon Choate wrote:
> On 01/17/2012 10:49 AM, Livnat Peer wrote:
>> On 17/01/12 17:04, Omer Frenkel wrote:
>>>
>>> ----- Original Message -----
>>>> From: "Jon Choate"<jchoate at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Sent: Tuesday, January 17, 2012 3:52:34 PM
>>>> Subject: Re: [Engine-devel] the future of template cloning
>>>>
>>>> On 01/17/2012 07:29 AM, Livnat Peer wrote:
>>>>> On 17/01/12 11:45, Ayal Baron wrote:
>>>>>> ----- Original Message -----
>>>>>>> On 17/01/12 10:46, Itamar Heim wrote:
>>>>>>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>>>>>>>>> ----- Original Message -----
>>>>>>>>>> From: "Itamar Heim"<iheim at redhat.com>
>>>>>>>>>> To: "Jon Choate"<jchoate at redhat.com>
>>>>>>>>>> Cc: engine-devel at ovirt.org
>>>>>>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
>>>>>>>>>> Subject: Re: [Engine-devel] the future of template cloning
>>>>>>>>>>
>>>>>>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>>>>>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>>>>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>>>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>>>>> We are going to be able to store the disks for a
>>>>>>>>>>>>>>>> template
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> different storage domains due to the multiple storage
>>>>>>>>>>>>>>>> domain
>>>>>>>>>>>>>>>> feature. Cloning a template will still be possible, but
>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> provide any value? Thoughts?
>>>>>>>>>>>>>>> I see no relation between the two options.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Scenario 1: I can create a VM with a single disk and
>>>>>>>>>>>>>>> create
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> template from it.
>>>>>>>>>>>>>>> I would still want to be able to clone the template in
>>>>>>>>>>>>>>> order
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> provision VMs from it on different domains.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Scenario 2: same thing with multiple disks on same
>>>>>>>>>>>>>>> domain.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>>>>>>>>>> domains
>>>>>>>>>>>>>>> (domain A and domain B) and I want to have another copy
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> template on domain C and domain D
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Jon,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> After talking to Michael Pasternak it seems that we did
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>> implemented
>>>>>>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
>>>>>>>>>>>>>> have.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This gap is playing in our favor, we can remove the
>>>>>>>>>>>>>> copyTemplate
>>>>>>>>>>>>>> verb
>>>>>>>>>>>>>> and introduce copyDisk verb.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The template disks can be copied to another SD.
>>>>>>>>>>>>>> When creating a VM from template the user can choose per
>>>>>>>>>>>>>> disk
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> destination SD (only SD with the disks are eligible
>>>>>>>>>>>>>> candidates).
>>>>>>>>>>>>> wait, when creating a VM from a template, the user won't
>>>>>>>>>>>>> get a
>>>>>>>>>>>>> choice
>>>>>>>>>>>>> will they? Won't the VM disks have to go on the same
>>>>>>>>>>>>> storage
>>>>>>>>>>>>> domain as
>>>>>>>>>>>>> the template disks they were created from?
>>>>>>>>>>>> yes, but the template disks can be copied to multiple
>>>>>>>>>>>> storage
>>>>>>>>>>>> domains,
>>>>>>>>>>>> so the user can choose for the VM/disk which storage domain
>>>>>>>>>>>> to
>>>>>>>>>>>> create
>>>>>>>>>>>> them from (per storage domains that have copies of that
>>>>>>>>>>>> disk)
>>>>>>>>>>> OH! I totally misunderstood. So what you are saying is that a
>>>>>>>>>>> template
>>>>>>>>>>> can have N number of copies of the same disk each on a
>>>>>>>>>>> different
>>>>>>>>>>> storage
>>>>>>>>>>> domain. I had thought that if you wanted that type of
>>>>>>>>>>> situation
>>>>>>>>>>> you
>>>>>>>>>>> would have multiple copies of the template itself too.
>>>>>>>>> yes, one copy of disk per domain though.
>>>>>>>>>
>>>>>>>>>>> Just to be clear, does this mean that the plan is to phase
>>>>>>>>>>> out
>>>>>>>>>>> the
>>>>>>>>>>> current clone template command and instead implementing a
>>>>>>>>>>> clone
>>>>>>>>>>> disk
>>>>>>>>>>> command so that a template can duplicate its disks
>>>>>>>>>>> individually?
>>>>>>>>>> pretty much, yes.
>>>>>>>>>> though i'd imagine 'clone template' would still be useful to
>>>>>>>>>> have
>>>>>>>>>> for
>>>>>>>>>> the user. not sure if it implies core should expose it as well
>>>>>>>>>> to
>>>>>>>>>> allow
>>>>>>>>>> easier usage at UI level for such a task.
>>>>>>>>> we can leave it untouched - means copyTemplate get 1
>>>>>>>>> destination
>>>>>>>>> domain, and copies all disks to it,
>>>>>>>>> but i think it will be unusable (and problematic - what if one
>>>>>>>>> of
>>>>>>>>> the
>>>>>>>>> disks already exists on the destination?),
>>>>>>>> then don't copy it, it is already there
>>>>>>>>
>>>>>>> I agree with Omer, there is no reason to support copy template,
>>>>>>> if
>>>>>>> the
>>>>>>> user wants to clone all the disks he can use multiple actions, we
>>>>>>> don't
>>>>>>> need a specific verb for this.
>>>>>> Reason or lack of depends on the common usage.
>>>>>> If we assume that normally all disks of a template would reside on
>>>>>> the same domain then it makes sense to have a verb to copy the
>>>>>> template in its entirety and not burden the user.
>>>>>> The general recommendation should be to use a single storage
>>>>>> domain, so I think there is room for such a verb.
>>>>>>
>>>>>>> If the UI chooses to expose such operation it will use the
>>>>>>> multipleRunAction API which makes it easier to expose to the user
>>>>>>> partial success, we could clone disk A and Disk B but Disk C
>>>>>>> failed
>>>>>>> etc.
>>>>>> The multipleRunAction is for user marking multiple objects in GUI
>>>>>> and running an action on all of these objects.
>>>>> Exactly, choosing the disks to copy to the storage domain.
>>>>>
>>>>>> Here however, the action the user wants is to copy 1 object (the
>>>>>> template) which has sub objects and it should run as a single
>>>>>> action.
>>>>> We are not cloning the template itself we are only cloning the
>>>>> template
>>>>> disks.
>>>>>
>>>>>
>>>>>> For example, if there is enough space on the target domain for 2/4
>>>>>> disks then using multipleRunAction would result in 2 disks being
>>>>>> copied and 2 failing.
>>>>>> If however it is a single action then the free space test would
>>>>>> fail the entire action and user would be able to choose if he
>>>>>> wants to copy just 2.
>>>>>> Note that in this case, actually performing the copy of the 2
>>>>>> disks is detrimental as it can negatively affect VMs on this
>>>>>> domain.
>>>>>>
>>>>>>>
>>>>>>>>> what the user really wants is to specify which disks to copy
>>>>>>>>> and destination per disk, and i don't see a reason to create a
>>>>>>>>> backend
>>>>>>>>> command to do it
>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Engine-devel mailing list
>>>>>>>>>> Engine-devel at ovirt.org
>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Engine-devel mailing list
>>>>>>>> Engine-devel at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>> _______________________________________________
>>>>>>> Engine-devel mailing list
>>>>>>> Engine-devel at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>> I would like this discussion to continue but I need some short-term
>>>> closure since I only have two days to get something into code. He is
>>>> what I am proposing to do:
>>>>
>>>> 1) leave clone template as it is. It will try to copy all of the
>>>> disks
>>>> to the same storage domain.
>>>> 2) Expose a copy disk command so that the user can select a single
>>>> disk
>>>> and create a copy of it on another storage domain.
>>>>
>>>> Is everyone ok with starting there and then refining once we can
>>>> reach
>>>> more of a consensus on what the end product should be?
>>> fine by me, although i think we should remove the clone template
>>> (actually called copy template),
>>> as i think the copy disk gives a way to do it, with more granularity,
>>> and the user will know exactly what to expect.
>>>
>>
>> We are removing the cloneTemplate and introducing cloneTemplatedisk,
>> following the above discussion does anyone have strong objection for
>> doing this?
>>
> 
> We can't remove cloneTemplate until it is removed from the UI or else we
> will break functionality.  For now we are just going to ensure that it
> works as it always has until the UI is ready to remove it.
> 

If by ensure you mean you are going to do adjustments for the
cloneTemplate code to work then it is not needed.

You can either send a patch to disable/remove this button from the UI
or synchronize your patch with a patch from the UI that removes this button.

I don't like the approach of leaving the code because many times we end
up with a code that is not used and not maintained.


>>> unfortunately, there is another issue,
>>> currently, since template can be fully on a domain or not,
>>> remove template can get list of storage domains to remove the
>>> template from,
>>> and if the list contains all the domain the template is on, the
>>> template is fully deleted from the db,
>>> if its partial, then only the copies of the disks are removed from
>>> the domain.
>>>
>> Worth Noting that we have another REST API gap that we can use in our
>> favor, the delete template from storage domain is not modeled yet and we
>> don't need to support it.
>>
>>> what i suggest is, making it simple:
>>> remove template will remove the template, and all of the disks copies
>>> from all the domains.
>> +1 for that approach.
>>
>>> we will need new removeTemplateDisk command that will remove a disk
>>> from a domain,
>>> as long there is another copy of the disk.
>>> if its the last copy it should fail, as removing the disk from all
>>> the domains will change the template,
>>> which is not a supported flow.
>>>
>>> thoughts?
>> I would implement a single verb - removeDisk, it has optional parameter
>> storage domain.
>> If the storage domain is not specified it is simply removing a disk (for
>> template, VM or floating) if a SD is specified in case there is only one
>> image that represent this disk (the only use case for VMs) we remove the
>> disk (same as no passing no SD except for extra validation) if more than
>> one image is associated with this disk (the image-SD map has more than
>> one entry) then remove the relevant mapping.
>>
>> Note: in case of deleting the disk and there is only one image
>> associated with the disk we should remove the device from the vm_devices
>> table as well.
>>
>> Implementing a single generic verb makes it easier to keep backwards
>> compatibility when the model changes.
>>
>> Livnat
>>
>>
>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 



From oschreib at redhat.com  Tue Jan 17 16:52:47 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Tue, 17 Jan 2012 11:52:47 -0500 (EST)
Subject: [Engine-devel] New oVirt-engine RPMs available
Message-ID: <084e01ccd538$73143210$593c9630$@redhat.com>

Hi all,

We've just finished uploading a new set of oVirt-engine rpms into
ovirt.org.
In this version, we're very excited to introduce the support for JBoss
AS7, the latest release of the JBoss community.
These rpms are considered as candidates for our first release, aimed for
January 31st, and will be tested during our official test day [1]
tomorrow.

Please note that due to the nature of nightly builds, and due to the
massive changes required in order to support JBoss AS7, we recommend a
clean environment to be used with the new set of rpms.

In order to install the RPMs, please follow the instructions described in
http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
Feedback, comments and bug reports are always welcome.

The oVirt-Engine team

[1] http://www.ovirt.org/wiki/Testing/OvirtTestDay 


From lpeer at redhat.com  Tue Jan 17 17:46:17 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 17 Jan 2012 19:46:17 +0200
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <20120117140823.GE20867@redhat.com>
References: <20120108130813.GC24375@redhat.com>
	<20120117140823.GE20867@redhat.com>
Message-ID: <4F15B3E9.8040506@redhat.com>

On 17/01/12 16:08, Dan Kenigsberg wrote:
> On Sun, Jan 08, 2012 at 03:08:13PM +0200, Dan Kenigsberg wrote:
>> Hi Lists,
>>
>> One cannot create a PV on a partitioned device, and therefor such
>> devices where not reported to Engine. This proved surprising to users
>> who woder where their LUN disappeared.
>>
>> Vdsm should report all devices, and ovirt-engine should mark partitioned
>> devices as unworthy of a PV. In the future, Vdsm may allow to forcefully
>> remove a partition table from a device, to make it usable as a PV.
>>
>> Douglas (CCed) would take resposibilty on the Vdsm side. Initial patch at
>> http://gerrit.ovirt.org/944 sports a backword-compatible API. Who's taking this
>> on Engine? This involves GUI, too, as partitioned devices should probably be
>> displayed greyed-out.
> 
> Livnat, is there any taker for this on your side?
> 
> Since the change is backward-compatible, I would push it if there are no
> objections.
> 
> Dan.

Hi Danken,

How did you mark a partitioned device? (How will the engine will spot
these LUNs)

Thanks, Livnat


From ykaul at redhat.com  Tue Jan 17 18:42:12 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Tue, 17 Jan 2012 20:42:12 +0200
Subject: [Engine-devel] Failure to deploy (with JBoss 7.1) [SOLVED]
In-Reply-To: <483fd37c-e875-448f-9d0a-922efdb0132c@zmail02.collab.prod.int.phx2.redhat.com>
References: <483fd37c-e875-448f-9d0a-922efdb0132c@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <4F15C104.2020501@redhat.com>

On 01/17/2012 03:41 PM, Oved Ourfalli wrote:
> I saw that error before, and running mvn clean install -Psetup,dep fixed this issue.

No, that didn't help, neither did re-running JBoss.
Deleting the engine.ear.failed file under deployment directory solved 
the problem.
Y.

> But, I would like to examine it a bit before I tell you to do that.
> Can you send me the details to your environment offline, and I'll do some testing there?
>
> ----- Original Message -----
>> From: "Yaniv Kaul"<ykaul at redhat.com>
>> To: "Oved Ourfalli"<ovedo at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Tuesday, January 17, 2012 3:27:50 PM
>> Subject: Re: [Engine-devel] Failure to deploy (with JBoss 7.1)
>>
>> On 01/17/2012 03:05 PM, Oved Ourfalli wrote:
>>> The question is what did you have in the previous environment (as5)
>>> in postgres-ds.xml file.
>> I've had the 'password' field enabled there. I suspect it's because I
>> did 'setup' in my deployment with Jboss 7.
>> I've edited standalone.xml, and I'm now into my next issue:
>> 2012-01-17 15:24:34,678 INFO
>>   [org.jboss.as.server.deployment.scanner]
>> (DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed
>> deployment engine.ear
>> 2012-01-17 15:24:34,747 INFO
>>   [org.jboss.as.server.deployment.scanner]
>> (DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in
>> deployment directory. To trigger deployment create a file called
>> engine.ear.dodeploy
>> 2012-01-17 15:24:34,758 ERROR [org.jboss.as.controller]
>> (DeploymentScanner-threads - 2) JBAS014612: Operation ("add") failed
>> -
>> address: ([("deployment" =>  "engine.ear")]):
>> java.lang.IllegalStateException: JBAS014666: Duplicate resource
>> [("deployment" =>  "engine.ear")]
>>       at
>> org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286)
>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>       at
>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
>>       at
>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
>>       at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>> [:1.6.0_22]
>>       at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>       [:1.6.0_22]
>>       at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>> [:1.6.0_22]
>>       at
>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>> [:1.6.0_22]
>>       at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> [:1.6.0_22]
>>       at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> [:1.6.0_22]
>>       at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>>       at org.jboss.threads.JBossThread.run(JBossThread.java:122)
>> [jboss-threads-2.0.0.GA.jar:]
>>
>> 2012-01-17 15:24:34,761 ERROR
>> [org.jboss.as.server.deployment.scanner]
>> (DeploymentScanner-threads - 1) JBAS014654: Composite operation was
>> rolled back
>> 2012-01-17 15:24:34,762 ERROR
>> [org.jboss.as.server.deployment.scanner]
>> (DeploymentScanner-threads - 1) {"JBAS014653: Composite operation
>> failed
>> and was rolled back. Steps that failed:" =>  {"Operation step-1" =>
>> "JBAS014749: Operation handler failed: JBAS014666: Duplicate resource
>> [(\"deployment\" =>  \"engine.ear\")]"}}
>>
>>>
>>> ----- Original Message -----
>>>> From: "Yaniv Kaul"<ykaul at redhat.com>
>>>> To: "Oved Ourfalli"<ovedo at redhat.com>
>>>> Cc: engine-devel at ovirt.org
>>>> Sent: Tuesday, January 17, 2012 2:43:24 PM
>>>> Subject: Re: [Engine-devel] Failure to deploy (with JBoss 7.1)
>>>>
>>>> On 01/17/2012 10:57 AM, Oved Ourfalli wrote:
>>>>> Did you use a username+password in the previous datasource (in
>>>>> AS5)
>>>>> or was a trust defined for the postgres instance?
>>>>>
>>>>> If needed, the username+password must be defined in
>>>>> $JBOSS_HOME/standalone/configuration/standalone.xml file:
>>>>> 1. If encrypted then in the EncryptDBPassword section, and
>>>>> reference it in the data source, by using the following instead
>>>>> of
>>>>> username:
>>>>>                            <security-domain>
>>>>>                                EncryptDBPassword
>>>>>                            </security-domain>
>>>>> 2. If not encrypted then just put the password in the datasource
>>>>> definition.
>>>> This is what I have:
>>>> <security>
>>>> <user-name>
>>>>                                postgres
>>>> </user-name>
>>>> <!-- IF NEEDED CHANGE THIS TO THE CORRECT PASSWORD/SECURITY DOMAIN
>>>> -->
>>>> <!--password>
>>>>                                password
>>>> </password-->
>>>> <!--security-domain>
>>>>                                EncryptDBPassword
>>>> </security-domain-->
>>>> </security>
>>>>
>>>> So as far as I understand, both options are actually commented
>>>> out.
>>>> Y.
>>>>
>>>>> I can connect to your machine and help if you want.
>>>>>
>>>>> Oved
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Yaniv Kaul"<ykaul at redhat.com>
>>>>>> To: engine-devel at ovirt.org
>>>>>> Sent: Tuesday, January 17, 2012 10:46:55 AM
>>>>>> Subject: [Engine-devel] Failure to deploy (with JBoss 7.1)
>>>>>>
>>>>>> After successful compilation and deployment, on a host that ran
>>>>>> previously fine, I'm getting some error on executing.
>>>>>> Not sure what happened to postgres access - it ran fine
>>>>>> previously
>>>>>> (previous JBoss):
>>>>>>
>>>>>> [ykaul at ykaul ear]$
>>>>>> /usr/local/jboss-as-7.1.0.Beta1b/bin/standalone.sh
>>>>>> =========================================================================
>>>>>>
>>>>>>       JBoss Bootstrap Environment
>>>>>>
>>>>>>       JBOSS_HOME: /usr/local/jboss-as-7.1.0.Beta1b
>>>>>>
>>>>>>       JAVA: java
>>>>>>
>>>>>>       JAVA_OPTS: -server -Xms64m -Xmx512m -XX:MaxPermSize=256m
>>>>>> -Djava.net.preferIPv4Stack=true
>>>>>> -Dorg.jboss.resolver.warning=true
>>>>>> -Dsun.rmi.dgc.client.gcInterval=3600000
>>>>>> -Dsun.rmi.dgc.server.gcInterval=3600000
>>>>>> -Djboss.modules.system.pkgs=org.jboss.byteman
>>>>>> -Djava.awt.headless=true
>>>>>>
>>>>>> =========================================================================
>>>>>>
>>>>>> 10:42:57,373 INFO  [org.jboss.modules] JBoss Modules version
>>>>>> 1.1.0.CR4
>>>>>> 10:42:57,608 INFO  [org.jboss.msc] JBoss MSC version 1.0.1.GA
>>>>>> 10:42:57,665 INFO  [org.jboss.as] JBoss AS 7.1.0.Beta1b "Tesla"
>>>>>> starting
>>>>>> 10:42:58,429 INFO  [org.jboss.as] Creating http management
>>>>>> service
>>>>>> using  socket-binding (management-http)
>>>>>> 10:42:58,434 INFO  [org.xnio] XNIO Version 3.0.0.CR5
>>>>>> 10:42:58,461 INFO  [org.xnio.nio] XNIO NIO Implementation
>>>>>> Version
>>>>>> 3.0.0.CR5
>>>>>> 10:42:58,471 INFO  [org.jboss.remoting] JBoss Remoting version
>>>>>> 3.2.0.CR6
>>>>>> 10:42:58,477 INFO  [org.jboss.as.logging] JBAS011502: Removing
>>>>>> bootstrap
>>>>>> log handlers
>>>>>> 2012-01-17 10:42:58,502 INFO  [org.jboss.as.clustering]
>>>>>> (ServerService
>>>>>> Thread Pool -- 28) JBAS010300: Activating Infinispan subsystem.
>>>>>> 2012-01-17 10:42:58,551 INFO  [org.jboss.as.connector] (MSC
>>>>>> service
>>>>>> thread 1-1) JBAS010408: Starting JCA Subsystem (JBoss
>>>>>> IronJacamar
>>>>>> 1.0.5.Final)
>>>>>> 2012-01-17 10:42:58,564 INFO  [org.jboss.as.naming]
>>>>>> (ServerService
>>>>>> Thread Pool -- 35) JBAS011800: Activating Naming Subsystem
>>>>>> 2012-01-17 10:42:58,592 INFO  [org.jboss.as.osgi] (ServerService
>>>>>> Thread
>>>>>> Pool -- 36) JBAS011910: Activating OSGi Subsystem
>>>>>> 2012-01-17 10:42:58,602 INFO  [org.jboss.as.naming] (MSC service
>>>>>> thread
>>>>>> 1-1) JBAS011802: Starting Naming Service
>>>>>> 2012-01-17 10:42:58,605 INFO  [org.jboss.as.security]
>>>>>> (ServerService
>>>>>> Thread Pool -- 41) Activating Security Subsystem
>>>>>> 2012-01-17 10:42:58,614 INFO  [org.jboss.as.mail.extension] (MSC
>>>>>> service
>>>>>> thread 1-1) JBAS015400: Bound mail session
>>>>>> [java:jboss/mail/Default]
>>>>>> 2012-01-17 10:42:58,615 INFO  [org.jboss.as.security] (MSC
>>>>>> service
>>>>>> thread 1-1) Picketbox version=4.0.6.Beta1
>>>>>> 2012-01-17 10:42:58,623 INFO
>>>>>> [org.jboss.as.connector.subsystems.datasources] (ServerService
>>>>>> Thread
>>>>>> Pool -- 24) JBAS010404: Deploying non-JDBC-compliant driver
>>>>>> class
>>>>>> org.postgresql.Driver (version 8.4)
>>>>>> 2012-01-17 10:42:58,694 INFO  [org.jboss.as.remoting] (MSC
>>>>>> service
>>>>>> thread 1-1) Listening on /127.0.0.1:9999
>>>>>> 2012-01-17 10:42:58,697 INFO  [org.jboss.as.remoting] (MSC
>>>>>> service
>>>>>> thread 1-2) Listening on /0.0.0.0:4447
>>>>>> 2012-01-17 10:42:58,785 INFO
>>>>>> [org.apache.catalina.core.AprLifecycleListener] (MSC service
>>>>>> thread
>>>>>> 1-4)
>>>>>> The Apache Tomcat Native library which allows optimal
>>>>>> performance
>>>>>> in
>>>>>> production environments was not found on the java.library.path:
>>>>>> /usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/usr/local/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
>>>>>> 2012-01-17 10:42:58,864 INFO
>>>>>>     [org.apache.coyote.http11.Http11Protocol]
>>>>>> (MSC service thread 1-4) Starting Coyote HTTP/1.1 on
>>>>>> http--0.0.0.0-8080
>>>>>> 2012-01-17 10:42:59,312 INFO
>>>>>>     [org.jboss.as.server.deployment.scanner]
>>>>>> (MSC service thread 1-4) JBAS015012: Started
>>>>>> FileSystemDeploymentService
>>>>>> for directory
>>>>>> /usr/local/jboss-as-7.1.0.Beta1b/standalone/deployments
>>>>>> 2012-01-17 10:42:59,341 INFO
>>>>>> [org.jboss.as.connector.subsystems.datasources] (MSC service
>>>>>> thread
>>>>>> 1-4)
>>>>>> JBAS010400: Bound data source [java:/ENGINEDataSource]
>>>>>> 2012-01-17 10:42:59,355 INFO  [org.jboss.as] (Controller Boot
>>>>>> Thread)
>>>>>> JBoss AS 7.1.0.Beta1b "Tesla" started in 2128ms - Started 129 of
>>>>>> 190
>>>>>> services (60 services are passive or on-demand)
>>>>>> 2012-01-17 10:42:59,420 INFO
>>>>>>     [org.jboss.as.server.deployment.scanner]
>>>>>> (DeploymentScanner-threads - 1) JBAS015014: Re-attempting failed
>>>>>> deployment engine.ear
>>>>>> 2012-01-17 10:42:59,459 WARN
>>>>>> [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool]
>>>>>> (JCA
>>>>>> PoolFiller) IJ000610: Unable to fill pool:
>>>>>> javax.resource.ResourceException: Could not create connection
>>>>>> at
>>>>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277)
>>>>>>         at
>>>>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235)
>>>>>>         at
>>>>>> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:736)
>>>>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>>>>>         at
>>>>>> org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.fillToMin(SemaphoreArrayListManagedConnectionPool.java:683)
>>>>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>>>>>         at
>>>>>> org.jboss.jca.core.connectionmanager.pool.mcp.PoolFiller.run(PoolFiller.java:97)
>>>>>> [ironjacamar-core-impl-1.0.5.Final.jar:]
>>>>>>         at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>>>>>> Caused by: org.postgresql.util.PSQLException: FATAL: password
>>>>>> authentication failed for user "postgres"
>>>>>>         at
>>>>>> org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
>>>>>>         at
>>>>>> org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
>>>>>>         at
>>>>>> org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
>>>>>>         at
>>>>>> org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
>>>>>>         at
>>>>>> org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
>>>>>>         at
>>>>>> org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22)
>>>>>>         at
>>>>>> org.postgresql.jdbc4.AbstractJdbc4Connection.<init>(AbstractJdbc4Connection.java:30)
>>>>>>         at
>>>>>>         org.postgresql.jdbc4.Jdbc4Connection.<init>(Jdbc4Connection.java:24)
>>>>>>         at org.postgresql.Driver.makeConnection(Driver.java:393)
>>>>>>         at org.postgresql.Driver.connect(Driver.java:267)
>>>>>>         at
>>>>>> org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249)
>>>>>>         ... 5 more
>>>>>>
>>>>>> 2012-01-17 10:42:59,491 INFO
>>>>>>     [org.jboss.as.server.deployment.scanner]
>>>>>> (DeploymentScanner-threads - 1) JBAS015003: Found engine.ear in
>>>>>> deployment directory. To trigger deployment create a file called
>>>>>> engine.ear.dodeploy
>>>>>> 2012-01-17 10:42:59,502 ERROR [org.jboss.as.controller]
>>>>>> (DeploymentScanner-threads - 2) JBAS014612: Operation ("add")
>>>>>> failed
>>>>>> -
>>>>>> address: ([("deployment" =>    "engine.ear")]):
>>>>>> java.lang.IllegalStateException: JBAS014666: Duplicate resource
>>>>>> [("deployment" =>    "engine.ear")]
>>>>>>         at
>>>>>> org.jboss.as.controller.OperationContextImpl.addResource(OperationContextImpl.java:503)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.OperationContextImpl.createResource(OperationContextImpl.java:471)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:170)
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.server.deployment.DeploymentDeployHandler.execute(DeploymentDeployHandler.java:68)
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.server.deployment.DeploymentAddHandler.execute(DeploymentAddHandler.java:183)
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.CompositeOperationHandler.execute(CompositeOperationHandler.java:84)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:426)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:322)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:216)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:152)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:115)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:296)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:286)
>>>>>> [jboss-as-controller-7.1.0.Beta1b.jar:]
>>>>>>         at
>>>>>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1178)
>>>>>>         at
>>>>>> org.jboss.as.server.deployment.scanner.FileSystemDeploymentService$DeploymentTask.call(FileSystemDeploymentService.java:1168)
>>>>>>         at
>>>>>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>>>>>> [:1.6.0_22]
>>>>>>         at
>>>>>>         java.util.concurrent.FutureTask.run(FutureTask.java:166)
>>>>>>         [:1.6.0_22]
>>>>>>         at
>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>>>>>> [:1.6.0_22]
>>>>>>         at
>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>>>>>> [:1.6.0_22]
>>>>>>         at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>>>>>> [:1.6.0_22]
>>>>>>         at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>>>>>> [:1.6.0_22]
>>>>>>         at java.lang.Thread.run(Thread.java:679) [:1.6.0_22]
>>>>>>         at
>>>>>>         org.jboss.threads.JBossThread.run(JBossThread.java:122)
>>>>>> [jboss-threads-2.0.0.GA.jar:]
>>>>>>
>>>>>> 2012-01-17 10:42:59,505 ERROR
>>>>>> [org.jboss.as.server.deployment.scanner]
>>>>>> (DeploymentScanner-threads - 1) JBAS014654: Composite operation
>>>>>> was
>>>>>> rolled back
>>>>>> 2012-01-17 10:42:59,506 ERROR
>>>>>> [org.jboss.as.server.deployment.scanner]
>>>>>> (DeploymentScanner-threads - 1) {"JBAS014653: Composite
>>>>>> operation
>>>>>> failed
>>>>>> and was rolled back. Steps that failed:" =>    {"Operation
>>>>>> step-1"
>>>>>> =>
>>>>>> "JBAS014749: Operation handler failed: JBAS014666: Duplicate
>>>>>> resource
>>>>>> [(\"deployment\" =>    \"engine.ear\")]"}}
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>



From abaron at redhat.com  Tue Jan 17 20:14:34 2012
From: abaron at redhat.com (Ayal Baron)
Date: Tue, 17 Jan 2012 15:14:34 -0500 (EST)
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <39727574-be45-443d-8dfe-f9b7df1ae52b@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <8b6cfe3d-b9ee-4334-870a-2e0763e735f6@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Maor" <mlipchuk at redhat.com>
> > To: engine-devel at ovirt.org
> > Sent: Tuesday, January 17, 2012 4:33:19 AM
> > Subject: Re: [Engine-devel] a different approach to the command
> > classes
> > 
> > On 01/17/2012 09:24 AM, Ayal Baron wrote:
> > > 
> > > 
> > > ----- Original Message -----
> > >> On 17/01/12 04:58, Jon Choate wrote:
> > >>> The way the command classes are written has bothered me for a
> > >>> while.
> > >>> While implementing the multiple storage domain features I am
> > >>> presented
> > >>> with the opportunity to create a new command from scratch.  I
> > >>> gave
> > >>> some
> > >>> thought to what I would like the command classes to look like
> > >>> while
> > >>> balancing that the class must still fit in with the existing
> > >>> structure.
> > >>> So here is what I came up with. I'd appreciate any feedback.
> > >>>
> > >>> The Command class encompasses only the rules of what needs to
> > >>> be
> > >>> done.
> > >>> It relies upon Validator classes to determine if the
> > >>> canDoAction
> > >>> conditions have been met.
> > >>>
> > >>>     @Override
> > >>>     public boolean canDoAction() {
> > >>>       ...
> > >>>         checkTargetDomainHasSpace();
> > >>>         checkTargetDomainIsValidTarget();
> > >>>         checkSourceDomainIsValidSource();
> > >>>         checkSourceAndTargetAreDifferent();
> > >>>      ...
> > >>> }
> > >>>
> > >>> ...
> > >>>
> > >>>   private void checkTargetDomainHasSpace() {
> > >>>         if(!actionAllowed) return;
> > >>>        
> > >>> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
> > >>> {
> > >>>          
> > >>> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
> > >>>           actionAllowed = false;
> > >>>         }
> > >>>     }
> > >>>
> > >>>
> > >>> Each of the checks follows a similar pattern of
> > >>>     - guard clause to see of we already failed and bail if we
> > >>>     did
> > >>>     - test for failure of the condition
> > >>>     - add failure message if needed
> > >>>     - set variable to failed if needed
> > >>>
> > >>> Storing the success flag in a variable allows us to keep the
> > >>> canDoAction
> > >>> method readable as a series of commands and to allow it to be
> > >>> accessed
> > >>> by all the private methods without them having to pass it
> > >>> around.
> > >>>
> > >>> The execution of the command will follow a similar pattern
> > >>> where
> > >>> the
> > >>> command class will only know how to describe what needs to be
> > >>> done
> > >>> and
> > >>> to rely on supporting objects to handle the implementation of
> > >>> these
> > >>> steps.  Getting the implementation out of the command classes
> > >>> will
> > >>> allow
> > >>> the commands to share validation and implementation details and
> > >>> remove a
> > >>> lot of the duplication that currently exists within the
> > >>> commands.
> > >>>
> > >>>
> > >>> How do people feel about this approach?
> > >>
> > >>
> > >> Hi Jon,
> > >>
> > >> The scope of your proposal is changing the implementation of the
> > >> canDoAction method, I think that the title of this mail is a bit
> > >> misleading.
> > > 
> > > Actually I think Jon was suggesting to do the same in the command
> > > itself.
> 
> 
> Yes I am.  I just haven't written that code yet!
> 
> 
> > 
> > I think, using shared canDoAction validation between commands might
> > be a
> > good idea also, it can be seen in operations such as add/update
> > commands.
> > In most cases, the update validation, is already based on the add
> > command validation, sometime it is implemented with
> > inheritance/external
> > class, in other cases it is just implemented with multiple code.
> > > 
> > >>
> > >> Basically what you are suggesting is to split the canDoAction
> > >> implementation into methods and then extract them from the
> > >> command
> > >> class
> > >> to a shared class so they can be reused.
> > >>
> > >> In many cases we can use (are using) inheritance for reusing
> > >> code,
> > >> there
> > >> are cases where inheritance does not do the work and we can
> > >> extract
> > >> to
> > >> external classes.
> > >>
> 
> Our overuse of inheritance is one of the things I'm trying to
> rectify.  Inheritance wasn't added
> to the language to facilitate reuse. It is there to represent object
> hierarchies.  In some cases
> we abuse this to leverage code reuse.  This is often a code smell
> that the code is in the wrong
> place to begin with.  If both classes need the code and they are not
> logically related in an object
> hierarchy, the code really needs to be outside the classes.
> 
> We have cases where the use of inheritance for reuse have gone too
> far. For instance:
> 
> public class RemoveVdsSpmIdCommand<T extends VdsActionParameters>
> extends AddVdsSpmIdCommand<T>
> 
> So this says that a RemoveVdsSmpIdCommand is a type of
> AddVdsSpmIdCommand?  That implies that I can
> use a RemoveVdsSmpIdCommand anywhere that an AddVdsSpmIdCommand is
> expected. Otherwise we have broken
> one of the fundamental contracts of object oriented programming.
> 
> > >> I think such a change is welcomed but on a needed basis, I think
> > >> it
> > >> is
> > >> overkill for the general use case and will make the code more
> > >> cumbersome
> > >> (if the original canDoAction was 4-5 lines long...).
> 
> From what I have seen those canDoActions are in the minority.  The
> overall goals are to increase
> the readability and maintainability of the code so I would suggest
> being pragmatic about any approach.
> If doing it doesn't help achieve the goal, then don't do it.
> 
> 
> One of the ideas I'm trying to convey here is that the command
> classes should be fairly ignorant.
> They should be responsible for knowing the list of steps involved in
> a workflow, but not the details
> of how those steps are carried out. Imo knowing both the steps and
> their implementation violates the SRP.
> 
> 
> > > 
> > > Jon, I think the burden of proof is on you here to show a real
> > > example and how it makes the code clearer (e.g. change 2 commands
> > > which share similar checks).
> > > Without 'seeing' it I don't think we would be able to appreciate
> > > the advantage of your approach.
> > > 
> 
> I need to get the multiple storage domains feature put away and then
> I will
> definitely do some refactoring to illustrate the value. I
> wholeheartedly agree
> that having actual code to kick around is better. I just can't do it
> right now!
> 
> 
> > >>
> > >> One thing I don't like in the above suggestion is the way you
> > >> validate
> > >> that the previous condition succeeded/failed. Having this
> > >> condition
> > >> at
> > >> the beginning of each validation method is not a good approach
> > >> IMO.
> 
> Can you elaborate on your opposition to the use of guard clauses?
> They have
> been considered a best practice for quite some time.
> 
> > > 
> > > +1, if the previous check failed it should raise an exception,
> > > not
> > > rely on the next check to bail.
> > > It would be easier (less code, clearer) to wrap all the calls
> > > with
> > > a try catch clause (1 for all the calls), catch the specific
> > > exception that says the check failed and return whatever you want
> > > to return.
> 
> I'm not really a fan of using exceptions for controlling flow of
> execution.  Failing one of these
> checks is not an exceptional event. Its an expected part of the
> workflow and should
> be handled as a normal occurrence.
> 
> I would argue that wrapping the entire canDoAction method in a
> try/catch would not make the code clearer:
> 
> 
>   public boolean canDoAction() {
>       boolean succeeded = true;
>       try {
>          do1();
>          do2();
>          do3();
>       catch(ValidationException e) {
>          succeeded = false;
>       }
>       return succeeded;
>   }
> 
> 
> has a lot more noise than:
> 
>   private boolean canDoAction = true;
> 
>   public boolean canDoAction() {
>     do1();
>     do2();
>     do3();
>     return canDoAction;
>   }
> 

You're forgetting the cost:

instead of:
public void do1() {
    ...
}

public void do2() {
    ...
}

public void do3() {
    ...
}

public void do1() {
    if (!globalSuccessVar) {
       return
    }
    ...
}

public void do2() {
    if (!globalSuccessVar) {
       return
    }
    ...
}

public void do3() {
    if (!globalSuccessVar) {
       return
    }
    ...
}

I on purpose did not call it 'actionAllowed' to stress the fact that it's a global var (another thing I don't like).

And before you go to the one liner option per function, I consider 'if (!globalSuccessVar) return' bad form as it's not easy to discern what the condition is and what the action is.


> In the second form, there is no extra indentation or curly braces.
>  The validation steps
> look just like a list of steps and imo it makes the code really easy
> to understand by a maintainer.
> 
> > > 
> > >>
> > >>
> > >> Livnat
> > >>
> > >>
> > >> _______________________________________________
> > >> Engine-devel mailing list
> > >> Engine-devel at ovirt.org
> > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From abaron at redhat.com  Tue Jan 17 20:21:29 2012
From: abaron at redhat.com (Ayal Baron)
Date: Tue, 17 Jan 2012 15:21:29 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export (was:
	Re: move disk command)
In-Reply-To: <4F157520.9090906@redhat.com>
Message-ID: <e1dcafd9-1eb0-4743-847c-adbc5dad39e6@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/17/2012 11:57 AM, Omer Frenkel wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Ayal Baron" <abaron at redhat.com>
> >> To: "Omer Frenkel" <ofrenkel at redhat.com>
> >> Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>
> >> Sent: Tuesday, January 17, 2012 10:45:53 AM
> >> Subject: Shared disk / Floating disk import/export (was: Re:
> >> [Engine-devel] move disk command)
> >>
> >>
> >> [SNIP]
> >>
> >>>>> This may be unrelated, but user would be allowed to export and
> >>>>> import a floating disk, right?
> >>>>> I would like the ability to import *any* disk in the export
> >>>>> domain
> >>>>> as a floating disk, but in the least, export and import disks
> >>>>> not
> >>>>> associated with a VM.
> >>>
> >>> you are right, it is unrelated, this thread is about move disk of
> >>> a
> >>> vm between SDs,
> >>> export and import is copy, and floating disks is part of the
> >>> shared
> >>> disk feature,
> >>> this indeed need to be discussed in that scope.
> >>
> >> Ok, so I've changed the subject, now let's discuss it.
> >>
> > 
> > Adding Maor,
> > not sure if we have any plan regard export/import of floating disk,
> > or in general, exporting disk without it's vm (if I understood Ayal
> > correctly)
> > 
> > Maor, any comments?
> I remember, we mentioned export/import issue in our last meeting with
> Andrew in the context of shared raw disk.
> It was decided then, that export/import will not be supported by
> shared
> raw disk, I'm not sure if we also decided that for floating disk,
> but I think its a PM decision.
> 
> Support import/export domain might evolve to a big change, since if
> we
> want to export disk, we might also want to reflect all its
> configuration
> and functionality there, also reflect disks which are attached to VMs
> and templates as well.

What properties does a disk have?
Interface is what type of connection is used when plugging the disk in the computer (VM) so it's not really a disk property.
Address is also VM specific
Description is already stored in storage side.
What else do you have for disk?

Note that being able to import floating disks means that we'd be able to take any data storage domain and import any disk(s) from it (assuming user changes domain type to export domain, which users already know how to do)



From mkenneth at redhat.com  Tue Jan 17 21:00:16 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Tue, 17 Jan 2012 16:00:16 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <a00e9a8b-2754-421c-a874-1f340bfdafe9@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <ef38bd6f-b448-43bc-b3b3-0b4dfda334ef@mkenneth.csb>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Livnat Peer" <lpeer at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Tuesday, January 17, 2012 11:45:33 AM
> Subject: Re: [Engine-devel] the future of template cloning
> 
> 
> 
> ----- Original Message -----
> > On 17/01/12 10:46, Itamar Heim wrote:
> > > On 01/17/2012 10:32 AM, Omer Frenkel wrote:
> > >>
> > >>
> > >> ----- Original Message -----
> > >>> From: "Itamar Heim"<iheim at redhat.com>
> > >>> To: "Jon Choate"<jchoate at redhat.com>
> > >>> Cc: engine-devel at ovirt.org
> > >>> Sent: Monday, January 16, 2012 7:26:24 PM
> > >>> Subject: Re: [Engine-devel] the future of template cloning
> > >>>
> > >>> On 01/16/2012 06:16 PM, Jon Choate wrote:
> > >>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
> > >>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
> > >>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
> > >>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
> > >>>>>>>>
> > >>>>>>>> ----- Original Message -----
> > >>>>>>>>> We are going to be able to store the disks for a template
> > >>>>>>>>> on
> > >>>>>>>>> different storage domains due to the multiple storage
> > >>>>>>>>> domain
> > >>>>>>>>> feature. Cloning a template will still be possible, but
> > >>>>>>>>> will
> > >>>>>>>>> it
> > >>>>>>>>> provide any value? Thoughts?
> > >>>>>>>> I see no relation between the two options.
> > >>>>>>>>
> > >>>>>>>> Scenario 1: I can create a VM with a single disk and
> > >>>>>>>> create
> > >>>>>>>> a
> > >>>>>>>> template from it.
> > >>>>>>>> I would still want to be able to clone the template in
> > >>>>>>>> order
> > >>>>>>>> to
> > >>>>>>>> provision VMs from it on different domains.
> > >>>>>>>>
> > >>>>>>>> Scenario 2: same thing with multiple disks on same domain.
> > >>>>>>>>
> > >>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
> > >>>>>>>> domains
> > >>>>>>>> (domain A and domain B) and I want to have another copy of
> > >>>>>>>> the
> > >>>>>>>> template on domain C and domain D
> > >>>>>>>>
> > >>>>>>> Hi Jon,
> > >>>>>>>
> > >>>>>>> After talking to Michael Pasternak it seems that we did not
> > >>>>>>> implemented
> > >>>>>>> copyTemplate in the REST API, it seems to be a gap that we
> > >>>>>>> have.
> > >>>>>>>
> > >>>>>>> This gap is playing in our favor, we can remove the
> > >>>>>>> copyTemplate
> > >>>>>>> verb
> > >>>>>>> and introduce copyDisk verb.
> > >>>>>>>
> > >>>>>>> The template disks can be copied to another SD.
> > >>>>>>> When creating a VM from template the user can choose per
> > >>>>>>> disk
> > >>>>>>> the
> > >>>>>>> destination SD (only SD with the disks are eligible
> > >>>>>>> candidates).
> > >>>>>> wait, when creating a VM from a template, the user won't get
> > >>>>>> a
> > >>>>>> choice
> > >>>>>> will they? Won't the VM disks have to go on the same storage
> > >>>>>> domain as
> > >>>>>> the template disks they were created from?
> > >>>>>
> > >>>>> yes, but the template disks can be copied to multiple storage
> > >>>>> domains,
> > >>>>> so the user can choose for the VM/disk which storage domain
> > >>>>> to
> > >>>>> create
> > >>>>> them from (per storage domains that have copies of that disk)
> > >>>> OH! I totally misunderstood. So what you are saying is that a
> > >>>> template
> > >>>> can have N number of copies of the same disk each on a
> > >>>> different
> > >>>> storage
> > >>>> domain. I had thought that if you wanted that type of
> > >>>> situation
> > >>>> you
> > >>>> would have multiple copies of the template itself too.
> > >>
> > >> yes, one copy of disk per domain though.
> > >>
> > >>>>
> > >>>> Just to be clear, does this mean that the plan is to phase out
> > >>>> the
> > >>>> current clone template command and instead implementing a
> > >>>> clone
> > >>>> disk
> > >>>> command so that a template can duplicate its disks
> > >>>> individually?
> > >>>
> > >>> pretty much, yes.
> > >>> though i'd imagine 'clone template' would still be useful to
> > >>> have
> > >>> for
> > >>> the user. not sure if it implies core should expose it as well
> > >>> to
> > >>> allow
> > >>> easier usage at UI level for such a task.
> > >>
> > >> we can leave it untouched - means copyTemplate get 1 destination
> > >> domain, and copies all disks to it,
> > >> but i think it will be unusable (and problematic - what if one
> > >> of
> > >> the
> > >> disks already exists on the destination?),
> > > 
> > > then don't copy it, it is already there
> > > 
> > 
> > I agree with Omer, there is no reason to support copy template, if
> > the
> > user wants to clone all the disks he can use multiple actions, we
> > don't
> > need a specific verb for this.
> 
> Reason or lack of depends on the common usage.
> If we assume that normally all disks of a template would reside on
> the same domain then it makes sense to have a verb to copy the
> template in its entirety and not burden the user.
> The general recommendation should be to use a single storage domain,
> so I think there is room for such a verb.
I agree. This is the common use case, all disk resides on the same SD.
and that is why we need a verb for it.
However, for more "trick"/special cases we need to support multi domains.
I also think it would be easier from api perspective to use a single copy verb.
> 
> > If the UI chooses to expose such operation it will use the
> > multipleRunAction API which makes it easier to expose to the user
> > partial success, we could clone disk A and Disk B but Disk C failed
> > etc.
> 
> The multipleRunAction is for user marking multiple objects in GUI and
> running an action on all of these objects.
> Here however, the action the user wants is to copy 1 object (the
> template) which has sub objects and it should run as a single
> action.
> For example, if there is enough space on the target domain for 2/4
> disks then using multipleRunAction would result in 2 disks being
> copied and 2 failing.
> If however it is a single action then the free space test would fail
> the entire action and user would be able to choose if he wants to
> copy just 2.
> Note that in this case, actually performing the copy of the 2 disks
> is detrimental as it can negatively affect VMs on this domain.
> 
> > 
> > 
> > 
> > >> what the user really wants is to specify which disks to copy
> > >> and destination per disk, and i don't see a reason to create a
> > >> backend
> > >> command to do it
> > >>
> > >>> _______________________________________________
> > >>> Engine-devel mailing list
> > >>> Engine-devel at ovirt.org
> > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>>
> > > 
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkenneth at redhat.com  Tue Jan 17 21:00:29 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Tue, 17 Jan 2012 16:00:29 -0500 (EST)
Subject: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
In-Reply-To: <12a24462-26b8-4b46-a3d4-07a18d2f03ca@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <09f0d675-441c-4ba7-ba2a-efe610b8c257@mkenneth.csb>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Michael Kublin" <mkublin at redhat.com>
> Cc: dlaor at redhat.com, engine-devel at ovirt.org
> Sent: Sunday, January 15, 2012 12:35:26 PM
> Subject: Re: [Engine-devel] Low Level design for HotPlug/HotUnplug feature
> 
> [SNIP]
> 
> > > >>>> - Any CanDoAction to allow/disallow plug/unplug from
> > > >>>> specific
> > > >>>> systems?
> > > > The operation will be allowed only for RHEL 5/6, Windows Server
> > > > 2003 and Windows server 2008 operating systems, added to design
> 
> IMO this behaviour is wrong.  We should not prevent user from running
> hot plug on an unsupported system, just warn that this operation
> might fail and can crash the VM (downstream versions can warn that
> it is unsupported).
> E.g. if user installed a service pack on a windows not listed that
> adds support for hotplug, why should we prevent running the action?
> or worse, a new windows version is released and users of the existing
> system would not be able to hot plug.
> 
> Alternatively, the list of OSs on which this works can be
> customizable by user in which case a knowledgeable user would modify
> at her own risk.
> 
> IIRC there is an 'other' os type when creating a VM, why block on
> this? again, either warn or allow customizing behaviour.
I second. I think that we are being "over protective" on the admin. 
I tend to agree that warning should be a better behavior (if the VM is not damaged,
just BSOD, than this is up to the admin to handle it)

> 
> > > >>>> - I suspect we'd be happier with some agent cooperation
> > > >>>> before
> > > >>>> unplugging - is this done by QEMU? Not detailed anywhere.
> 
> This is not currently planned, but should definitely be on the road
> map (need to add to the feature page and state that it will not be
> covered in 3.1)
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkenneth at redhat.com  Tue Jan 17 21:00:54 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Tue, 17 Jan 2012 16:00:54 -0500 (EST)
Subject: [Engine-devel] move disk command
In-Reply-To: <af7e51b5-b30d-42ea-80c2-ca015e844729@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <ffc8d1f5-a5fd-41a2-b370-096a8e134628@mkenneth.csb>



----- Original Message -----
> From: "Omer Frenkel" <ofrenkel at redhat.com>
> To: "Jon Choate" <jchoate at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Tuesday, January 17, 2012 10:41:43 AM
> Subject: Re: [Engine-devel] move disk command
> 
> 
> 
> ----- Original Message -----
> > From: "Jon Choate" <jchoate at redhat.com>
> > To: "Ayal Baron" <abaron at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Monday, January 16, 2012 9:39:55 PM
> > Subject: Re: [Engine-devel] move disk command
> > 
> > On 01/16/2012 02:26 PM, Ayal Baron wrote:
> > >
> > > ----- Original Message -----
> > >> On 01/16/2012 05:26 PM, Jon Choate wrote:
> > >>> As part of the multiple storage domains feature there will be
> > >>> new
> > >>> functionality added to allow users to move individual disks.
> > >>>
> > >>> What are the prerequisites for moving a disk?
> > >>>
> > >>> 1) the disk must exist
> > >>> 2) the associated VM must be down
> > >> this can't just be a CanDoAction check - the lock has to be real
> > >> to
> > >> prevent a race from starting the VM after the validation.
> > >>
> > > Either down or disk is unplugged.
> > >
> > >>> 3) the associated VM must not be locked
> > >>> 4) the source storage domain must exist
> > >>> 5) the source storage domain must be available
> > >>> 6) the target domain must exist
> > >>> 7) the target domain must be available
> > >>> 8) the target domain must have adequate disk space
> > >>> 9) the target domain cannot be an ISO or export domain
> > >>> 10) the source domain cannot be an ISO or export domain
> > > This may be unrelated, but user would be allowed to export and
> > > import a floating disk, right?
> > > I would like the ability to import *any* disk in the export
> > > domain
> > > as a floating disk, but in the least, export and import disks not
> > > associated with a VM.
> 
> you are right, it is unrelated, this thread is about move disk of a
> vm between SDs,
> export and import is copy, and floating disks is part of the shared
> disk feature,
> this indeed need to be discussed in that scope.
So what is the behavior of a move of a floating disk? can I move any type of disk?
(shared?) between SDs? 
> 
> > This was not in scope for the work I am currently doing.  If this
> > is
> > something desirable I think it needs to be prioritized and worked
> > in
> > at
> > a later time.  If it does need to happen now then we are going to
> > need
> > to be able to do full crud for a floating disk I would think.
> > 
> > >> user must provide same/other quota for the target domain which
> > >> has
> > >> enough quota left for the requested size.
> > >>
> > >>> What am I missing?
> > >>>
> > >>> Also, should we allow the moving of a template disk that has VM
> > >>> disks based on it? Unless I'm wrong this would require all of
> > >>> the
> > >>> disks based on the template to be moved as well.
> > >> I'd say no. you can only move a template disk if it is not used.
> > >> I'd go further and say one should copy the template disk and
> > >> delete,
> > >> rather than support move for it at all (not relevant for VM
> > >> disk,
> > >> since
> > >> we don't have the same concept of multiple copies for it).
> > > As long as you can delete a copy of the disk from a domain where
> > > there are no VM disks derived from it.
> > >
> > >>> thoughts?
> > >>> _______________________________________________
> > >>> Engine-devel mailing list
> > >>> Engine-devel at ovirt.org
> > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >> _______________________________________________
> > >> Engine-devel mailing list
> > >> Engine-devel at ovirt.org
> > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From iheim at redhat.com  Tue Jan 17 22:46:57 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 18 Jan 2012 00:46:57 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F15A342.20802@redhat.com>
References: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>
	<4F159878.7000501@redhat.com> <4F159C1A.20209@redhat.com>
	<4F15A342.20802@redhat.com>
Message-ID: <4F15FA61.4050307@redhat.com>

On 01/17/2012 06:35 PM, Livnat Peer wrote:
...

>> We can't remove cloneTemplate until it is removed from the UI or else we
>> will break functionality.  For now we are just going to ensure that it
>> works as it always has until the UI is ready to remove it.
>>
>
> If by ensure you mean you are going to do adjustments for the
> cloneTemplate code to work then it is not needed.
>
> You can either send a patch to disable/remove this button from the UI
> or synchronize your patch with a patch from the UI that removes this button.
>
> I don't like the approach of leaving the code because many times we end
> up with a code that is not used and not maintained.

I don't think a patch which would make existing functionality disabled 
is the way to go.
I expect a lot of times a new functionality will be added, and the old 
one should be flagged deprecated with a comment as to why (or open a BZ 
to self to clean it up), or collaborate with someone more knowledgeable 
on the specific component to make the change in sync, or do the patch 
yourself.
to make less code deprecated, the old api could wrap the new 
functionality (call removeDisk in a loop, etc.).
it should still be flagged as something to clean up once the api calling 
it is removed.
obviously, collaborating (or doing it thyself) on a sync'd change is 
easiest, but not always possible.


From abaron at redhat.com  Wed Jan 18 00:22:55 2012
From: abaron at redhat.com (Ayal Baron)
Date: Tue, 17 Jan 2012 19:22:55 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F1569B7.9050100@redhat.com>
Message-ID: <910118d7-ed84-4010-8cba-6c2bd54dd207@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 17/01/12 11:45, Ayal Baron wrote:
> > 
> > 
> > ----- Original Message -----
> >> On 17/01/12 10:46, Itamar Heim wrote:
> >>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
> >>>>
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Itamar Heim"<iheim at redhat.com>
> >>>>> To: "Jon Choate"<jchoate at redhat.com>
> >>>>> Cc: engine-devel at ovirt.org
> >>>>> Sent: Monday, January 16, 2012 7:26:24 PM
> >>>>> Subject: Re: [Engine-devel] the future of template cloning
> >>>>>
> >>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
> >>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
> >>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
> >>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
> >>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
> >>>>>>>>>>
> >>>>>>>>>> ----- Original Message -----
> >>>>>>>>>>> We are going to be able to store the disks for a template
> >>>>>>>>>>> on
> >>>>>>>>>>> different storage domains due to the multiple storage
> >>>>>>>>>>> domain
> >>>>>>>>>>> feature. Cloning a template will still be possible, but
> >>>>>>>>>>> will
> >>>>>>>>>>> it
> >>>>>>>>>>> provide any value? Thoughts?
> >>>>>>>>>> I see no relation between the two options.
> >>>>>>>>>>
> >>>>>>>>>> Scenario 1: I can create a VM with a single disk and
> >>>>>>>>>> create
> >>>>>>>>>> a
> >>>>>>>>>> template from it.
> >>>>>>>>>> I would still want to be able to clone the template in
> >>>>>>>>>> order
> >>>>>>>>>> to
> >>>>>>>>>> provision VMs from it on different domains.
> >>>>>>>>>>
> >>>>>>>>>> Scenario 2: same thing with multiple disks on same domain.
> >>>>>>>>>>
> >>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
> >>>>>>>>>> domains
> >>>>>>>>>> (domain A and domain B) and I want to have another copy of
> >>>>>>>>>> the
> >>>>>>>>>> template on domain C and domain D
> >>>>>>>>>>
> >>>>>>>>> Hi Jon,
> >>>>>>>>>
> >>>>>>>>> After talking to Michael Pasternak it seems that we did not
> >>>>>>>>> implemented
> >>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
> >>>>>>>>> have.
> >>>>>>>>>
> >>>>>>>>> This gap is playing in our favor, we can remove the
> >>>>>>>>> copyTemplate
> >>>>>>>>> verb
> >>>>>>>>> and introduce copyDisk verb.
> >>>>>>>>>
> >>>>>>>>> The template disks can be copied to another SD.
> >>>>>>>>> When creating a VM from template the user can choose per
> >>>>>>>>> disk
> >>>>>>>>> the
> >>>>>>>>> destination SD (only SD with the disks are eligible
> >>>>>>>>> candidates).
> >>>>>>>> wait, when creating a VM from a template, the user won't get
> >>>>>>>> a
> >>>>>>>> choice
> >>>>>>>> will they? Won't the VM disks have to go on the same storage
> >>>>>>>> domain as
> >>>>>>>> the template disks they were created from?
> >>>>>>>
> >>>>>>> yes, but the template disks can be copied to multiple storage
> >>>>>>> domains,
> >>>>>>> so the user can choose for the VM/disk which storage domain
> >>>>>>> to
> >>>>>>> create
> >>>>>>> them from (per storage domains that have copies of that disk)
> >>>>>> OH! I totally misunderstood. So what you are saying is that a
> >>>>>> template
> >>>>>> can have N number of copies of the same disk each on a
> >>>>>> different
> >>>>>> storage
> >>>>>> domain. I had thought that if you wanted that type of
> >>>>>> situation
> >>>>>> you
> >>>>>> would have multiple copies of the template itself too.
> >>>>
> >>>> yes, one copy of disk per domain though.
> >>>>
> >>>>>>
> >>>>>> Just to be clear, does this mean that the plan is to phase out
> >>>>>> the
> >>>>>> current clone template command and instead implementing a
> >>>>>> clone
> >>>>>> disk
> >>>>>> command so that a template can duplicate its disks
> >>>>>> individually?
> >>>>>
> >>>>> pretty much, yes.
> >>>>> though i'd imagine 'clone template' would still be useful to
> >>>>> have
> >>>>> for
> >>>>> the user. not sure if it implies core should expose it as well
> >>>>> to
> >>>>> allow
> >>>>> easier usage at UI level for such a task.
> >>>>
> >>>> we can leave it untouched - means copyTemplate get 1 destination
> >>>> domain, and copies all disks to it,
> >>>> but i think it will be unusable (and problematic - what if one
> >>>> of
> >>>> the
> >>>> disks already exists on the destination?),
> >>>
> >>> then don't copy it, it is already there
> >>>
> >>
> >> I agree with Omer, there is no reason to support copy template, if
> >> the
> >> user wants to clone all the disks he can use multiple actions, we
> >> don't
> >> need a specific verb for this.
> > 
> > Reason or lack of depends on the common usage.
> > If we assume that normally all disks of a template would reside on
> > the same domain then it makes sense to have a verb to copy the
> > template in its entirety and not burden the user.
> > The general recommendation should be to use a single storage
> > domain, so I think there is room for such a verb.
> > 
> 
> 
> >> If the UI chooses to expose such operation it will use the
> >> multipleRunAction API which makes it easier to expose to the user
> >> partial success, we could clone disk A and Disk B but Disk C
> >> failed
> >> etc.
> > 
> > The multipleRunAction is for user marking multiple objects in GUI
> > and running an action on all of these objects.
> 
> Exactly, choosing the disks to copy to the storage domain.

Which is wrong because multipleRunAction assumes there are no dependencies between the actions.  In the case of copyTemplate the user sees copying all disks as a single operation which should either fail or succeed.  Partial results are not clear and can cause other problems.
Validations in this case should be on the entire set together, not one by one, etc.

> 
> > Here however, the action the user wants is to copy 1 object (the
> > template) which has sub objects and it should run as a single
> > action.
> 
> We are not cloning the template itself we are only cloning the
> template
> disks.
> 
> 
> > For example, if there is enough space on the target domain for 2/4
> > disks then using multipleRunAction would result in 2 disks being
> > copied and 2 failing.
> 
> > If however it is a single action then the free space test would
> > fail the entire action and user would be able to choose if he
> > wants to copy just 2.
> > Note that in this case, actually performing the copy of the 2 disks
> > is detrimental as it can negatively affect VMs on this domain.
> > 
> >>
> >>
> >>
> >>>> what the user really wants is to specify which disks to copy
> >>>> and destination per disk, and i don't see a reason to create a
> >>>> backend
> >>>> command to do it
> >>>>
> >>>>> _______________________________________________
> >>>>> Engine-devel mailing list
> >>>>> Engine-devel at ovirt.org
> >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>>
> >>>
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> 
> 


From lpeer at redhat.com  Wed Jan 18 07:04:59 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 09:04:59 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <910118d7-ed84-4010-8cba-6c2bd54dd207@zmail13.collab.prod.int.phx2.redhat.com>
References: <910118d7-ed84-4010-8cba-6c2bd54dd207@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F166F1B.1010408@redhat.com>

On 18/01/12 02:22, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> On 17/01/12 11:45, Ayal Baron wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> On 17/01/12 10:46, Itamar Heim wrote:
>>>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Itamar Heim"<iheim at redhat.com>
>>>>>>> To: "Jon Choate"<jchoate at redhat.com>
>>>>>>> Cc: engine-devel at ovirt.org
>>>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
>>>>>>> Subject: Re: [Engine-devel] the future of template cloning
>>>>>>>
>>>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>>> We are going to be able to store the disks for a template
>>>>>>>>>>>>> on
>>>>>>>>>>>>> different storage domains due to the multiple storage
>>>>>>>>>>>>> domain
>>>>>>>>>>>>> feature. Cloning a template will still be possible, but
>>>>>>>>>>>>> will
>>>>>>>>>>>>> it
>>>>>>>>>>>>> provide any value? Thoughts?
>>>>>>>>>>>> I see no relation between the two options.
>>>>>>>>>>>>
>>>>>>>>>>>> Scenario 1: I can create a VM with a single disk and
>>>>>>>>>>>> create
>>>>>>>>>>>> a
>>>>>>>>>>>> template from it.
>>>>>>>>>>>> I would still want to be able to clone the template in
>>>>>>>>>>>> order
>>>>>>>>>>>> to
>>>>>>>>>>>> provision VMs from it on different domains.
>>>>>>>>>>>>
>>>>>>>>>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>>>>>>>>>
>>>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>>>>>>> domains
>>>>>>>>>>>> (domain A and domain B) and I want to have another copy of
>>>>>>>>>>>> the
>>>>>>>>>>>> template on domain C and domain D
>>>>>>>>>>>>
>>>>>>>>>>> Hi Jon,
>>>>>>>>>>>
>>>>>>>>>>> After talking to Michael Pasternak it seems that we did not
>>>>>>>>>>> implemented
>>>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
>>>>>>>>>>> have.
>>>>>>>>>>>
>>>>>>>>>>> This gap is playing in our favor, we can remove the
>>>>>>>>>>> copyTemplate
>>>>>>>>>>> verb
>>>>>>>>>>> and introduce copyDisk verb.
>>>>>>>>>>>
>>>>>>>>>>> The template disks can be copied to another SD.
>>>>>>>>>>> When creating a VM from template the user can choose per
>>>>>>>>>>> disk
>>>>>>>>>>> the
>>>>>>>>>>> destination SD (only SD with the disks are eligible
>>>>>>>>>>> candidates).
>>>>>>>>>> wait, when creating a VM from a template, the user won't get
>>>>>>>>>> a
>>>>>>>>>> choice
>>>>>>>>>> will they? Won't the VM disks have to go on the same storage
>>>>>>>>>> domain as
>>>>>>>>>> the template disks they were created from?
>>>>>>>>>
>>>>>>>>> yes, but the template disks can be copied to multiple storage
>>>>>>>>> domains,
>>>>>>>>> so the user can choose for the VM/disk which storage domain
>>>>>>>>> to
>>>>>>>>> create
>>>>>>>>> them from (per storage domains that have copies of that disk)
>>>>>>>> OH! I totally misunderstood. So what you are saying is that a
>>>>>>>> template
>>>>>>>> can have N number of copies of the same disk each on a
>>>>>>>> different
>>>>>>>> storage
>>>>>>>> domain. I had thought that if you wanted that type of
>>>>>>>> situation
>>>>>>>> you
>>>>>>>> would have multiple copies of the template itself too.
>>>>>>
>>>>>> yes, one copy of disk per domain though.
>>>>>>
>>>>>>>>
>>>>>>>> Just to be clear, does this mean that the plan is to phase out
>>>>>>>> the
>>>>>>>> current clone template command and instead implementing a
>>>>>>>> clone
>>>>>>>> disk
>>>>>>>> command so that a template can duplicate its disks
>>>>>>>> individually?
>>>>>>>
>>>>>>> pretty much, yes.
>>>>>>> though i'd imagine 'clone template' would still be useful to
>>>>>>> have
>>>>>>> for
>>>>>>> the user. not sure if it implies core should expose it as well
>>>>>>> to
>>>>>>> allow
>>>>>>> easier usage at UI level for such a task.
>>>>>>
>>>>>> we can leave it untouched - means copyTemplate get 1 destination
>>>>>> domain, and copies all disks to it,
>>>>>> but i think it will be unusable (and problematic - what if one
>>>>>> of
>>>>>> the
>>>>>> disks already exists on the destination?),
>>>>>
>>>>> then don't copy it, it is already there
>>>>>
>>>>
>>>> I agree with Omer, there is no reason to support copy template, if
>>>> the
>>>> user wants to clone all the disks he can use multiple actions, we
>>>> don't
>>>> need a specific verb for this.
>>>
>>> Reason or lack of depends on the common usage.
>>> If we assume that normally all disks of a template would reside on
>>> the same domain then it makes sense to have a verb to copy the
>>> template in its entirety and not burden the user.
>>> The general recommendation should be to use a single storage
>>> domain, so I think there is room for such a verb.
>>>
>>
>>
>>>> If the UI chooses to expose such operation it will use the
>>>> multipleRunAction API which makes it easier to expose to the user
>>>> partial success, we could clone disk A and Disk B but Disk C
>>>> failed
>>>> etc.
>>>
>>> The multipleRunAction is for user marking multiple objects in GUI
>>> and running an action on all of these objects.
>>
>> Exactly, choosing the disks to copy to the storage domain.
> 
> Which is wrong because multipleRunAction assumes there are no dependencies between the actions.  In the case of copyTemplate the user sees copying all disks as a single operation which should either fail or succeed.  Partial results are not clear and can cause other problems.
> Validations in this case should be on the entire set together, not one by one, etc.
> 

I think that using the multi action approach is the better behavior for
the user.
If the user clone a template with 3 disks and the engine was able to
clone only two out of three (got an error on the third copy), As a user
i would rather get a chance to clone the third one again and not
re-clone the other two (it takes time...).

So my vote goes to multi actions.

>>
>>> Here however, the action the user wants is to copy 1 object (the
>>> template) which has sub objects and it should run as a single
>>> action.
>>
>> We are not cloning the template itself we are only cloning the
>> template
>> disks.
>>
>>
>>> For example, if there is enough space on the target domain for 2/4
>>> disks then using multipleRunAction would result in 2 disks being
>>> copied and 2 failing.
>>
>>> If however it is a single action then the free space test would
>>> fail the entire action and user would be able to choose if he
>>> wants to copy just 2.
>>> Note that in this case, actually performing the copy of the 2 disks
>>> is detrimental as it can negatively affect VMs on this domain.
>>>
>>>>
>>>>
>>>>
>>>>>> what the user really wants is to specify which disks to copy
>>>>>> and destination per disk, and i don't see a reason to create a
>>>>>> backend
>>>>>> command to do it
>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Engine-devel mailing list
>>>>>>> Engine-devel at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>
>>



From lpeer at redhat.com  Wed Jan 18 07:16:58 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 09:16:58 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <ef38bd6f-b448-43bc-b3b3-0b4dfda334ef@mkenneth.csb>
References: <ef38bd6f-b448-43bc-b3b3-0b4dfda334ef@mkenneth.csb>
Message-ID: <4F1671EA.3000703@redhat.com>

On 17/01/12 23:00, Miki Kenneth wrote:
> 
> 
> ----- Original Message -----
>> From: "Ayal Baron" <abaron at redhat.com>
>> To: "Livnat Peer" <lpeer at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Tuesday, January 17, 2012 11:45:33 AM
>> Subject: Re: [Engine-devel] the future of template cloning
>>
>>
>>
>> ----- Original Message -----
>>> On 17/01/12 10:46, Itamar Heim wrote:
>>>> On 01/17/2012 10:32 AM, Omer Frenkel wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Itamar Heim"<iheim at redhat.com>
>>>>>> To: "Jon Choate"<jchoate at redhat.com>
>>>>>> Cc: engine-devel at ovirt.org
>>>>>> Sent: Monday, January 16, 2012 7:26:24 PM
>>>>>> Subject: Re: [Engine-devel] the future of template cloning
>>>>>>
>>>>>> On 01/16/2012 06:16 PM, Jon Choate wrote:
>>>>>>> On 01/16/2012 10:58 AM, Itamar Heim wrote:
>>>>>>>> On 01/16/2012 05:46 PM, Jon Choate wrote:
>>>>>>>>> On 01/16/2012 09:46 AM, Livnat Peer wrote:
>>>>>>>>>> On 12/01/12 22:45, Ayal Baron wrote:
>>>>>>>>>>>
>>>>>>>>>>> ----- Original Message -----
>>>>>>>>>>>> We are going to be able to store the disks for a template
>>>>>>>>>>>> on
>>>>>>>>>>>> different storage domains due to the multiple storage
>>>>>>>>>>>> domain
>>>>>>>>>>>> feature. Cloning a template will still be possible, but
>>>>>>>>>>>> will
>>>>>>>>>>>> it
>>>>>>>>>>>> provide any value? Thoughts?
>>>>>>>>>>> I see no relation between the two options.
>>>>>>>>>>>
>>>>>>>>>>> Scenario 1: I can create a VM with a single disk and
>>>>>>>>>>> create
>>>>>>>>>>> a
>>>>>>>>>>> template from it.
>>>>>>>>>>> I would still want to be able to clone the template in
>>>>>>>>>>> order
>>>>>>>>>>> to
>>>>>>>>>>> provision VMs from it on different domains.
>>>>>>>>>>>
>>>>>>>>>>> Scenario 2: same thing with multiple disks on same domain.
>>>>>>>>>>>
>>>>>>>>>>> Scenario 3: I have a template with 2 disks on 2 different
>>>>>>>>>>> domains
>>>>>>>>>>> (domain A and domain B) and I want to have another copy of
>>>>>>>>>>> the
>>>>>>>>>>> template on domain C and domain D
>>>>>>>>>>>
>>>>>>>>>> Hi Jon,
>>>>>>>>>>
>>>>>>>>>> After talking to Michael Pasternak it seems that we did not
>>>>>>>>>> implemented
>>>>>>>>>> copyTemplate in the REST API, it seems to be a gap that we
>>>>>>>>>> have.
>>>>>>>>>>
>>>>>>>>>> This gap is playing in our favor, we can remove the
>>>>>>>>>> copyTemplate
>>>>>>>>>> verb
>>>>>>>>>> and introduce copyDisk verb.
>>>>>>>>>>
>>>>>>>>>> The template disks can be copied to another SD.
>>>>>>>>>> When creating a VM from template the user can choose per
>>>>>>>>>> disk
>>>>>>>>>> the
>>>>>>>>>> destination SD (only SD with the disks are eligible
>>>>>>>>>> candidates).
>>>>>>>>> wait, when creating a VM from a template, the user won't get
>>>>>>>>> a
>>>>>>>>> choice
>>>>>>>>> will they? Won't the VM disks have to go on the same storage
>>>>>>>>> domain as
>>>>>>>>> the template disks they were created from?
>>>>>>>>
>>>>>>>> yes, but the template disks can be copied to multiple storage
>>>>>>>> domains,
>>>>>>>> so the user can choose for the VM/disk which storage domain
>>>>>>>> to
>>>>>>>> create
>>>>>>>> them from (per storage domains that have copies of that disk)
>>>>>>> OH! I totally misunderstood. So what you are saying is that a
>>>>>>> template
>>>>>>> can have N number of copies of the same disk each on a
>>>>>>> different
>>>>>>> storage
>>>>>>> domain. I had thought that if you wanted that type of
>>>>>>> situation
>>>>>>> you
>>>>>>> would have multiple copies of the template itself too.
>>>>>
>>>>> yes, one copy of disk per domain though.
>>>>>
>>>>>>>
>>>>>>> Just to be clear, does this mean that the plan is to phase out
>>>>>>> the
>>>>>>> current clone template command and instead implementing a
>>>>>>> clone
>>>>>>> disk
>>>>>>> command so that a template can duplicate its disks
>>>>>>> individually?
>>>>>>
>>>>>> pretty much, yes.
>>>>>> though i'd imagine 'clone template' would still be useful to
>>>>>> have
>>>>>> for
>>>>>> the user. not sure if it implies core should expose it as well
>>>>>> to
>>>>>> allow
>>>>>> easier usage at UI level for such a task.
>>>>>
>>>>> we can leave it untouched - means copyTemplate get 1 destination
>>>>> domain, and copies all disks to it,
>>>>> but i think it will be unusable (and problematic - what if one
>>>>> of
>>>>> the
>>>>> disks already exists on the destination?),
>>>>
>>>> then don't copy it, it is already there
>>>>
>>>
>>> I agree with Omer, there is no reason to support copy template, if
>>> the
>>> user wants to clone all the disks he can use multiple actions, we
>>> don't
>>> need a specific verb for this.
>>
>> Reason or lack of depends on the common usage.
>> If we assume that normally all disks of a template would reside on
>> the same domain then it makes sense to have a verb to copy the
>> template in its entirety and not burden the user.
>> The general recommendation should be to use a single storage domain,
>> so I think there is room for such a verb.

> I agree. This is the common use case, all disk resides on the same SD.
> and that is why we need a verb for it.
> However, for more "trick"/special cases we need to support multi domains.
> I also think it would be easier from api perspective to use a single copy verb.

The question is what would we like to be the behavior when the operation
fails, the different approaches are mostly around error flows:

- If the clone template is a single action, the error handling would be
to roll back (if the engine failed copying all the images, the images
that were copied successfully to the destination will be deleted).

- If we use the multiple action approach it means that each image that
was copied successfully will stay there and be presented to the user as
success and for those we failed to copy the user will get an error message.





From lpeer at redhat.com  Wed Jan 18 07:38:35 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 09:38:35 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F15FA61.4050307@redhat.com>
References: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>
	<4F159878.7000501@redhat.com> <4F159C1A.20209@redhat.com>
	<4F15A342.20802@redhat.com> <4F15FA61.4050307@redhat.com>
Message-ID: <4F1676FB.9090405@redhat.com>

On 18/01/12 00:46, Itamar Heim wrote:
> On 01/17/2012 06:35 PM, Livnat Peer wrote:
> ...
> 
>>> We can't remove cloneTemplate until it is removed from the UI or else we
>>> will break functionality.  For now we are just going to ensure that it
>>> works as it always has until the UI is ready to remove it.
>>>
>>
>> If by ensure you mean you are going to do adjustments for the
>> cloneTemplate code to work then it is not needed.
>>
>> You can either send a patch to disable/remove this button from the UI
>> or synchronize your patch with a patch from the UI that removes this
>> button.
>>
>> I don't like the approach of leaving the code because many times we end
>> up with a code that is not used and not maintained.
> 
> I don't think a patch which would make existing functionality disabled
> is the way to go.
> I expect a lot of times a new functionality will be added, and the old
> one should be flagged deprecated with a comment as to why (or open a BZ
> to self to clean it up), 

I don't expect that we'll remove functionality a lot of times, this was
a gap in the API modeling that enabled us doing that.

If i understand correctly then you expect to add the clone disk
functionality to the UI before removing the clone template button.
Let's assume this requires a rewrite of the command to work with the new
DB schema, this would be a throw away code, should it be done? I think not.

We have upstream releases which should be stable - preferably with no
partial functionality, other than that there is a development going on
in upstream and from time to time some of the functionality won't be
fully ready, it does not mean the application is broken but a disabled
button in the UI or something similar can be a valid state IMO.



or collaborate with someone more knowledgeable
> on the specific component to make the change in sync, or do the patch
> yourself.


> to make less code deprecated, the old api could wrap the new
> functionality (call removeDisk in a loop, etc.).
> it should still be flagged as something to clean up once the api calling
> it is removed.
> obviously, collaborating (or doing it thyself) on a sync'd change is
> easiest, but not always possible.



From lpeer at redhat.com  Wed Jan 18 07:49:25 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 09:49:25 +0200
Subject: [Engine-devel] agenda for today's engine core meeting
Message-ID: <4F167985.3050601@redhat.com>

Hi All,

The agenda for today -

1. Close the clone template discussion.
2. Introduce the new setup networks API.
3. Short summary of the stable device address (managed and unmanaged
devices)

----If we have time ------

4. Discussing the clone VM from snapshot functionality.



Livnat



From iheim at redhat.com  Wed Jan 18 07:52:29 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 18 Jan 2012 09:52:29 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F1676FB.9090405@redhat.com>
References: <cb97f81f-3bdf-4f94-856d-51d86b4b70b4@zmail07.collab.prod.int.phx2.redhat.com>
	<4F159878.7000501@redhat.com> <4F159C1A.20209@redhat.com>
	<4F15A342.20802@redhat.com> <4F15FA61.4050307@redhat.com>
	<4F1676FB.9090405@redhat.com>
Message-ID: <4F167A3D.6040809@redhat.com>

On 01/18/2012 09:38 AM, Livnat Peer wrote:
> On 18/01/12 00:46, Itamar Heim wrote:
>> On 01/17/2012 06:35 PM, Livnat Peer wrote:
>> ...
>>
>>>> We can't remove cloneTemplate until it is removed from the UI or else we
>>>> will break functionality.  For now we are just going to ensure that it
>>>> works as it always has until the UI is ready to remove it.
>>>>
>>>
>>> If by ensure you mean you are going to do adjustments for the
>>> cloneTemplate code to work then it is not needed.
>>>
>>> You can either send a patch to disable/remove this button from the UI
>>> or synchronize your patch with a patch from the UI that removes this
>>> button.
>>>
>>> I don't like the approach of leaving the code because many times we end
>>> up with a code that is not used and not maintained.
>>
>> I don't think a patch which would make existing functionality disabled
>> is the way to go.
>> I expect a lot of times a new functionality will be added, and the old
>> one should be flagged deprecated with a comment as to why (or open a BZ
>> to self to clean it up),
>
> I don't expect that we'll remove functionality a lot of times, this was
> a gap in the API modeling that enabled us doing that.
>
> If i understand correctly then you expect to add the clone disk
> functionality to the UI before removing the clone template button.
> Let's assume this requires a rewrite of the command to work with the new
> DB schema, this would be a throw away code, should it be done? I think not.
>
> We have upstream releases which should be stable - preferably with no
> partial functionality, other than that there is a development going on
> in upstream and from time to time some of the functionality won't be
> fully ready, it does not mean the application is broken but a disabled
> button in the UI or something similar can be a valid state IMO.

I think the assumption is someone should always be able to take upstream 
HEAD and build/base a version on it, other than the formal versions.


From ofrenkel at redhat.com  Wed Jan 18 07:56:02 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Wed, 18 Jan 2012 02:56:02 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export (was:
	Re: move disk command)
In-Reply-To: <e1dcafd9-1eb0-4743-847c-adbc5dad39e6@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <23812bcd-b7a7-4520-b0d6-7d511a18ddb1@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Maor" <mlipchuk at redhat.com>
> Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>, "Omer Frenkel" <ofrenkel at redhat.com>
> Sent: Tuesday, January 17, 2012 10:21:29 PM
> Subject: Re: Shared disk / Floating disk import/export (was: Re: [Engine-devel] move disk command)
> 
> 
> 
> ----- Original Message -----
> > On 01/17/2012 11:57 AM, Omer Frenkel wrote:
> > > 
> > > 
> > > ----- Original Message -----
> > >> From: "Ayal Baron" <abaron at redhat.com>
> > >> To: "Omer Frenkel" <ofrenkel at redhat.com>
> > >> Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>
> > >> Sent: Tuesday, January 17, 2012 10:45:53 AM
> > >> Subject: Shared disk / Floating disk import/export (was: Re:
> > >> [Engine-devel] move disk command)
> > >>
> > >>
> > >> [SNIP]
> > >>
> > >>>>> This may be unrelated, but user would be allowed to export
> > >>>>> and
> > >>>>> import a floating disk, right?
> > >>>>> I would like the ability to import *any* disk in the export
> > >>>>> domain
> > >>>>> as a floating disk, but in the least, export and import disks
> > >>>>> not
> > >>>>> associated with a VM.
> > >>>
> > >>> you are right, it is unrelated, this thread is about move disk
> > >>> of
> > >>> a
> > >>> vm between SDs,
> > >>> export and import is copy, and floating disks is part of the
> > >>> shared
> > >>> disk feature,
> > >>> this indeed need to be discussed in that scope.
> > >>
> > >> Ok, so I've changed the subject, now let's discuss it.
> > >>
> > > 
> > > Adding Maor,
> > > not sure if we have any plan regard export/import of floating
> > > disk,
> > > or in general, exporting disk without it's vm (if I understood
> > > Ayal
> > > correctly)
> > > 
> > > Maor, any comments?
> > I remember, we mentioned export/import issue in our last meeting
> > with
> > Andrew in the context of shared raw disk.
> > It was decided then, that export/import will not be supported by
> > shared
> > raw disk, I'm not sure if we also decided that for floating disk,
> > but I think its a PM decision.
> > 
> > Support import/export domain might evolve to a big change, since if
> > we
> > want to export disk, we might also want to reflect all its
> > configuration
> > and functionality there, also reflect disks which are attached to
> > VMs
> > and templates as well.
> 
> What properties does a disk have?
> Interface is what type of connection is used when plugging the disk
> in the computer (VM) so it's not really a disk property.
> Address is also VM specific
> Description is already stored in storage side.
> What else do you have for disk?

type (system/data..)
post-zero
name(?)
creation-date
last modify date
application-list
volume type and format

> 
> Note that being able to import floating disks means that we'd be able
> to take any data storage domain and import any disk(s) from it
> (assuming user changes domain type to export domain, which users
> already know how to do)
> 
> 


From abaron at redhat.com  Wed Jan 18 08:20:01 2012
From: abaron at redhat.com (Ayal Baron)
Date: Wed, 18 Jan 2012 03:20:01 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export (was:
	Re: move disk command)
In-Reply-To: <23812bcd-b7a7-4520-b0d6-7d511a18ddb1@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <d2fdfadc-a2f3-4397-863f-4907c99f3c49@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Ayal Baron" <abaron at redhat.com>
> > To: "Maor" <mlipchuk at redhat.com>
> > Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>,
> > "Omer Frenkel" <ofrenkel at redhat.com>
> > Sent: Tuesday, January 17, 2012 10:21:29 PM
> > Subject: Re: Shared disk / Floating disk import/export (was: Re:
> > [Engine-devel] move disk command)
> > 
> > 
> > 
> > ----- Original Message -----
> > > On 01/17/2012 11:57 AM, Omer Frenkel wrote:
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > >> From: "Ayal Baron" <abaron at redhat.com>
> > > >> To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > >> Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>
> > > >> Sent: Tuesday, January 17, 2012 10:45:53 AM
> > > >> Subject: Shared disk / Floating disk import/export (was: Re:
> > > >> [Engine-devel] move disk command)
> > > >>
> > > >>
> > > >> [SNIP]
> > > >>
> > > >>>>> This may be unrelated, but user would be allowed to export
> > > >>>>> and
> > > >>>>> import a floating disk, right?
> > > >>>>> I would like the ability to import *any* disk in the export
> > > >>>>> domain
> > > >>>>> as a floating disk, but in the least, export and import
> > > >>>>> disks
> > > >>>>> not
> > > >>>>> associated with a VM.
> > > >>>
> > > >>> you are right, it is unrelated, this thread is about move
> > > >>> disk
> > > >>> of
> > > >>> a
> > > >>> vm between SDs,
> > > >>> export and import is copy, and floating disks is part of the
> > > >>> shared
> > > >>> disk feature,
> > > >>> this indeed need to be discussed in that scope.
> > > >>
> > > >> Ok, so I've changed the subject, now let's discuss it.
> > > >>
> > > > 
> > > > Adding Maor,
> > > > not sure if we have any plan regard export/import of floating
> > > > disk,
> > > > or in general, exporting disk without it's vm (if I understood
> > > > Ayal
> > > > correctly)
> > > > 
> > > > Maor, any comments?
> > > I remember, we mentioned export/import issue in our last meeting
> > > with
> > > Andrew in the context of shared raw disk.
> > > It was decided then, that export/import will not be supported by
> > > shared
> > > raw disk, I'm not sure if we also decided that for floating disk,
> > > but I think its a PM decision.
> > > 
> > > Support import/export domain might evolve to a big change, since
> > > if
> > > we
> > > want to export disk, we might also want to reflect all its
> > > configuration
> > > and functionality there, also reflect disks which are attached to
> > > VMs
> > > and templates as well.
> > 
> > What properties does a disk have?
> > Interface is what type of connection is used when plugging the disk
> > in the computer (VM) so it's not really a disk property.
> > Address is also VM specific
> > Description is already stored in storage side.
> > What else do you have for disk?
> 
> type (system/data..)

Didn't we recently have a discussion about this being irrelevant?

> post-zero

Can assume always true

> name(?)

This exists on storage side

> creation-date

When imported

> last modify date

This exists on storage side

> application-list

Has nothing to do with disk, it's a VM property

> volume type and format

This exists on storage side

> 
> > 
> > Note that being able to import floating disks means that we'd be
> > able
> > to take any data storage domain and import any disk(s) from it
> > (assuming user changes domain type to export domain, which users
> > already know how to do)
> > 
> > 
> 


From ofrenkel at redhat.com  Wed Jan 18 08:24:34 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Wed, 18 Jan 2012 03:24:34 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export (was:
	Re: move disk command)
In-Reply-To: <d2fdfadc-a2f3-4397-863f-4907c99f3c49@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <34b7b2b1-9a7f-471e-ae39-69cf38e29ef4@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Omer Frenkel" <ofrenkel at redhat.com>
> Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>, "Maor" <mlipchuk at redhat.com>
> Sent: Wednesday, January 18, 2012 10:20:01 AM
> Subject: Re: Shared disk / Floating disk import/export (was: Re: [Engine-devel] move disk command)
> 
> 
> 
> ----- Original Message -----
> > 
> > 
> > ----- Original Message -----
> > > From: "Ayal Baron" <abaron at redhat.com>
> > > To: "Maor" <mlipchuk at redhat.com>
> > > Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>,
> > > "Omer Frenkel" <ofrenkel at redhat.com>
> > > Sent: Tuesday, January 17, 2012 10:21:29 PM
> > > Subject: Re: Shared disk / Floating disk import/export (was: Re:
> > > [Engine-devel] move disk command)
> > > 
> > > 
> > > 
> > > ----- Original Message -----
> > > > On 01/17/2012 11:57 AM, Omer Frenkel wrote:
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > >> From: "Ayal Baron" <abaron at redhat.com>
> > > > >> To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > >> Cc: engine-devel at ovirt.org, "Jon Choate"
> > > > >> <jchoate at redhat.com>
> > > > >> Sent: Tuesday, January 17, 2012 10:45:53 AM
> > > > >> Subject: Shared disk / Floating disk import/export (was: Re:
> > > > >> [Engine-devel] move disk command)
> > > > >>
> > > > >>
> > > > >> [SNIP]
> > > > >>
> > > > >>>>> This may be unrelated, but user would be allowed to
> > > > >>>>> export
> > > > >>>>> and
> > > > >>>>> import a floating disk, right?
> > > > >>>>> I would like the ability to import *any* disk in the
> > > > >>>>> export
> > > > >>>>> domain
> > > > >>>>> as a floating disk, but in the least, export and import
> > > > >>>>> disks
> > > > >>>>> not
> > > > >>>>> associated with a VM.
> > > > >>>
> > > > >>> you are right, it is unrelated, this thread is about move
> > > > >>> disk
> > > > >>> of
> > > > >>> a
> > > > >>> vm between SDs,
> > > > >>> export and import is copy, and floating disks is part of
> > > > >>> the
> > > > >>> shared
> > > > >>> disk feature,
> > > > >>> this indeed need to be discussed in that scope.
> > > > >>
> > > > >> Ok, so I've changed the subject, now let's discuss it.
> > > > >>
> > > > > 
> > > > > Adding Maor,
> > > > > not sure if we have any plan regard export/import of floating
> > > > > disk,
> > > > > or in general, exporting disk without it's vm (if I
> > > > > understood
> > > > > Ayal
> > > > > correctly)
> > > > > 
> > > > > Maor, any comments?
> > > > I remember, we mentioned export/import issue in our last
> > > > meeting
> > > > with
> > > > Andrew in the context of shared raw disk.
> > > > It was decided then, that export/import will not be supported
> > > > by
> > > > shared
> > > > raw disk, I'm not sure if we also decided that for floating
> > > > disk,
> > > > but I think its a PM decision.
> > > > 
> > > > Support import/export domain might evolve to a big change,
> > > > since
> > > > if
> > > > we
> > > > want to export disk, we might also want to reflect all its
> > > > configuration
> > > > and functionality there, also reflect disks which are attached
> > > > to
> > > > VMs
> > > > and templates as well.
> > > 
> > > What properties does a disk have?
> > > Interface is what type of connection is used when plugging the
> > > disk
> > > in the computer (VM) so it's not really a disk property.
> > > Address is also VM specific
> > > Description is already stored in storage side.
> > > What else do you have for disk?
> > 
> > type (system/data..)
> 
> Didn't we recently have a discussion about this being irrelevant?

as long it is visible to the user, no.

> 
> > post-zero
> 
> Can assume always true

i disagree

> 
> > name(?)
> 
> This exists on storage side

in addition to description?

> 
> > creation-date
> 
> When imported

no!

> 
> > last modify date
> 
> This exists on storage side
> 
> > application-list
> 
> Has nothing to do with disk, it's a VM property

no!

> 
> > volume type and format
> 
> This exists on storage side
> 
> > 
> > > 
> > > Note that being able to import floating disks means that we'd be
> > > able
> > > to take any data storage domain and import any disk(s) from it
> > > (assuming user changes domain type to export domain, which users
> > > already know how to do)
> > > 
> > > 
> > 
> 


From abaron at redhat.com  Wed Jan 18 08:26:18 2012
From: abaron at redhat.com (Ayal Baron)
Date: Wed, 18 Jan 2012 03:26:18 -0500 (EST)
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <4F166F1B.1010408@redhat.com>
Message-ID: <a968c256-fae0-4cc4-bbf5-f53207997bfb@zmail13.collab.prod.int.phx2.redhat.com>

<SNIP>

> >>>>>>>>
> >>>>>>>> Just to be clear, does this mean that the plan is to phase
> >>>>>>>> out
> >>>>>>>> the
> >>>>>>>> current clone template command and instead implementing a
> >>>>>>>> clone
> >>>>>>>> disk
> >>>>>>>> command so that a template can duplicate its disks
> >>>>>>>> individually?
> >>>>>>>
> >>>>>>> pretty much, yes.
> >>>>>>> though i'd imagine 'clone template' would still be useful to
> >>>>>>> have
> >>>>>>> for
> >>>>>>> the user. not sure if it implies core should expose it as
> >>>>>>> well
> >>>>>>> to
> >>>>>>> allow
> >>>>>>> easier usage at UI level for such a task.
> >>>>>>
> >>>>>> we can leave it untouched - means copyTemplate get 1
> >>>>>> destination
> >>>>>> domain, and copies all disks to it,
> >>>>>> but i think it will be unusable (and problematic - what if one
> >>>>>> of
> >>>>>> the
> >>>>>> disks already exists on the destination?),
> >>>>>
> >>>>> then don't copy it, it is already there
> >>>>>
> >>>>
> >>>> I agree with Omer, there is no reason to support copy template,
> >>>> if
> >>>> the
> >>>> user wants to clone all the disks he can use multiple actions,
> >>>> we
> >>>> don't
> >>>> need a specific verb for this.
> >>>
> >>> Reason or lack of depends on the common usage.
> >>> If we assume that normally all disks of a template would reside
> >>> on
> >>> the same domain then it makes sense to have a verb to copy the
> >>> template in its entirety and not burden the user.
> >>> The general recommendation should be to use a single storage
> >>> domain, so I think there is room for such a verb.
> >>>
> >>
> >>
> >>>> If the UI chooses to expose such operation it will use the
> >>>> multipleRunAction API which makes it easier to expose to the
> >>>> user
> >>>> partial success, we could clone disk A and Disk B but Disk C
> >>>> failed
> >>>> etc.
> >>>
> >>> The multipleRunAction is for user marking multiple objects in GUI
> >>> and running an action on all of these objects.
> >>
> >> Exactly, choosing the disks to copy to the storage domain.
> > 
> > Which is wrong because multipleRunAction assumes there are no
> > dependencies between the actions.  In the case of copyTemplate the
> > user sees copying all disks as a single operation which should
> > either fail or succeed.  Partial results are not clear and can
> > cause other problems.
> > Validations in this case should be on the entire set together, not
> > one by one, etc.
> > 
> 
> I think that using the multi action approach is the better behavior
> for
> the user.
> If the user clone a template with 3 disks and the engine was able to
> clone only two out of three (got an error on the third copy), As a
> user
> i would rather get a chance to clone the third one again and not
> re-clone the other two (it takes time...).
> 
> So my vote goes to multi actions.
> 

You're missing the point, if it's a single action then the canDoAction would validate free space for all disks together and wouldn't copy a single byte before failing.

On the other hand, I do agree that if canDoAction succeeds and then one of the copies fails then you have to rollback which is not nice.
If you can do an aggregated canDoAction for copying multiple disks using the multipleRunAction (i.e. calculate space needed for *all* disks on each target domain and warn user before starting or fail) and GUI-wise the operation is done on the disks themselves and not on the template then I'm fine with that.


<SNIP>


From abaron at redhat.com  Wed Jan 18 08:32:39 2012
From: abaron at redhat.com (Ayal Baron)
Date: Wed, 18 Jan 2012 03:32:39 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export (was:
	Re: move disk command)
In-Reply-To: <34b7b2b1-9a7f-471e-ae39-69cf38e29ef4@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <1339c6ca-cf7a-4a58-89c6-0a289456c5ac@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Ayal Baron" <abaron at redhat.com>
> > To: "Omer Frenkel" <ofrenkel at redhat.com>
> > Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>,
> > "Maor" <mlipchuk at redhat.com>
> > Sent: Wednesday, January 18, 2012 10:20:01 AM
> > Subject: Re: Shared disk / Floating disk import/export (was: Re:
> > [Engine-devel] move disk command)
> > 
> > 
> > 
> > ----- Original Message -----
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Ayal Baron" <abaron at redhat.com>
> > > > To: "Maor" <mlipchuk at redhat.com>
> > > > Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>,
> > > > "Omer Frenkel" <ofrenkel at redhat.com>
> > > > Sent: Tuesday, January 17, 2012 10:21:29 PM
> > > > Subject: Re: Shared disk / Floating disk import/export (was:
> > > > Re:
> > > > [Engine-devel] move disk command)
> > > > 
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > On 01/17/2012 11:57 AM, Omer Frenkel wrote:
> > > > > > 
> > > > > > 
> > > > > > ----- Original Message -----
> > > > > >> From: "Ayal Baron" <abaron at redhat.com>
> > > > > >> To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > > >> Cc: engine-devel at ovirt.org, "Jon Choate"
> > > > > >> <jchoate at redhat.com>
> > > > > >> Sent: Tuesday, January 17, 2012 10:45:53 AM
> > > > > >> Subject: Shared disk / Floating disk import/export (was:
> > > > > >> Re:
> > > > > >> [Engine-devel] move disk command)
> > > > > >>
> > > > > >>
> > > > > >> [SNIP]
> > > > > >>
> > > > > >>>>> This may be unrelated, but user would be allowed to
> > > > > >>>>> export
> > > > > >>>>> and
> > > > > >>>>> import a floating disk, right?
> > > > > >>>>> I would like the ability to import *any* disk in the
> > > > > >>>>> export
> > > > > >>>>> domain
> > > > > >>>>> as a floating disk, but in the least, export and import
> > > > > >>>>> disks
> > > > > >>>>> not
> > > > > >>>>> associated with a VM.
> > > > > >>>
> > > > > >>> you are right, it is unrelated, this thread is about move
> > > > > >>> disk
> > > > > >>> of
> > > > > >>> a
> > > > > >>> vm between SDs,
> > > > > >>> export and import is copy, and floating disks is part of
> > > > > >>> the
> > > > > >>> shared
> > > > > >>> disk feature,
> > > > > >>> this indeed need to be discussed in that scope.
> > > > > >>
> > > > > >> Ok, so I've changed the subject, now let's discuss it.
> > > > > >>
> > > > > > 
> > > > > > Adding Maor,
> > > > > > not sure if we have any plan regard export/import of
> > > > > > floating
> > > > > > disk,
> > > > > > or in general, exporting disk without it's vm (if I
> > > > > > understood
> > > > > > Ayal
> > > > > > correctly)
> > > > > > 
> > > > > > Maor, any comments?
> > > > > I remember, we mentioned export/import issue in our last
> > > > > meeting
> > > > > with
> > > > > Andrew in the context of shared raw disk.
> > > > > It was decided then, that export/import will not be supported
> > > > > by
> > > > > shared
> > > > > raw disk, I'm not sure if we also decided that for floating
> > > > > disk,
> > > > > but I think its a PM decision.
> > > > > 
> > > > > Support import/export domain might evolve to a big change,
> > > > > since
> > > > > if
> > > > > we
> > > > > want to export disk, we might also want to reflect all its
> > > > > configuration
> > > > > and functionality there, also reflect disks which are
> > > > > attached
> > > > > to
> > > > > VMs
> > > > > and templates as well.
> > > > 
> > > > What properties does a disk have?
> > > > Interface is what type of connection is used when plugging the
> > > > disk
> > > > in the computer (VM) so it's not really a disk property.
> > > > Address is also VM specific
> > > > Description is already stored in storage side.
> > > > What else do you have for disk?
> > > 
> > > type (system/data..)
> > 
> > Didn't we recently have a discussion about this being irrelevant?
> 
> as long it is visible to the user, no.

It's meaningless.

> 
> > 
> > > post-zero
> > 
> > Can assume always true
> 
> i disagree

It's the default behaviour for files, no reason to not allow users to import disk because of this.

> 
> > 
> > > name(?)
> > 
> > This exists on storage side
> 
> in addition to description?

there is uuid and description, do you store in addition another name?

> 
> > 
> > > creation-date
> > 
> > When imported
> 
> no!

Hmmm, I love how elaborate your explanations are.
It's common with files, no reason to treat differently.

> 
> > 
> > > last modify date
> > 
> > This exists on storage side
> > 
> > > application-list
> > 
> > Has nothing to do with disk, it's a VM property
> 
> no!

Again, very elaborate...
VM has 3 disks, guest reports app-list for the VM, how do you determine which disk has which apps?
If this is currently a disk property in engine then it's a bug.

> 
> > 
> > > volume type and format
> > 
> > This exists on storage side
> > 
> > > 
> > > > 
> > > > Note that being able to import floating disks means that we'd
> > > > be
> > > > able
> > > > to take any data storage domain and import any disk(s) from it
> > > > (assuming user changes domain type to export domain, which
> > > > users
> > > > already know how to do)
> > > > 
> > > > 
> > > 
> > 
> 


From yzaslavs at redhat.com  Wed Jan 18 08:53:03 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Wed, 18 Jan 2012 10:53:03 +0200
Subject: [Engine-devel] CloneVMFromSnapshot - feature document
Message-ID: <4F16886F.1050005@redhat.com>

Hi all,
You can look at http://ovirt.org/wiki/Features/CloneVmFromSnapshot

And see the feature document for Clone VM from Snapshot.

Comments are more than welcome (planning on providing a the detailed
document soon)

Kind regards,
Yair


From mkolesni at redhat.com  Wed Jan 18 09:26:54 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Wed, 18 Jan 2012 04:26:54 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export
	(was:	Re: move disk command)
In-Reply-To: <1339c6ca-cf7a-4a58-89c6-0a289456c5ac@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <e1dea435-3f02-4f66-a72e-64600e39f65c@zmail14.collab.prod.int.phx2.redhat.com>


----- Original Message -----
> 
> 
> ----- Original Message -----
> > 
> > 
> > ----- Original Message -----
> > > From: "Ayal Baron" <abaron at redhat.com>
> > > To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>,
> > > "Maor" <mlipchuk at redhat.com>
> > > Sent: Wednesday, January 18, 2012 10:20:01 AM
> > > Subject: Re: Shared disk / Floating disk import/export (was: Re:
> > > [Engine-devel] move disk command)
> > > 
> > > 
> > > 
> > > ----- Original Message -----
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > From: "Ayal Baron" <abaron at redhat.com>
> > > > > To: "Maor" <mlipchuk at redhat.com>
> > > > > Cc: engine-devel at ovirt.org, "Jon Choate"
> > > > > <jchoate at redhat.com>,
> > > > > "Omer Frenkel" <ofrenkel at redhat.com>
> > > > > Sent: Tuesday, January 17, 2012 10:21:29 PM
> > > > > Subject: Re: Shared disk / Floating disk import/export (was:
> > > > > Re:
> > > > > [Engine-devel] move disk command)
> > > > > 
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > > > On 01/17/2012 11:57 AM, Omer Frenkel wrote:
> > > > > > > 
> > > > > > > 
> > > > > > > ----- Original Message -----
> > > > > > >> From: "Ayal Baron" <abaron at redhat.com>
> > > > > > >> To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > > > >> Cc: engine-devel at ovirt.org, "Jon Choate"
> > > > > > >> <jchoate at redhat.com>
> > > > > > >> Sent: Tuesday, January 17, 2012 10:45:53 AM
> > > > > > >> Subject: Shared disk / Floating disk import/export (was:
> > > > > > >> Re:
> > > > > > >> [Engine-devel] move disk command)
> > > > > > >>
> > > > > > >>
> > > > > > >> [SNIP]
> > > > > > >>
> > > > > > >>>>> This may be unrelated, but user would be allowed to
> > > > > > >>>>> export
> > > > > > >>>>> and
> > > > > > >>>>> import a floating disk, right?
> > > > > > >>>>> I would like the ability to import *any* disk in the
> > > > > > >>>>> export
> > > > > > >>>>> domain
> > > > > > >>>>> as a floating disk, but in the least, export and
> > > > > > >>>>> import
> > > > > > >>>>> disks
> > > > > > >>>>> not
> > > > > > >>>>> associated with a VM.
> > > > > > >>>
> > > > > > >>> you are right, it is unrelated, this thread is about
> > > > > > >>> move
> > > > > > >>> disk
> > > > > > >>> of
> > > > > > >>> a
> > > > > > >>> vm between SDs,
> > > > > > >>> export and import is copy, and floating disks is part
> > > > > > >>> of
> > > > > > >>> the
> > > > > > >>> shared
> > > > > > >>> disk feature,
> > > > > > >>> this indeed need to be discussed in that scope.
> > > > > > >>
> > > > > > >> Ok, so I've changed the subject, now let's discuss it.
> > > > > > >>
> > > > > > > 
> > > > > > > Adding Maor,
> > > > > > > not sure if we have any plan regard export/import of
> > > > > > > floating
> > > > > > > disk,
> > > > > > > or in general, exporting disk without it's vm (if I
> > > > > > > understood
> > > > > > > Ayal
> > > > > > > correctly)
> > > > > > > 
> > > > > > > Maor, any comments?
> > > > > > I remember, we mentioned export/import issue in our last
> > > > > > meeting
> > > > > > with
> > > > > > Andrew in the context of shared raw disk.
> > > > > > It was decided then, that export/import will not be
> > > > > > supported
> > > > > > by
> > > > > > shared
> > > > > > raw disk, I'm not sure if we also decided that for floating
> > > > > > disk,
> > > > > > but I think its a PM decision.
> > > > > > 
> > > > > > Support import/export domain might evolve to a big change,
> > > > > > since
> > > > > > if
> > > > > > we
> > > > > > want to export disk, we might also want to reflect all its
> > > > > > configuration
> > > > > > and functionality there, also reflect disks which are
> > > > > > attached
> > > > > > to
> > > > > > VMs
> > > > > > and templates as well.
> > > > > 
> > > > > What properties does a disk have?
> > > > > Interface is what type of connection is used when plugging
> > > > > the
> > > > > disk
> > > > > in the computer (VM) so it's not really a disk property.
> > > > > Address is also VM specific
> > > > > Description is already stored in storage side.
> > > > > What else do you have for disk?
> > > > 
> > > > type (system/data..)
> > > 
> > > Didn't we recently have a discussion about this being irrelevant?
> > 
> > as long it is visible to the user, no.
> 
> It's meaningless.
> 
> > 
> > > 
> > > > post-zero
> > > 
> > > Can assume always true
> > 
> > i disagree
> 
> It's the default behaviour for files, no reason to not allow users to
> import disk because of this.
> 
> > 
> > > 
> > > > name(?)
> > > 
> > > This exists on storage side
> > 
> > in addition to description?
> 
> there is uuid and description, do you store in addition another name?
> 
> > 
> > > 
> > > > creation-date
> > > 
> > > When imported
> > 
> > no!
> 
> Hmmm, I love how elaborate your explanations are.
> It's common with files, no reason to treat differently.
> 
> > 
> > > 
> > > > last modify date
> > > 
> > > This exists on storage side
> > > 
> > > > application-list
> > > 
> > > Has nothing to do with disk, it's a VM property
> > 
> > no!
> 
> Again, very elaborate...
> VM has 3 disks, guest reports app-list for the VM, how do you
> determine which disk has which apps?
> If this is currently a disk property in engine then it's a bug.

I think this is a snapshot's property.

> 
> > 
> > > 
> > > > volume type and format
> > > 
> > > This exists on storage side
> > > 
> > > > 
> > > > > 
> > > > > Note that being able to import floating disks means that we'd
> > > > > be
> > > > > able
> > > > > to take any data storage domain and import any disk(s) from
> > > > > it
> > > > > (assuming user changes domain type to export domain, which
> > > > > users
> > > > > already know how to do)
> > > > > 
> > > > > 
> > > > 
> > > 
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From juan.hernandez at redhat.com  Wed Jan 18 09:43:34 2012
From: juan.hernandez at redhat.com (Juan Hernandez)
Date: Wed, 18 Jan 2012 10:43:34 +0100
Subject: [Engine-devel] Any way to get more than 100 VMs with the REST API?
Message-ID: <4F169446.2010305@redhat.com>

Hello,

Is there any way to get more than 100 VMs (or whatever the value of the
option "SearchResultsLimit" is) using the REST API?

Thanks in advance,
Juan Hernandez


From danken at redhat.com  Wed Jan 18 10:44:23 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Wed, 18 Jan 2012 12:44:23 +0200
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <4F15B3E9.8040506@redhat.com>
References: <20120108130813.GC24375@redhat.com>
	<20120117140823.GE20867@redhat.com> <4F15B3E9.8040506@redhat.com>
Message-ID: <20120118104422.GA30396@redhat.com>

On Tue, Jan 17, 2012 at 07:46:17PM +0200, Livnat Peer wrote:
> On 17/01/12 16:08, Dan Kenigsberg wrote:
> > On Sun, Jan 08, 2012 at 03:08:13PM +0200, Dan Kenigsberg wrote:
> >> Hi Lists,
> >>
> >> One cannot create a PV on a partitioned device, and therefor such
> >> devices where not reported to Engine. This proved surprising to users
> >> who woder where their LUN disappeared.
> >>
> >> Vdsm should report all devices, and ovirt-engine should mark partitioned
> >> devices as unworthy of a PV. In the future, Vdsm may allow to forcefully
> >> remove a partition table from a device, to make it usable as a PV.
> >>
> >> Douglas (CCed) would take resposibilty on the Vdsm side. Initial patch at
> >> http://gerrit.ovirt.org/944 sports a backword-compatible API. Who's taking this
> >> on Engine? This involves GUI, too, as partitioned devices should probably be
> >> displayed greyed-out.
> > 
> > Livnat, is there any taker for this on your side?
> > 
> > Since the change is backward-compatible, I would push it if there are no
> > objections.
> > 
> > Dan.
> 
> Hi Danken,
> 
> How did you mark a partitioned device? (How will the engine will spot
> these LUNs)

Each device has a field 'partitioned'. Currently, Engine ignores it (as it is
always False). With my little patch, devices with partitioned=True would be
reported, too.

Dan.


From lpeer at redhat.com  Wed Jan 18 11:05:42 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 13:05:42 +0200
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <20120118104422.GA30396@redhat.com>
References: <20120108130813.GC24375@redhat.com>
	<20120117140823.GE20867@redhat.com> <4F15B3E9.8040506@redhat.com>
	<20120118104422.GA30396@redhat.com>
Message-ID: <4F16A786.20800@redhat.com>

On 18/01/12 12:44, Dan Kenigsberg wrote:
> On Tue, Jan 17, 2012 at 07:46:17PM +0200, Livnat Peer wrote:
>> On 17/01/12 16:08, Dan Kenigsberg wrote:
>>> On Sun, Jan 08, 2012 at 03:08:13PM +0200, Dan Kenigsberg wrote:
>>>> Hi Lists,
>>>>
>>>> One cannot create a PV on a partitioned device, and therefor such
>>>> devices where not reported to Engine. This proved surprising to users
>>>> who woder where their LUN disappeared.
>>>>
>>>> Vdsm should report all devices, and ovirt-engine should mark partitioned
>>>> devices as unworthy of a PV. In the future, Vdsm may allow to forcefully
>>>> remove a partition table from a device, to make it usable as a PV.
>>>>
>>>> Douglas (CCed) would take resposibilty on the Vdsm side. Initial patch at
>>>> http://gerrit.ovirt.org/944 sports a backword-compatible API. Who's taking this
>>>> on Engine? This involves GUI, too, as partitioned devices should probably be
>>>> displayed greyed-out.
>>>
>>> Livnat, is there any taker for this on your side?
>>>
>>> Since the change is backward-compatible, I would push it if there are no
>>> objections.
>>>
>>> Dan.
>>
>> Hi Danken,
>>
>> How did you mark a partitioned device? (How will the engine will spot
>> these LUNs)
> 
> Each device has a field 'partitioned'. Currently, Engine ignores it (as it is
> always False). With my little patch, devices with partitioned=True would be
> reported, too.
> 
> Dan.

ok great thanks.
you can push it and will take care of the engine side.

Thanks, Livnat




From abaron at redhat.com  Wed Jan 18 11:17:53 2012
From: abaron at redhat.com (Ayal Baron)
Date: Wed, 18 Jan 2012 06:17:53 -0500 (EST)
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <20120118104422.GA30396@redhat.com>
Message-ID: <32db41e6-8fec-4da0-8880-d0f616836bf9@zmail13.collab.prod.int.phx2.redhat.com>

Can we broaden the scope and also allow passing createVG partitioned devices with an override flag or something? (we'd need to check the devices and run "kpartx -d" and fdisk to clean the devices before calling pvcreate).

----- Original Message -----
> On Tue, Jan 17, 2012 at 07:46:17PM +0200, Livnat Peer wrote:
> > On 17/01/12 16:08, Dan Kenigsberg wrote:
> > > On Sun, Jan 08, 2012 at 03:08:13PM +0200, Dan Kenigsberg wrote:
> > >> Hi Lists,
> > >>
> > >> One cannot create a PV on a partitioned device, and therefor
> > >> such
> > >> devices where not reported to Engine. This proved surprising to
> > >> users
> > >> who woder where their LUN disappeared.
> > >>
> > >> Vdsm should report all devices, and ovirt-engine should mark
> > >> partitioned
> > >> devices as unworthy of a PV. In the future, Vdsm may allow to
> > >> forcefully
> > >> remove a partition table from a device, to make it usable as a
> > >> PV.
> > >>
> > >> Douglas (CCed) would take resposibilty on the Vdsm side. Initial
> > >> patch at
> > >> http://gerrit.ovirt.org/944 sports a backword-compatible API.
> > >> Who's taking this
> > >> on Engine? This involves GUI, too, as partitioned devices should
> > >> probably be
> > >> displayed greyed-out.
> > > 
> > > Livnat, is there any taker for this on your side?
> > > 
> > > Since the change is backward-compatible, I would push it if there
> > > are no
> > > objections.
> > > 
> > > Dan.
> > 
> > Hi Danken,
> > 
> > How did you mark a partitioned device? (How will the engine will
> > spot
> > these LUNs)
> 
> Each device has a field 'partitioned'. Currently, Engine ignores it
> (as it is
> always False). With my little patch, devices with partitioned=True
> would be
> reported, too.
> 
> Dan.
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> 


From abaron at redhat.com  Wed Jan 18 11:23:28 2012
From: abaron at redhat.com (Ayal Baron)
Date: Wed, 18 Jan 2012 06:23:28 -0500 (EST)
Subject: [Engine-devel] Shared disk / Floating disk import/export
	(was:	Re: move disk command)
In-Reply-To: <e1dea435-3f02-4f66-a72e-64600e39f65c@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <a7eb4922-1fb6-4fbf-a80c-4bfec12691b7@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> ----- Original Message -----
> > 
> > 
> > ----- Original Message -----
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Ayal Baron" <abaron at redhat.com>
> > > > To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > Cc: engine-devel at ovirt.org, "Jon Choate" <jchoate at redhat.com>,
> > > > "Maor" <mlipchuk at redhat.com>
> > > > Sent: Wednesday, January 18, 2012 10:20:01 AM
> > > > Subject: Re: Shared disk / Floating disk import/export (was:
> > > > Re:
> > > > [Engine-devel] move disk command)
> > > > 
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > > > From: "Ayal Baron" <abaron at redhat.com>
> > > > > > To: "Maor" <mlipchuk at redhat.com>
> > > > > > Cc: engine-devel at ovirt.org, "Jon Choate"
> > > > > > <jchoate at redhat.com>,
> > > > > > "Omer Frenkel" <ofrenkel at redhat.com>
> > > > > > Sent: Tuesday, January 17, 2012 10:21:29 PM
> > > > > > Subject: Re: Shared disk / Floating disk import/export
> > > > > > (was:
> > > > > > Re:
> > > > > > [Engine-devel] move disk command)
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > ----- Original Message -----
> > > > > > > On 01/17/2012 11:57 AM, Omer Frenkel wrote:
> > > > > > > > 
> > > > > > > > 
> > > > > > > > ----- Original Message -----
> > > > > > > >> From: "Ayal Baron" <abaron at redhat.com>
> > > > > > > >> To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > > > > >> Cc: engine-devel at ovirt.org, "Jon Choate"
> > > > > > > >> <jchoate at redhat.com>
> > > > > > > >> Sent: Tuesday, January 17, 2012 10:45:53 AM
> > > > > > > >> Subject: Shared disk / Floating disk import/export
> > > > > > > >> (was:
> > > > > > > >> Re:
> > > > > > > >> [Engine-devel] move disk command)
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> [SNIP]
> > > > > > > >>
> > > > > > > >>>>> This may be unrelated, but user would be allowed to
> > > > > > > >>>>> export
> > > > > > > >>>>> and
> > > > > > > >>>>> import a floating disk, right?
> > > > > > > >>>>> I would like the ability to import *any* disk in
> > > > > > > >>>>> the
> > > > > > > >>>>> export
> > > > > > > >>>>> domain
> > > > > > > >>>>> as a floating disk, but in the least, export and
> > > > > > > >>>>> import
> > > > > > > >>>>> disks
> > > > > > > >>>>> not
> > > > > > > >>>>> associated with a VM.
> > > > > > > >>>
> > > > > > > >>> you are right, it is unrelated, this thread is about
> > > > > > > >>> move
> > > > > > > >>> disk
> > > > > > > >>> of
> > > > > > > >>> a
> > > > > > > >>> vm between SDs,
> > > > > > > >>> export and import is copy, and floating disks is part
> > > > > > > >>> of
> > > > > > > >>> the
> > > > > > > >>> shared
> > > > > > > >>> disk feature,
> > > > > > > >>> this indeed need to be discussed in that scope.
> > > > > > > >>
> > > > > > > >> Ok, so I've changed the subject, now let's discuss it.
> > > > > > > >>
> > > > > > > > 
> > > > > > > > Adding Maor,
> > > > > > > > not sure if we have any plan regard export/import of
> > > > > > > > floating
> > > > > > > > disk,
> > > > > > > > or in general, exporting disk without it's vm (if I
> > > > > > > > understood
> > > > > > > > Ayal
> > > > > > > > correctly)
> > > > > > > > 
> > > > > > > > Maor, any comments?
> > > > > > > I remember, we mentioned export/import issue in our last
> > > > > > > meeting
> > > > > > > with
> > > > > > > Andrew in the context of shared raw disk.
> > > > > > > It was decided then, that export/import will not be
> > > > > > > supported
> > > > > > > by
> > > > > > > shared
> > > > > > > raw disk, I'm not sure if we also decided that for
> > > > > > > floating
> > > > > > > disk,
> > > > > > > but I think its a PM decision.
> > > > > > > 
> > > > > > > Support import/export domain might evolve to a big
> > > > > > > change,
> > > > > > > since
> > > > > > > if
> > > > > > > we
> > > > > > > want to export disk, we might also want to reflect all
> > > > > > > its
> > > > > > > configuration
> > > > > > > and functionality there, also reflect disks which are
> > > > > > > attached
> > > > > > > to
> > > > > > > VMs
> > > > > > > and templates as well.
> > > > > > 
> > > > > > What properties does a disk have?
> > > > > > Interface is what type of connection is used when plugging
> > > > > > the
> > > > > > disk
> > > > > > in the computer (VM) so it's not really a disk property.
> > > > > > Address is also VM specific
> > > > > > Description is already stored in storage side.
> > > > > > What else do you have for disk?
> > > > > 
> > > > > type (system/data..)
> > > > 
> > > > Didn't we recently have a discussion about this being
> > > > irrelevant?
> > > 
> > > as long it is visible to the user, no.
> > 
> > It's meaningless.
> > 
> > > 
> > > > 
> > > > > post-zero
> > > > 
> > > > Can assume always true
> > > 
> > > i disagree
> > 
> > It's the default behaviour for files, no reason to not allow users
> > to
> > import disk because of this.
> > 
> > > 
> > > > 
> > > > > name(?)
> > > > 
> > > > This exists on storage side
> > > 
> > > in addition to description?
> > 
> > there is uuid and description, do you store in addition another
> > name?
> > 
> > > 
> > > > 
> > > > > creation-date
> > > > 
> > > > When imported
> > > 
> > > no!
> > 
> > Hmmm, I love how elaborate your explanations are.
> > It's common with files, no reason to treat differently.
> > 
> > > 
> > > > 
> > > > > last modify date
> > > > 
> > > > This exists on storage side
> > > > 
> > > > > application-list
> > > > 
> > > > Has nothing to do with disk, it's a VM property
> > > 
> > > no!
> > 
> > Again, very elaborate...
> > VM has 3 disks, guest reports app-list for the VM, how do you
> > determine which disk has which apps?
> > If this is currently a disk property in engine then it's a bug.
> 
> I think this is a snapshot's property.

Snapshot in itself is a VM property, not a disk property

> 
> > 
> > > 
> > > > 
> > > > > volume type and format
> > > > 
> > > > This exists on storage side
> > > > 
> > > > > 
> > > > > > 
> > > > > > Note that being able to import floating disks means that
> > > > > > we'd
> > > > > > be
> > > > > > able
> > > > > > to take any data storage domain and import any disk(s) from
> > > > > > it
> > > > > > (assuming user changes domain type to export domain, which
> > > > > > users
> > > > > > already know how to do)
> > > > > > 
> > > > > > 
> > > > > 
> > > > 
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 


From jchoate at redhat.com  Wed Jan 18 11:41:45 2012
From: jchoate at redhat.com (Jon Choate)
Date: Wed, 18 Jan 2012 06:41:45 -0500
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <8b6cfe3d-b9ee-4334-870a-2e0763e735f6@zmail13.collab.prod.int.phx2.redhat.com>
References: <8b6cfe3d-b9ee-4334-870a-2e0763e735f6@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F16AFF9.6030408@redhat.com>

On 01/17/2012 03:14 PM, Ayal Baron wrote:
>
> ----- Original Message -----
>>
>> ----- Original Message -----
>>> From: "Maor"<mlipchuk at redhat.com>
>>> To: engine-devel at ovirt.org
>>> Sent: Tuesday, January 17, 2012 4:33:19 AM
>>> Subject: Re: [Engine-devel] a different approach to the command
>>> classes
>>>
>>> On 01/17/2012 09:24 AM, Ayal Baron wrote:
>>>>
>>>> ----- Original Message -----
>>>>> On 17/01/12 04:58, Jon Choate wrote:
>>>>>> The way the command classes are written has bothered me for a
>>>>>> while.
>>>>>> While implementing the multiple storage domain features I am
>>>>>> presented
>>>>>> with the opportunity to create a new command from scratch.  I
>>>>>> gave
>>>>>> some
>>>>>> thought to what I would like the command classes to look like
>>>>>> while
>>>>>> balancing that the class must still fit in with the existing
>>>>>> structure.
>>>>>> So here is what I came up with. I'd appreciate any feedback.
>>>>>>
>>>>>> The Command class encompasses only the rules of what needs to
>>>>>> be
>>>>>> done.
>>>>>> It relies upon Validator classes to determine if the
>>>>>> canDoAction
>>>>>> conditions have been met.
>>>>>>
>>>>>>      @Override
>>>>>>      public boolean canDoAction() {
>>>>>>        ...
>>>>>>          checkTargetDomainHasSpace();
>>>>>>          checkTargetDomainIsValidTarget();
>>>>>>          checkSourceDomainIsValidSource();
>>>>>>          checkSourceAndTargetAreDifferent();
>>>>>>       ...
>>>>>> }
>>>>>>
>>>>>> ...
>>>>>>
>>>>>>    private void checkTargetDomainHasSpace() {
>>>>>>          if(!actionAllowed) return;
>>>>>>
>>>>>> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
>>>>>> {
>>>>>>
>>>>>> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
>>>>>>            actionAllowed = false;
>>>>>>          }
>>>>>>      }
>>>>>>
>>>>>>
>>>>>> Each of the checks follows a similar pattern of
>>>>>>      - guard clause to see of we already failed and bail if we
>>>>>>      did
>>>>>>      - test for failure of the condition
>>>>>>      - add failure message if needed
>>>>>>      - set variable to failed if needed
>>>>>>
>>>>>> Storing the success flag in a variable allows us to keep the
>>>>>> canDoAction
>>>>>> method readable as a series of commands and to allow it to be
>>>>>> accessed
>>>>>> by all the private methods without them having to pass it
>>>>>> around.
>>>>>>
>>>>>> The execution of the command will follow a similar pattern
>>>>>> where
>>>>>> the
>>>>>> command class will only know how to describe what needs to be
>>>>>> done
>>>>>> and
>>>>>> to rely on supporting objects to handle the implementation of
>>>>>> these
>>>>>> steps.  Getting the implementation out of the command classes
>>>>>> will
>>>>>> allow
>>>>>> the commands to share validation and implementation details and
>>>>>> remove a
>>>>>> lot of the duplication that currently exists within the
>>>>>> commands.
>>>>>>
>>>>>>
>>>>>> How do people feel about this approach?
>>>>>
>>>>> Hi Jon,
>>>>>
>>>>> The scope of your proposal is changing the implementation of the
>>>>> canDoAction method, I think that the title of this mail is a bit
>>>>> misleading.
>>>> Actually I think Jon was suggesting to do the same in the command
>>>> itself.
>>
>> Yes I am.  I just haven't written that code yet!
>>
>>
>>> I think, using shared canDoAction validation between commands might
>>> be a
>>> good idea also, it can be seen in operations such as add/update
>>> commands.
>>> In most cases, the update validation, is already based on the add
>>> command validation, sometime it is implemented with
>>> inheritance/external
>>> class, in other cases it is just implemented with multiple code.
>>>>> Basically what you are suggesting is to split the canDoAction
>>>>> implementation into methods and then extract them from the
>>>>> command
>>>>> class
>>>>> to a shared class so they can be reused.
>>>>>
>>>>> In many cases we can use (are using) inheritance for reusing
>>>>> code,
>>>>> there
>>>>> are cases where inheritance does not do the work and we can
>>>>> extract
>>>>> to
>>>>> external classes.
>>>>>
>> Our overuse of inheritance is one of the things I'm trying to
>> rectify.  Inheritance wasn't added
>> to the language to facilitate reuse. It is there to represent object
>> hierarchies.  In some cases
>> we abuse this to leverage code reuse.  This is often a code smell
>> that the code is in the wrong
>> place to begin with.  If both classes need the code and they are not
>> logically related in an object
>> hierarchy, the code really needs to be outside the classes.
>>
>> We have cases where the use of inheritance for reuse have gone too
>> far. For instance:
>>
>> public class RemoveVdsSpmIdCommand<T extends VdsActionParameters>
>> extends AddVdsSpmIdCommand<T>
>>
>> So this says that a RemoveVdsSmpIdCommand is a type of
>> AddVdsSpmIdCommand?  That implies that I can
>> use a RemoveVdsSmpIdCommand anywhere that an AddVdsSpmIdCommand is
>> expected. Otherwise we have broken
>> one of the fundamental contracts of object oriented programming.
>>
>>>>> I think such a change is welcomed but on a needed basis, I think
>>>>> it
>>>>> is
>>>>> overkill for the general use case and will make the code more
>>>>> cumbersome
>>>>> (if the original canDoAction was 4-5 lines long...).
>>  From what I have seen those canDoActions are in the minority.  The
>> overall goals are to increase
>> the readability and maintainability of the code so I would suggest
>> being pragmatic about any approach.
>> If doing it doesn't help achieve the goal, then don't do it.
>>
>>
>> One of the ideas I'm trying to convey here is that the command
>> classes should be fairly ignorant.
>> They should be responsible for knowing the list of steps involved in
>> a workflow, but not the details
>> of how those steps are carried out. Imo knowing both the steps and
>> their implementation violates the SRP.
>>
>>
>>>> Jon, I think the burden of proof is on you here to show a real
>>>> example and how it makes the code clearer (e.g. change 2 commands
>>>> which share similar checks).
>>>> Without 'seeing' it I don't think we would be able to appreciate
>>>> the advantage of your approach.
>>>>
>> I need to get the multiple storage domains feature put away and then
>> I will
>> definitely do some refactoring to illustrate the value. I
>> wholeheartedly agree
>> that having actual code to kick around is better. I just can't do it
>> right now!
>>
>>
>>>>> One thing I don't like in the above suggestion is the way you
>>>>> validate
>>>>> that the previous condition succeeded/failed. Having this
>>>>> condition
>>>>> at
>>>>> the beginning of each validation method is not a good approach
>>>>> IMO.
>> Can you elaborate on your opposition to the use of guard clauses?
>> They have
>> been considered a best practice for quite some time.
>>
>>>> +1, if the previous check failed it should raise an exception,
>>>> not
>>>> rely on the next check to bail.
>>>> It would be easier (less code, clearer) to wrap all the calls
>>>> with
>>>> a try catch clause (1 for all the calls), catch the specific
>>>> exception that says the check failed and return whatever you want
>>>> to return.
>> I'm not really a fan of using exceptions for controlling flow of
>> execution.  Failing one of these
>> checks is not an exceptional event. Its an expected part of the
>> workflow and should
>> be handled as a normal occurrence.
>>
>> I would argue that wrapping the entire canDoAction method in a
>> try/catch would not make the code clearer:
>>
>>
>>    public boolean canDoAction() {
>>        boolean succeeded = true;
>>        try {
>>           do1();
>>           do2();
>>           do3();
>>        catch(ValidationException e) {
>>           succeeded = false;
>>        }
>>        return succeeded;
>>    }
>>
>>
>> has a lot more noise than:
>>
>>    private boolean canDoAction = true;
>>
>>    public boolean canDoAction() {
>>      do1();
>>      do2();
>>      do3();
>>      return canDoAction;
>>    }
>>
> You're forgetting the cost:
>
> instead of:
> public void do1() {
>      ...
> }
>
> public void do2() {
>      ...
> }
>
> public void do3() {
>      ...
> }
>
> public void do1() {
>      if (!globalSuccessVar) {
>         return
>      }
>      ...
> }
>
> public void do2() {
>      if (!globalSuccessVar) {
>         return
>      }
>      ...
> }
>
> public void do3() {
>      if (!globalSuccessVar) {
>         return
>      }
>      ...
> }

I don't necessarily see this as a cost in the same way that you do.  My 
focus is to make the code readable and easy to understand.  One of the 
main methods in a command class that readers are going to focus on is 
canDoAction. Imo in most cases the reader is going to want to know what 
the conditions are for success not the details of how they are all 
determined.  Any noise in that method that detracts from this 
(try/catches, nested ifs, local booleans that get set and checked a lot) 
detract from the readability of the code.

I would much rather use guard clauses in the detail method where people 
are less likely to be reading.  Guard clauses also don't detract from 
readability much since they tend to be one liners at the top of methods. 
They are a common pattern so readers are generally not surprised to see 
them.

My general approach is to keep the top-level methods as clean as 
possible and push the details down.  This makes the top-level methods 
readable and allows interested readers to drill down only where they are 
interested.


>
> I on purpose did not call it 'actionAllowed' to stress the fact that it's a global var (another thing I don't like).

I don't think global is really the right word since it would be an 
instance variable not a static one.  It is unclear to me why you don't 
like instance variables.  An instance of a command class represents a 
single attempt at running a command.  It ability to succeed or fail is 
an instance-level concept so why not store it as an instance variable 
that can then be used as shared state across multiple methods.


>
> And before you go to the one liner option per function, I consider 'if (!globalSuccessVar) return' bad form as it's not easy to discern what the condition is and what the action is.
>

Not sure what you mean by this. Can you elaborate?

>> In the second form, there is no extra indentation or curly braces.
>>   The validation steps
>> look just like a list of steps and imo it makes the code really easy
>> to understand by a maintainer.
>>
>>>>>
>>>>> Livnat
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From lpeer at redhat.com  Wed Jan 18 12:06:37 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 14:06:37 +0200
Subject: [Engine-devel] move disk command
In-Reply-To: <ffc8d1f5-a5fd-41a2-b370-096a8e134628@mkenneth.csb>
References: <ffc8d1f5-a5fd-41a2-b370-096a8e134628@mkenneth.csb>
Message-ID: <4F16B5CD.40405@redhat.com>

On 17/01/12 23:00, Miki Kenneth wrote:
> 
> 
> ----- Original Message -----
>> From: "Omer Frenkel" <ofrenkel at redhat.com>
>> To: "Jon Choate" <jchoate at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Tuesday, January 17, 2012 10:41:43 AM
>> Subject: Re: [Engine-devel] move disk command
>>
>>
>>
>> ----- Original Message -----
>>> From: "Jon Choate" <jchoate at redhat.com>
>>> To: "Ayal Baron" <abaron at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Monday, January 16, 2012 9:39:55 PM
>>> Subject: Re: [Engine-devel] move disk command
>>>
>>> On 01/16/2012 02:26 PM, Ayal Baron wrote:
>>>>
>>>> ----- Original Message -----
>>>>> On 01/16/2012 05:26 PM, Jon Choate wrote:
>>>>>> As part of the multiple storage domains feature there will be
>>>>>> new
>>>>>> functionality added to allow users to move individual disks.
>>>>>>
>>>>>> What are the prerequisites for moving a disk?
>>>>>>
>>>>>> 1) the disk must exist
>>>>>> 2) the associated VM must be down
>>>>> this can't just be a CanDoAction check - the lock has to be real
>>>>> to
>>>>> prevent a race from starting the VM after the validation.
>>>>>
>>>> Either down or disk is unplugged.
>>>>
>>>>>> 3) the associated VM must not be locked
>>>>>> 4) the source storage domain must exist
>>>>>> 5) the source storage domain must be available
>>>>>> 6) the target domain must exist
>>>>>> 7) the target domain must be available
>>>>>> 8) the target domain must have adequate disk space
>>>>>> 9) the target domain cannot be an ISO or export domain
>>>>>> 10) the source domain cannot be an ISO or export domain
>>>> This may be unrelated, but user would be allowed to export and
>>>> import a floating disk, right?
>>>> I would like the ability to import *any* disk in the export
>>>> domain
>>>> as a floating disk, but in the least, export and import disks not
>>>> associated with a VM.
>>
>> you are right, it is unrelated, this thread is about move disk of a
>> vm between SDs,
>> export and import is copy, and floating disks is part of the shared
>> disk feature,
>> this indeed need to be discussed in that scope.
> So what is the behavior of a move of a floating disk? can I move any type of disk?
> (shared?) between SDs? 

You can move around all disks, except:
- Template disk - which has VMs based on this disk in this SD.
- VM disk - which is based on a template disk that does not exist on the
destination domain.







From jhernand at redhat.com  Wed Jan 18 09:42:49 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Wed, 18 Jan 2012 10:42:49 +0100
Subject: [Engine-devel] Any way to get more than 100 VMs with the REST API?
Message-ID: <4F169419.7000409@redhat.com>

Hello,

Is there any way to get more than 100 VMs (or whatever the value of the
option "SearchResultsLimit" is) using the REST API?

Thanks in advance,
Juan Hernandez
-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From lpeer at redhat.com  Wed Jan 18 13:30:26 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 15:30:26 +0200
Subject: [Engine-devel] the future of template cloning
In-Reply-To: <a968c256-fae0-4cc4-bbf5-f53207997bfb@zmail13.collab.prod.int.phx2.redhat.com>
References: <a968c256-fae0-4cc4-bbf5-f53207997bfb@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F16C972.4080807@redhat.com>


>>
>> I think that using the multi action approach is the better behavior
>> for
>> the user.
>> If the user clone a template with 3 disks and the engine was able to
>> clone only two out of three (got an error on the third copy), As a
>> user
>> i would rather get a chance to clone the third one again and not
>> re-clone the other two (it takes time...).
>>
>> So my vote goes to multi actions.
>>
> 
> You're missing the point, if it's a single action then the canDoAction would validate free space for all disks together and wouldn't copy a single byte before failing.
> 

I think the main point is the roll back behavior as i mentioned earlier,
and less the validation of the canDoAction.


> On the other hand, I do agree that if canDoAction succeeds and then one of the copies fails then you have to rollback which is not nice.
> If you can do an aggregated canDoAction for copying multiple disks using the multipleRunAction (i.e. calculate space needed for *all* disks on each target domain and warn user before starting or fail) and GUI-wise the operation is done on the disks themselves and not on the template then I'm fine with that.
> 

The engine has built-in support for adding shared logic for multiple
actions.
I think it makes sense to add a validation for the storage capacity on
the summary of all storage requests.

I rather get another input on this before going with the shared logic -

If implementing the above behavior it means we are going to fail cloning
all disks if there is not enough space to clone one of them regardless
of the destination storage domain.

For example:

- There are 2 domains each has 10G free space
- There are 3 disks each of them takes 6G
- The engine gets multiple-action command:
	* clone disk1 to domain1
	* clone disk2 to domain1
        * clone disk3 to domain2

Result:
- engine fails all command although we could have cloned disk3.

I am ok with the above scenario just want to make clear what is going to
be the behavior.


> 
> <SNIP>



From ovedo at redhat.com  Wed Jan 18 15:41:59 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Wed, 18 Jan 2012 10:41:59 -0500 (EST)
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <5169bf7b-0c8f-4319-8aa0-9cccb6479dc3@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <138723b0-43ed-4271-a4ef-1eaecf5807e4@zmail02.collab.prod.int.phx2.redhat.com>

Hey all,

Continuing the discussion about Aeolus instance data injection to a VM (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html) we propose a new VM Payload feature.

The following wiki page contains a description page of the feature.
http://www.ovirt.org/wiki/Features/VMPayload

Please read and review.
There are several approaches there, and we wish to head your opinions and thoughts about them.

Once we agree on an approach, we will start designing.

Thank you,
Oved


From lpeer at redhat.com  Wed Jan 18 15:53:48 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 17:53:48 +0200
Subject: [Engine-devel] ovirt core MOM
Message-ID: <4F16EB0C.2030208@redhat.com>

Hi All,

This is what we've discussed in the meeting today:

Multiple storage domain:
- Should have a single generic verb for removing a disk.
- We block removing the last template disk - template is immutable.
- We add a shared logic for multiple actions of cloning a disk.

Clone VM from Snapshot:
- Shared disk and direct LUN are open issues, Yair will start a
discussion on this upstream.

- The suggestion was to mark direct LUN and shared disk as unplugged
disks when taking a snapshot.

- We had no time to discuss setup networks and stable device addressed,
we'll discuss this on the next meeting.

Thanks, Livnat


From lpeer at redhat.com  Wed Jan 18 16:00:41 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 18 Jan 2012 18:00:41 +0200
Subject: [Engine-devel] Any way to get more than 100 VMs with the REST
 API?
In-Reply-To: <4F169419.7000409@redhat.com>
References: <4F169419.7000409@redhat.com>
Message-ID: <4F16ECA9.6030000@redhat.com>

On 18/01/12 11:42, Juan Hernandez wrote:
> Hello,
> 
> Is there any way to get more than 100 VMs (or whatever the value of the
> option "SearchResultsLimit" is) using the REST API?
> 

I think it is not modeled in the API.
Michael - do we expose a way to set search page size to REST API users?

Livnat

> Thanks in advance,
> Juan Hernandez



From sanjal at redhat.com  Wed Jan 18 13:23:05 2012
From: sanjal at redhat.com (Shireesh Anjal)
Date: Wed, 18 Jan 2012 18:53:05 +0530
Subject: [Engine-devel] a different approach to the command classes
In-Reply-To: <39727574-be45-443d-8dfe-f9b7df1ae52b@zmail15.collab.prod.int.phx2.redhat.com>
References: <39727574-be45-443d-8dfe-f9b7df1ae52b@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <4F16C7B9.4000602@redhat.com>

This is a really interesting discussion! Though I'm pretty new to oVirt, 
here (inline) are my two cents :)

On Tuesday 17 January 2012 07:07 PM, Jon Choate wrote:
>
> ----- Original Message -----
>> From: "Maor"<mlipchuk at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Tuesday, January 17, 2012 4:33:19 AM
>> Subject: Re: [Engine-devel] a different approach to the command classes
>>
>> On 01/17/2012 09:24 AM, Ayal Baron wrote:
>>>
>>> ----- Original Message -----
>>>> On 17/01/12 04:58, Jon Choate wrote:
>>>>> The way the command classes are written has bothered me for a
>>>>> while.
>>>>> While implementing the multiple storage domain features I am
>>>>> presented
>>>>> with the opportunity to create a new command from scratch.  I
>>>>> gave
>>>>> some
>>>>> thought to what I would like the command classes to look like
>>>>> while
>>>>> balancing that the class must still fit in with the existing
>>>>> structure.
>>>>> So here is what I came up with. I'd appreciate any feedback.
>>>>>
>>>>> The Command class encompasses only the rules of what needs to be
>>>>> done.
>>>>> It relies upon Validator classes to determine if the canDoAction
>>>>> conditions have been met.
>>>>>
>>>>>      @Override
>>>>>      public boolean canDoAction() {
>>>>>        ...
>>>>>          checkTargetDomainHasSpace();
>>>>>          checkTargetDomainIsValidTarget();
>>>>>          checkSourceDomainIsValidSource();
>>>>>          checkSourceAndTargetAreDifferent();
>>>>>       ...
>>>>> }
>>>>>
>>>>> ...
>>>>>
>>>>>    private void checkTargetDomainHasSpace() {
>>>>>          if(!actionAllowed) return;
>>>>>
>>>>> if(!targetDomainValidator.hasSpace(getParameters().getDiskImageId()))
>>>>> {
>>>>>
>>>>> addCanDoActionMessage(VdcBllMessages.ACTION_TYPE_FAILED_DISK_SPACE_LOW);
>>>>>            actionAllowed = false;
>>>>>          }
>>>>>      }
>>>>>
>>>>>
>>>>> Each of the checks follows a similar pattern of
>>>>>      - guard clause to see of we already failed and bail if we did
>>>>>      - test for failure of the condition
>>>>>      - add failure message if needed
>>>>>      - set variable to failed if needed
>>>>>
>>>>> Storing the success flag in a variable allows us to keep the
>>>>> canDoAction
>>>>> method readable as a series of commands and to allow it to be
>>>>> accessed
>>>>> by all the private methods without them having to pass it around.
>>>>>
>>>>> The execution of the command will follow a similar pattern where
>>>>> the
>>>>> command class will only know how to describe what needs to be
>>>>> done
>>>>> and
>>>>> to rely on supporting objects to handle the implementation of
>>>>> these
>>>>> steps.  Getting the implementation out of the command classes
>>>>> will
>>>>> allow
>>>>> the commands to share validation and implementation details and
>>>>> remove a
>>>>> lot of the duplication that currently exists within the commands.
>>>>>
>>>>>
>>>>> How do people feel about this approach?
>>>>
>>>> Hi Jon,
>>>>
>>>> The scope of your proposal is changing the implementation of the
>>>> canDoAction method, I think that the title of this mail is a bit
>>>> misleading.
>>> Actually I think Jon was suggesting to do the same in the command
>>> itself.
>
> Yes I am.  I just haven't written that code yet!
>
>
>> I think, using shared canDoAction validation between commands might
>> be a
>> good idea also, it can be seen in operations such as add/update
>> commands.
>> In most cases, the update validation, is already based on the add
>> command validation, sometime it is implemented with
>> inheritance/external
>> class, in other cases it is just implemented with multiple code.
>>>> Basically what you are suggesting is to split the canDoAction
>>>> implementation into methods and then extract them from the command
>>>> class
>>>> to a shared class so they can be reused.
>>>>
>>>> In many cases we can use (are using) inheritance for reusing code,
>>>> there
>>>> are cases where inheritance does not do the work and we can
>>>> extract
>>>> to
>>>> external classes.
>>>>
> Our overuse of inheritance is one of the things I'm trying to rectify.  Inheritance wasn't added
> to the language to facilitate reuse. It is there to represent object hierarchies.  In some cases
> we abuse this to leverage code reuse.  This is often a code smell that the code is in the wrong
> place to begin with.  If both classes need the code and they are not logically related in an object
> hierarchy, the code really needs to be outside the classes.
>
> We have cases where the use of inheritance for reuse have gone too far. For instance:
>
> public class RemoveVdsSpmIdCommand<T extends VdsActionParameters>  extends AddVdsSpmIdCommand<T>
>
> So this says that a RemoveVdsSmpIdCommand is a type of AddVdsSpmIdCommand?  That implies that I can
> use a RemoveVdsSmpIdCommand anywhere that an AddVdsSpmIdCommand is expected. Otherwise we have broken
> one of the fundamental contracts of object oriented programming.
>
>>>> I think such a change is welcomed but on a needed basis, I think
>>>> it
>>>> is
>>>> overkill for the general use case and will make the code more
>>>> cumbersome
>>>> (if the original canDoAction was 4-5 lines long...).
>  From what I have seen those canDoActions are in the minority.  The overall goals are to increase
> the readability and maintainability of the code so I would suggest being pragmatic about any approach.
> If doing it doesn't help achieve the goal, then don't do it.
>
>
> One of the ideas I'm trying to convey here is that the command classes should be fairly ignorant.
> They should be responsible for knowing the list of steps involved in a workflow, but not the details
> of how those steps are carried out. Imo knowing both the steps and their implementation violates the SRP.
>
>
>>> Jon, I think the burden of proof is on you here to show a real
>>> example and how it makes the code clearer (e.g. change 2 commands
>>> which share similar checks).
>>> Without 'seeing' it I don't think we would be able to appreciate
>>> the advantage of your approach.
>>>
> I need to get the multiple storage domains feature put away and then I will
> definitely do some refactoring to illustrate the value. I wholeheartedly agree
> that having actual code to kick around is better. I just can't do it right now!
>
>
>>>> One thing I don't like in the above suggestion is the way you
>>>> validate
>>>> that the previous condition succeeded/failed. Having this
>>>> condition
>>>> at
>>>> the beginning of each validation method is not a good approach
>>>> IMO.
> Can you elaborate on your opposition to the use of guard clauses? They have
> been considered a best practice for quite some time.
>
>>> +1, if the previous check failed it should raise an exception, not
>>> rely on the next check to bail.
>>> It would be easier (less code, clearer) to wrap all the calls with
>>> a try catch clause (1 for all the calls), catch the specific
>>> exception that says the check failed and return whatever you want
>>> to return.
> I'm not really a fan of using exceptions for controlling flow of execution.  Failing one of these
> checks is not an exceptional event. Its an expected part of the workflow and should
> be handled as a normal occurrence.
I'm not sure if all validation failures can be called as "expected part 
of the workflow". If the caller didn't pass right parameters, why not 
consider it as an "unexpected" event and throw a "Validation Exception" ?
>
> I would argue that wrapping the entire canDoAction method in a try/catch would not make the code clearer:
>
>
>    public boolean canDoAction() {
>        boolean succeeded = true;
>        try {
>           do1();
>           do2();
>           do3();
>        catch(ValidationException e) {
>           succeeded = false;
>        }
>        return succeeded;
>    }
>
>
> has a lot more noise than:
>
>    private boolean canDoAction = true;
>
>    public boolean canDoAction() {
>      do1();
>      do2();
>      do3();
>      return canDoAction;
>    }
>
> In the second form, there is no extra indentation or curly braces.  The validation steps
> look just like a list of steps and imo it makes the code really easy to understand by a maintainer.
This is true. In general checked exceptions and consequent try/catch 
blocks add a lot of noise to the code, and in many cases result in 
repetitive code and/or unintentional eating up of exceptions (empty or 
just-log-the-exception kind of catch blocks that happen in the frenzy of 
writing too much code in too little time). IMO, a good approach is to 
throw a ValidationException (which extends from RuntimeException) from 
validation methods, and handle this at framework level (not inside the 
"canDoAction" method). That way, the main validation method will look 
clean, and all validation errors will be handled at a central place in a 
consistent way. This would also mean that instead of a "boolean 
canDoAction", we have a "void validateActionParams", something like:

public void validateActionParams() {
     validate1();
     validate2();
     validate3();
}

public void validate1() {
     if(somethingIsWrong) {
         throw new ValidationException(withAllRelevantInfo);
     }
}

...

There can be another variant of the ValidationException that takes a 
"list" of validation errors, thus capturing multiple validation issues.

>
>>>>
>>>> Livnat
>>>>
>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
-- 
Regards,
Shireesh



From iheim at redhat.com  Wed Jan 18 19:40:41 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 18 Jan 2012 21:40:41 +0200
Subject: [Engine-devel] Any way to get more than 100 VMs with the REST
 API?
In-Reply-To: <4F16ECA9.6030000@redhat.com>
References: <4F169419.7000409@redhat.com> <4F16ECA9.6030000@redhat.com>
Message-ID: <4F172039.1090101@redhat.com>

On 01/18/2012 06:00 PM, Livnat Peer wrote:
> On 18/01/12 11:42, Juan Hernandez wrote:
>> Hello,
>>
>> Is there any way to get more than 100 VMs (or whatever the value of the
>> option "SearchResultsLimit" is) using the REST API?
>>
>
> I think it is not modeled in the API.
> Michael - do we expose a way to set search page size to REST API users?

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Virtualization/3.0/html-single/REST_API_Guide/index.html#sect-REST_API_Guide-Common_Features-Collections-Querying_Collections

look at:
7.2.3.3. Pagination


From yzaslavs at redhat.com  Thu Jan 19 06:38:40 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Thu, 19 Jan 2012 08:38:40 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
Message-ID: <4F17BA70.2060800@redhat.com>

Hi all,
Following the upstream meeting dated Wednesday January 18th, 2012 -
I presented the clone VM from snpashot feature and we discussed the
feature behaviour.

Two issues that were raised are the behaviour of the feature in context
of shared disks and direct LUNs-based disks -
On one hand, if we copy&collapse such images - this may yield to data
corruption (think of a scenario where the source and destination VMs use
the same disk).
On the other hand - if we decide not to copy&collapse  - the target VM
will have missing VM and its state will not totally reflect the logical
state.
One of the solution raises is to mark such disks (at the destination) as
unplugged, allowing the administrator the ability to plug them
(understanding of course the consequences).

I would like to receive inputs on this issue


Kind regards,

Yair


From lpeer at redhat.com  Thu Jan 19 07:19:52 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Thu, 19 Jan 2012 09:19:52 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <4F17BA70.2060800@redhat.com>
References: <4F17BA70.2060800@redhat.com>
Message-ID: <4F17C418.30004@redhat.com>

On 19/01/12 08:38, Yair Zaslavsky wrote:
> Hi all,
> Following the upstream meeting dated Wednesday January 18th, 2012 -
> I presented the clone VM from snpashot feature and we discussed the
> feature behaviour.
> 
> Two issues that were raised are the behaviour of the feature in context
> of shared disks and direct LUNs-based disks -
> On one hand, if we copy&collapse such images - this may yield to data
> corruption (think of a scenario where the source and destination VMs use
> the same disk).
> On the other hand - if we decide not to copy&collapse  - the target VM
> will have missing VM and its state will not totally reflect the logical
> state.
> One of the solution raises is to mark such disks (at the destination) as
> unplugged, allowing the administrator the ability to plug them
> (understanding of course the consequences).
> 
> I would like to receive inputs on this issue
> 
> 
> Kind regards,
> 
> Yair

Hi Yair,

Some clarifications on the above issue.
Currently when taking a snapshot on a VM with shared disks or direct LUN
disk there are 3 optional behaviors:

1. Blocking the snapshot action. (User can not take a snapshot of the VM
if it has plugged shared or direct LUN disks)

2. Taking the snapshot and marking the shared disk and direct LUN disks
as unplugged (in the VM snapshot configuration) and marking the snapshot
state as partial.

3. Taking the snapshot of the VM as is, leaving the VM configuration
with plugged disks.


The issue with including these disks in the snapshot is that they are
not really being snapshotted, they are not capturing the point in time
we are trying to achieve in the snapshot.

Enabling the snapshot action in such a state is a bit misleading to the
user.

If we do allow taking the snapshot we should mark the snapshot as
partial to indicate that the snapshot did not capture the point in time
as the user intended.

I have no preference with regards to the second and third approach, the
second approach is a bit more safe, we basically force the user to plug
the disks and be sure that he knows what he is doing and the third
approach is less safe and less annoying to the user (he took the
snapshot, cloned it and wants to start the VM - don't require extra actions)

Kolesnik - please note when starting VM in a preview mode we should
mount the disks in read-only mode (if supported).


Livnat







From ofrenkel at redhat.com  Thu Jan 19 08:25:54 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Thu, 19 Jan 2012 03:25:54 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
	context of shared disks and direct LUNs-based disks
In-Reply-To: <4F17C418.30004@redhat.com>
Message-ID: <ee6d5729-138a-40c4-85f4-0a5ee6113421@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik" <mkolesni at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Thursday, January 19, 2012 9:19:52 AM
> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot feature in context of shared disks and direct
> LUNs-based disks
> 
> On 19/01/12 08:38, Yair Zaslavsky wrote:
> > Hi all,
> > Following the upstream meeting dated Wednesday January 18th, 2012 -
> > I presented the clone VM from snpashot feature and we discussed the
> > feature behaviour.
> > 
> > Two issues that were raised are the behaviour of the feature in
> > context
> > of shared disks and direct LUNs-based disks -
> > On one hand, if we copy&collapse such images - this may yield to
> > data
> > corruption (think of a scenario where the source and destination
> > VMs use
> > the same disk).
> > On the other hand - if we decide not to copy&collapse  - the target
> > VM
> > will have missing VM and its state will not totally reflect the
> > logical
> > state.
> > One of the solution raises is to mark such disks (at the
> > destination) as
> > unplugged, allowing the administrator the ability to plug them
> > (understanding of course the consequences).
> > 
> > I would like to receive inputs on this issue
> > 
> > 
> > Kind regards,
> > 
> > Yair
> 
> Hi Yair,
> 
> Some clarifications on the above issue.
> Currently when taking a snapshot on a VM with shared disks or direct
> LUN
> disk there are 3 optional behaviors:
> 
> 1. Blocking the snapshot action. (User can not take a snapshot of the
> VM
> if it has plugged shared or direct LUN disks)
> 
> 2. Taking the snapshot and marking the shared disk and direct LUN
> disks
> as unplugged (in the VM snapshot configuration) and marking the
> snapshot
> state as partial.
> 
> 3. Taking the snapshot of the VM as is, leaving the VM configuration
> with plugged disks.
> 
> 
> The issue with including these disks in the snapshot is that they are
> not really being snapshotted, they are not capturing the point in
> time
> we are trying to achieve in the snapshot.
> 
> Enabling the snapshot action in such a state is a bit misleading to
> the
> user.
> 
> If we do allow taking the snapshot we should mark the snapshot as
> partial to indicate that the snapshot did not capture the point in
> time
> as the user intended.
> 
> I have no preference with regards to the second and third approach,
> the
> second approach is a bit more safe, we basically force the user to
> plug
> the disks and be sure that he knows what he is doing and the third
> approach is less safe and less annoying to the user (he took the
> snapshot, cloned it and wants to start the VM - don't require extra
> actions)
> 
> Kolesnik - please note when starting VM in a preview mode we should
> mount the disks in read-only mode (if supported).
> 
> 
> Livnat
> 
> 
> 
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 

+1 for option 3


From mkolesni at redhat.com  Thu Jan 19 08:32:54 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Thu, 19 Jan 2012 03:32:54 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
	context of shared disks and direct LUNs-based disks
In-Reply-To: <ee6d5729-138a-40c4-85f4-0a5ee6113421@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <34406727-ee63-465f-8250-56a9d3c7731f@zmail14.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Livnat Peer" <lpeer at redhat.com>
> > To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik"
> > <mkolesni at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Thursday, January 19, 2012 9:19:52 AM
> > Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
> > feature in context of shared disks and direct
> > LUNs-based disks
> > 
> > On 19/01/12 08:38, Yair Zaslavsky wrote:
> > > Hi all,
> > > Following the upstream meeting dated Wednesday January 18th, 2012
> > > -
> > > I presented the clone VM from snpashot feature and we discussed
> > > the
> > > feature behaviour.
> > > 
> > > Two issues that were raised are the behaviour of the feature in
> > > context
> > > of shared disks and direct LUNs-based disks -
> > > On one hand, if we copy&collapse such images - this may yield to
> > > data
> > > corruption (think of a scenario where the source and destination
> > > VMs use
> > > the same disk).
> > > On the other hand - if we decide not to copy&collapse  - the
> > > target
> > > VM
> > > will have missing VM and its state will not totally reflect the
> > > logical
> > > state.
> > > One of the solution raises is to mark such disks (at the
> > > destination) as
> > > unplugged, allowing the administrator the ability to plug them
> > > (understanding of course the consequences).
> > > 
> > > I would like to receive inputs on this issue
> > > 
> > > 
> > > Kind regards,
> > > 
> > > Yair
> > 
> > Hi Yair,
> > 
> > Some clarifications on the above issue.
> > Currently when taking a snapshot on a VM with shared disks or
> > direct
> > LUN
> > disk there are 3 optional behaviors:
> > 
> > 1. Blocking the snapshot action. (User can not take a snapshot of
> > the
> > VM
> > if it has plugged shared or direct LUN disks)
> > 
> > 2. Taking the snapshot and marking the shared disk and direct LUN
> > disks
> > as unplugged (in the VM snapshot configuration) and marking the
> > snapshot
> > state as partial.
> > 
> > 3. Taking the snapshot of the VM as is, leaving the VM
> > configuration
> > with plugged disks.
> > 
> > 
> > The issue with including these disks in the snapshot is that they
> > are
> > not really being snapshotted, they are not capturing the point in
> > time
> > we are trying to achieve in the snapshot.
> > 
> > Enabling the snapshot action in such a state is a bit misleading to
> > the
> > user.
> > 
> > If we do allow taking the snapshot we should mark the snapshot as
> > partial to indicate that the snapshot did not capture the point in
> > time
> > as the user intended.
> > 
> > I have no preference with regards to the second and third approach,
> > the
> > second approach is a bit more safe, we basically force the user to
> > plug
> > the disks and be sure that he knows what he is doing and the third
> > approach is less safe and less annoying to the user (he took the
> > snapshot, cloned it and wants to start the VM - don't require extra
> > actions)
> > 
> > Kolesnik - please note when starting VM in a preview mode we should
> > mount the disks in read-only mode (if supported).

I don't understand this, can you please elaborate why and in which case? 
  The disk is plugged/unplugged? 
  What happens when you commit? It becomes r/w?

> > 
> > 
> > Livnat
> > 
> > 
> > 
> 
> +1 for option 3
> 

+1 for option 3 as well (also good with option 1, but I think this will hinder usability).


Regards,
Mike


From mpastern at redhat.com  Thu Jan 19 08:40:28 2012
From: mpastern at redhat.com (Michael Pasternak)
Date: Thu, 19 Jan 2012 10:40:28 +0200
Subject: [Engine-devel] Any way to get more than 100 VMs with the REST
 API?
In-Reply-To: <4F172039.1090101@redhat.com>
References: <4F169419.7000409@redhat.com> <4F16ECA9.6030000@redhat.com>
	<4F172039.1090101@redhat.com>
Message-ID: <4F17D6FC.2000105@redhat.com>

On 01/18/2012 09:40 PM, Itamar Heim wrote:
> On 01/18/2012 06:00 PM, Livnat Peer wrote:
>> On 18/01/12 11:42, Juan Hernandez wrote:
>>> Hello,
>>>
>>> Is there any way to get more than 100 VMs (or whatever the value of the
>>> option "SearchResultsLimit" is) using the REST API?
>>>
>>
>> I think it is not modeled in the API.
>> Michael - do we expose a way to set search page size to REST API users?

only if this is (oVirt) searchable collection, i have RFE for general pagination support.

> 
> http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Virtualization/3.0/html-single/REST_API_Guide/index.html#sect-REST_API_Guide-Common_Features-Collections-Querying_Collections
> 
> 
> look at:
> 7.2.3.3. Pagination


-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D


From abaron at redhat.com  Thu Jan 19 09:22:01 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 19 Jan 2012 04:22:01 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature
	in	context of shared disks and direct LUNs-based disks
In-Reply-To: <34406727-ee63-465f-8250-56a9d3c7731f@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <7a2ce992-ddc8-4a7c-972c-fdb26761d903@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > 
> > 
> > ----- Original Message -----
> > > From: "Livnat Peer" <lpeer at redhat.com>
> > > To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik"
> > > <mkolesni at redhat.com>
> > > Cc: engine-devel at ovirt.org
> > > Sent: Thursday, January 19, 2012 9:19:52 AM
> > > Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
> > > feature in context of shared disks and direct
> > > LUNs-based disks
> > > 
> > > On 19/01/12 08:38, Yair Zaslavsky wrote:
> > > > Hi all,
> > > > Following the upstream meeting dated Wednesday January 18th,
> > > > 2012
> > > > -
> > > > I presented the clone VM from snpashot feature and we discussed
> > > > the
> > > > feature behaviour.
> > > > 
> > > > Two issues that were raised are the behaviour of the feature in
> > > > context
> > > > of shared disks and direct LUNs-based disks -
> > > > On one hand, if we copy&collapse such images - this may yield
> > > > to
> > > > data
> > > > corruption (think of a scenario where the source and
> > > > destination
> > > > VMs use
> > > > the same disk).
> > > > On the other hand - if we decide not to copy&collapse  - the
> > > > target
> > > > VM
> > > > will have missing VM and its state will not totally reflect the
> > > > logical
> > > > state.
> > > > One of the solution raises is to mark such disks (at the
> > > > destination) as
> > > > unplugged, allowing the administrator the ability to plug them
> > > > (understanding of course the consequences).
> > > > 
> > > > I would like to receive inputs on this issue
> > > > 
> > > > 
> > > > Kind regards,
> > > > 
> > > > Yair
> > > 
> > > Hi Yair,
> > > 
> > > Some clarifications on the above issue.
> > > Currently when taking a snapshot on a VM with shared disks or
> > > direct
> > > LUN
> > > disk there are 3 optional behaviors:
> > > 
> > > 1. Blocking the snapshot action. (User can not take a snapshot of
> > > the
> > > VM
> > > if it has plugged shared or direct LUN disks)

-1, user should be able to take snapshots.
Note that once we have integration with hardware side snapshots, we will be able to snapshot direct LUNs (when supported).

> > > 
> > > 2. Taking the snapshot and marking the shared disk and direct LUN
> > > disks
> > > as unplugged (in the VM snapshot configuration) and marking the
> > > snapshot
> > > state as partial.

+1, user has to specifically say 'I know that the direct LUN is not in the same state as it was when I took the snapshot but I know what I'm doing and I want to run the snapshot with the LUN as is'

> > > 
> > > 3. Taking the snapshot of the VM as is, leaving the VM
> > > configuration
> > > with plugged disks.

-1, this will lead to accidental corruptions.

> > > 
> > > 
> > > The issue with including these disks in the snapshot is that they
> > > are
> > > not really being snapshotted, they are not capturing the point in
> > > time
> > > we are trying to achieve in the snapshot.
> > > 
> > > Enabling the snapshot action in such a state is a bit misleading
> > > to
> > > the
> > > user.
> > > 
> > > If we do allow taking the snapshot we should mark the snapshot as
> > > partial to indicate that the snapshot did not capture the point
> > > in
> > > time
> > > as the user intended.

+1, the user should be aware of what she did.

> > > 
> > > I have no preference with regards to the second and third
> > > approach,
> > > the
> > > second approach is a bit more safe, we basically force the user
> > > to
> > > plug
> > > the disks and be sure that he knows what he is doing and the
> > > third
> > > approach is less safe and less annoying to the user (he took the
> > > snapshot, cloned it and wants to start the VM - don't require
> > > extra
> > > actions)
> > > 
> > > Kolesnik - please note when starting VM in a preview mode we
> > > should
> > > mount the disks in read-only mode (if supported).

Note that if the LUN contains a file system to be mounted and the disk is plugged in read only mode the guest will likely have a kernel panic when trying to mount the fs, I'm not sure this is the behaviour we want.

> 
> I don't understand this, can you please elaborate why and in which
> case?
>   The disk is plugged/unplugged?
>   What happens when you commit? It becomes r/w?

No, you mark it as read-only in the configuration when you create the snapshot.
User would have to manually change this to r/w.

I'm not sure we should both mark it as unplugged and r/o (although this is the safest option).
As it would be annoying to get the disk to actually work in r/w (plug + make r/w).
Andy, your thoughts on this?

> 
> > > 
> > > 
> > > Livnat
> > > 
> > > 
> > > 
> > 
> > +1 for option 3
> > 
> 
> +1 for option 3 as well (also good with option 1, but I think this
> will hinder usability).
> 
> 
> Regards,
> Mike
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From iheim at redhat.com  Thu Jan 19 09:49:54 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 19 Jan 2012 11:49:54 +0200
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <4F16EB0C.2030208@redhat.com>
References: <4F16EB0C.2030208@redhat.com>
Message-ID: <4F17E742.4090805@redhat.com>

On 01/18/2012 05:53 PM, Livnat Peer wrote:
> Hi All,
>
> This is what we've discussed in the meeting today:
>
> Multiple storage domain:
> - Should have a single generic verb for removing a disk.
> - We block removing the last template disk - template is immutable.

but it will be deleted when deleting the template, right?


From iheim at redhat.com  Thu Jan 19 09:53:52 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 19 Jan 2012 11:53:52 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <4F17C418.30004@redhat.com>
References: <4F17BA70.2060800@redhat.com> <4F17C418.30004@redhat.com>
Message-ID: <4F17E830.3050704@redhat.com>

On 01/19/2012 09:19 AM, Livnat Peer wrote:
> On 19/01/12 08:38, Yair Zaslavsky wrote:
>> Hi all,
>> Following the upstream meeting dated Wednesday January 18th, 2012 -
>> I presented the clone VM from snpashot feature and we discussed the
>> feature behaviour.
>>
>> Two issues that were raised are the behaviour of the feature in context
>> of shared disks and direct LUNs-based disks -
>> On one hand, if we copy&collapse such images - this may yield to data
>> corruption (think of a scenario where the source and destination VMs use
>> the same disk).
>> On the other hand - if we decide not to copy&collapse  - the target VM
>> will have missing VM and its state will not totally reflect the logical
>> state.
>> One of the solution raises is to mark such disks (at the destination) as
>> unplugged, allowing the administrator the ability to plug them
>> (understanding of course the consequences).
>>
>> I would like to receive inputs on this issue
>>
>>
>> Kind regards,
>>
>> Yair
>
> Hi Yair,
>
> Some clarifications on the above issue.
> Currently when taking a snapshot on a VM with shared disks or direct LUN
> disk there are 3 optional behaviors:
>
> 1. Blocking the snapshot action. (User can not take a snapshot of the VM
> if it has plugged shared or direct LUN disks)
>
> 2. Taking the snapshot and marking the shared disk and direct LUN disks
> as unplugged (in the VM snapshot configuration) and marking the snapshot
> state as partial.

just making sure - this wouldn't change the active running config (when 
taking a live snapshot)?
doesn't it mean we could take partial snapshot for any type of disks 
then (internal ones as well)?


From abaron at redhat.com  Thu Jan 19 09:58:11 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 19 Jan 2012 04:58:11 -0500 (EST)
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <4F17E742.4090805@redhat.com>
Message-ID: <746eed7f-20a0-4a81-9b83-3974c6d2098c@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/18/2012 05:53 PM, Livnat Peer wrote:
> > Hi All,
> >
> > This is what we've discussed in the meeting today:
> >
> > Multiple storage domain:
> > - Should have a single generic verb for removing a disk.
> > - We block removing the last template disk - template is immutable.
> 
> but it will be deleted when deleting the template, right?

Of course.
The point is that the template is an immutable object and should not change (until we support editing a template at which point the user would have to change the template to edit mode before being able to make such changes and maybe also be able to run it and make changes internally?).

> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From abaron at redhat.com  Thu Jan 19 10:03:44 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 19 Jan 2012 05:03:44 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
	context of shared disks and direct LUNs-based disks
In-Reply-To: <4F17E830.3050704@redhat.com>
Message-ID: <f292085a-76a1-4208-ac60-5c68b67a489a@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/19/2012 09:19 AM, Livnat Peer wrote:
> > On 19/01/12 08:38, Yair Zaslavsky wrote:
> >> Hi all,
> >> Following the upstream meeting dated Wednesday January 18th, 2012
> >> -
> >> I presented the clone VM from snpashot feature and we discussed
> >> the
> >> feature behaviour.
> >>
> >> Two issues that were raised are the behaviour of the feature in
> >> context
> >> of shared disks and direct LUNs-based disks -
> >> On one hand, if we copy&collapse such images - this may yield to
> >> data
> >> corruption (think of a scenario where the source and destination
> >> VMs use
> >> the same disk).
> >> On the other hand - if we decide not to copy&collapse  - the
> >> target VM
> >> will have missing VM and its state will not totally reflect the
> >> logical
> >> state.
> >> One of the solution raises is to mark such disks (at the
> >> destination) as
> >> unplugged, allowing the administrator the ability to plug them
> >> (understanding of course the consequences).
> >>
> >> I would like to receive inputs on this issue
> >>
> >>
> >> Kind regards,
> >>
> >> Yair
> >
> > Hi Yair,
> >
> > Some clarifications on the above issue.
> > Currently when taking a snapshot on a VM with shared disks or
> > direct LUN
> > disk there are 3 optional behaviors:
> >
> > 1. Blocking the snapshot action. (User can not take a snapshot of
> > the VM
> > if it has plugged shared or direct LUN disks)
> >
> > 2. Taking the snapshot and marking the shared disk and direct LUN
> > disks
> > as unplugged (in the VM snapshot configuration) and marking the
> > snapshot
> > state as partial.
> 
> just making sure - this wouldn't change the active running config
> (when
> taking a live snapshot)?

no, only the config of the snapshot.

> doesn't it mean we could take partial snapshot for any type of disks
> then (internal ones as well)?

To do this properly you would need to drive it from the GUI.  Currently the behaviour was defined as implicit (engine would automatically do this according to disk type).

In any event, even if partial snapshots will not be available in 3.1, the API should get the list of LUNs to snapshot and just raise an error if not all devices were passed.
Otherwise we'd need a new API for partial snapshots (I'm talking about REST API in case this isn't clear).

> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From eedri at redhat.com  Thu Jan 19 10:07:24 2012
From: eedri at redhat.com (Eyal Edri)
Date: Thu, 19 Jan 2012 05:07:24 -0500 (EST)
Subject: [Engine-devel] Jenkins Continuous Integration Server for oVirt is
	up and running!
In-Reply-To: <923b70d3-3243-4d94-b36a-4b58b54e056b@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <ab2db7af-60e7-41ba-ba00-f12d57e8a328@zmail13.collab.prod.int.phx2.redhat.com>

fyi,

oVirt project now has a Jenkins CI server[1] on http://jenkins.ovirt.org.

The CI server runs various jobs on oVirt components *[2] such as ovirt-engine,ovirt-node,etc..  
Every commit to gerrit.ovirt.org will trigger the job 'ovirt_engine' which will run 'maven' build and verify that the commit didn't break the code.
If the commit did break the code, it will send an alert email to "engine-patches.ovirt.org" and to the commiter with a link to a log console containing the error.
On success, the job will trigger other jobs such as "find_bugs", "gwt profiles", "create db", "unit-tests", each testing a different part of the code. 

In time, more and more jobs will be added to jenkins, which will allow us to catch bugs much faster than before, and to improve code quality even more.
If you have questions, don't hesitate to ask me or infra at ovirt.org. 

[1] http://jenkins-ci.org/
[2] currently only ovirt-engine is configured and working. 


Eyal Edri
oVirt infrastructure team


From lpeer at redhat.com  Thu Jan 19 11:45:58 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Thu, 19 Jan 2012 13:45:58 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <f292085a-76a1-4208-ac60-5c68b67a489a@zmail13.collab.prod.int.phx2.redhat.com>
References: <f292085a-76a1-4208-ac60-5c68b67a489a@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F180276.3040706@redhat.com>

On 19/01/12 12:03, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> On 01/19/2012 09:19 AM, Livnat Peer wrote:
>>> On 19/01/12 08:38, Yair Zaslavsky wrote:
>>>> Hi all,
>>>> Following the upstream meeting dated Wednesday January 18th, 2012
>>>> -
>>>> I presented the clone VM from snpashot feature and we discussed
>>>> the
>>>> feature behaviour.
>>>>
>>>> Two issues that were raised are the behaviour of the feature in
>>>> context
>>>> of shared disks and direct LUNs-based disks -
>>>> On one hand, if we copy&collapse such images - this may yield to
>>>> data
>>>> corruption (think of a scenario where the source and destination
>>>> VMs use
>>>> the same disk).
>>>> On the other hand - if we decide not to copy&collapse  - the
>>>> target VM
>>>> will have missing VM and its state will not totally reflect the
>>>> logical
>>>> state.
>>>> One of the solution raises is to mark such disks (at the
>>>> destination) as
>>>> unplugged, allowing the administrator the ability to plug them
>>>> (understanding of course the consequences).
>>>>
>>>> I would like to receive inputs on this issue
>>>>
>>>>
>>>> Kind regards,
>>>>
>>>> Yair
>>>
>>> Hi Yair,
>>>
>>> Some clarifications on the above issue.
>>> Currently when taking a snapshot on a VM with shared disks or
>>> direct LUN
>>> disk there are 3 optional behaviors:
>>>
>>> 1. Blocking the snapshot action. (User can not take a snapshot of
>>> the VM
>>> if it has plugged shared or direct LUN disks)
>>>
>>> 2. Taking the snapshot and marking the shared disk and direct LUN
>>> disks
>>> as unplugged (in the VM snapshot configuration) and marking the
>>> snapshot
>>> state as partial.
>>
>> just making sure - this wouldn't change the active running config
>> (when
>> taking a live snapshot)?
> 
> no, only the config of the snapshot.
> 
>> doesn't it mean we could take partial snapshot for any type of disks
>> then (internal ones as well)?

Not in this version, we added the concept of partial snapshot to enable
deleting a disk from a VM with snapshots. We mark all snapshots that
used this disk as partial.

> 
> To do this properly you would need to drive it from the GUI.  Currently the behaviour was defined as implicit (engine would automatically do this according to disk type).
> 
> In any event, even if partial snapshots will not be available in 3.1, the API should get the list of LUNs to snapshot and just raise an error if not all devices were passed.
> Otherwise we'd need a new API for partial snapshots (I'm talking about REST API in case this isn't clear).
> 
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From yzaslavs at redhat.com  Thu Jan 19 11:54:56 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Thu, 19 Jan 2012 13:54:56 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <34406727-ee63-465f-8250-56a9d3c7731f@zmail14.collab.prod.int.phx2.redhat.com>
References: <34406727-ee63-465f-8250-56a9d3c7731f@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F180490.9090602@redhat.com>

On 01/19/2012 10:32 AM, Mike Kolesnik wrote:
> 
> 
> ----- Original Message -----
>>
>>
>> ----- Original Message -----
>>> From: "Livnat Peer" <lpeer at redhat.com>
>>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik"
>>> <mkolesni at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Thursday, January 19, 2012 9:19:52 AM
>>> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
>>> feature in context of shared disks and direct
>>> LUNs-based disks
>>>
>>> On 19/01/12 08:38, Yair Zaslavsky wrote:
>>>> Hi all,
>>>> Following the upstream meeting dated Wednesday January 18th, 2012
>>>> -
>>>> I presented the clone VM from snpashot feature and we discussed
>>>> the
>>>> feature behaviour.
>>>>
>>>> Two issues that were raised are the behaviour of the feature in
>>>> context
>>>> of shared disks and direct LUNs-based disks -
>>>> On one hand, if we copy&collapse such images - this may yield to
>>>> data
>>>> corruption (think of a scenario where the source and destination
>>>> VMs use
>>>> the same disk).
>>>> On the other hand - if we decide not to copy&collapse  - the
>>>> target
>>>> VM
>>>> will have missing VM and its state will not totally reflect the
>>>> logical
>>>> state.
>>>> One of the solution raises is to mark such disks (at the
>>>> destination) as
>>>> unplugged, allowing the administrator the ability to plug them
>>>> (understanding of course the consequences).
>>>>
>>>> I would like to receive inputs on this issue
>>>>
>>>>
>>>> Kind regards,
>>>>
>>>> Yair
>>>
>>> Hi Yair,
>>>
>>> Some clarifications on the above issue.
>>> Currently when taking a snapshot on a VM with shared disks or
>>> direct
>>> LUN
>>> disk there are 3 optional behaviors:
>>>
>>> 1. Blocking the snapshot action. (User can not take a snapshot of
>>> the
>>> VM
>>> if it has plugged shared or direct LUN disks)
>>>
>>> 2. Taking the snapshot and marking the shared disk and direct LUN
>>> disks
>>> as unplugged (in the VM snapshot configuration) and marking the
>>> snapshot
>>> state as partial.
>>>
>>> 3. Taking the snapshot of the VM as is, leaving the VM
>>> configuration
>>> with plugged disks.
>>>
>>>
>>> The issue with including these disks in the snapshot is that they
>>> are
>>> not really being snapshotted, they are not capturing the point in
>>> time
>>> we are trying to achieve in the snapshot.
>>>
>>> Enabling the snapshot action in such a state is a bit misleading to
>>> the
>>> user.
>>>
>>> If we do allow taking the snapshot we should mark the snapshot as
>>> partial to indicate that the snapshot did not capture the point in
>>> time
>>> as the user intended.
>>>
>>> I have no preference with regards to the second and third approach,
>>> the
>>> second approach is a bit more safe, we basically force the user to
>>> plug
>>> the disks and be sure that he knows what he is doing and the third
>>> approach is less safe and less annoying to the user (he took the
>>> snapshot, cloned it and wants to start the VM - don't require extra
>>> actions)
>>>
>>> Kolesnik - please note when starting VM in a preview mode we should
>>> mount the disks in read-only mode (if supported).
> 
> I don't understand this, can you please elaborate why and in which case? 
>   The disk is plugged/unplugged? 
>   What happens when you commit? It becomes r/w?
> 
>>>
>>>
>>> Livnat
>>>
>>>
>>>
>>
>> +1 for option 3
>>
> 
> +1 for option 3 as well (also good with option 1, but I think this will hinder usability).
I agree with Mike - I think option one is too "aggressive" in protecting
us, this is why I prefer 3.  +1 on option 3
> 
> 
> Regards,
> Mike



From abaron at redhat.com  Thu Jan 19 12:04:02 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 19 Jan 2012 07:04:02 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
	context of shared disks and direct LUNs-based disks
In-Reply-To: <4F180490.9090602@redhat.com>
Message-ID: <415a49ae-2b7b-41ae-ac4f-5849a012dcfe@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/19/2012 10:32 AM, Mike Kolesnik wrote:
> > 
> > 
> > ----- Original Message -----
> >>
> >>
> >> ----- Original Message -----
> >>> From: "Livnat Peer" <lpeer at redhat.com>
> >>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik"
> >>> <mkolesni at redhat.com>
> >>> Cc: engine-devel at ovirt.org
> >>> Sent: Thursday, January 19, 2012 9:19:52 AM
> >>> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
> >>> feature in context of shared disks and direct
> >>> LUNs-based disks
> >>>
> >>> On 19/01/12 08:38, Yair Zaslavsky wrote:
> >>>> Hi all,
> >>>> Following the upstream meeting dated Wednesday January 18th,
> >>>> 2012
> >>>> -
> >>>> I presented the clone VM from snpashot feature and we discussed
> >>>> the
> >>>> feature behaviour.
> >>>>
> >>>> Two issues that were raised are the behaviour of the feature in
> >>>> context
> >>>> of shared disks and direct LUNs-based disks -
> >>>> On one hand, if we copy&collapse such images - this may yield to
> >>>> data
> >>>> corruption (think of a scenario where the source and destination
> >>>> VMs use
> >>>> the same disk).
> >>>> On the other hand - if we decide not to copy&collapse  - the
> >>>> target
> >>>> VM
> >>>> will have missing VM and its state will not totally reflect the
> >>>> logical
> >>>> state.
> >>>> One of the solution raises is to mark such disks (at the
> >>>> destination) as
> >>>> unplugged, allowing the administrator the ability to plug them
> >>>> (understanding of course the consequences).
> >>>>
> >>>> I would like to receive inputs on this issue
> >>>>
> >>>>
> >>>> Kind regards,
> >>>>
> >>>> Yair
> >>>
> >>> Hi Yair,
> >>>
> >>> Some clarifications on the above issue.
> >>> Currently when taking a snapshot on a VM with shared disks or
> >>> direct
> >>> LUN
> >>> disk there are 3 optional behaviors:
> >>>
> >>> 1. Blocking the snapshot action. (User can not take a snapshot of
> >>> the
> >>> VM
> >>> if it has plugged shared or direct LUN disks)
> >>>
> >>> 2. Taking the snapshot and marking the shared disk and direct LUN
> >>> disks
> >>> as unplugged (in the VM snapshot configuration) and marking the
> >>> snapshot
> >>> state as partial.
> >>>
> >>> 3. Taking the snapshot of the VM as is, leaving the VM
> >>> configuration
> >>> with plugged disks.
> >>>
> >>>
> >>> The issue with including these disks in the snapshot is that they
> >>> are
> >>> not really being snapshotted, they are not capturing the point in
> >>> time
> >>> we are trying to achieve in the snapshot.
> >>>
> >>> Enabling the snapshot action in such a state is a bit misleading
> >>> to
> >>> the
> >>> user.
> >>>
> >>> If we do allow taking the snapshot we should mark the snapshot as
> >>> partial to indicate that the snapshot did not capture the point
> >>> in
> >>> time
> >>> as the user intended.
> >>>
> >>> I have no preference with regards to the second and third
> >>> approach,
> >>> the
> >>> second approach is a bit more safe, we basically force the user
> >>> to
> >>> plug
> >>> the disks and be sure that he knows what he is doing and the
> >>> third
> >>> approach is less safe and less annoying to the user (he took the
> >>> snapshot, cloned it and wants to start the VM - don't require
> >>> extra
> >>> actions)
> >>>
> >>> Kolesnik - please note when starting VM in a preview mode we
> >>> should
> >>> mount the disks in read-only mode (if supported).
> > 
> > I don't understand this, can you please elaborate why and in which
> > case?
> >   The disk is plugged/unplugged?
> >   What happens when you commit? It becomes r/w?
> > 
> >>>
> >>>
> >>> Livnat
> >>>
> >>>
> >>>
> >>
> >> +1 for option 3
> >>
> > 
> > +1 for option 3 as well (also good with option 1, but I think this
> > will hinder usability).
> I agree with Mike - I think option one is too "aggressive" in
> protecting
> us, this is why I prefer 3.  +1 on option 3

This would only be acceptable if the disk is marked read only.
Just leaving the disk plugged means many users *will* corrupt their VMs.
That trumps the need to mark a checkbox if you want it available.

As I said before, readonly has its own problems and in fact, IMO the behaviour is even more difficult to explain (yeah, you might corrupt the running VM if it's r/w so we made it r/o and now if you start up your clone / preview you will get a kernel panic).

> > 
> > 
> > Regards,
> > Mike
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From oschreib at redhat.com  Thu Jan 19 13:25:26 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Thu, 19 Jan 2012 08:25:26 -0500 (EST)
Subject: [Engine-devel] First release blockers
Message-ID: <017d01ccd6ad$cf0af5b0$6d20e110$@redhat.com>

As part of summarizing the awesome test day we had yesterday (Thanks Moran
and all participants), I've created
http://www.ovirt.org/wiki/Releases/First_Release_Blockers.

If you think you noticed/opened/fixed a critical bug, please add it to the
wiki above.

Thanks for cooperation,

--
Ofer Schreiber 
Red Hat Israel, Ra'anana

+972-9-7692347
www.redhat.com




From abaron at redhat.com  Thu Jan 19 14:05:08 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 19 Jan 2012 09:05:08 -0500 (EST)
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <138723b0-43ed-4271-a4ef-1eaecf5807e4@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <1cdfa509-3d19-469f-a74e-ca009fcd89bf@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> Hey all,
> 
> Continuing the discussion about Aeolus instance data injection to a
> VM
> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> we propose a new VM Payload feature.
> 
> The following wiki page contains a description page of the feature.
> http://www.ovirt.org/wiki/Features/VMPayload
> 
> Please read and review.
> There are several approaches there, and we wish to head your opinions
> and thoughts about them.
> 
> Once we agree on an approach, we will start designing.

Permanent payload availability requires determining where the payload is stored.
Makes sense to me to store it together with the VM disks on the storage domain, but that requires the small object store which will not be available in the coming version (payloads can be large and keeping them in the DB and passing over the net every time the VM is run doesn't make much sense).

Wrt availability, I don't see a reason to exclude attaching both a CD and a payload via another CD at the same time (i.e. multiple devices).

> 
> Thank you,
> Oved
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkenneth at redhat.com  Thu Jan 19 18:41:07 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Thu, 19 Jan 2012 13:41:07 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature
	in	context of shared disks and direct LUNs-based disks
In-Reply-To: <415a49ae-2b7b-41ae-ac4f-5849a012dcfe@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <64f84f20-e078-4867-afe6-d84d9dedf8f5@mkenneth.csb>

Top Posting:

>From user POV I think that option 2 is the only one that make sense. We try to do as much as we can, 
and on each "problematic" case, we make him aware and let him decide.

Miki



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Yair Zaslavsky" <yzaslavs at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Thursday, January 19, 2012 4:04:02 AM
> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot feature in	context of shared disks and direct
> LUNs-based disks
> 
> 
> 
> ----- Original Message -----
> > On 01/19/2012 10:32 AM, Mike Kolesnik wrote:
> > > 
> > > 
> > > ----- Original Message -----
> > >>
> > >>
> > >> ----- Original Message -----
> > >>> From: "Livnat Peer" <lpeer at redhat.com>
> > >>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik"
> > >>> <mkolesni at redhat.com>
> > >>> Cc: engine-devel at ovirt.org
> > >>> Sent: Thursday, January 19, 2012 9:19:52 AM
> > >>> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
> > >>> feature in context of shared disks and direct
> > >>> LUNs-based disks
> > >>>
> > >>> On 19/01/12 08:38, Yair Zaslavsky wrote:
> > >>>> Hi all,
> > >>>> Following the upstream meeting dated Wednesday January 18th,
> > >>>> 2012
> > >>>> -
> > >>>> I presented the clone VM from snpashot feature and we
> > >>>> discussed
> > >>>> the
> > >>>> feature behaviour.
> > >>>>
> > >>>> Two issues that were raised are the behaviour of the feature
> > >>>> in
> > >>>> context
> > >>>> of shared disks and direct LUNs-based disks -
> > >>>> On one hand, if we copy&collapse such images - this may yield
> > >>>> to
> > >>>> data
> > >>>> corruption (think of a scenario where the source and
> > >>>> destination
> > >>>> VMs use
> > >>>> the same disk).
> > >>>> On the other hand - if we decide not to copy&collapse  - the
> > >>>> target
> > >>>> VM
> > >>>> will have missing VM and its state will not totally reflect
> > >>>> the
> > >>>> logical
> > >>>> state.
> > >>>> One of the solution raises is to mark such disks (at the
> > >>>> destination) as
> > >>>> unplugged, allowing the administrator the ability to plug them
> > >>>> (understanding of course the consequences).
> > >>>>
> > >>>> I would like to receive inputs on this issue
> > >>>>
> > >>>>
> > >>>> Kind regards,
> > >>>>
> > >>>> Yair
> > >>>
> > >>> Hi Yair,
> > >>>
> > >>> Some clarifications on the above issue.
> > >>> Currently when taking a snapshot on a VM with shared disks or
> > >>> direct
> > >>> LUN
> > >>> disk there are 3 optional behaviors:
> > >>>
> > >>> 1. Blocking the snapshot action. (User can not take a snapshot
> > >>> of
> > >>> the
> > >>> VM
> > >>> if it has plugged shared or direct LUN disks)
> > >>>
> > >>> 2. Taking the snapshot and marking the shared disk and direct
> > >>> LUN
> > >>> disks
> > >>> as unplugged (in the VM snapshot configuration) and marking the
> > >>> snapshot
> > >>> state as partial.
> > >>>
> > >>> 3. Taking the snapshot of the VM as is, leaving the VM
> > >>> configuration
> > >>> with plugged disks.
> > >>>
> > >>>
> > >>> The issue with including these disks in the snapshot is that
> > >>> they
> > >>> are
> > >>> not really being snapshotted, they are not capturing the point
> > >>> in
> > >>> time
> > >>> we are trying to achieve in the snapshot.
> > >>>
> > >>> Enabling the snapshot action in such a state is a bit
> > >>> misleading
> > >>> to
> > >>> the
> > >>> user.
> > >>>
> > >>> If we do allow taking the snapshot we should mark the snapshot
> > >>> as
> > >>> partial to indicate that the snapshot did not capture the point
> > >>> in
> > >>> time
> > >>> as the user intended.
> > >>>
> > >>> I have no preference with regards to the second and third
> > >>> approach,
> > >>> the
> > >>> second approach is a bit more safe, we basically force the user
> > >>> to
> > >>> plug
> > >>> the disks and be sure that he knows what he is doing and the
> > >>> third
> > >>> approach is less safe and less annoying to the user (he took
> > >>> the
> > >>> snapshot, cloned it and wants to start the VM - don't require
> > >>> extra
> > >>> actions)
> > >>>
> > >>> Kolesnik - please note when starting VM in a preview mode we
> > >>> should
> > >>> mount the disks in read-only mode (if supported).
> > > 
> > > I don't understand this, can you please elaborate why and in
> > > which
> > > case?
> > >   The disk is plugged/unplugged?
> > >   What happens when you commit? It becomes r/w?
> > > 
> > >>>
> > >>>
> > >>> Livnat
> > >>>
> > >>>
> > >>>
> > >>
> > >> +1 for option 3
> > >>
> > > 
> > > +1 for option 3 as well (also good with option 1, but I think
> > > this
> > > will hinder usability).
> > I agree with Mike - I think option one is too "aggressive" in
> > protecting
> > us, this is why I prefer 3.  +1 on option 3
> 
> This would only be acceptable if the disk is marked read only.
> Just leaving the disk plugged means many users *will* corrupt their
> VMs.
> That trumps the need to mark a checkbox if you want it available.
> 
> As I said before, readonly has its own problems and in fact, IMO the
> behaviour is even more difficult to explain (yeah, you might corrupt
> the running VM if it's r/w so we made it r/o and now if you start up
> your clone / preview you will get a kernel panic).
> 
> > > 
> > > 
> > > Regards,
> > > Mike
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From danken at redhat.com  Thu Jan 19 23:56:11 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Fri, 20 Jan 2012 01:56:11 +0200
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <32db41e6-8fec-4da0-8880-d0f616836bf9@zmail13.collab.prod.int.phx2.redhat.com>
References: <20120118104422.GA30396@redhat.com>
	<32db41e6-8fec-4da0-8880-d0f616836bf9@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <20120119235610.GH25773@redhat.com>

On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
> Can we broaden the scope and also allow passing createVG partitioned devices with an override flag or something? (we'd need to check the devices and run "kpartx -d" and fdisk to clean the devices before calling pvcreate).

We can, and we should. My initial patch is just the bare minimum; I'd like
Douglas to carry it on, and I am still waiting to meet his Engine counterpart.
Currently, a LUN that was once used as a raw hard disk cannot be used by RHEV;
that's sad.

How about this for API:

   createVG(self, vgname, devlist, options={trashpart_devlist: []})

createVG would honor an optional list of devices (subset of devlist) whose
partition tables should be trashed.

Dan.


From abaron at redhat.com  Fri Jan 20 07:07:05 2012
From: abaron at redhat.com (Ayal Baron)
Date: Fri, 20 Jan 2012 02:07:05 -0500 (EST)
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <20120119235610.GH25773@redhat.com>
Message-ID: <a184a066-5115-4c17-9072-190e6f914777@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
> > Can we broaden the scope and also allow passing createVG
> > partitioned devices with an override flag or something? (we'd need
> > to check the devices and run "kpartx -d" and fdisk to clean the
> > devices before calling pvcreate).
> 
> We can, and we should. My initial patch is just the bare minimum; I'd
> like
> Douglas to carry it on, and I am still waiting to meet his Engine
> counterpart.
> Currently, a LUN that was once used as a raw hard disk cannot be used
> by RHEV;
> that's sad.
> 
> How about this for API:
> 
>    createVG(self, vgname, devlist, options={trashpart_devlist: []})

No, stop using options as a trash can.
If we're changing the API, it should already be just passing a LUNs dictionary to createStorageDomain and start deprecating createVG

> 
> createVG would honor an optional list of devices (subset of devlist)
> whose
> partition tables should be trashed.
> 
> Dan.
> 


From abaron at redhat.com  Fri Jan 20 07:35:41 2012
From: abaron at redhat.com (Ayal Baron)
Date: Fri, 20 Jan 2012 02:35:41 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature
	in	context of shared disks and direct LUNs-based disks
In-Reply-To: <64f84f20-e078-4867-afe6-d84d9dedf8f5@mkenneth.csb>
Message-ID: <e46da1a1-4bc0-4992-aa41-6bff7076c144@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> Top Posting:
> 
> From user POV I think that option 2 is the only one that make sense.
> We try to do as much as we can,
> and on each "problematic" case, we make him aware and let him decide.
> 

Yep, +1.

> Miki
> 
> 
> 
> ----- Original Message -----
> > From: "Ayal Baron" <abaron at redhat.com>
> > To: "Yair Zaslavsky" <yzaslavs at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Thursday, January 19, 2012 4:04:02 AM
> > Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
> > feature in	context of shared disks and direct
> > LUNs-based disks
> > 
> > 
> > 
> > ----- Original Message -----
> > > On 01/19/2012 10:32 AM, Mike Kolesnik wrote:
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > >>
> > > >>
> > > >> ----- Original Message -----
> > > >>> From: "Livnat Peer" <lpeer at redhat.com>
> > > >>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik"
> > > >>> <mkolesni at redhat.com>
> > > >>> Cc: engine-devel at ovirt.org
> > > >>> Sent: Thursday, January 19, 2012 9:19:52 AM
> > > >>> Subject: Re: [Engine-devel] Question about
> > > >>> CloneVMFromSnapshot
> > > >>> feature in context of shared disks and direct
> > > >>> LUNs-based disks
> > > >>>
> > > >>> On 19/01/12 08:38, Yair Zaslavsky wrote:
> > > >>>> Hi all,
> > > >>>> Following the upstream meeting dated Wednesday January 18th,
> > > >>>> 2012
> > > >>>> -
> > > >>>> I presented the clone VM from snpashot feature and we
> > > >>>> discussed
> > > >>>> the
> > > >>>> feature behaviour.
> > > >>>>
> > > >>>> Two issues that were raised are the behaviour of the feature
> > > >>>> in
> > > >>>> context
> > > >>>> of shared disks and direct LUNs-based disks -
> > > >>>> On one hand, if we copy&collapse such images - this may
> > > >>>> yield
> > > >>>> to
> > > >>>> data
> > > >>>> corruption (think of a scenario where the source and
> > > >>>> destination
> > > >>>> VMs use
> > > >>>> the same disk).
> > > >>>> On the other hand - if we decide not to copy&collapse  - the
> > > >>>> target
> > > >>>> VM
> > > >>>> will have missing VM and its state will not totally reflect
> > > >>>> the
> > > >>>> logical
> > > >>>> state.
> > > >>>> One of the solution raises is to mark such disks (at the
> > > >>>> destination) as
> > > >>>> unplugged, allowing the administrator the ability to plug
> > > >>>> them
> > > >>>> (understanding of course the consequences).
> > > >>>>
> > > >>>> I would like to receive inputs on this issue
> > > >>>>
> > > >>>>
> > > >>>> Kind regards,
> > > >>>>
> > > >>>> Yair
> > > >>>
> > > >>> Hi Yair,
> > > >>>
> > > >>> Some clarifications on the above issue.
> > > >>> Currently when taking a snapshot on a VM with shared disks or
> > > >>> direct
> > > >>> LUN
> > > >>> disk there are 3 optional behaviors:
> > > >>>
> > > >>> 1. Blocking the snapshot action. (User can not take a
> > > >>> snapshot
> > > >>> of
> > > >>> the
> > > >>> VM
> > > >>> if it has plugged shared or direct LUN disks)
> > > >>>
> > > >>> 2. Taking the snapshot and marking the shared disk and direct
> > > >>> LUN
> > > >>> disks
> > > >>> as unplugged (in the VM snapshot configuration) and marking
> > > >>> the
> > > >>> snapshot
> > > >>> state as partial.
> > > >>>
> > > >>> 3. Taking the snapshot of the VM as is, leaving the VM
> > > >>> configuration
> > > >>> with plugged disks.
> > > >>>
> > > >>>
> > > >>> The issue with including these disks in the snapshot is that
> > > >>> they
> > > >>> are
> > > >>> not really being snapshotted, they are not capturing the
> > > >>> point
> > > >>> in
> > > >>> time
> > > >>> we are trying to achieve in the snapshot.
> > > >>>
> > > >>> Enabling the snapshot action in such a state is a bit
> > > >>> misleading
> > > >>> to
> > > >>> the
> > > >>> user.
> > > >>>
> > > >>> If we do allow taking the snapshot we should mark the
> > > >>> snapshot
> > > >>> as
> > > >>> partial to indicate that the snapshot did not capture the
> > > >>> point
> > > >>> in
> > > >>> time
> > > >>> as the user intended.
> > > >>>
> > > >>> I have no preference with regards to the second and third
> > > >>> approach,
> > > >>> the
> > > >>> second approach is a bit more safe, we basically force the
> > > >>> user
> > > >>> to
> > > >>> plug
> > > >>> the disks and be sure that he knows what he is doing and the
> > > >>> third
> > > >>> approach is less safe and less annoying to the user (he took
> > > >>> the
> > > >>> snapshot, cloned it and wants to start the VM - don't require
> > > >>> extra
> > > >>> actions)
> > > >>>
> > > >>> Kolesnik - please note when starting VM in a preview mode we
> > > >>> should
> > > >>> mount the disks in read-only mode (if supported).
> > > > 
> > > > I don't understand this, can you please elaborate why and in
> > > > which
> > > > case?
> > > >   The disk is plugged/unplugged?
> > > >   What happens when you commit? It becomes r/w?
> > > > 
> > > >>>
> > > >>>
> > > >>> Livnat
> > > >>>
> > > >>>
> > > >>>
> > > >>
> > > >> +1 for option 3
> > > >>
> > > > 
> > > > +1 for option 3 as well (also good with option 1, but I think
> > > > this
> > > > will hinder usability).
> > > I agree with Mike - I think option one is too "aggressive" in
> > > protecting
> > > us, this is why I prefer 3.  +1 on option 3
> > 
> > This would only be acceptable if the disk is marked read only.
> > Just leaving the disk plugged means many users *will* corrupt their
> > VMs.
> > That trumps the need to mark a checkbox if you want it available.
> > 
> > As I said before, readonly has its own problems and in fact, IMO
> > the
> > behaviour is even more difficult to explain (yeah, you might
> > corrupt
> > the running VM if it's r/w so we made it r/o and now if you start
> > up
> > your clone / preview you will get a kernel panic).
> > 
> > > > 
> > > > 
> > > > Regards,
> > > > Mike
> > > 
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 


From lpeer at redhat.com  Fri Jan 20 10:01:02 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Fri, 20 Jan 2012 12:01:02 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <e46da1a1-4bc0-4992-aa41-6bff7076c144@zmail13.collab.prod.int.phx2.redhat.com>
References: <e46da1a1-4bc0-4992-aa41-6bff7076c144@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F193B5E.1050200@redhat.com>

On 20/01/12 09:35, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> Top Posting:
>>
>> From user POV I think that option 2 is the only one that make sense.
>> We try to do as much as we can,
>> and on each "problematic" case, we make him aware and let him decide.
>>
> 
> Yep, +1.
> 

Trying to get to a conclusion here,
3 different people said on this thread that they think that from the
user perspective leaving the shared devices plugged is what they think
is the best behavior to the user. (Omer, Kolesnik, Yair)

On the other hand we have 2 people who think that protecting the user is
more important than leaving the VM configuration as it was in the
original VM (Miki, Ayal).

Ayal/Miki can you please specify what are we protecting the user from?


I think that because we are not snapshotting the shared disk and the
direct LUN they should not be part of the VM configuration (in the
snapshot) at all. we can not promise the user that the disk will be
there and if it is there we can not guarantee it is in the same state as
it was when we took the snapshot.


Another issue,

I can not see a reason to limit this feature to creating a VM from
snapshot and not a template? Almost no extra work is needed for
supporting templates as well.

Any reason not to?

Livnat


>> Miki
>>
>>
>>
>> ----- Original Message -----
>>> From: "Ayal Baron" <abaron at redhat.com>
>>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Thursday, January 19, 2012 4:04:02 AM
>>> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
>>> feature in	context of shared disks and direct
>>> LUNs-based disks
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> On 01/19/2012 10:32 AM, Mike Kolesnik wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>>>>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik"
>>>>>>> <mkolesni at redhat.com>
>>>>>>> Cc: engine-devel at ovirt.org
>>>>>>> Sent: Thursday, January 19, 2012 9:19:52 AM
>>>>>>> Subject: Re: [Engine-devel] Question about
>>>>>>> CloneVMFromSnapshot
>>>>>>> feature in context of shared disks and direct
>>>>>>> LUNs-based disks
>>>>>>>
>>>>>>> On 19/01/12 08:38, Yair Zaslavsky wrote:
>>>>>>>> Hi all,
>>>>>>>> Following the upstream meeting dated Wednesday January 18th,
>>>>>>>> 2012
>>>>>>>> -
>>>>>>>> I presented the clone VM from snpashot feature and we
>>>>>>>> discussed
>>>>>>>> the
>>>>>>>> feature behaviour.
>>>>>>>>
>>>>>>>> Two issues that were raised are the behaviour of the feature
>>>>>>>> in
>>>>>>>> context
>>>>>>>> of shared disks and direct LUNs-based disks -
>>>>>>>> On one hand, if we copy&collapse such images - this may
>>>>>>>> yield
>>>>>>>> to
>>>>>>>> data
>>>>>>>> corruption (think of a scenario where the source and
>>>>>>>> destination
>>>>>>>> VMs use
>>>>>>>> the same disk).
>>>>>>>> On the other hand - if we decide not to copy&collapse  - the
>>>>>>>> target
>>>>>>>> VM
>>>>>>>> will have missing VM and its state will not totally reflect
>>>>>>>> the
>>>>>>>> logical
>>>>>>>> state.
>>>>>>>> One of the solution raises is to mark such disks (at the
>>>>>>>> destination) as
>>>>>>>> unplugged, allowing the administrator the ability to plug
>>>>>>>> them
>>>>>>>> (understanding of course the consequences).
>>>>>>>>
>>>>>>>> I would like to receive inputs on this issue
>>>>>>>>
>>>>>>>>
>>>>>>>> Kind regards,
>>>>>>>>
>>>>>>>> Yair
>>>>>>>
>>>>>>> Hi Yair,
>>>>>>>
>>>>>>> Some clarifications on the above issue.
>>>>>>> Currently when taking a snapshot on a VM with shared disks or
>>>>>>> direct
>>>>>>> LUN
>>>>>>> disk there are 3 optional behaviors:
>>>>>>>
>>>>>>> 1. Blocking the snapshot action. (User can not take a
>>>>>>> snapshot
>>>>>>> of
>>>>>>> the
>>>>>>> VM
>>>>>>> if it has plugged shared or direct LUN disks)
>>>>>>>
>>>>>>> 2. Taking the snapshot and marking the shared disk and direct
>>>>>>> LUN
>>>>>>> disks
>>>>>>> as unplugged (in the VM snapshot configuration) and marking
>>>>>>> the
>>>>>>> snapshot
>>>>>>> state as partial.
>>>>>>>
>>>>>>> 3. Taking the snapshot of the VM as is, leaving the VM
>>>>>>> configuration
>>>>>>> with plugged disks.
>>>>>>>
>>>>>>>
>>>>>>> The issue with including these disks in the snapshot is that
>>>>>>> they
>>>>>>> are
>>>>>>> not really being snapshotted, they are not capturing the
>>>>>>> point
>>>>>>> in
>>>>>>> time
>>>>>>> we are trying to achieve in the snapshot.
>>>>>>>
>>>>>>> Enabling the snapshot action in such a state is a bit
>>>>>>> misleading
>>>>>>> to
>>>>>>> the
>>>>>>> user.
>>>>>>>
>>>>>>> If we do allow taking the snapshot we should mark the
>>>>>>> snapshot
>>>>>>> as
>>>>>>> partial to indicate that the snapshot did not capture the
>>>>>>> point
>>>>>>> in
>>>>>>> time
>>>>>>> as the user intended.
>>>>>>>
>>>>>>> I have no preference with regards to the second and third
>>>>>>> approach,
>>>>>>> the
>>>>>>> second approach is a bit more safe, we basically force the
>>>>>>> user
>>>>>>> to
>>>>>>> plug
>>>>>>> the disks and be sure that he knows what he is doing and the
>>>>>>> third
>>>>>>> approach is less safe and less annoying to the user (he took
>>>>>>> the
>>>>>>> snapshot, cloned it and wants to start the VM - don't require
>>>>>>> extra
>>>>>>> actions)
>>>>>>>
>>>>>>> Kolesnik - please note when starting VM in a preview mode we
>>>>>>> should
>>>>>>> mount the disks in read-only mode (if supported).
>>>>>
>>>>> I don't understand this, can you please elaborate why and in
>>>>> which
>>>>> case?
>>>>>   The disk is plugged/unplugged?
>>>>>   What happens when you commit? It becomes r/w?
>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Livnat
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> +1 for option 3
>>>>>>
>>>>>
>>>>> +1 for option 3 as well (also good with option 1, but I think
>>>>> this
>>>>> will hinder usability).
>>>> I agree with Mike - I think option one is too "aggressive" in
>>>> protecting
>>>> us, this is why I prefer 3.  +1 on option 3
>>>
>>> This would only be acceptable if the disk is marked read only.
>>> Just leaving the disk plugged means many users *will* corrupt their
>>> VMs.
>>> That trumps the need to mark a checkbox if you want it available.
>>>
>>> As I said before, readonly has its own problems and in fact, IMO
>>> the
>>> behaviour is even more difficult to explain (yeah, you might
>>> corrupt
>>> the running VM if it's r/w so we made it r/o and now if you start
>>> up
>>> your clone / preview you will get a kernel panic).
>>>
>>>>>
>>>>>
>>>>> Regards,
>>>>> Mike
>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From iheim at redhat.com  Fri Jan 20 10:12:27 2012
From: iheim at redhat.com (Itamar Heim)
Date: Fri, 20 Jan 2012 12:12:27 +0200
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <746eed7f-20a0-4a81-9b83-3974c6d2098c@zmail13.collab.prod.int.phx2.redhat.com>
References: <746eed7f-20a0-4a81-9b83-3974c6d2098c@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F193E0B.8080307@redhat.com>

On 01/19/2012 11:58 AM, Ayal Baron wrote:
>
>
> ----- Original Message -----
>> On 01/18/2012 05:53 PM, Livnat Peer wrote:
>>> Hi All,
>>>
>>> This is what we've discussed in the meeting today:
>>>
>>> Multiple storage domain:
>>> - Should have a single generic verb for removing a disk.
>>> - We block removing the last template disk - template is immutable.
>>
>> but it will be deleted when deleting the template, right?
>
> Of course.
> The point is that the template is an immutable object and should not change (until we support editing a template at which point the user would have to change the template to edit mode before being able to make such changes and maybe also be able to run it and make changes internally?).

When i hear "edit a template" i don't expect replacing the files.
I expect a new edition of disks appearing as a new version of the 
template. but they don't have to derive from same original template.
say i want to create a "Fedora 16 template", then update it every month 
with latest "yum update".
it doesn't matter if i use a VM from same template or just create a new one.
then specify it is V2 of the "Fedora 16 template".
when someone creates a VM from this template, default version would be 
latest (but we can let them choose specific older versions as well)


From yzaslavs at redhat.com  Fri Jan 20 11:52:20 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Fri, 20 Jan 2012 13:52:20 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <64f84f20-e078-4867-afe6-d84d9dedf8f5@mkenneth.csb>
References: <64f84f20-e078-4867-afe6-d84d9dedf8f5@mkenneth.csb>
Message-ID: <4F195574.2070309@redhat.com>

On 01/19/2012 08:41 PM, Miki Kenneth wrote:
> Top Posting:
> 
> From user POV I think that option 2 is the only one that make sense. We try to do as much as we can, 
> and on each "problematic" case, we make him aware and let him decide.
> 
> Miki
Miki, just to clear /be sure - you're talking about taking the snapshot
as is, living the shared disk as "plugged" on destination VM?

Yair

> 
> 
> 
> ----- Original Message -----
>> From: "Ayal Baron" <abaron at redhat.com>
>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Thursday, January 19, 2012 4:04:02 AM
>> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot feature in	context of shared disks and direct
>> LUNs-based disks
>>
>>
>>
>> ----- Original Message -----
>>> On 01/19/2012 10:32 AM, Mike Kolesnik wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>>>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Mike Kolesnik"
>>>>>> <mkolesni at redhat.com>
>>>>>> Cc: engine-devel at ovirt.org
>>>>>> Sent: Thursday, January 19, 2012 9:19:52 AM
>>>>>> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
>>>>>> feature in context of shared disks and direct
>>>>>> LUNs-based disks
>>>>>>
>>>>>> On 19/01/12 08:38, Yair Zaslavsky wrote:
>>>>>>> Hi all,
>>>>>>> Following the upstream meeting dated Wednesday January 18th,
>>>>>>> 2012
>>>>>>> -
>>>>>>> I presented the clone VM from snpashot feature and we
>>>>>>> discussed
>>>>>>> the
>>>>>>> feature behaviour.
>>>>>>>
>>>>>>> Two issues that were raised are the behaviour of the feature
>>>>>>> in
>>>>>>> context
>>>>>>> of shared disks and direct LUNs-based disks -
>>>>>>> On one hand, if we copy&collapse such images - this may yield
>>>>>>> to
>>>>>>> data
>>>>>>> corruption (think of a scenario where the source and
>>>>>>> destination
>>>>>>> VMs use
>>>>>>> the same disk).
>>>>>>> On the other hand - if we decide not to copy&collapse  - the
>>>>>>> target
>>>>>>> VM
>>>>>>> will have missing VM and its state will not totally reflect
>>>>>>> the
>>>>>>> logical
>>>>>>> state.
>>>>>>> One of the solution raises is to mark such disks (at the
>>>>>>> destination) as
>>>>>>> unplugged, allowing the administrator the ability to plug them
>>>>>>> (understanding of course the consequences).
>>>>>>>
>>>>>>> I would like to receive inputs on this issue
>>>>>>>
>>>>>>>
>>>>>>> Kind regards,
>>>>>>>
>>>>>>> Yair
>>>>>>
>>>>>> Hi Yair,
>>>>>>
>>>>>> Some clarifications on the above issue.
>>>>>> Currently when taking a snapshot on a VM with shared disks or
>>>>>> direct
>>>>>> LUN
>>>>>> disk there are 3 optional behaviors:
>>>>>>
>>>>>> 1. Blocking the snapshot action. (User can not take a snapshot
>>>>>> of
>>>>>> the
>>>>>> VM
>>>>>> if it has plugged shared or direct LUN disks)
>>>>>>
>>>>>> 2. Taking the snapshot and marking the shared disk and direct
>>>>>> LUN
>>>>>> disks
>>>>>> as unplugged (in the VM snapshot configuration) and marking the
>>>>>> snapshot
>>>>>> state as partial.
>>>>>>
>>>>>> 3. Taking the snapshot of the VM as is, leaving the VM
>>>>>> configuration
>>>>>> with plugged disks.
>>>>>>
>>>>>>
>>>>>> The issue with including these disks in the snapshot is that
>>>>>> they
>>>>>> are
>>>>>> not really being snapshotted, they are not capturing the point
>>>>>> in
>>>>>> time
>>>>>> we are trying to achieve in the snapshot.
>>>>>>
>>>>>> Enabling the snapshot action in such a state is a bit
>>>>>> misleading
>>>>>> to
>>>>>> the
>>>>>> user.
>>>>>>
>>>>>> If we do allow taking the snapshot we should mark the snapshot
>>>>>> as
>>>>>> partial to indicate that the snapshot did not capture the point
>>>>>> in
>>>>>> time
>>>>>> as the user intended.
>>>>>>
>>>>>> I have no preference with regards to the second and third
>>>>>> approach,
>>>>>> the
>>>>>> second approach is a bit more safe, we basically force the user
>>>>>> to
>>>>>> plug
>>>>>> the disks and be sure that he knows what he is doing and the
>>>>>> third
>>>>>> approach is less safe and less annoying to the user (he took
>>>>>> the
>>>>>> snapshot, cloned it and wants to start the VM - don't require
>>>>>> extra
>>>>>> actions)
>>>>>>
>>>>>> Kolesnik - please note when starting VM in a preview mode we
>>>>>> should
>>>>>> mount the disks in read-only mode (if supported).
>>>>
>>>> I don't understand this, can you please elaborate why and in
>>>> which
>>>> case?
>>>>   The disk is plugged/unplugged?
>>>>   What happens when you commit? It becomes r/w?
>>>>
>>>>>>
>>>>>>
>>>>>> Livnat
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> +1 for option 3
>>>>>
>>>>
>>>> +1 for option 3 as well (also good with option 1, but I think
>>>> this
>>>> will hinder usability).
>>> I agree with Mike - I think option one is too "aggressive" in
>>> protecting
>>> us, this is why I prefer 3.  +1 on option 3
>>
>> This would only be acceptable if the disk is marked read only.
>> Just leaving the disk plugged means many users *will* corrupt their
>> VMs.
>> That trumps the need to mark a checkbox if you want it available.
>>
>> As I said before, readonly has its own problems and in fact, IMO the
>> behaviour is even more difficult to explain (yeah, you might corrupt
>> the running VM if it's r/w so we made it r/o and now if you start up
>> your clone / preview you will get a kernel panic).
>>
>>>>
>>>>
>>>> Regards,
>>>> Mike
>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From jchoate at redhat.com  Fri Jan 20 12:28:50 2012
From: jchoate at redhat.com (Jon Choate)
Date: Fri, 20 Jan 2012 07:28:50 -0500
Subject: [Engine-devel] Moving to Jboss AS7
In-Reply-To: <803c568c-626c-4bd9-8287-60c4601e815a@zmail02.collab.prod.int.phx2.redhat.com>
References: <803c568c-626c-4bd9-8287-60c4601e815a@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <4F195E02.8010402@redhat.com>

On 01/15/2012 04:05 AM, Oved Ourfalli wrote:
> Hey all,
> The patches are now pushed!
>
> So please, if you fetch from now on you'll have to perform the actions below.
> The wiki was updated as well.
>
> Short list of steps:
> 1. Download jboss 7.1.0 Beta1b (wget http://download.jboss.org/jbossas/7.1/jboss-as-7.1.0.Beta1b/jboss-as-7.1.0.Beta1b.tar.gz)
> 2. Fetch the latest changes from our git repository
> 3. Change the Jboss home to the new path, both in the JBOSS_HOME
> environment variable, and in maven settings file
> (~/.m2/settings.xml)
> 4. Build the engine, with profiles "dep" and "setup". This will put
> all the proper configuration files, postgresql driver and make all
> the other needed changes in Jboss in order to make it work properly
> mvn clean install -Psetup,dep .......
> 5. In order to run Jboss go to JBOSS_HOME/bin and run ./standalone.sh
>
> A more descriptive set of steps and notes can be found in my previous E-mail below.
>
> I'm here if you need any help.
>
> Thank you,
> Oved
>
> ----- Original Message -----
>> From: "Oved Ourfalli"<ovedo at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Wednesday, January 11, 2012 2:57:19 PM
>> Subject: Moving to Jboss AS7
>>
>> Hey all,
>>
>> The code changes required to make the engine work on Jboss AS7 will
>> soon be push
>> It will, of course, require you to install it, and start working with
>> it :-)
>>
>> A separate E-mail will be sent to notify you all once pushed, and
>> then you'll have to perform the following steps:
>>
>> 1. Download jboss 7.1.0 Beta1b
>> (http://download.jboss.org/jbossas/7.1/jboss-as-7.1.0.Beta1b/jboss-as-7.1.0.Beta1b.tar.gz)
>> - There is a newer version, but it has issues in the REST-API, so we
>> decided to work with the beta version until a proper fix will be
>> released.
>> 2. Fetch the latest changes from our git repository
>> 3. Change the Jboss home to the new path, both in the JBOSS_HOME
>> environment variable, and in maven settings file
>> (~/.m2/settings.xml)
>> 4. Build the engine, with profiles "dep" and "setup". This will put
>> all the proper configuration files, postgresql driver and make all
>> the other needed changes in Jboss in order to make it work properly
>> mvn clean install -Psetup,dep .......
>> 5. In order to run Jboss go to JBOSS_HOME/bin and run ./standalone.sh
>> 6. Look inside the JBOSS_HOME/bin/standalone.conf file in order to
>> enable jboss debugging (just uncomment the line
>> JAVA_OPTS="$JAVA_OPTS
>> -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n")
>> 7. If you have a krb5.conf file you are working with, put it in
>> JBOSS_HOME/standalone/configuration directory
>> 8. Run Jboss (and be impressed by the startup speed!)
>>
>> The above will be also added to the wiki page for building the engine
>> as soon as the patches will be pushed upstream.
>>
>> Some facts about Jboss 7:
>> 1. It supports both standalone deployment, and domain deployment. We
>> use the standalone one.
>> 2. Stuff related to the standalone mode can be found in the
>> JBOSS_HOME/standalone directory:
>>   * configuration - contains the main configuration file called
>>   standalone.xml file, plus some other configuration files
>>   * deployments - that's where your deployments should be. When adding
>>   a new one, don't forget to create a file called
>>   "<deployment-name>.dodeploy" in order to make it deploy.
>>   * log - that's where the log files are written (unless stated
>>   differently in the standalone.xml file).
>> 3. The different modules that come with Jboss 7 are located in
>> JBOSS_HOME/modules directory
>>   * No more common/lib directory.
>>   * Every module has a module.xml file which contains it's
>>   dependencies in other modules, the jars that are part of the
>>   module, and etc.
>>   * In order to use a dependency from there you have to add
>>   "Dependencies" section to your manifest file (do git grep
>>   "Dependencies" to take a look at some examples done in the engine
>>   source code).
>> 4. Useful links:
>>   * Documentation -
>>   https://docs.jboss.org/author/display/AS7/Documentation
>>   * Class loading changes -
>>   https://docs.jboss.org/author/display/AS7/Class+Loading+in+AS7
>>   * The Jboss community - http://community.jboss.org/wiki
>>
>>
>> Please send issues/feedback to this mailing list.
>>
>> Thank you all,
>> Oved
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
I didn't realize until now that these changes would prevent deploying to 
earlier jboss versions.  Is this intentional? I had thought we would 
continue supporting earlier versions and use profiles for deployments.



From jchoate at redhat.com  Fri Jan 20 13:46:39 2012
From: jchoate at redhat.com (Jon Choate)
Date: Fri, 20 Jan 2012 08:46:39 -0500
Subject: [Engine-devel] running jboss as 7 with openjdk
Message-ID: <4F19703F.7010402@redhat.com>

I keep getting a message that it can't find 
sun.security.krb5.KrbException. Any ideas? Jboss-as-5 ran ovirt-engine 
just fine with openjdk.


From iheim at redhat.com  Fri Jan 20 15:21:34 2012
From: iheim at redhat.com (Itamar Heim)
Date: Fri, 20 Jan 2012 17:21:34 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <4F193B5E.1050200@redhat.com>
References: <e46da1a1-4bc0-4992-aa41-6bff7076c144@zmail13.collab.prod.int.phx2.redhat.com>
	<4F193B5E.1050200@redhat.com>
Message-ID: <4F19867E.3030005@redhat.com>

On 01/20/2012 12:01 PM, Livnat Peer wrote:
> On 20/01/12 09:35, Ayal Baron wrote:
>>
>>
>> ----- Original Message -----
>>> Top Posting:
>>>
>>>  From user POV I think that option 2 is the only one that make sense.
>>> We try to do as much as we can,
>>> and on each "problematic" case, we make him aware and let him decide.
>>>
>>
>> Yep, +1.
>>
>
> Trying to get to a conclusion here,
> 3 different people said on this thread that they think that from the
> user perspective leaving the shared devices plugged is what they think
> is the best behavior to the user. (Omer, Kolesnik, Yair)
>
> On the other hand we have 2 people who think that protecting the user is
> more important than leaving the VM configuration as it was in the
> original VM (Miki, Ayal).
>
> Ayal/Miki can you please specify what are we protecting the user from?
>
>
> I think that because we are not snapshotting the shared disk and the
> direct LUN they should not be part of the VM configuration (in the
> snapshot) at all. we can not promise the user that the disk will be
> there and if it is there we can not guarantee it is in the same state as
> it was when we took the snapshot.
>
>
> Another issue,
>
> I can not see a reason to limit this feature to creating a VM from
> snapshot and not a template? Almost no extra work is needed for
> supporting templates as well.

I assume you meant, creating a VM from another VM (if it is down)?
It should be supported.


From mkenneth at redhat.com  Fri Jan 20 18:50:14 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Fri, 20 Jan 2012 13:50:14 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
	context of shared disks and direct LUNs-based disks
In-Reply-To: <4F19867E.3030005@redhat.com>
Message-ID: <0a8555d3-5e18-4946-b4bd-105521329d60@mkenneth.csb>

Sorry guys if I was not clear or maybe I missed something...

Let's take a use case:
- User like to create a VM for instance Win 2008, and would like to attach a shared disk to it.
- User liked to create multiple copies of this VM.
(all vms shared the same disk, and run same OS).
so do I do that in oVirt.... we can do either option 2 or 3.

Option 2 as I read it:
Taking the snapshot and marking the shared disk and direct LUN
disks as unplugged (in the VM snapshot configuration) and marking the snapshot state as partial.

my understanding was:
1. we is clone the vm configuration as is.
2. we try to clone the different disks
3. if there is shared raw disk/direct LUN, we do not clone them, we "unplug" them.
4. the (poor) user, will have to plug these vms manually, in order to assure connectivity and raise awareness that these disks are "special". This is nice but not great.

Option 3 as I read it:
Taking the snapshot and marking the shared disk and direct LUN
disks as plugged (in the VM snapshot configuration) and marking these disks as read only.

my understanding was:
1. we is clone the vm configuration as is.
2. we try to clone the different disks
3. if there is shared raw disk/direct LUN, we do not clone them , we make them read only(?), and they remain plugged.
4. user is happy.
5. only issue is how we have to make the user aware that these disks are shared/read only???
if this is possible, I agree to vote for third option :)

You might want to have a look at:
http://www.symantec.com/connect/articles/building-vmware-shared-disk
(look at the configuration file in Vmware:
disk.locking = "FALSE"
diskLib.dataCacheMaxSize = "0"
#scsi1 data storage
scsi1.present = "TRUE"
scsi1.virtualDev = "lsilogic"
scsi1.sharedbus = "none"
scsi1:0.present = "TRUE"
scsi1:0.fileName = " D:\Virtual Machines\Shared Disk\SHARED-DISK.vmdk "
scsi1:0.mode = "independent-persistent"
scsi1:0.shared = "TRUE"      
scsi1:0.redo = ""
The shared flag is set for shared file, indicating "no locking"

I would like to re-ephasize that the user does not know the snapshotting mechanics. He would like to "copy" the VM as is. We have to do our best, and highlights the issues/sensitive points he has to take care of. 


does that make sense?

Miki



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Livnat Peer" <lpeer at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Friday, January 20, 2012 7:21:34 AM
> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot feature in context of shared disks and direct
> LUNs-based disks
> 
> On 01/20/2012 12:01 PM, Livnat Peer wrote:
> > On 20/01/12 09:35, Ayal Baron wrote:
> >>
> >>
> >> ----- Original Message -----
> >>> Top Posting:
> >>>
> >>>  From user POV I think that option 2 is the only one that make
> >>>  sense.
> >>> We try to do as much as we can,
> >>> and on each "problematic" case, we make him aware and let him
> >>> decide.
> >>>
> >>
> >> Yep, +1.
> >>
> >
> > Trying to get to a conclusion here,
> > 3 different people said on this thread that they think that from
> > the
> > user perspective leaving the shared devices plugged is what they
> > think
> > is the best behavior to the user. (Omer, Kolesnik, Yair)
> >
> > On the other hand we have 2 people who think that protecting the
> > user is
> > more important than leaving the VM configuration as it was in the
> > original VM (Miki, Ayal).
> >
> > Ayal/Miki can you please specify what are we protecting the user
> > from?
> >
> >
> > I think that because we are not snapshotting the shared disk and
> > the
> > direct LUN they should not be part of the VM configuration (in the
> > snapshot) at all. we can not promise the user that the disk will be
> > there and if it is there we can not guarantee it is in the same
> > state as
> > it was when we took the snapshot.
> >
> >
> > Another issue,
> >
> > I can not see a reason to limit this feature to creating a VM from
> > snapshot and not a template? Almost no extra work is needed for
> > supporting templates as well.
> 
> I assume you meant, creating a VM from another VM (if it is down)?
> It should be supported.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From abaron at redhat.com  Fri Jan 20 21:37:54 2012
From: abaron at redhat.com (Ayal Baron)
Date: Fri, 20 Jan 2012 16:37:54 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature
	in	context of shared disks and direct LUNs-based disks
In-Reply-To: <0a8555d3-5e18-4946-b4bd-105521329d60@mkenneth.csb>
Message-ID: <f9eca839-f990-4869-b4c5-338d28673734@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> Sorry guys if I was not clear or maybe I missed something...
> 
> Let's take a use case:
> - User like to create a VM for instance Win 2008, and would like to
> attach a shared disk to it.
> - User liked to create multiple copies of this VM.
> (all vms shared the same disk, and run same OS).
> so do I do that in oVirt.... we can do either option 2 or 3.
> 
> Option 2 as I read it:
> Taking the snapshot and marking the shared disk and direct LUN
> disks as unplugged (in the VM snapshot configuration) and marking the
> snapshot state as partial.
> 
> my understanding was:
> 1. we is clone the vm configuration as is.
> 2. we try to clone the different disks
> 3. if there is shared raw disk/direct LUN, we do not clone them, we
> "unplug" them.
> 4. the (poor) user, will have to plug these vms manually, in order to
> assure connectivity and raise awareness that these disks are
> "special". This is nice but not great.

Correct 

> 
> Option 3 as I read it:
> Taking the snapshot and marking the shared disk and direct LUN
> disks as plugged (in the VM snapshot configuration) and marking these
> disks as read only.
> 
> my understanding was:
> 1. we is clone the vm configuration as is.
> 2. we try to clone the different disks
> 3. if there is shared raw disk/direct LUN, we do not clone them , we
> make them read only(?), and they remain plugged.
> 4. user is happy.
> 5. only issue is how we have to make the user aware that these disks
> are shared/read only???
> if this is possible, I agree to vote for third option :)

This will become apparent to the user once he boots the machine and gets the kernel panic :)

> 
> You might want to have a look at:
> http://www.symantec.com/connect/articles/building-vmware-shared-disk
> (look at the configuration file in Vmware:
> disk.locking = "FALSE"
> diskLib.dataCacheMaxSize = "0"
> #scsi1 data storage
> scsi1.present = "TRUE"
> scsi1.virtualDev = "lsilogic"
> scsi1.sharedbus = "none"
> scsi1:0.present = "TRUE"
> scsi1:0.fileName = " D:\Virtual Machines\Shared Disk\SHARED-DISK.vmdk
> "
> scsi1:0.mode = "independent-persistent"
> scsi1:0.shared = "TRUE"
> scsi1:0.redo = ""
> The shared flag is set for shared file, indicating "no locking"

This is shared disk, what about RDM?

> 
> I would like to re-ephasize that the user does not know the
> snapshotting mechanics. He would like to "copy" the VM as is. We
> have to do our best, and highlights the issues/sensitive points he
> has to take care of.
> 
> 
> does that make sense?
> 
> Miki
> 
> 
> 
> ----- Original Message -----
> > From: "Itamar Heim" <iheim at redhat.com>
> > To: "Livnat Peer" <lpeer at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Friday, January 20, 2012 7:21:34 AM
> > Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
> > feature in context of shared disks and direct
> > LUNs-based disks
> > 
> > On 01/20/2012 12:01 PM, Livnat Peer wrote:
> > > On 20/01/12 09:35, Ayal Baron wrote:
> > >>
> > >>
> > >> ----- Original Message -----
> > >>> Top Posting:
> > >>>
> > >>>  From user POV I think that option 2 is the only one that make
> > >>>  sense.
> > >>> We try to do as much as we can,
> > >>> and on each "problematic" case, we make him aware and let him
> > >>> decide.
> > >>>
> > >>
> > >> Yep, +1.
> > >>
> > >
> > > Trying to get to a conclusion here,
> > > 3 different people said on this thread that they think that from
> > > the
> > > user perspective leaving the shared devices plugged is what they
> > > think
> > > is the best behavior to the user. (Omer, Kolesnik, Yair)
> > >
> > > On the other hand we have 2 people who think that protecting the
> > > user is
> > > more important than leaving the VM configuration as it was in the
> > > original VM (Miki, Ayal).
> > >
> > > Ayal/Miki can you please specify what are we protecting the user
> > > from?
> > >
> > >
> > > I think that because we are not snapshotting the shared disk and
> > > the
> > > direct LUN they should not be part of the VM configuration (in
> > > the
> > > snapshot) at all. we can not promise the user that the disk will
> > > be
> > > there and if it is there we can not guarantee it is in the same
> > > state as
> > > it was when we took the snapshot.
> > >
> > >
> > > Another issue,
> > >
> > > I can not see a reason to limit this feature to creating a VM
> > > from
> > > snapshot and not a template? Almost no extra work is needed for
> > > supporting templates as well.
> > 
> > I assume you meant, creating a VM from another VM (if it is down)?
> > It should be supported.
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkenneth at redhat.com  Fri Jan 20 21:42:05 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Fri, 20 Jan 2012 16:42:05 -0500 (EST)
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <4F193E0B.8080307@redhat.com>
Message-ID: <e1ebf203-dac2-4a5b-880d-1ab323122305@mkenneth.csb>



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Ayal Baron" <abaron at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Friday, January 20, 2012 2:12:27 AM
> Subject: Re: [Engine-devel] ovirt core MOM
> 
> On 01/19/2012 11:58 AM, Ayal Baron wrote:
> >
> >
> > ----- Original Message -----
> >> On 01/18/2012 05:53 PM, Livnat Peer wrote:
> >>> Hi All,
> >>>
> >>> This is what we've discussed in the meeting today:
> >>>
> >>> Multiple storage domain:
> >>> - Should have a single generic verb for removing a disk.
> >>> - We block removing the last template disk - template is
> >>> immutable.
> >>
> >> but it will be deleted when deleting the template, right?
> >
> > Of course.
> > The point is that the template is an immutable object and should
> > not change (until we support editing a template at which point the
> > user would have to change the template to edit mode before being
> > able to make such changes and maybe also be able to run it and
> > make changes internally?).
> 
> When i hear "edit a template" i don't expect replacing the files.
> I expect a new edition of disks appearing as a new version of the
> template. but they don't have to derive from same original template.
> say i want to create a "Fedora 16 template", then update it every
> month
> with latest "yum update".
> it doesn't matter if i use a VM from same template or just create a
> new one.
> then specify it is V2 of the "Fedora 16 template".
> when someone creates a VM from this template, default version would
> be
> latest (but we can let them choose specific older versions as well)
+1. Nicely put.
And just to add another common use case is the pool usage.
When we creating stateless VM pool from the template,
it would be nice to be able to update the template to V2, 
and have all the newly created VMs dynamically based to the new template.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkenneth at redhat.com  Fri Jan 20 21:51:40 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Fri, 20 Jan 2012 16:51:40 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature
	in	context of shared disks and direct LUNs-based disks
In-Reply-To: <f9eca839-f990-4869-b4c5-338d28673734@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <2be91893-2625-42db-a7ee-5fef45cf1222@mkenneth.csb>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Miki Kenneth" <mkenneth at redhat.com>
> Cc: engine-devel at ovirt.org, "Itamar Heim" <iheim at redhat.com>
> Sent: Friday, January 20, 2012 1:37:54 PM
> Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot feature in	context of shared disks and direct
> LUNs-based disks
> 
> 
> 
> ----- Original Message -----
> > Sorry guys if I was not clear or maybe I missed something...
> > 
> > Let's take a use case:
> > - User like to create a VM for instance Win 2008, and would like to
> > attach a shared disk to it.
> > - User liked to create multiple copies of this VM.
> > (all vms shared the same disk, and run same OS).
> > so do I do that in oVirt.... we can do either option 2 or 3.
> > 
> > Option 2 as I read it:
> > Taking the snapshot and marking the shared disk and direct LUN
> > disks as unplugged (in the VM snapshot configuration) and marking
> > the
> > snapshot state as partial.
> > 
> > my understanding was:
> > 1. we is clone the vm configuration as is.
> > 2. we try to clone the different disks
> > 3. if there is shared raw disk/direct LUN, we do not clone them, we
> > "unplug" them.
> > 4. the (poor) user, will have to plug these vms manually, in order
> > to
> > assure connectivity and raise awareness that these disks are
> > "special". This is nice but not great.
> 
> Correct
> 
> > 
> > Option 3 as I read it:
> > Taking the snapshot and marking the shared disk and direct LUN
> > disks as plugged (in the VM snapshot configuration) and marking
> > these
> > disks as read only.
> > 
> > my understanding was:
> > 1. we is clone the vm configuration as is.
> > 2. we try to clone the different disks
> > 3. if there is shared raw disk/direct LUN, we do not clone them ,
> > we
> > make them read only(?), and they remain plugged.
> > 4. user is happy.
> > 5. only issue is how we have to make the user aware that these
> > disks
> > are shared/read only???
> > if this is possible, I agree to vote for third option :)
> 
> This will become apparent to the user once he boots the machine and
> gets the kernel panic :)
Of course, this is no acceptable!
> 
> > 
> > You might want to have a look at:
> > http://www.symantec.com/connect/articles/building-vmware-shared-disk
> > (look at the configuration file in Vmware:
> > disk.locking = "FALSE"
> > diskLib.dataCacheMaxSize = "0"
> > #scsi1 data storage
> > scsi1.present = "TRUE"
> > scsi1.virtualDev = "lsilogic"
> > scsi1.sharedbus = "none"
> > scsi1:0.present = "TRUE"
> > scsi1:0.fileName = " D:\Virtual Machines\Shared
> > Disk\SHARED-DISK.vmdk
> > "
> > scsi1:0.mode = "independent-persistent"
> > scsi1:0.shared = "TRUE"
> > scsi1:0.redo = ""
> > The shared flag is set for shared file, indicating "no locking"
> 
> This is shared disk, what about RDM?
Don't know, will try to find out.
> 
> > 
> > I would like to re-ephasize that the user does not know the
> > snapshotting mechanics. He would like to "copy" the VM as is. We
> > have to do our best, and highlights the issues/sensitive points he
> > has to take care of.
> > 
> > 
> > does that make sense?
> > 
> > Miki
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Itamar Heim" <iheim at redhat.com>
> > > To: "Livnat Peer" <lpeer at redhat.com>
> > > Cc: engine-devel at ovirt.org
> > > Sent: Friday, January 20, 2012 7:21:34 AM
> > > Subject: Re: [Engine-devel] Question about CloneVMFromSnapshot
> > > feature in context of shared disks and direct
> > > LUNs-based disks
> > > 
> > > On 01/20/2012 12:01 PM, Livnat Peer wrote:
> > > > On 20/01/12 09:35, Ayal Baron wrote:
> > > >>
> > > >>
> > > >> ----- Original Message -----
> > > >>> Top Posting:
> > > >>>
> > > >>>  From user POV I think that option 2 is the only one that
> > > >>>  make
> > > >>>  sense.
> > > >>> We try to do as much as we can,
> > > >>> and on each "problematic" case, we make him aware and let him
> > > >>> decide.
> > > >>>
> > > >>
> > > >> Yep, +1.
> > > >>
> > > >
> > > > Trying to get to a conclusion here,
> > > > 3 different people said on this thread that they think that
> > > > from
> > > > the
> > > > user perspective leaving the shared devices plugged is what
> > > > they
> > > > think
> > > > is the best behavior to the user. (Omer, Kolesnik, Yair)
> > > >
> > > > On the other hand we have 2 people who think that protecting
> > > > the
> > > > user is
> > > > more important than leaving the VM configuration as it was in
> > > > the
> > > > original VM (Miki, Ayal).
> > > >
> > > > Ayal/Miki can you please specify what are we protecting the
> > > > user
> > > > from?
> > > >
> > > >
> > > > I think that because we are not snapshotting the shared disk
> > > > and
> > > > the
> > > > direct LUN they should not be part of the VM configuration (in
> > > > the
> > > > snapshot) at all. we can not promise the user that the disk
> > > > will
> > > > be
> > > > there and if it is there we can not guarantee it is in the same
> > > > state as
> > > > it was when we took the snapshot.
> > > >
> > > >
> > > > Another issue,
> > > >
> > > > I can not see a reason to limit this feature to creating a VM
> > > > from
> > > > snapshot and not a template? Almost no extra work is needed for
> > > > supporting templates as well.
> > > 
> > > I assume you meant, creating a VM from another VM (if it is
> > > down)?
> > > It should be supported.
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 


From iheim at redhat.com  Sat Jan 21 03:51:14 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sat, 21 Jan 2012 05:51:14 +0200
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <e1ebf203-dac2-4a5b-880d-1ab323122305@mkenneth.csb>
References: <e1ebf203-dac2-4a5b-880d-1ab323122305@mkenneth.csb>
Message-ID: <4F1A3632.8080107@redhat.com>

On 01/20/2012 11:42 PM, Miki Kenneth wrote:
>
>
> ----- Original Message -----
>> From: "Itamar Heim"<iheim at redhat.com>
>> To: "Ayal Baron"<abaron at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Friday, January 20, 2012 2:12:27 AM
>> Subject: Re: [Engine-devel] ovirt core MOM
>>
>> On 01/19/2012 11:58 AM, Ayal Baron wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> On 01/18/2012 05:53 PM, Livnat Peer wrote:
>>>>> Hi All,
>>>>>
>>>>> This is what we've discussed in the meeting today:
>>>>>
>>>>> Multiple storage domain:
>>>>> - Should have a single generic verb for removing a disk.
>>>>> - We block removing the last template disk - template is
>>>>> immutable.
>>>>
>>>> but it will be deleted when deleting the template, right?
>>>
>>> Of course.
>>> The point is that the template is an immutable object and should
>>> not change (until we support editing a template at which point the
>>> user would have to change the template to edit mode before being
>>> able to make such changes and maybe also be able to run it and
>>> make changes internally?).
>>
>> When i hear "edit a template" i don't expect replacing the files.
>> I expect a new edition of disks appearing as a new version of the
>> template. but they don't have to derive from same original template.
>> say i want to create a "Fedora 16 template", then update it every
>> month
>> with latest "yum update".
>> it doesn't matter if i use a VM from same template or just create a
>> new one.
>> then specify it is V2 of the "Fedora 16 template".
>> when someone creates a VM from this template, default version would
>> be
>> latest (but we can let them choose specific older versions as well)
> +1. Nicely put.
> And just to add another common use case is the pool usage.
> When we creating stateless VM pool from the template,
> it would be nice to be able to update the template to V2,
> and have all the newly created VMs dynamically based to the new template.

that is indeed where i was going with it as well, but not as trivial, 
since need to wait for VMs to stop and return to pool and create new 
ones and remove old ones.
also, creating new ones may involve an admin action of first boot + take 
of first snapshot

(hence i stopped the previous description before this part, but since 
you opened the door...)




From iheim at redhat.com  Sat Jan 21 21:57:23 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sat, 21 Jan 2012 23:57:23 +0200
Subject: [Engine-devel] CloneVMFromSnapshot - feature document
In-Reply-To: <4F16886F.1050005@redhat.com>
References: <4F16886F.1050005@redhat.com>
Message-ID: <4F1B34C3.90305@redhat.com>

On 01/18/2012 10:53 AM, Yair Zaslavsky wrote:
> Hi all,
> You can look at http://ovirt.org/wiki/Features/CloneVmFromSnapshot
>
> And see the feature document for Clone VM from Snapshot.
>
> Comments are more than welcome (planning on providing a the detailed
> document soon)

apart from the ongoing discussion on this, my main comment is it should 
be possible to clone a VM in down state directly, without a need to 
create snapshots on it.

This is possible today by creating a template from the VM, then 
instantiating VMs from the template via thin or clone.
Clone VM should allow to remove the need of creating a template.

Only clone would be supported of course, though  the VM is thinly 
provisioned from a template, then cloning only the thinly provisioned 
disks rather than the entire disks would be good as well)


From lpeer at redhat.com  Sun Jan 22 07:26:44 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 22 Jan 2012 09:26:44 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <4F19867E.3030005@redhat.com>
References: <e46da1a1-4bc0-4992-aa41-6bff7076c144@zmail13.collab.prod.int.phx2.redhat.com>
	<4F193B5E.1050200@redhat.com> <4F19867E.3030005@redhat.com>
Message-ID: <4F1BBA34.6040106@redhat.com>

On 20/01/12 17:21, Itamar Heim wrote:
> On 01/20/2012 12:01 PM, Livnat Peer wrote:
>> On 20/01/12 09:35, Ayal Baron wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> Top Posting:
>>>>
>>>>  From user POV I think that option 2 is the only one that make sense.
>>>> We try to do as much as we can,
>>>> and on each "problematic" case, we make him aware and let him decide.
>>>>
>>>
>>> Yep, +1.
>>>
>>
>> Trying to get to a conclusion here,
>> 3 different people said on this thread that they think that from the
>> user perspective leaving the shared devices plugged is what they think
>> is the best behavior to the user. (Omer, Kolesnik, Yair)
>>
>> On the other hand we have 2 people who think that protecting the user is
>> more important than leaving the VM configuration as it was in the
>> original VM (Miki, Ayal).
>>
>> Ayal/Miki can you please specify what are we protecting the user from?
>>
>>
>> I think that because we are not snapshotting the shared disk and the
>> direct LUN they should not be part of the VM configuration (in the
>> snapshot) at all. we can not promise the user that the disk will be
>> there and if it is there we can not guarantee it is in the same state as
>> it was when we took the snapshot.
>>
>>
>> Another issue,
>>
>> I can not see a reason to limit this feature to creating a VM from
>> snapshot and not a template? Almost no extra work is needed for
>> supporting templates as well.
> 
> I assume you meant, creating a VM from another VM (if it is down)?
> It should be supported.

Actually I meant creating a Template from Snapshot.

What you suggested is creating a VM from VM.
Although I see how the two are connected, I think they should be modeled
as two different API calls.
There is a difference in the flow, behavior, locks and parameters
between the two.

Behavior:
- Original VM has to be down for creating a VM from VM, not the case for
creating a VM from snapshot.

parameters:
- Creating VM from snapshot should support getting a snapshot-ID,
Creating VM from VM get a VM id

Locks:
- When creating a VM from VM, we need to lock the original VM as a
whole, we can not edit the VM, take snapshot or any other VM level
action while such operation is active.
While for creating the VM from snapshot we can take more fine-grained
locks (only image related locks).

Implementation:
Well it is simply another implementation.


Livnat



From lpeer at redhat.com  Sun Jan 22 07:29:03 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 22 Jan 2012 09:29:03 +0200
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <7c36dec5-9c7c-4947-9a8a-b94e76c255d8@zmail16.collab.prod.int.phx2.redhat.com>
References: <7c36dec5-9c7c-4947-9a8a-b94e76c255d8@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1BBABF.1090907@redhat.com>

On 22/01/12 08:28, Eduardo Warszawski wrote:
> 
> 
> ----- Original Message -----
>>
>>
>> ----- Original Message -----
>>> On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
>>>> Can we broaden the scope and also allow passing createVG
>>>> partitioned devices with an override flag or something? (we'd
>>>> need
>>>> to check the devices and run "kpartx -d" and fdisk to clean the
>>>> devices before calling pvcreate).
>>>
>>> We can, and we should. My initial patch is just the bare minimum;
>>> I'd
>>> like
>>> Douglas to carry it on, and I am still waiting to meet his Engine
>>> counterpart.
>>> Currently, a LUN that was once used as a raw hard disk cannot be
>>> used
>>> by RHEV;
>>> that's sad.
>>>
>>> How about this for API:
>>>
>>>    createVG(self, vgname, devlist, options={trashpart_devlist: []})
>>
>> No, stop using options as a trash can.
>> If we're changing the API, it should already be just passing a LUNs
>> dictionary to createStorageDomain and start deprecating createVG
> 
> Was agreed that createVG and all vg-uuid aware functions will be removed shortly.
> Use only createStorageDomain.
> 

When you write 'removed'? you don't actually mean remove, right?


>>
>>>
>>> createVG would honor an optional list of devices (subset of
>>> devlist)
>>> whose
>>> partition tables should be trashed.
>>>
>>> Dan.
>>>
>>



From ovedo at redhat.com  Sun Jan 22 07:31:28 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Sun, 22 Jan 2012 02:31:28 -0500 (EST)
Subject: [Engine-devel] running jboss as 7 with openjdk
In-Reply-To: <4F19703F.7010402@redhat.com>
Message-ID: <d3cac700-5fae-4345-a063-ad6e945be366@zmail02.collab.prod.int.phx2.redhat.com>

Can you attach the log?
Did you use the maven setup profile the first time you started using jboss as7?

The reason I'm asking is that this step adds the following lines to $JBOSS_HOME/modules/sun/jdk/main/module.xml, which is supposed to fix this error:
                <path name="sun/security"/>
                <path name="sun/security/krb5"/>

Let me know if these lines are there, and if adding them there fixes your issue.

Oved


----- Original Message -----
> From: "Jon Choate" <jchoate at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Friday, January 20, 2012 3:46:39 PM
> Subject: [Engine-devel] running jboss as 7 with openjdk
> 
> I keep getting a message that it can't find
> sun.security.krb5.KrbException. Any ideas? Jboss-as-5 ran
> ovirt-engine
> just fine with openjdk.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From lpeer at redhat.com  Sun Jan 22 07:37:26 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 22 Jan 2012 09:37:26 +0200
Subject: [Engine-devel] CloneVMFromSnapshot - feature document
In-Reply-To: <4F1B34C3.90305@redhat.com>
References: <4F16886F.1050005@redhat.com> <4F1B34C3.90305@redhat.com>
Message-ID: <4F1BBCB6.9090501@redhat.com>

On 21/01/12 23:57, Itamar Heim wrote:
> On 01/18/2012 10:53 AM, Yair Zaslavsky wrote:
>> Hi all,
>> You can look at http://ovirt.org/wiki/Features/CloneVmFromSnapshot
>>
>> And see the feature document for Clone VM from Snapshot.
>>
>> Comments are more than welcome (planning on providing a the detailed
>> document soon)
> 
> apart from the ongoing discussion on this, my main comment is it should
> be possible to clone a VM in down state directly, without a need to
> create snapshots on it.
> 
> This is possible today by creating a template from the VM, then
> instantiating VMs from the template via thin or clone.
> Clone VM should allow to remove the need of creating a template.
> 
> Only clone would be supported of course, though  the VM is thinly
> provisioned from a template, then cloning only the thinly provisioned
> disks rather than the entire disks would be good as well)


I commented on this requirement on the other thread.
If it is ok with you let's try to have all related discussions on the
other thread for making it easy to track.


> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From lpeer at redhat.com  Sun Jan 22 07:42:06 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 22 Jan 2012 09:42:06 +0200
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <2fe24fa1-84e1-4375-8ed4-25de7cde8d8c@zmail16.collab.prod.int.phx2.redhat.com>
References: <2fe24fa1-84e1-4375-8ed4-25de7cde8d8c@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1BBDCE.6000904@redhat.com>

On 22/01/12 09:39, Eduardo Warszawski wrote:
> 
> 
> ----- Original Message -----
>> On 22/01/12 08:28, Eduardo Warszawski wrote:
>>>
>>>
>>> ----- Original Message -----
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
>>>>>> Can we broaden the scope and also allow passing createVG
>>>>>> partitioned devices with an override flag or something? (we'd
>>>>>> need
>>>>>> to check the devices and run "kpartx -d" and fdisk to clean the
>>>>>> devices before calling pvcreate).
>>>>>
>>>>> We can, and we should. My initial patch is just the bare minimum;
>>>>> I'd
>>>>> like
>>>>> Douglas to carry it on, and I am still waiting to meet his Engine
>>>>> counterpart.
>>>>> Currently, a LUN that was once used as a raw hard disk cannot be
>>>>> used
>>>>> by RHEV;
>>>>> that's sad.
>>>>>
>>>>> How about this for API:
>>>>>
>>>>>    createVG(self, vgname, devlist, options={trashpart_devlist:
>>>>>    []})
>>>>
>>>> No, stop using options as a trash can.
>>>> If we're changing the API, it should already be just passing a
>>>> LUNs
>>>> dictionary to createStorageDomain and start deprecating createVG
>>>
>>> Was agreed that createVG and all vg-uuid aware functions will be
>>> removed shortly.
>>> Use only createStorageDomain.
>>>
>>
>> When you write 'removed'? you don't actually mean remove, right?
> Yes, I do.
> The flows are simpler from the RHEV-M and VDSM sides.

I agree.

> VG's can be created, removed, etc. using lvm tools.
> VDSM is only about Storage Domains and not a VG manager.

I agree

> What's the point of creating a VG using VDSM if not for building a SD?

Backwards compatibility.

> Therefore it should be a unique operation in the API.
> 
>>
>>
>>>>
>>>>>
>>>>> createVG would honor an optional list of devices (subset of
>>>>> devlist)
>>>>> whose
>>>>> partition tables should be trashed.
>>>>>
>>>>> Dan.
>>>>>
>>>>
>>
>>



From ovedo at redhat.com  Sun Jan 22 08:09:21 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Sun, 22 Jan 2012 03:09:21 -0500 (EST)
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <1cdfa509-3d19-469f-a74e-ca009fcd89bf@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <ed78c818-830e-41bd-abf2-c861390d062b@zmail02.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Oved Ourfalli" <ovedo at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Thursday, January 19, 2012 4:05:08 PM
> Subject: Re: [Engine-devel] VM Payload feature
> 
> 
> 
> ----- Original Message -----
> > Hey all,
> > 
> > Continuing the discussion about Aeolus instance data injection to a
> > VM
> > (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> > we propose a new VM Payload feature.
> > 
> > The following wiki page contains a description page of the feature.
> > http://www.ovirt.org/wiki/Features/VMPayload
> > 
> > Please read and review.
> > There are several approaches there, and we wish to head your
> > opinions
> > and thoughts about them.
> > 
> > Once we agree on an approach, we will start designing.
> 
> Permanent payload availability requires determining where the payload
> is stored.
> Makes sense to me to store it together with the VM disks on the
> storage domain, but that requires the small object store which will
> not be available in the coming version (payloads can be large and
> keeping them in the DB and passing over the net every time the VM is
> run doesn't make much sense).
> 
I guess we can start with storing it in the database, with some size limitation, and move it to the storage domain later on.

> Wrt availability, I don't see a reason to exclude attaching both a CD
> and a payload via another CD at the same time (i.e. multiple
> devices).
> 
> > 
> > Thank you,
> > Oved
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From lpeer at redhat.com  Sun Jan 22 08:15:59 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 22 Jan 2012 10:15:59 +0200
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <f2875751-7797-4003-b24a-36eff6037bb0@zmail16.collab.prod.int.phx2.redhat.com>
References: <f2875751-7797-4003-b24a-36eff6037bb0@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1BC5BF.2010808@redhat.com>

On 22/01/12 10:05, Eduardo Warszawski wrote:
> 
> 
> ----- Original Message -----
>> On 22/01/12 09:39, Eduardo Warszawski wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> On 22/01/12 08:28, Eduardo Warszawski wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
>>>>>>>> Can we broaden the scope and also allow passing createVG
>>>>>>>> partitioned devices with an override flag or something? (we'd
>>>>>>>> need
>>>>>>>> to check the devices and run "kpartx -d" and fdisk to clean
>>>>>>>> the
>>>>>>>> devices before calling pvcreate).
>>>>>>>
>>>>>>> We can, and we should. My initial patch is just the bare
>>>>>>> minimum;
>>>>>>> I'd
>>>>>>> like
>>>>>>> Douglas to carry it on, and I am still waiting to meet his
>>>>>>> Engine
>>>>>>> counterpart.
>>>>>>> Currently, a LUN that was once used as a raw hard disk cannot
>>>>>>> be
>>>>>>> used
>>>>>>> by RHEV;
>>>>>>> that's sad.
>>>>>>>
>>>>>>> How about this for API:
>>>>>>>
>>>>>>>    createVG(self, vgname, devlist, options={trashpart_devlist:
>>>>>>>    []})
>>>>>>
>>>>>> No, stop using options as a trash can.
>>>>>> If we're changing the API, it should already be just passing a
>>>>>> LUNs
>>>>>> dictionary to createStorageDomain and start deprecating createVG
>>>>>
>>>>> Was agreed that createVG and all vg-uuid aware functions will be
>>>>> removed shortly.
>>>>> Use only createStorageDomain.
>>>>>
>>>>
>>>> When you write 'removed'? you don't actually mean remove, right?
>>> Yes, I do.
>>> The flows are simpler from the RHEV-M and VDSM sides.
>>
>> I agree.
>>
>>> VG's can be created, removed, etc. using lvm tools.
>>> VDSM is only about Storage Domains and not a VG manager.
>>
>> I agree
>>
>>> What's the point of creating a VG using VDSM if not for building a
>>> SD?
>>
>> Backwards compatibility.
> For backwards compatibility we don't need to develop further the createVG interface.
> If the API is changed createStorageDomain should receive the device dict 
> (as Ayal said), and then we can mockup the createVG part, as we already do with the 
> formatStorageDomain-removeVG functions.

I don't expect you to further develop the createVG verb only not to
remove it.
In addition you should take into consideration that the engine is going
to keep using the current createVG flow, not because we like it simply
because making the change for running another flow is not prioritized
ATM, so please make sure everything is working the same.


> 
>>
>>> Therefore it should be a unique operation in the API.
>>>
>>>>
>>>>
>>>>>>
>>>>>>>
>>>>>>> createVG would honor an optional list of devices (subset of
>>>>>>> devlist)
>>>>>>> whose
>>>>>>> partition tables should be trashed.
>>>>>>>
>>>>>>> Dan.
>>>>>>>
>>>>>>
>>>>
>>>>
>>
>>



From ewarszaw at redhat.com  Sun Jan 22 06:28:10 2012
From: ewarszaw at redhat.com (Eduardo Warszawski)
Date: Sun, 22 Jan 2012 01:28:10 -0500 (EST)
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <a184a066-5115-4c17-9072-190e6f914777@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <7c36dec5-9c7c-4947-9a8a-b94e76c255d8@zmail16.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
> > > Can we broaden the scope and also allow passing createVG
> > > partitioned devices with an override flag or something? (we'd
> > > need
> > > to check the devices and run "kpartx -d" and fdisk to clean the
> > > devices before calling pvcreate).
> > 
> > We can, and we should. My initial patch is just the bare minimum;
> > I'd
> > like
> > Douglas to carry it on, and I am still waiting to meet his Engine
> > counterpart.
> > Currently, a LUN that was once used as a raw hard disk cannot be
> > used
> > by RHEV;
> > that's sad.
> > 
> > How about this for API:
> > 
> >    createVG(self, vgname, devlist, options={trashpart_devlist: []})
> 
> No, stop using options as a trash can.
> If we're changing the API, it should already be just passing a LUNs
> dictionary to createStorageDomain and start deprecating createVG

Was agreed that createVG and all vg-uuid aware functions will be removed shortly.
Use only createStorageDomain.

> 
> > 
> > createVG would honor an optional list of devices (subset of
> > devlist)
> > whose
> > partition tables should be trashed.
> > 
> > Dan.
> > 
> 


From ewarszaw at redhat.com  Sun Jan 22 07:39:36 2012
From: ewarszaw at redhat.com (Eduardo Warszawski)
Date: Sun, 22 Jan 2012 02:39:36 -0500 (EST)
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <4F1BBABF.1090907@redhat.com>
Message-ID: <2fe24fa1-84e1-4375-8ed4-25de7cde8d8c@zmail16.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 22/01/12 08:28, Eduardo Warszawski wrote:
> > 
> > 
> > ----- Original Message -----
> >>
> >>
> >> ----- Original Message -----
> >>> On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
> >>>> Can we broaden the scope and also allow passing createVG
> >>>> partitioned devices with an override flag or something? (we'd
> >>>> need
> >>>> to check the devices and run "kpartx -d" and fdisk to clean the
> >>>> devices before calling pvcreate).
> >>>
> >>> We can, and we should. My initial patch is just the bare minimum;
> >>> I'd
> >>> like
> >>> Douglas to carry it on, and I am still waiting to meet his Engine
> >>> counterpart.
> >>> Currently, a LUN that was once used as a raw hard disk cannot be
> >>> used
> >>> by RHEV;
> >>> that's sad.
> >>>
> >>> How about this for API:
> >>>
> >>>    createVG(self, vgname, devlist, options={trashpart_devlist:
> >>>    []})
> >>
> >> No, stop using options as a trash can.
> >> If we're changing the API, it should already be just passing a
> >> LUNs
> >> dictionary to createStorageDomain and start deprecating createVG
> > 
> > Was agreed that createVG and all vg-uuid aware functions will be
> > removed shortly.
> > Use only createStorageDomain.
> > 
> 
> When you write 'removed'? you don't actually mean remove, right?
Yes, I do.
The flows are simpler from the RHEV-M and VDSM sides.
VG's can be created, removed, etc. using lvm tools.
VDSM is only about Storage Domains and not a VG manager.
What's the point of creating a VG using VDSM if not for building a SD?
Therefore it should be a unique operation in the API.

> 
> 
> >>
> >>>
> >>> createVG would honor an optional list of devices (subset of
> >>> devlist)
> >>> whose
> >>> partition tables should be trashed.
> >>>
> >>> Dan.
> >>>
> >>
> 
> 


From ewarszaw at redhat.com  Sun Jan 22 08:05:13 2012
From: ewarszaw at redhat.com (Eduardo Warszawski)
Date: Sun, 22 Jan 2012 03:05:13 -0500 (EST)
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <4F1BBDCE.6000904@redhat.com>
Message-ID: <f2875751-7797-4003-b24a-36eff6037bb0@zmail16.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 22/01/12 09:39, Eduardo Warszawski wrote:
> > 
> > 
> > ----- Original Message -----
> >> On 22/01/12 08:28, Eduardo Warszawski wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>>
> >>>>
> >>>> ----- Original Message -----
> >>>>> On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
> >>>>>> Can we broaden the scope and also allow passing createVG
> >>>>>> partitioned devices with an override flag or something? (we'd
> >>>>>> need
> >>>>>> to check the devices and run "kpartx -d" and fdisk to clean
> >>>>>> the
> >>>>>> devices before calling pvcreate).
> >>>>>
> >>>>> We can, and we should. My initial patch is just the bare
> >>>>> minimum;
> >>>>> I'd
> >>>>> like
> >>>>> Douglas to carry it on, and I am still waiting to meet his
> >>>>> Engine
> >>>>> counterpart.
> >>>>> Currently, a LUN that was once used as a raw hard disk cannot
> >>>>> be
> >>>>> used
> >>>>> by RHEV;
> >>>>> that's sad.
> >>>>>
> >>>>> How about this for API:
> >>>>>
> >>>>>    createVG(self, vgname, devlist, options={trashpart_devlist:
> >>>>>    []})
> >>>>
> >>>> No, stop using options as a trash can.
> >>>> If we're changing the API, it should already be just passing a
> >>>> LUNs
> >>>> dictionary to createStorageDomain and start deprecating createVG
> >>>
> >>> Was agreed that createVG and all vg-uuid aware functions will be
> >>> removed shortly.
> >>> Use only createStorageDomain.
> >>>
> >>
> >> When you write 'removed'? you don't actually mean remove, right?
> > Yes, I do.
> > The flows are simpler from the RHEV-M and VDSM sides.
> 
> I agree.
> 
> > VG's can be created, removed, etc. using lvm tools.
> > VDSM is only about Storage Domains and not a VG manager.
> 
> I agree
> 
> > What's the point of creating a VG using VDSM if not for building a
> > SD?
> 
> Backwards compatibility.
For backwards compatibility we don't need to develop further the createVG interface.
If the API is changed createStorageDomain should receive the device dict 
(as Ayal said), and then we can mockup the createVG part, as we already do with the 
formatStorageDomain-removeVG functions.

> 
> > Therefore it should be a unique operation in the API.
> > 
> >>
> >>
> >>>>
> >>>>>
> >>>>> createVG would honor an optional list of devices (subset of
> >>>>> devlist)
> >>>>> whose
> >>>>> partition tables should be trashed.
> >>>>>
> >>>>> Dan.
> >>>>>
> >>>>
> >>
> >>
> 
> 


From yzaslavs at redhat.com  Sun Jan 22 09:26:13 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Sun, 22 Jan 2012 11:26:13 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <4F1BBA34.6040106@redhat.com>
References: <e46da1a1-4bc0-4992-aa41-6bff7076c144@zmail13.collab.prod.int.phx2.redhat.com>	<4F193B5E.1050200@redhat.com>
	<4F19867E.3030005@redhat.com> <4F1BBA34.6040106@redhat.com>
Message-ID: <4F1BD635.9040302@redhat.com>

On 01/22/2012 09:26 AM, Livnat Peer wrote:
> On 20/01/12 17:21, Itamar Heim wrote:
>> On 01/20/2012 12:01 PM, Livnat Peer wrote:
>>> On 20/01/12 09:35, Ayal Baron wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> Top Posting:
>>>>>
>>>>>  From user POV I think that option 2 is the only one that make sense.
>>>>> We try to do as much as we can,
>>>>> and on each "problematic" case, we make him aware and let him decide.
>>>>>
>>>>
>>>> Yep, +1.
>>>>
>>>
>>> Trying to get to a conclusion here,
>>> 3 different people said on this thread that they think that from the
>>> user perspective leaving the shared devices plugged is what they think
>>> is the best behavior to the user. (Omer, Kolesnik, Yair)
>>>
>>> On the other hand we have 2 people who think that protecting the user is
>>> more important than leaving the VM configuration as it was in the
>>> original VM (Miki, Ayal).
>>>
>>> Ayal/Miki can you please specify what are we protecting the user from?
>>>
>>>
>>> I think that because we are not snapshotting the shared disk and the
>>> direct LUN they should not be part of the VM configuration (in the
>>> snapshot) at all. we can not promise the user that the disk will be
>>> there and if it is there we can not guarantee it is in the same state as
>>> it was when we took the snapshot.
>>>
>>>
>>> Another issue,
>>>
>>> I can not see a reason to limit this feature to creating a VM from
>>> snapshot and not a template? Almost no extra work is needed for
>>> supporting templates as well.
>>
>> I assume you meant, creating a VM from another VM (if it is down)?
>> It should be supported.
> 
> Actually I meant creating a Template from Snapshot.
Livnat - I think that in case of creating a template from snapshot we
should should have new API/Command, that will probably have lots in
common with Create VM from snapshot.

> 
> What you suggested is creating a VM from VM.
> Although I see how the two are connected, I think they should be modeled
> as two different API calls.
> There is a difference in the flow, behavior, locks and parameters
> between the two.
> 
> Behavior:
> - Original VM has to be down for creating a VM from VM, not the case for
> creating a VM from snapshot.
> 
> parameters:
> - Creating VM from snapshot should support getting a snapshot-ID,
> Creating VM from VM get a VM id
> 
> Locks:
> - When creating a VM from VM, we need to lock the original VM as a
> whole, we can not edit the VM, take snapshot or any other VM level
> action while such operation is active.
> While for creating the VM from snapshot we can take more fine-grained
> locks (only image related locks).
> 
> Implementation:
> Well it is simply another implementation.
+1 on Livnat's explanation - I do see a (design/implementation wise) an
option for some code reuse, but IMHO - this should be a new command with
new API modelling
> 
> 
> Livnat
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From emesika at redhat.com  Sun Jan 22 09:48:00 2012
From: emesika at redhat.com (Eli Mesika)
Date: Sun, 22 Jan 2012 04:48:00 -0500 (EST)
Subject: [Engine-devel] backup/restore your git branch out of git repository
In-Reply-To: <081b4f10-c434-4e5c-b167-3b3e36c94852@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <c83b4550-7961-4ced-ba71-dbc592541c78@zmail13.collab.prod.int.phx2.redhat.com>

Hi

I had last week a corrupted git repository caused possibly by a forced shutdown, I had to re-clone my repository to a new place and apply all my changes.
Lucky me , I had a backup out of the git repository.

The following takes 1 min to apply and may save you lot of work in case something gone wrong.


The following command will create a directory named as your current git branch under ~/backup/patch and will backup there all your branch work

copy the following lines to your ~/.bashrc

alias gitbackup='br=$(git symbolic-ref HEAD 2>/dev/null | cut -d "/" -f3);rm -f *.patch;git format-patch origin/master;mkdir -p ~/backup/patch/$br;mv *.patch ~/backup/patch/$br'

alias gitrestore='br=$(git symbolic-ref HEAD 2>/dev/null | cut -d "/" -f3);git stash; git checkout master;git checkout -b "$br-restore-$(date "+%d-%m-%y--%H-%M")";cp ~/backup/patch/$br/*.patch .;for f in $(ls -1 *.patch) ;do git am --ignore-whitespace $f;done'

> source ~/.bashrc

to backup your branch just run from your current git branch 

> gitbackup

In any case you have messed up with your branch (and you have a backup), you can run from your current git branch
> gitrestore

this will create a new restored branch named <your-original-branch-name>-restore-<date> and apply on it your backed up patches.




From abaron at redhat.com  Sun Jan 22 10:14:33 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 22 Jan 2012 05:14:33 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
	context of shared disks and direct LUNs-based disks
In-Reply-To: <4F1BBA34.6040106@redhat.com>
Message-ID: <f79f1b98-71e2-4ed9-a236-239c5c5498aa@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 20/01/12 17:21, Itamar Heim wrote:
> > On 01/20/2012 12:01 PM, Livnat Peer wrote:
> >> On 20/01/12 09:35, Ayal Baron wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> Top Posting:
> >>>>
> >>>>  From user POV I think that option 2 is the only one that make
> >>>>  sense.
> >>>> We try to do as much as we can,
> >>>> and on each "problematic" case, we make him aware and let him
> >>>> decide.
> >>>>
> >>>
> >>> Yep, +1.
> >>>
> >>
> >> Trying to get to a conclusion here,
> >> 3 different people said on this thread that they think that from
> >> the
> >> user perspective leaving the shared devices plugged is what they
> >> think
> >> is the best behavior to the user. (Omer, Kolesnik, Yair)
> >>
> >> On the other hand we have 2 people who think that protecting the
> >> user is
> >> more important than leaving the VM configuration as it was in the
> >> original VM (Miki, Ayal).
> >>
> >> Ayal/Miki can you please specify what are we protecting the user
> >> from?
> >>
> >>
> >> I think that because we are not snapshotting the shared disk and
> >> the
> >> direct LUN they should not be part of the VM configuration (in the
> >> snapshot) at all. we can not promise the user that the disk will
> >> be
> >> there and if it is there we can not guarantee it is in the same
> >> state as
> >> it was when we took the snapshot.
> >>
> >>
> >> Another issue,
> >>
> >> I can not see a reason to limit this feature to creating a VM from
> >> snapshot and not a template? Almost no extra work is needed for
> >> supporting templates as well.
> > 
> > I assume you meant, creating a VM from another VM (if it is down)?
> > It should be supported.
> 
> Actually I meant creating a Template from Snapshot.

+1

> 
> What you suggested is creating a VM from VM.
> Although I see how the two are connected, I think they should be
> modeled
> as two different API calls.
> There is a difference in the flow, behavior, locks and parameters
> between the two.
> 
> Behavior:
> - Original VM has to be down for creating a VM from VM, not the case
> for
> creating a VM from snapshot.
> 
> parameters:
> - Creating VM from snapshot should support getting a snapshot-ID,
> Creating VM from VM get a VM id

What's the difference?

> 
> Locks:
> - When creating a VM from VM, we need to lock the original VM as a
> whole, we can not edit the VM, take snapshot or any other VM level
> action while such operation is active.
> While for creating the VM from snapshot we can take more fine-grained
> locks (only image related locks).

I would suggest we change the clone VM flow to be:
1. create a snapshot in original VM
2. clone the snapshot
This way, while this is going on, the user *can* run the VM and do everything else and behaviour becomes much nicer and not 'you have to wait 4 hours until your clone ends before running the VM'.
3. delete the snapshot

This would also mean that both ops would be collapsed to one and there would be only 1 flow.

> 
> Implementation:
> Well it is simply another implementation.
> 
> 
> Livnat
> 
> 


From abaron at redhat.com  Sun Jan 22 10:15:46 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 22 Jan 2012 05:15:46 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
	context of shared disks and direct LUNs-based disks
In-Reply-To: <4F1BD635.9040302@redhat.com>
Message-ID: <20f90a49-e2f8-4b58-ae95-a696af5a7fa5@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/22/2012 09:26 AM, Livnat Peer wrote:
> > On 20/01/12 17:21, Itamar Heim wrote:
> >> On 01/20/2012 12:01 PM, Livnat Peer wrote:
> >>> On 20/01/12 09:35, Ayal Baron wrote:
> >>>>
> >>>>
> >>>> ----- Original Message -----
> >>>>> Top Posting:
> >>>>>
> >>>>>  From user POV I think that option 2 is the only one that make
> >>>>>  sense.
> >>>>> We try to do as much as we can,
> >>>>> and on each "problematic" case, we make him aware and let him
> >>>>> decide.
> >>>>>
> >>>>
> >>>> Yep, +1.
> >>>>
> >>>
> >>> Trying to get to a conclusion here,
> >>> 3 different people said on this thread that they think that from
> >>> the
> >>> user perspective leaving the shared devices plugged is what they
> >>> think
> >>> is the best behavior to the user. (Omer, Kolesnik, Yair)
> >>>
> >>> On the other hand we have 2 people who think that protecting the
> >>> user is
> >>> more important than leaving the VM configuration as it was in the
> >>> original VM (Miki, Ayal).
> >>>
> >>> Ayal/Miki can you please specify what are we protecting the user
> >>> from?
> >>>
> >>>
> >>> I think that because we are not snapshotting the shared disk and
> >>> the
> >>> direct LUN they should not be part of the VM configuration (in
> >>> the
> >>> snapshot) at all. we can not promise the user that the disk will
> >>> be
> >>> there and if it is there we can not guarantee it is in the same
> >>> state as
> >>> it was when we took the snapshot.
> >>>
> >>>
> >>> Another issue,
> >>>
> >>> I can not see a reason to limit this feature to creating a VM
> >>> from
> >>> snapshot and not a template? Almost no extra work is needed for
> >>> supporting templates as well.
> >>
> >> I assume you meant, creating a VM from another VM (if it is down)?
> >> It should be supported.
> > 
> > Actually I meant creating a Template from Snapshot.
> Livnat - I think that in case of creating a template from snapshot we
> should should have new API/Command, that will probably have lots in
> common with Create VM from snapshot.

Why?

> 
> > 
> > What you suggested is creating a VM from VM.
> > Although I see how the two are connected, I think they should be
> > modeled
> > as two different API calls.
> > There is a difference in the flow, behavior, locks and parameters
> > between the two.
> > 
> > Behavior:
> > - Original VM has to be down for creating a VM from VM, not the
> > case for
> > creating a VM from snapshot.
> > 
> > parameters:
> > - Creating VM from snapshot should support getting a snapshot-ID,
> > Creating VM from VM get a VM id
> > 
> > Locks:
> > - When creating a VM from VM, we need to lock the original VM as a
> > whole, we can not edit the VM, take snapshot or any other VM level
> > action while such operation is active.
> > While for creating the VM from snapshot we can take more
> > fine-grained
> > locks (only image related locks).
> > 
> > Implementation:
> > Well it is simply another implementation.
> +1 on Livnat's explanation - I do see a (design/implementation wise)
> an
> option for some code reuse, but IMHO - this should be a new command
> with
> new API modelling
> > 
> > 
> > Livnat
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From abaron at redhat.com  Sun Jan 22 10:26:26 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 22 Jan 2012 05:26:26 -0500 (EST)
Subject: [Engine-devel] getDeviceList should report partitioned devices,
	too
In-Reply-To: <4F1BC5BF.2010808@redhat.com>
Message-ID: <76bcf328-a292-4829-b05f-519746a274c6@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 22/01/12 10:05, Eduardo Warszawski wrote:
> > 
> > 
> > ----- Original Message -----
> >> On 22/01/12 09:39, Eduardo Warszawski wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> On 22/01/12 08:28, Eduardo Warszawski wrote:
> >>>>>
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>>
> >>>>>>
> >>>>>> ----- Original Message -----
> >>>>>>> On Wed, Jan 18, 2012 at 06:17:53AM -0500, Ayal Baron wrote:
> >>>>>>>> Can we broaden the scope and also allow passing createVG
> >>>>>>>> partitioned devices with an override flag or something?
> >>>>>>>> (we'd
> >>>>>>>> need
> >>>>>>>> to check the devices and run "kpartx -d" and fdisk to clean
> >>>>>>>> the
> >>>>>>>> devices before calling pvcreate).
> >>>>>>>
> >>>>>>> We can, and we should. My initial patch is just the bare
> >>>>>>> minimum;
> >>>>>>> I'd
> >>>>>>> like
> >>>>>>> Douglas to carry it on, and I am still waiting to meet his
> >>>>>>> Engine
> >>>>>>> counterpart.
> >>>>>>> Currently, a LUN that was once used as a raw hard disk cannot
> >>>>>>> be
> >>>>>>> used
> >>>>>>> by RHEV;
> >>>>>>> that's sad.
> >>>>>>>
> >>>>>>> How about this for API:
> >>>>>>>
> >>>>>>>    createVG(self, vgname, devlist,
> >>>>>>>    options={trashpart_devlist:
> >>>>>>>    []})
> >>>>>>
> >>>>>> No, stop using options as a trash can.
> >>>>>> If we're changing the API, it should already be just passing a
> >>>>>> LUNs
> >>>>>> dictionary to createStorageDomain and start deprecating
> >>>>>> createVG
> >>>>>
> >>>>> Was agreed that createVG and all vg-uuid aware functions will
> >>>>> be
> >>>>> removed shortly.
> >>>>> Use only createStorageDomain.
> >>>>>
> >>>>
> >>>> When you write 'removed'? you don't actually mean remove, right?
> >>> Yes, I do.
> >>> The flows are simpler from the RHEV-M and VDSM sides.
> >>
> >> I agree.
> >>
> >>> VG's can be created, removed, etc. using lvm tools.
> >>> VDSM is only about Storage Domains and not a VG manager.
> >>
> >> I agree
> >>
> >>> What's the point of creating a VG using VDSM if not for building
> >>> a
> >>> SD?
> >>
> >> Backwards compatibility.
> > For backwards compatibility we don't need to develop further the
> > createVG interface.
> > If the API is changed createStorageDomain should receive the device
> > dict
> > (as Ayal said), and then we can mockup the createVG part, as we
> > already do with the
> > formatStorageDomain-removeVG functions.
> 
> I don't expect you to further develop the createVG verb only not to
> remove it.
> In addition you should take into consideration that the engine is
> going
> to keep using the current createVG flow, not because we like it
> simply
> because making the change for running another flow is not prioritized
> ATM, so please make sure everything is working the same.

You're asking the wrong question hence getting the wrong answers.
vdsm is committed to backward compatibility of 1 version, hence no verb will be actually 'removed' from vdsm API until a new verb has been introduced and used side by side for at least 1 version.
This is *always* true and if this is broken it is a bug that will be fixed.

Now that we've got that out of the way, wrt the current flow - no changes should be made in createVG, only in createStorageDomain.

> 
> 
> > 
> >>
> >>> Therefore it should be a unique operation in the API.
> >>>
> >>>>
> >>>>
> >>>>>>
> >>>>>>>
> >>>>>>> createVG would honor an optional list of devices (subset of
> >>>>>>> devlist)
> >>>>>>> whose
> >>>>>>> partition tables should be trashed.
> >>>>>>>
> >>>>>>> Dan.
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
> >>
> 
> 


From lpeer at redhat.com  Sun Jan 22 11:12:17 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 22 Jan 2012 13:12:17 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <f79f1b98-71e2-4ed9-a236-239c5c5498aa@zmail13.collab.prod.int.phx2.redhat.com>
References: <f79f1b98-71e2-4ed9-a236-239c5c5498aa@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1BEF11.5020300@redhat.com>

On 22/01/12 12:14, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> On 20/01/12 17:21, Itamar Heim wrote:
>>> On 01/20/2012 12:01 PM, Livnat Peer wrote:
>>>> On 20/01/12 09:35, Ayal Baron wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> Top Posting:
>>>>>>
>>>>>>  From user POV I think that option 2 is the only one that make
>>>>>>  sense.
>>>>>> We try to do as much as we can,
>>>>>> and on each "problematic" case, we make him aware and let him
>>>>>> decide.
>>>>>>
>>>>>
>>>>> Yep, +1.
>>>>>
>>>>
>>>> Trying to get to a conclusion here,
>>>> 3 different people said on this thread that they think that from
>>>> the
>>>> user perspective leaving the shared devices plugged is what they
>>>> think
>>>> is the best behavior to the user. (Omer, Kolesnik, Yair)
>>>>
>>>> On the other hand we have 2 people who think that protecting the
>>>> user is
>>>> more important than leaving the VM configuration as it was in the
>>>> original VM (Miki, Ayal).
>>>>
>>>> Ayal/Miki can you please specify what are we protecting the user
>>>> from?
>>>>
>>>>
>>>> I think that because we are not snapshotting the shared disk and
>>>> the
>>>> direct LUN they should not be part of the VM configuration (in the
>>>> snapshot) at all. we can not promise the user that the disk will
>>>> be
>>>> there and if it is there we can not guarantee it is in the same
>>>> state as
>>>> it was when we took the snapshot.
>>>>
>>>>
>>>> Another issue,
>>>>
>>>> I can not see a reason to limit this feature to creating a VM from
>>>> snapshot and not a template? Almost no extra work is needed for
>>>> supporting templates as well.
>>>
>>> I assume you meant, creating a VM from another VM (if it is down)?
>>> It should be supported.
>>
>> Actually I meant creating a Template from Snapshot.
> 
> +1
> 
>>
>> What you suggested is creating a VM from VM.
>> Although I see how the two are connected, I think they should be
>> modeled
>> as two different API calls.
>> There is a difference in the flow, behavior, locks and parameters
>> between the two.
>>
>> Behavior:
>> - Original VM has to be down for creating a VM from VM, not the case
>> for
>> creating a VM from snapshot.
>>
>> parameters:
>> - Creating VM from snapshot should support getting a snapshot-ID,
>> Creating VM from VM get a VM id
> 
> What's the difference?
> 
>>
>> Locks:
>> - When creating a VM from VM, we need to lock the original VM as a
>> whole, we can not edit the VM, take snapshot or any other VM level
>> action while such operation is active.
>> While for creating the VM from snapshot we can take more fine-grained
>> locks (only image related locks).
> 
> I would suggest we change the clone VM flow to be:
> 1. create a snapshot in original VM
> 2. clone the snapshot
> This way, while this is going on, the user *can* run the VM and do everything else and behaviour becomes much nicer and not 'you have to wait 4 hours until your clone ends before running the VM'.
> 3. delete the snapshot
> 
> This would also mean that both ops would be collapsed to one and there would be only 1 flow.
> 

I am not sure this is the right way to support creating a VM from VM flow.

Let's say I have a server VM with RAW disks (for performance), I would
not want to take a snapshot from this VM to clone it.



>>
>> Implementation:
>> Well it is simply another implementation.
>>
>>
>> Livnat
>>
>>



From abaron at redhat.com  Sun Jan 22 13:21:10 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 22 Jan 2012 08:21:10 -0500 (EST)
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
	context of shared disks and direct LUNs-based disks
In-Reply-To: <4F1BEF11.5020300@redhat.com>
Message-ID: <584e85e8-9f43-456f-9130-981ed8211083@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 22/01/12 12:14, Ayal Baron wrote:
> > 
> > 
> > ----- Original Message -----
> >> On 20/01/12 17:21, Itamar Heim wrote:
> >>> On 01/20/2012 12:01 PM, Livnat Peer wrote:
> >>>> On 20/01/12 09:35, Ayal Baron wrote:
> >>>>>
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> Top Posting:
> >>>>>>
> >>>>>>  From user POV I think that option 2 is the only one that make
> >>>>>>  sense.
> >>>>>> We try to do as much as we can,
> >>>>>> and on each "problematic" case, we make him aware and let him
> >>>>>> decide.
> >>>>>>
> >>>>>
> >>>>> Yep, +1.
> >>>>>
> >>>>
> >>>> Trying to get to a conclusion here,
> >>>> 3 different people said on this thread that they think that from
> >>>> the
> >>>> user perspective leaving the shared devices plugged is what they
> >>>> think
> >>>> is the best behavior to the user. (Omer, Kolesnik, Yair)
> >>>>
> >>>> On the other hand we have 2 people who think that protecting the
> >>>> user is
> >>>> more important than leaving the VM configuration as it was in
> >>>> the
> >>>> original VM (Miki, Ayal).
> >>>>
> >>>> Ayal/Miki can you please specify what are we protecting the user
> >>>> from?
> >>>>
> >>>>
> >>>> I think that because we are not snapshotting the shared disk and
> >>>> the
> >>>> direct LUN they should not be part of the VM configuration (in
> >>>> the
> >>>> snapshot) at all. we can not promise the user that the disk will
> >>>> be
> >>>> there and if it is there we can not guarantee it is in the same
> >>>> state as
> >>>> it was when we took the snapshot.
> >>>>
> >>>>
> >>>> Another issue,
> >>>>
> >>>> I can not see a reason to limit this feature to creating a VM
> >>>> from
> >>>> snapshot and not a template? Almost no extra work is needed for
> >>>> supporting templates as well.
> >>>
> >>> I assume you meant, creating a VM from another VM (if it is
> >>> down)?
> >>> It should be supported.
> >>
> >> Actually I meant creating a Template from Snapshot.
> > 
> > +1
> > 
> >>
> >> What you suggested is creating a VM from VM.
> >> Although I see how the two are connected, I think they should be
> >> modeled
> >> as two different API calls.
> >> There is a difference in the flow, behavior, locks and parameters
> >> between the two.
> >>
> >> Behavior:
> >> - Original VM has to be down for creating a VM from VM, not the
> >> case
> >> for
> >> creating a VM from snapshot.
> >>
> >> parameters:
> >> - Creating VM from snapshot should support getting a snapshot-ID,
> >> Creating VM from VM get a VM id
> > 
> > What's the difference?
> > 
> >>
> >> Locks:
> >> - When creating a VM from VM, we need to lock the original VM as a
> >> whole, we can not edit the VM, take snapshot or any other VM level
> >> action while such operation is active.
> >> While for creating the VM from snapshot we can take more
> >> fine-grained
> >> locks (only image related locks).
> > 
> > I would suggest we change the clone VM flow to be:
> > 1. create a snapshot in original VM
> > 2. clone the snapshot
> > This way, while this is going on, the user *can* run the VM and do
> > everything else and behaviour becomes much nicer and not 'you have
> > to wait 4 hours until your clone ends before running the VM'.
> > 3. delete the snapshot
> > 
> > This would also mean that both ops would be collapsed to one and
> > there would be only 1 flow.
> > 
> 
> I am not sure this is the right way to support creating a VM from VM
> flow.
> 
> Let's say I have a server VM with RAW disks (for performance), I
> would
> not want to take a snapshot from this VM to clone it.

There is only a performance problem if the VM is running.  
Keeping as suggested before would prevent running the VM.
So what I'm suggesting is an improvement (when the VM is running it gives much better performance compared to when it is not).
Then if the user ran the VM then there are 2 options at the end of the clone:
1. shutdown the VM and merge back (again, as opposed to not being able to run the VM - an improvement)
2. live merge which would be costly but VM would go on running (also note that next version will support merge in both directions so even better)



> 
> 
> 
> >>
> >> Implementation:
> >> Well it is simply another implementation.
> >>
> >>
> >> Livnat
> >>
> >>
> 
> 


From iheim at redhat.com  Sun Jan 22 15:09:02 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sun, 22 Jan 2012 17:09:02 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <4F1BBA34.6040106@redhat.com>
References: <e46da1a1-4bc0-4992-aa41-6bff7076c144@zmail13.collab.prod.int.phx2.redhat.com>
	<4F193B5E.1050200@redhat.com> <4F19867E.3030005@redhat.com>
	<4F1BBA34.6040106@redhat.com>
Message-ID: <4F1C268E.8010501@redhat.com>

On 01/22/2012 09:26 AM, Livnat Peer wrote:
> On 20/01/12 17:21, Itamar Heim wrote:
>> On 01/20/2012 12:01 PM, Livnat Peer wrote:
>>> On 20/01/12 09:35, Ayal Baron wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> Top Posting:
>>>>>
>>>>>   From user POV I think that option 2 is the only one that make sense.
>>>>> We try to do as much as we can,
>>>>> and on each "problematic" case, we make him aware and let him decide.
>>>>>
>>>>
>>>> Yep, +1.
>>>>
>>>
>>> Trying to get to a conclusion here,
>>> 3 different people said on this thread that they think that from the
>>> user perspective leaving the shared devices plugged is what they think
>>> is the best behavior to the user. (Omer, Kolesnik, Yair)
>>>
>>> On the other hand we have 2 people who think that protecting the user is
>>> more important than leaving the VM configuration as it was in the
>>> original VM (Miki, Ayal).
>>>
>>> Ayal/Miki can you please specify what are we protecting the user from?
>>>
>>>
>>> I think that because we are not snapshotting the shared disk and the
>>> direct LUN they should not be part of the VM configuration (in the
>>> snapshot) at all. we can not promise the user that the disk will be
>>> there and if it is there we can not guarantee it is in the same state as
>>> it was when we took the snapshot.
>>>
>>>
>>> Another issue,
>>>
>>> I can not see a reason to limit this feature to creating a VM from
>>> snapshot and not a template? Almost no extra work is needed for
>>> supporting templates as well.
>>
>> I assume you meant, creating a VM from another VM (if it is down)?
>> It should be supported.
>
> Actually I meant creating a Template from Snapshot.
>
> What you suggested is creating a VM from VM.
> Although I see how the two are connected, I think they should be modeled
> as two different API calls.
> There is a difference in the flow, behavior, locks and parameters
> between the two.
>
> Behavior:
> - Original VM has to be down for creating a VM from VM, not the case for
> creating a VM from snapshot.
>
> parameters:
> - Creating VM from snapshot should support getting a snapshot-ID,
> Creating VM from VM get a VM id
>
> Locks:
> - When creating a VM from VM, we need to lock the original VM as a
> whole, we can not edit the VM, take snapshot or any other VM level
> action while such operation is active.
> While for creating the VM from snapshot we can take more fine-grained
> locks (only image related locks).
>
> Implementation:
> Well it is simply another implementation.

These would be the "technical details", but from user perspective, I'd 
argue cloning a snapshot is just cloning an older version of the VM.
i.e., i may pass to clone_vm an optional parameter to request cloning an 
older version (specific snapshot), vs. cloning the latest down or 
running VM.
for a running VM, ayal mentioned a possible flow (live snapshot, clone, 
live merge).
implementing this doesn't have to be in same phase, but the question is 
what is the right level of modeling for the API for the action.


From lpeer at redhat.com  Sun Jan 22 17:01:28 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 22 Jan 2012 19:01:28 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <4F1C268E.8010501@redhat.com>
References: <e46da1a1-4bc0-4992-aa41-6bff7076c144@zmail13.collab.prod.int.phx2.redhat.com>
	<4F193B5E.1050200@redhat.com> <4F19867E.3030005@redhat.com>
	<4F1BBA34.6040106@redhat.com> <4F1C268E.8010501@redhat.com>
Message-ID: <4F1C40E8.9070203@redhat.com>

On 22/01/12 17:09, Itamar Heim wrote:
> On 01/22/2012 09:26 AM, Livnat Peer wrote:
>> On 20/01/12 17:21, Itamar Heim wrote:
>>> On 01/20/2012 12:01 PM, Livnat Peer wrote:
>>>> On 20/01/12 09:35, Ayal Baron wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> Top Posting:
>>>>>>
>>>>>>   From user POV I think that option 2 is the only one that make
>>>>>> sense.
>>>>>> We try to do as much as we can,
>>>>>> and on each "problematic" case, we make him aware and let him decide.
>>>>>>
>>>>>
>>>>> Yep, +1.
>>>>>
>>>>
>>>> Trying to get to a conclusion here,
>>>> 3 different people said on this thread that they think that from the
>>>> user perspective leaving the shared devices plugged is what they think
>>>> is the best behavior to the user. (Omer, Kolesnik, Yair)
>>>>
>>>> On the other hand we have 2 people who think that protecting the
>>>> user is
>>>> more important than leaving the VM configuration as it was in the
>>>> original VM (Miki, Ayal).
>>>>
>>>> Ayal/Miki can you please specify what are we protecting the user from?
>>>>
>>>>
>>>> I think that because we are not snapshotting the shared disk and the
>>>> direct LUN they should not be part of the VM configuration (in the
>>>> snapshot) at all. we can not promise the user that the disk will be
>>>> there and if it is there we can not guarantee it is in the same
>>>> state as
>>>> it was when we took the snapshot.
>>>>
>>>>
>>>> Another issue,
>>>>
>>>> I can not see a reason to limit this feature to creating a VM from
>>>> snapshot and not a template? Almost no extra work is needed for
>>>> supporting templates as well.
>>>
>>> I assume you meant, creating a VM from another VM (if it is down)?
>>> It should be supported.
>>
>> Actually I meant creating a Template from Snapshot.
>>
>> What you suggested is creating a VM from VM.
>> Although I see how the two are connected, I think they should be modeled
>> as two different API calls.
>> There is a difference in the flow, behavior, locks and parameters
>> between the two.
>>
>> Behavior:
>> - Original VM has to be down for creating a VM from VM, not the case for
>> creating a VM from snapshot.
>>
>> parameters:
>> - Creating VM from snapshot should support getting a snapshot-ID,
>> Creating VM from VM get a VM id
>>
>> Locks:
>> - When creating a VM from VM, we need to lock the original VM as a
>> whole, we can not edit the VM, take snapshot or any other VM level
>> action while such operation is active.
>> While for creating the VM from snapshot we can take more fine-grained
>> locks (only image related locks).
>>
>> Implementation:
>> Well it is simply another implementation.
> 
> These would be the "technical details", but from user perspective, I'd
> argue cloning a snapshot is just cloning an older version of the VM.
> i.e., i may pass to clone_vm an optional parameter to request cloning an
> older version (specific snapshot), vs. cloning the latest down or
> running VM.
> for a running VM, ayal mentioned a possible flow (live snapshot, clone,
> live merge).
> implementing this doesn't have to be in same phase, but the question is
> what is the right level of modeling for the API for the action.

I agree that modeling the API should drive the decision here and not the
implementation.

I argue that because of the different behavior of the two actions we
should model it as two different operations.

If we force the VM to be down and we are locking the VM through out the
operation of cloning VM from VM while we don't have to do the above for
cloning VM from snapshot we should separate the two.

I think there are flaws with the flow Ayal suggested, I'll detailed them
in the context of his reply.









From lpeer at redhat.com  Sun Jan 22 17:06:57 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sun, 22 Jan 2012 19:06:57 +0200
Subject: [Engine-devel] Question about CloneVMFromSnapshot feature in
 context of shared disks and direct LUNs-based disks
In-Reply-To: <584e85e8-9f43-456f-9130-981ed8211083@zmail13.collab.prod.int.phx2.redhat.com>
References: <584e85e8-9f43-456f-9130-981ed8211083@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1C4231.5060705@redhat.com>

On 22/01/12 15:21, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>> On 22/01/12 12:14, Ayal Baron wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> On 20/01/12 17:21, Itamar Heim wrote:
>>>>> On 01/20/2012 12:01 PM, Livnat Peer wrote:
>>>>>> On 20/01/12 09:35, Ayal Baron wrote:
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> Top Posting:
>>>>>>>>
>>>>>>>>  From user POV I think that option 2 is the only one that make
>>>>>>>>  sense.
>>>>>>>> We try to do as much as we can,
>>>>>>>> and on each "problematic" case, we make him aware and let him
>>>>>>>> decide.
>>>>>>>>
>>>>>>>
>>>>>>> Yep, +1.
>>>>>>>
>>>>>>
>>>>>> Trying to get to a conclusion here,
>>>>>> 3 different people said on this thread that they think that from
>>>>>> the
>>>>>> user perspective leaving the shared devices plugged is what they
>>>>>> think
>>>>>> is the best behavior to the user. (Omer, Kolesnik, Yair)
>>>>>>
>>>>>> On the other hand we have 2 people who think that protecting the
>>>>>> user is
>>>>>> more important than leaving the VM configuration as it was in
>>>>>> the
>>>>>> original VM (Miki, Ayal).
>>>>>>
>>>>>> Ayal/Miki can you please specify what are we protecting the user
>>>>>> from?
>>>>>>
>>>>>>
>>>>>> I think that because we are not snapshotting the shared disk and
>>>>>> the
>>>>>> direct LUN they should not be part of the VM configuration (in
>>>>>> the
>>>>>> snapshot) at all. we can not promise the user that the disk will
>>>>>> be
>>>>>> there and if it is there we can not guarantee it is in the same
>>>>>> state as
>>>>>> it was when we took the snapshot.
>>>>>>
>>>>>>
>>>>>> Another issue,
>>>>>>
>>>>>> I can not see a reason to limit this feature to creating a VM
>>>>>> from
>>>>>> snapshot and not a template? Almost no extra work is needed for
>>>>>> supporting templates as well.
>>>>>
>>>>> I assume you meant, creating a VM from another VM (if it is
>>>>> down)?
>>>>> It should be supported.
>>>>
>>>> Actually I meant creating a Template from Snapshot.
>>>
>>> +1
>>>
>>>>
>>>> What you suggested is creating a VM from VM.
>>>> Although I see how the two are connected, I think they should be
>>>> modeled
>>>> as two different API calls.
>>>> There is a difference in the flow, behavior, locks and parameters
>>>> between the two.
>>>>
>>>> Behavior:
>>>> - Original VM has to be down for creating a VM from VM, not the
>>>> case
>>>> for
>>>> creating a VM from snapshot.
>>>>
>>>> parameters:
>>>> - Creating VM from snapshot should support getting a snapshot-ID,
>>>> Creating VM from VM get a VM id
>>>
>>> What's the difference?
>>>
>>>>
>>>> Locks:
>>>> - When creating a VM from VM, we need to lock the original VM as a
>>>> whole, we can not edit the VM, take snapshot or any other VM level
>>>> action while such operation is active.
>>>> While for creating the VM from snapshot we can take more
>>>> fine-grained
>>>> locks (only image related locks).
>>>
>>> I would suggest we change the clone VM flow to be:
>>> 1. create a snapshot in original VM
>>> 2. clone the snapshot
>>> This way, while this is going on, the user *can* run the VM and do
>>> everything else and behaviour becomes much nicer and not 'you have
>>> to wait 4 hours until your clone ends before running the VM'.
>>> 3. delete the snapshot
>>>
>>> This would also mean that both ops would be collapsed to one and
>>> there would be only 1 flow.
>>>
>>
>> I am not sure this is the right way to support creating a VM from VM
>> flow.
>>
>> Let's say I have a server VM with RAW disks (for performance), I
>> would
>> not want to take a snapshot from this VM to clone it.
> 
> There is only a performance problem if the VM is running.  
> Keeping as suggested before would prevent running the VM.
> So what I'm suggesting is an improvement (when the VM is running it gives much better performance compared to when it is not).
> Then if the user ran the VM then there are 2 options at the end of the clone:
> 1. shutdown the VM and merge back (again, as opposed to not being able to run the VM - an improvement)
> 2. live merge which would be costly but VM would go on running (also note that next version will support merge in both directions so even better)
> 
> 

And what about quotas? with the above flow for cloning VM the user needs
quota on the source domain, I find this not intuitive for the user, and
i expect we'll have hard time explaining it.


> 
>>
>>
>>
>>>>
>>>> Implementation:
>>>> Well it is simply another implementation.
>>>>
>>>>
>>>> Livnat
>>>>
>>>>
>>
>>



From hbrock at redhat.com  Sun Jan 22 17:04:35 2012
From: hbrock at redhat.com (Hugh Brock)
Date: Sun, 22 Jan 2012 12:04:35 -0500
Subject: [Engine-devel] Fwd:  VM Payload feature
In-Reply-To: <4F1B33FA.60707@redhat.com>
References: <138723b0-43ed-4271-a4ef-1eaecf5807e4@zmail02.collab.prod.int.phx2.redhat.com>
	<4F1B33FA.60707@redhat.com>
Message-ID: <20120122170434.GE17168@redhat.com>

On Sat, Jan 21, 2012 at 11:54:02PM +0200, Itamar Heim wrote:
> this is intended to cover the cloud forms requirement - please
> review and make sure it is in the right direction.
> (going with floppy/iso for this version, rather than IP/injection
> which could be optional future injection methods).
> please reply on thread upstream for the discussion.
> 
> Thanks,
>    Itamar

>From the Aeolus side I don't think we have a strong opinion which way
we do this -- it's more up to the Deltacloud guys. Neither approach
requires the guest agent to be installed, right?

In any case I'm glad to see this is in the works.

Take care,
--Hugh

> -------- Original Message --------
> Subject: [Engine-devel] VM Payload feature
> Date: Wed, 18 Jan 2012 10:41:59 -0500 (EST)
> From: Oved Ourfalli <ovedo at redhat.com>
> To: engine-devel at ovirt.org
> 
> Hey all,
> 
> Continuing the discussion about Aeolus instance data injection to a
> VM (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> we propose a new VM Payload feature.
> 
> The following wiki page contains a description page of the feature.
> http://www.ovirt.org/wiki/Features/VMPayload
> 
> Please read and review.
> There are several approaches there, and we wish to head your
> opinions and thoughts about them.
> 
> Once we agree on an approach, we will start designing.
> 
> Thank you,
> Oved
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

-- 
== Hugh Brock, hbrock at redhat.com                                   ==
== Engineering Manager, Cloud BU                                   ==
== Aeolus Project: Manage virtual infrastructure across clouds.    ==
== http://aeolusproject.org                                        ==

"I know that you believe you understand what you think I said, but I?m
not sure you realize that what you heard is not what I meant."
--Robert McCloskey


From abaron at redhat.com  Sun Jan 22 18:40:01 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 22 Jan 2012 13:40:01 -0500 (EST)
Subject: [Engine-devel] Fwd:  VM Payload feature
In-Reply-To: <20120122170434.GE17168@redhat.com>
Message-ID: <1e563907-ba41-40cd-a762-10e0f568a193@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On Sat, Jan 21, 2012 at 11:54:02PM +0200, Itamar Heim wrote:
> > this is intended to cover the cloud forms requirement - please
> > review and make sure it is in the right direction.
> > (going with floppy/iso for this version, rather than IP/injection
> > which could be optional future injection methods).
> > please reply on thread upstream for the discussion.
> > 
> > Thanks,
> >    Itamar
> 
> From the Aeolus side I don't think we have a strong opinion which way
> we do this -- it's more up to the Deltacloud guys. Neither approach
> requires the guest agent to be installed, right?

Correct.  Basically we're talking about exposing this either via a CD or via floppy (we'll support both) to the guest in the first phase, so the guest app that needs the data would just have to access the relevant device.

> 
> In any case I'm glad to see this is in the works.
> 
> Take care,
> --Hugh
> 
> > -------- Original Message --------
> > Subject: [Engine-devel] VM Payload feature
> > Date: Wed, 18 Jan 2012 10:41:59 -0500 (EST)
> > From: Oved Ourfalli <ovedo at redhat.com>
> > To: engine-devel at ovirt.org
> > 
> > Hey all,
> > 
> > Continuing the discussion about Aeolus instance data injection to a
> > VM
> > (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> > we propose a new VM Payload feature.
> > 
> > The following wiki page contains a description page of the feature.
> > http://www.ovirt.org/wiki/Features/VMPayload
> > 
> > Please read and review.
> > There are several approaches there, and we wish to head your
> > opinions and thoughts about them.
> > 
> > Once we agree on an approach, we will start designing.
> > 
> > Thank you,
> > Oved
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> --
> == Hugh Brock, hbrock at redhat.com                                   ==
> == Engineering Manager, Cloud BU                                   ==
> == Aeolus Project: Manage virtual infrastructure across clouds.    ==
> == http://aeolusproject.org                                        ==
> 
> "I know that you believe you understand what you think I said, but
> I?m
> not sure you realize that what you heard is not what I meant."
> --Robert McCloskey
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From abaron at redhat.com  Sun Jan 22 18:42:07 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 22 Jan 2012 13:42:07 -0500 (EST)
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <ed78c818-830e-41bd-abf2-c861390d062b@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <ae15611f-e655-45d0-8c5f-264daac1a1d7@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Ayal Baron" <abaron at redhat.com>
> > To: "Oved Ourfalli" <ovedo at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Thursday, January 19, 2012 4:05:08 PM
> > Subject: Re: [Engine-devel] VM Payload feature
> > 
> > 
> > 
> > ----- Original Message -----
> > > Hey all,
> > > 
> > > Continuing the discussion about Aeolus instance data injection to
> > > a
> > > VM
> > > (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> > > we propose a new VM Payload feature.
> > > 
> > > The following wiki page contains a description page of the
> > > feature.
> > > http://www.ovirt.org/wiki/Features/VMPayload
> > > 
> > > Please read and review.
> > > There are several approaches there, and we wish to head your
> > > opinions
> > > and thoughts about them.
> > > 
> > > Once we agree on an approach, we will start designing.
> > 
> > Permanent payload availability requires determining where the
> > payload
> > is stored.
> > Makes sense to me to store it together with the VM disks on the
> > storage domain, but that requires the small object store which will
> > not be available in the coming version (payloads can be large and
> > keeping them in the DB and passing over the net every time the VM
> > is
> > run doesn't make much sense).
> > 
> I guess we can start with storing it in the database, with some size
> limitation, and move it to the storage domain later on.

If Aeolus and Deltacloud don't need the permanent payload feature then no need to store it at all and then just add this capability later on properly.

> 
> > Wrt availability, I don't see a reason to exclude attaching both a
> > CD
> > and a payload via another CD at the same time (i.e. multiple
> > devices).
> > 
> > > 
> > > Thank you,
> > > Oved
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 


From jchoate at redhat.com  Sun Jan 22 18:43:47 2012
From: jchoate at redhat.com (Jon Choate)
Date: Sun, 22 Jan 2012 13:43:47 -0500
Subject: [Engine-devel] running jboss as 7 with openjdk
In-Reply-To: <d3cac700-5fae-4345-a063-ad6e945be366@zmail02.collab.prod.int.phx2.redhat.com>
References: <d3cac700-5fae-4345-a063-ad6e945be366@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1C58E3.3030401@redhat.com>

On 01/22/2012 02:31 AM, Oved Ourfalli wrote:
> Can you attach the log?
> Did you use the maven setup profile the first time you started using jboss as7?
>
> The reason I'm asking is that this step adds the following lines to $JBOSS_HOME/modules/sun/jdk/main/module.xml, which is supposed to fix this error:
>                  <path name="sun/security"/>
>                  <path name="sun/security/krb5"/>
>
> Let me know if these lines are there, and if adding them there fixes your issue.

Turns out the pom was not resolving ${jbossHome} so instead it was 
copying things to ./ear/${jbossHome}.  Strange but it was easy to fix.

> Oved
>
>
> ----- Original Message -----
>> From: "Jon Choate"<jchoate at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Friday, January 20, 2012 3:46:39 PM
>> Subject: [Engine-devel] running jboss as 7 with openjdk
>>
>> I keep getting a message that it can't find
>> sun.security.krb5.KrbException. Any ideas? Jboss-as-5 ran
>> ovirt-engine
>> just fine with openjdk.
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From abaron at redhat.com  Sun Jan 22 19:19:03 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 22 Jan 2012 14:19:03 -0500 (EST)
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <4F1A3632.8080107@redhat.com>
Message-ID: <43b7c858-4cd6-4ecb-95be-352d7a676227@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> On 01/20/2012 11:42 PM, Miki Kenneth wrote:
> >
> >
> > ----- Original Message -----
> >> From: "Itamar Heim"<iheim at redhat.com>
> >> To: "Ayal Baron"<abaron at redhat.com>
> >> Cc: engine-devel at ovirt.org
> >> Sent: Friday, January 20, 2012 2:12:27 AM
> >> Subject: Re: [Engine-devel] ovirt core MOM
> >>
> >> On 01/19/2012 11:58 AM, Ayal Baron wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> On 01/18/2012 05:53 PM, Livnat Peer wrote:
> >>>>> Hi All,
> >>>>>
> >>>>> This is what we've discussed in the meeting today:
> >>>>>
> >>>>> Multiple storage domain:
> >>>>> - Should have a single generic verb for removing a disk.
> >>>>> - We block removing the last template disk - template is
> >>>>> immutable.
> >>>>
> >>>> but it will be deleted when deleting the template, right?
> >>>
> >>> Of course.
> >>> The point is that the template is an immutable object and should
> >>> not change (until we support editing a template at which point
> >>> the
> >>> user would have to change the template to edit mode before being
> >>> able to make such changes and maybe also be able to run it and
> >>> make changes internally?).
> >>
> >> When i hear "edit a template" i don't expect replacing the files.
> >> I expect a new edition of disks appearing as a new version of the
> >> template. but they don't have to derive from same original
> >> template.
> >> say i want to create a "Fedora 16 template", then update it every
> >> month
> >> with latest "yum update".
> >> it doesn't matter if i use a VM from same template or just create
> >> a
> >> new one.
> >> then specify it is V2 of the "Fedora 16 template".
> >> when someone creates a VM from this template, default version
> >> would
> >> be
> >> latest (but we can let them choose specific older versions as
> >> well)
> > +1. Nicely put.
> > And just to add another common use case is the pool usage.
> > When we creating stateless VM pool from the template,
> > it would be nice to be able to update the template to V2,
> > and have all the newly created VMs dynamically based to the new
> > template.
> 
> that is indeed where i was going with it as well, but not as trivial,
> since need to wait for VMs to stop and return to pool and create new
> ones and remove old ones.
> also, creating new ones may involve an admin action of first boot +
> take
> of first snapshot
> 
> (hence i stopped the previous description before this part, but since
> you opened the door...)

Yes, but this all goes to template versioning (which is great and we need).
For the user though, creating a new template version like you described would be a long and wasteful process, and is not what I'm talking about.

Unless we support nested templates (second template would be a snapshot over the first one), then we're likely to require way too much space and creation process would be too slow (having to copy over all the bits).
I think the pool example is an excellent example where I would not want to have 2 copies of the template where the only difference between them is a set of security patches I've applied to the new template.

So the 2 options are for what I'm suggesting are:
1. decommission the old template by making in place changes
2. support template snapshots

Again, need to put the emphasis on fast provisioning of templates and VMs.
Applying updates to a pool should be a breeze (e.g. an automatic monthly process that unseals the template, updates it and reseals it).

> 
> 
> 


From ashakarc at redhat.com  Mon Jan 23 09:29:53 2012
From: ashakarc at redhat.com (Asaf Shakarchi)
Date: Mon, 23 Jan 2012 04:29:53 -0500 (EST)
Subject: [Engine-devel] Engine UI only: cleaning EAR from deployment folder
	may be required.
In-Reply-To: <60e2ba67-dd27-4ff8-9766-3f9415dc8d52@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <8d76777e-8a28-4155-a9b3-433f243a737e@zmail07.collab.prod.int.phx2.redhat.com>

Hello,

Some maven re-factoring (for UI modules only) patch (http://gerrit.ovirt.org/#change,689) has just merged,
You may encounter weird UI behaviours that might occur because some unnecessary data still exist in the deployed EAR folder.


If so please follow these steps:


1) Clean then build (the entire engine) with Pgwt-admin,gwt-user profiles
2) Fully remove the ovirt.ear from your JBOSS deployment folder.
3) cd OVIRT_ENGINE/ear then 'mvn install -Pdep'


Thanks,


Regards,
Asaf 





From ykaul at redhat.com  Mon Jan 23 09:31:14 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Mon, 23 Jan 2012 11:31:14 +0200
Subject: [Engine-devel] Engine UI only: cleaning EAR from deployment
 folder may be required.
In-Reply-To: <8d76777e-8a28-4155-a9b3-433f243a737e@zmail07.collab.prod.int.phx2.redhat.com>
References: <8d76777e-8a28-4155-a9b3-433f243a737e@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1D28E2.8000303@redhat.com>

On 01/23/2012 11:29 AM, Asaf Shakarchi wrote:
> Hello,
>
> Some maven re-factoring (for UI modules only) patch (http://gerrit.ovirt.org/#change,689) has just merged,
> You may encounter weird UI behaviours that might occur because some unnecessary data still exist in the deployed EAR folder.
>
>
> If so please follow these steps:
>
>
> 1) Clean then build (the entire engine) with Pgwt-admin,gwt-user profiles
> 2) Fully remove the ovirt.ear from your JBOSS deployment folder.
> 3) cd OVIRT_ENGINE/ear then 'mvn install -Pdep'

I'd appreciate a 'clean ear for dummies' - just provide the necessary 
commands to perform the above (the first two at least).
Y.

>
>
> Thanks,
>
>
> Regards,
> Asaf
>
>
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From ashakarc at redhat.com  Mon Jan 23 09:34:43 2012
From: ashakarc at redhat.com (Asaf Shakarchi)
Date: Mon, 23 Jan 2012 04:34:43 -0500 (EST)
Subject: [Engine-devel] Engine UI only: cleaning EAR from deployment
	folder may be required.
In-Reply-To: <4F1D28E2.8000303@redhat.com>
Message-ID: <09afae9a-64aa-458f-8d53-2a191ff585fe@zmail07.collab.prod.int.phx2.redhat.com>

----- Original Message -----
> On 01/23/2012 11:29 AM, Asaf Shakarchi wrote:
> > Hello,
> >
> > Some maven re-factoring (for UI modules only) patch
> > (http://gerrit.ovirt.org/#change,689) has just merged,
> > You may encounter weird UI behaviours that might occur because some
> > unnecessary data still exist in the deployed EAR folder.
> >
> >
> > If so please follow these steps:
> >
> >
> > 1) Clean then build (the entire engine) with Pgwt-admin,gwt-user
> > profiles
> > 2) Fully remove the ovirt.ear from your JBOSS deployment folder.
> > 3) cd OVIRT_ENGINE/ear then 'mvn install -Pdep'
> 
> I'd appreciate a 'clean ear for dummies' - just provide the necessary
> commands to perform the above (the first two at least).
> Y.

Sure,

- mvn clean install -Pgwt-admin,gwt-user -DskipTests=true
- rm -rf JBOSS_HOME/standalone/deployments/engine.ear (For Jboss 7)
- cd OVIRT_ENGINE/ear then 'mvn install -Pdep'

Thanks.
> 
> >
> >
> > Thanks,
> >
> >
> > Regards,
> > Asaf
> >
> >
> >
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkenneth at redhat.com  Mon Jan 23 13:18:31 2012
From: mkenneth at redhat.com (Miki Kenneth)
Date: Mon, 23 Jan 2012 08:18:31 -0500 (EST)
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <43b7c858-4cd6-4ecb-95be-352d7a676227@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <77ce0167-8c81-444a-8818-b83659f8a511@mkenneth.csb>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Itamar Heim" <iheim at redhat.com>
> Cc: engine-devel at ovirt.org, "Miki Kenneth" <mkenneth at redhat.com>
> Sent: Sunday, January 22, 2012 11:19:03 AM
> Subject: Re: [Engine-devel] ovirt core MOM
> 
> 
> 
> ----- Original Message -----
> > On 01/20/2012 11:42 PM, Miki Kenneth wrote:
> > >
> > >
> > > ----- Original Message -----
> > >> From: "Itamar Heim"<iheim at redhat.com>
> > >> To: "Ayal Baron"<abaron at redhat.com>
> > >> Cc: engine-devel at ovirt.org
> > >> Sent: Friday, January 20, 2012 2:12:27 AM
> > >> Subject: Re: [Engine-devel] ovirt core MOM
> > >>
> > >> On 01/19/2012 11:58 AM, Ayal Baron wrote:
> > >>>
> > >>>
> > >>> ----- Original Message -----
> > >>>> On 01/18/2012 05:53 PM, Livnat Peer wrote:
> > >>>>> Hi All,
> > >>>>>
> > >>>>> This is what we've discussed in the meeting today:
> > >>>>>
> > >>>>> Multiple storage domain:
> > >>>>> - Should have a single generic verb for removing a disk.
> > >>>>> - We block removing the last template disk - template is
> > >>>>> immutable.
> > >>>>
> > >>>> but it will be deleted when deleting the template, right?
> > >>>
> > >>> Of course.
> > >>> The point is that the template is an immutable object and
> > >>> should
> > >>> not change (until we support editing a template at which point
> > >>> the
> > >>> user would have to change the template to edit mode before
> > >>> being
> > >>> able to make such changes and maybe also be able to run it and
> > >>> make changes internally?).
> > >>
> > >> When i hear "edit a template" i don't expect replacing the
> > >> files.
> > >> I expect a new edition of disks appearing as a new version of
> > >> the
> > >> template. but they don't have to derive from same original
> > >> template.
> > >> say i want to create a "Fedora 16 template", then update it
> > >> every
> > >> month
> > >> with latest "yum update".
> > >> it doesn't matter if i use a VM from same template or just
> > >> create
> > >> a
> > >> new one.
> > >> then specify it is V2 of the "Fedora 16 template".
> > >> when someone creates a VM from this template, default version
> > >> would
> > >> be
> > >> latest (but we can let them choose specific older versions as
> > >> well)
> > > +1. Nicely put.
> > > And just to add another common use case is the pool usage.
> > > When we creating stateless VM pool from the template,
> > > it would be nice to be able to update the template to V2,
> > > and have all the newly created VMs dynamically based to the new
> > > template.
> > 
> > that is indeed where i was going with it as well, but not as
> > trivial,
> > since need to wait for VMs to stop and return to pool and create
> > new
> > ones and remove old ones.
> > also, creating new ones may involve an admin action of first boot +
> > take
> > of first snapshot
> > 
> > (hence i stopped the previous description before this part, but
> > since
> > you opened the door...)
> 
> Yes, but this all goes to template versioning (which is great and we
> need).
> For the user though, creating a new template version like you
> described would be a long and wasteful process, and is not what I'm
> talking about.
> 
> Unless we support nested templates (second template would be a
> snapshot over the first one), then we're likely to require way too
> much space and creation process would be too slow (having to copy
> over all the bits).
> I think the pool example is an excellent example where I would not
> want to have 2 copies of the template where the only difference
> between them is a set of security patches I've applied to the new
> template.
Not sure I understand how you do that while vms are still running on the original template?
> 
> So the 2 options are for what I'm suggesting are:
> 1. decommission the old template by making in place changes
> 2. support template snapshots
Not sure how this will work and what use case it serves?
> 
> Again, need to put the emphasis on fast provisioning of templates and
> VMs.
> Applying updates to a pool should be a breeze (e.g. an automatic
> monthly process that unseals the template, updates it and reseals
> it).
> 
> > 
> > 
> > 
> 


From jvlcek at redhat.com  Mon Jan 23 14:39:49 2012
From: jvlcek at redhat.com (Joseph VLcek)
Date: Mon, 23 Jan 2012 09:39:49 -0500
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <ed78c818-830e-41bd-abf2-c861390d062b@zmail02.collab.prod.int.phx2.redhat.com>
References: <ed78c818-830e-41bd-abf2-c861390d062b@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <2057230A-04E5-4F7D-BC90-45CCBA5663E6@redhat.com>


On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote:

> 
> 
> ----- Original Message -----
>> From: "Ayal Baron" <abaron at redhat.com>
>> To: "Oved Ourfalli" <ovedo at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Thursday, January 19, 2012 4:05:08 PM
>> Subject: Re: [Engine-devel] VM Payload feature
>> 
>> 
>> 
>> ----- Original Message -----
>>> Hey all,
>>> 
>>> Continuing the discussion about Aeolus instance data injection to a
>>> VM
>>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
>>> we propose a new VM Payload feature.
>>> 
>>> The following wiki page contains a description page of the feature.
>>> http://www.ovirt.org/wiki/Features/VMPayload
>>> 
>>> Please read and review.
>>> There are several approaches there, and we wish to head your
>>> opinions
>>> and thoughts about them.
>>> 
>>> Once we agree on an approach, we will start designing.
>> 
>> Permanent payload availability requires determining where the payload
>> is stored.
>> Makes sense to me to store it together with the VM disks on the
>> storage domain, but that requires the small object store which will
>> not be available in the coming version (payloads can be large and
>> keeping them in the DB and passing over the net every time the VM is
>> run doesn't make much sense).
>> 
> I guess we can start with storing it in the database, with some size limitation, and move it to the storage domain later on.
> 
>> Wrt availability, I don't see a reason to exclude attaching both a CD
>> and a payload via another CD at the same time (i.e. multiple
>> devices).
>> 
>>> 
>>> Thank you,
>>> Oved
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>> 
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>> 



My perspective is that of the end user, the instance retrieving the data.

From a functional standpoint I would like to see similar performance to
what EC2 provides. AWS EC2 user data is limited to 16K. This limit
applies to the data in raw form, not base64 encoded form.
see: 
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html

I am concerned about the 512k limit as mentioned in the notes
of: http://www.ovirt.org/wiki/Features/VMPayload
"if the content of the file is bigger the 512K it will pass an nfs
share for vdsm to fetch the file/s"

Please confirm:
- Will it be possible to pass user data to larger than 512k?
- If so what will the instance need to do in order to retrieve
   user-data bigger than 512k.
- What will the MAX size supported for the user-data?

Thank you.
    Joe VLcek

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120123/409c106c/attachment.html>

From shaharh at redhat.com  Mon Jan 23 15:07:30 2012
From: shaharh at redhat.com (Shahar Havivi)
Date: Mon, 23 Jan 2012 17:07:30 +0200
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <2057230A-04E5-4F7D-BC90-45CCBA5663E6@redhat.com>
References: <ed78c818-830e-41bd-abf2-c861390d062b@zmail02.collab.prod.int.phx2.redhat.com>
	<2057230A-04E5-4F7D-BC90-45CCBA5663E6@redhat.com>
Message-ID: <20120123150728.GA2300@redhat.com>

On 23.01.12 09:39, Joseph VLcek wrote:
> 
> On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote:
> 
> > 
> > 
> > ----- Original Message -----
> >> From: "Ayal Baron" <abaron at redhat.com>
> >> To: "Oved Ourfalli" <ovedo at redhat.com>
> >> Cc: engine-devel at ovirt.org
> >> Sent: Thursday, January 19, 2012 4:05:08 PM
> >> Subject: Re: [Engine-devel] VM Payload feature
> >> 
> >> 
> >> 
> >> ----- Original Message -----
> >>> Hey all,
> >>> 
> >>> Continuing the discussion about Aeolus instance data injection to a
> >>> VM
> >>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> >>> we propose a new VM Payload feature.
> >>> 
> >>> The following wiki page contains a description page of the feature.
> >>> http://www.ovirt.org/wiki/Features/VMPayload
> >>> 
> >>> Please read and review.
> >>> There are several approaches there, and we wish to head your
> >>> opinions
> >>> and thoughts about them.
> >>> 
> >>> Once we agree on an approach, we will start designing.
> >> 
> >> Permanent payload availability requires determining where the payload
> >> is stored.
> >> Makes sense to me to store it together with the VM disks on the
> >> storage domain, but that requires the small object store which will
> >> not be available in the coming version (payloads can be large and
> >> keeping them in the DB and passing over the net every time the VM is
> >> run doesn't make much sense).
> >> 
> > I guess we can start with storing it in the database, with some size limitation, and move it to the storage domain later on.
> > 
> >> Wrt availability, I don't see a reason to exclude attaching both a CD
> >> and a payload via another CD at the same time (i.e. multiple
> >> devices).
> >> 
> >>> 
> >>> Thank you,
> >>> Oved
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>> 
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >> 
> 
> 
> 
> My perspective is that of the end user, the instance retrieving the data.
> 
> From a functional standpoint I would like to see similar performance to
> what EC2 provides. AWS EC2 user data is limited to 16K. This limit
> applies to the data in raw form, not base64 encoded form.
> see: 
> http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html
> 
> I am concerned about the 512k limit as mentioned in the notes
> of: http://www.ovirt.org/wiki/Features/VMPayload
> "if the content of the file is bigger the 512K it will pass an nfs
> share for vdsm to fetch the file/s"
> 
> Please confirm:
> - Will it be possible to pass user data to larger than 512k?
> - If so what will the instance need to do in order to retrieve
>    user-data bigger than 512k.
> - What will the MAX size supported for the user-data?
512k is a suggestion,
we don't want to embed large files in the verb that ovirt calls vdsm, instead
if it bigger then 512k/1M we will pass urls/nfs path of the files and VDSM
will add them to the iso file.
there is not limitation of size...
> 
> Thank you.
>     Joe VLcek
> 


From acathrow at redhat.com  Mon Jan 23 15:11:42 2012
From: acathrow at redhat.com (Andrew Cathrow)
Date: Mon, 23 Jan 2012 10:11:42 -0500 (EST)
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <20120123150728.GA2300@redhat.com>
Message-ID: <e5e4e589-ef4a-4189-90be-90e3de76b9b5@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Shahar Havivi" <shaharh at redhat.com>
> To: "Joseph VLcek" <jvlcek at redhat.com>
> Cc: "Oved Ourfalli" <ovedo at redhat.com>, engine-devel at ovirt.org, "Michal Fojtik" <mfojtik at redhat.com>, "David
> Lutterkort" <dlutter at redhat.com>
> Sent: Monday, January 23, 2012 10:07:30 AM
> Subject: Re: [Engine-devel] VM Payload feature
> 
> On 23.01.12 09:39, Joseph VLcek wrote:
> > 
> > On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote:
> > 
> > > 
> > > 
> > > ----- Original Message -----
> > >> From: "Ayal Baron" <abaron at redhat.com>
> > >> To: "Oved Ourfalli" <ovedo at redhat.com>
> > >> Cc: engine-devel at ovirt.org
> > >> Sent: Thursday, January 19, 2012 4:05:08 PM
> > >> Subject: Re: [Engine-devel] VM Payload feature
> > >> 
> > >> 
> > >> 
> > >> ----- Original Message -----
> > >>> Hey all,
> > >>> 
> > >>> Continuing the discussion about Aeolus instance data injection
> > >>> to a
> > >>> VM
> > >>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> > >>> we propose a new VM Payload feature.
> > >>> 
> > >>> The following wiki page contains a description page of the
> > >>> feature.
> > >>> http://www.ovirt.org/wiki/Features/VMPayload
> > >>> 
> > >>> Please read and review.
> > >>> There are several approaches there, and we wish to head your
> > >>> opinions
> > >>> and thoughts about them.
> > >>> 
> > >>> Once we agree on an approach, we will start designing.
> > >> 
> > >> Permanent payload availability requires determining where the
> > >> payload
> > >> is stored.
> > >> Makes sense to me to store it together with the VM disks on the
> > >> storage domain, but that requires the small object store which
> > >> will
> > >> not be available in the coming version (payloads can be large
> > >> and
> > >> keeping them in the DB and passing over the net every time the
> > >> VM is
> > >> run doesn't make much sense).
> > >> 
> > > I guess we can start with storing it in the database, with some
> > > size limitation, and move it to the storage domain later on.
> > > 
> > >> Wrt availability, I don't see a reason to exclude attaching both
> > >> a CD
> > >> and a payload via another CD at the same time (i.e. multiple
> > >> devices).
> > >> 
> > >>> 
> > >>> Thank you,
> > >>> Oved
> > >>> _______________________________________________
> > >>> Engine-devel mailing list
> > >>> Engine-devel at ovirt.org
> > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>> 
> > >> _______________________________________________
> > >> Engine-devel mailing list
> > >> Engine-devel at ovirt.org
> > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >> 
> > 
> > 
> > 
> > My perspective is that of the end user, the instance retrieving the
> > data.
> > 
> > From a functional standpoint I would like to see similar
> > performance to
> > what EC2 provides. AWS EC2 user data is limited to 16K. This limit
> > applies to the data in raw form, not base64 encoded form.
> > see:
> > http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html
> > 
> > I am concerned about the 512k limit as mentioned in the notes
> > of: http://www.ovirt.org/wiki/Features/VMPayload
> > "if the content of the file is bigger the 512K it will pass an nfs
> > share for vdsm to fetch the file/s"
> > 
> > Please confirm:
> > - Will it be possible to pass user data to larger than 512k?
> > - If so what will the instance need to do in order to retrieve
> >    user-data bigger than 512k.
> > - What will the MAX size supported for the user-data?
> 512k is a suggestion,
> we don't want to embed large files in the verb that ovirt calls vdsm,
> instead
> if it bigger then 512k/1M we will pass urls/nfs path of the files and
> VDSM
> will add them to the iso file.
> there is not limitation of size...

If we're talking about URLs keep in mind SELinux restrictions (eg. passing a URL to a HTTP hosted IS will be blocked, iirc)

> > 
> > Thank you.
> >     Joe VLcek
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From shaharh at redhat.com  Mon Jan 23 15:12:24 2012
From: shaharh at redhat.com (Shahar Havivi)
Date: Mon, 23 Jan 2012 17:12:24 +0200
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <e5e4e589-ef4a-4189-90be-90e3de76b9b5@zmail07.collab.prod.int.phx2.redhat.com>
References: <20120123150728.GA2300@redhat.com>
	<e5e4e589-ef4a-4189-90be-90e3de76b9b5@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <20120123151223.GB2300@redhat.com>

On 23.01.12 10:11, Andrew Cathrow wrote:
> 
> 
> ----- Original Message -----
> > From: "Shahar Havivi" <shaharh at redhat.com>
> > To: "Joseph VLcek" <jvlcek at redhat.com>
> > Cc: "Oved Ourfalli" <ovedo at redhat.com>, engine-devel at ovirt.org, "Michal Fojtik" <mfojtik at redhat.com>, "David
> > Lutterkort" <dlutter at redhat.com>
> > Sent: Monday, January 23, 2012 10:07:30 AM
> > Subject: Re: [Engine-devel] VM Payload feature
> > 
> > On 23.01.12 09:39, Joseph VLcek wrote:
> > > 
> > > On Jan 22, 2012, at 3:09 AM, Oved Ourfalli wrote:
> > > 
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > >> From: "Ayal Baron" <abaron at redhat.com>
> > > >> To: "Oved Ourfalli" <ovedo at redhat.com>
> > > >> Cc: engine-devel at ovirt.org
> > > >> Sent: Thursday, January 19, 2012 4:05:08 PM
> > > >> Subject: Re: [Engine-devel] VM Payload feature
> > > >> 
> > > >> 
> > > >> 
> > > >> ----- Original Message -----
> > > >>> Hey all,
> > > >>> 
> > > >>> Continuing the discussion about Aeolus instance data injection
> > > >>> to a
> > > >>> VM
> > > >>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
> > > >>> we propose a new VM Payload feature.
> > > >>> 
> > > >>> The following wiki page contains a description page of the
> > > >>> feature.
> > > >>> http://www.ovirt.org/wiki/Features/VMPayload
> > > >>> 
> > > >>> Please read and review.
> > > >>> There are several approaches there, and we wish to head your
> > > >>> opinions
> > > >>> and thoughts about them.
> > > >>> 
> > > >>> Once we agree on an approach, we will start designing.
> > > >> 
> > > >> Permanent payload availability requires determining where the
> > > >> payload
> > > >> is stored.
> > > >> Makes sense to me to store it together with the VM disks on the
> > > >> storage domain, but that requires the small object store which
> > > >> will
> > > >> not be available in the coming version (payloads can be large
> > > >> and
> > > >> keeping them in the DB and passing over the net every time the
> > > >> VM is
> > > >> run doesn't make much sense).
> > > >> 
> > > > I guess we can start with storing it in the database, with some
> > > > size limitation, and move it to the storage domain later on.
> > > > 
> > > >> Wrt availability, I don't see a reason to exclude attaching both
> > > >> a CD
> > > >> and a payload via another CD at the same time (i.e. multiple
> > > >> devices).
> > > >> 
> > > >>> 
> > > >>> Thank you,
> > > >>> Oved
> > > >>> _______________________________________________
> > > >>> Engine-devel mailing list
> > > >>> Engine-devel at ovirt.org
> > > >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > >>> 
> > > >> _______________________________________________
> > > >> Engine-devel mailing list
> > > >> Engine-devel at ovirt.org
> > > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > >> 
> > > 
> > > 
> > > 
> > > My perspective is that of the end user, the instance retrieving the
> > > data.
> > > 
> > > From a functional standpoint I would like to see similar
> > > performance to
> > > what EC2 provides. AWS EC2 user data is limited to 16K. This limit
> > > applies to the data in raw form, not base64 encoded form.
> > > see:
> > > http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html
> > > 
> > > I am concerned about the 512k limit as mentioned in the notes
> > > of: http://www.ovirt.org/wiki/Features/VMPayload
> > > "if the content of the file is bigger the 512K it will pass an nfs
> > > share for vdsm to fetch the file/s"
> > > 
> > > Please confirm:
> > > - Will it be possible to pass user data to larger than 512k?
> > > - If so what will the instance need to do in order to retrieve
> > >    user-data bigger than 512k.
> > > - What will the MAX size supported for the user-data?
> > 512k is a suggestion,
> > we don't want to embed large files in the verb that ovirt calls vdsm,
> > instead
> > if it bigger then 512k/1M we will pass urls/nfs path of the files and
> > VDSM
> > will add them to the iso file.
> > there is not limitation of size...
> 
> If we're talking about URLs keep in mind SELinux restrictions (eg. passing a URL to a HTTP hosted IS will be blocked, iirc)
right,
its proffered to be common share directory
> 
> > > 
> > > Thank you.
> > >     Joe VLcek
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel


From yzaslavs at redhat.com  Mon Jan 23 15:38:20 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Mon, 23 Jan 2012 17:38:20 +0200
Subject: [Engine-devel] engine-core redundant instantiation of mappers
Message-ID: <4F1D7EEC.2060703@redhat.com>

Hi all,
I am refactoring now DiskImageDAODbFacadeImpl.
As part of my work, I'm also defining there a Spring-JDBC mapper ,
similar to other ParameterizedRowMapper based mappers we have in code
(i.e DbUserDAODbFacadeImpl has one)
Looks like in our DAO getXXXX methods we instantiate these mapper
objects (each method creates a new mapper object).
Can anyone see a reason , in case a mapper object is stateless (and from
what I see, they are) - why not to have a single instantiation per
mapper type (i.e - Have a MapperUtils class, with static methods like
MapperUtils.getDiskImageMapper() ?
IMHO this can save us some unnecessary instanatiations and improve
performance

Yair


From abaron at redhat.com  Mon Jan 23 21:01:31 2012
From: abaron at redhat.com (Ayal Baron)
Date: Mon, 23 Jan 2012 16:01:31 -0500 (EST)
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <77ce0167-8c81-444a-8818-b83659f8a511@mkenneth.csb>
Message-ID: <af92aa7c-2ee0-45b5-bb9d-0ba7823a1a5d@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Ayal Baron" <abaron at redhat.com>
> > To: "Itamar Heim" <iheim at redhat.com>
> > Cc: engine-devel at ovirt.org, "Miki Kenneth" <mkenneth at redhat.com>
> > Sent: Sunday, January 22, 2012 11:19:03 AM
> > Subject: Re: [Engine-devel] ovirt core MOM
> > 
> > 
> > 
> > ----- Original Message -----
> > > On 01/20/2012 11:42 PM, Miki Kenneth wrote:
> > > >
> > > >
> > > > ----- Original Message -----
> > > >> From: "Itamar Heim"<iheim at redhat.com>
> > > >> To: "Ayal Baron"<abaron at redhat.com>
> > > >> Cc: engine-devel at ovirt.org
> > > >> Sent: Friday, January 20, 2012 2:12:27 AM
> > > >> Subject: Re: [Engine-devel] ovirt core MOM
> > > >>
> > > >> On 01/19/2012 11:58 AM, Ayal Baron wrote:
> > > >>>
> > > >>>
> > > >>> ----- Original Message -----
> > > >>>> On 01/18/2012 05:53 PM, Livnat Peer wrote:
> > > >>>>> Hi All,
> > > >>>>>
> > > >>>>> This is what we've discussed in the meeting today:
> > > >>>>>
> > > >>>>> Multiple storage domain:
> > > >>>>> - Should have a single generic verb for removing a disk.
> > > >>>>> - We block removing the last template disk - template is
> > > >>>>> immutable.
> > > >>>>
> > > >>>> but it will be deleted when deleting the template, right?
> > > >>>
> > > >>> Of course.
> > > >>> The point is that the template is an immutable object and
> > > >>> should
> > > >>> not change (until we support editing a template at which
> > > >>> point
> > > >>> the
> > > >>> user would have to change the template to edit mode before
> > > >>> being
> > > >>> able to make such changes and maybe also be able to run it
> > > >>> and
> > > >>> make changes internally?).
> > > >>
> > > >> When i hear "edit a template" i don't expect replacing the
> > > >> files.
> > > >> I expect a new edition of disks appearing as a new version of
> > > >> the
> > > >> template. but they don't have to derive from same original
> > > >> template.
> > > >> say i want to create a "Fedora 16 template", then update it
> > > >> every
> > > >> month
> > > >> with latest "yum update".
> > > >> it doesn't matter if i use a VM from same template or just
> > > >> create
> > > >> a
> > > >> new one.
> > > >> then specify it is V2 of the "Fedora 16 template".
> > > >> when someone creates a VM from this template, default version
> > > >> would
> > > >> be
> > > >> latest (but we can let them choose specific older versions as
> > > >> well)
> > > > +1. Nicely put.
> > > > And just to add another common use case is the pool usage.
> > > > When we creating stateless VM pool from the template,
> > > > it would be nice to be able to update the template to V2,
> > > > and have all the newly created VMs dynamically based to the new
> > > > template.
> > > 
> > > that is indeed where i was going with it as well, but not as
> > > trivial,
> > > since need to wait for VMs to stop and return to pool and create
> > > new
> > > ones and remove old ones.
> > > also, creating new ones may involve an admin action of first boot
> > > +
> > > take
> > > of first snapshot
> > > 
> > > (hence i stopped the previous description before this part, but
> > > since
> > > you opened the door...)
> > 
> > Yes, but this all goes to template versioning (which is great and
> > we
> > need).
> > For the user though, creating a new template version like you
> > described would be a long and wasteful process, and is not what I'm
> > talking about.
> > 
> > Unless we support nested templates (second template would be a
> > snapshot over the first one), then we're likely to require way too
> > much space and creation process would be too slow (having to copy
> > over all the bits).
> > I think the pool example is an excellent example where I would not
> > want to have 2 copies of the template where the only difference
> > between them is a set of security patches I've applied to the new
> > template.
> Not sure I understand how you do that while vms are still running on
> the original template?

They either:
1. wouldn't be (if changes are in place)
2. if we support template over template (from snapshot) then no issue at all.
   Once all VMs stop running on previous template we can live merge the 2.

> > 
> > So the 2 options are for what I'm suggesting are:
> > 1. decommission the old template by making in place changes
> > 2. support template snapshots
> Not sure how this will work and what use case it serves?

number 1: changing the template for stateless pools.
number 2: for anything you want including template versioning.
Template versioning should have 2 flavours:
1. my golden image is outdated and I would like to decommission it and replace with a new one created from scratch (i.e. same name, new VMs would be derived from new template, no data dedup).
2. my golden image is outdated and I would like to update it internally - create a VM from it, make the changes, seal this VM as the new version of the template (not using the process we have today which copies all the data, just change it to be immutable).

The latter requires supporting trees.


> > 
> > Again, need to put the emphasis on fast provisioning of templates
> > and
> > VMs.
> > Applying updates to a pool should be a breeze (e.g. an automatic
> > monthly process that unseals the template, updates it and reseals
> > it).
> > 
> > > 
> > > 
> > > 
> > 
> 


From smizrahi at redhat.com  Mon Jan 23 21:54:10 2012
From: smizrahi at redhat.com (Saggi Mizrahi)
Date: Mon, 23 Jan 2012 16:54:10 -0500 (EST)
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <0a2b8e83-4b71-4c42-8bff-def24ba5dea8@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>

I have begun work at changing how API clients can control storage connections when interacting with VDSM.

Currently there are 2 API calls:
connectStorageServer() - Will connect to the storage target if the host is not already connected to it.
disconnectStorageServer() - Will disconnect from the storage target if the host is connected to it.

This API is very simple but is inappropriate when multiple clients and flows try to access the same storage.

This is currently solved by trying to synchronize things inside rhevm. This is hard and convoluted. It also brings out issues with other clients using the VDSM API.

Another problem is error recovery. Currently ovirt-engine(OE) has no way of monitoring the connections on all the hosts an if a connection disappears it's OE's responsibility to reconnect.

I suggest a different concept where VDSM 'manages' the connections. VDSM receives a manage request with the connection information and from that point forward VDSM will try to keep this connection alive. If the connection fails VDSM will automatically try and recover.

Every manage request will also have a connection ID(CID). This CID will be used when the same client asks to unamange the connection.
When multiple requests for manage are received to the same connection they all have to have their own unique CID. By internally mapping CIDs to actual connections VDSM can properly disconnect when no CID is addressing the connection. This allows each client and even each flow to have it's own CID effectively eliminating connect\disconnect races.

The change from (dis)connect to (un)manage also changes the semantics of the calls significantly.
Whereas connectStorageServer would have returned when the storage is either connected or failed to connect, manageStorageServer will return once VDSM registered the CID. This means that the connection might not be active immediately as the VDSM tries to connect. The connection might remain down for a long time if the storage target is down or is having issues.

This allows for VDSM to receive the manage request even if the storage is having issues and recover as soon as it's operational without user intervention.

In order for the client to query the current state of the connections I propose getStorageConnectionList(). This will return a mapping of CID to connection status. The status contains the connection info (excluding credentials), whether the connection is active, whether the connection is managed (unamanged connection are returned with transient IDs), and, if the connection is down, the last error information.

The same actual connection can return multiple times, once for each CID.

For cases where an operation requires a connection to be active a user can poll the status of the CID. The user can then choose to poll for a certain amount of time or until an error appears in the error field of the status. This will give you either a timeout or a "try once" semantic depending on the flows needs.

All connections that have been managed persist VDSM restart and will be managed until a corresponding unmanage command has been issued.

There is no concept of temporary connections as "temporary" is flow dependent and VDSM can't accommodate all interpretation of "temporary". An ad-hoc mechanism can be build using the CID field. For instance a client can manage a connection with "ENGINE_FLOW101_CON1". If the flow got interrupted the client can clean all IDs with certain flow IDs.

I think this API gives safety, robustness, and implementation freedom.


Nitty Gritty:

manageStorageServer
===================
Synopsis:
manageStorageServer(uri, connectionID):

Parameters:
uri - a uri pointing to a storage target (eg: nfs://server:export, iscsi://host/iqn;portal=1)
connectionID - string with any char except "/".

Description:
Tells VDSM to start managing the connection. From this moment on VDSM will try and have the connection available when needed. VDSM will monitor the connection and will automatically reconnect on failure.
Returns:
Success code if VDSM was able to manage the connection.
It usually just verifies that the arguments are sane and that the CID is not already in use.
This doesn't mean the host is connected.
----
unmanageStorageServer
=====================
Synopsis:
unmanageStorageServer(connectionID):

Parameters:
connectionID - string with any char except "/".

Descriptions:
Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.

Returns:
Success code if VDSM was able to unmanage the connection.
It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
----
getStorageServerList
====================
Synopsis:
getStorageServerList()

Description:
Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.

Results:VDSM was able to manage the connection.
It usually just verifies that the arguments are sane and that the CID is not already in use.
This doesn't mean the host is connected.
----
unmanageStorageServer
=====================
Synopsis:
unmanageStorageServer(connectionID):

Parameters:
connectionID - string with any char except "/".

Descriptions:
Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.

Returns:
Success code if VDSM was able to unmanage the connection.
It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
----
getStorageServerList
====================
Synopsis:
getStorageServerList()

Description:
Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.

Results:
A mapping between CIDs and the status.
example return value (Actual key names may differ)

{'conA': {'connected': True, 'managed': True, 'lastError': 0, 'connectionInfo': {
    'remotePath': 'server:/export
    'retrans': 3
    'version': 4
    }}
 'iscsi_session_34': {'connected': False, 'managed': False, 'lastError': 339, 'connectionIfno': {
    'hostname': 'dandylopn'
    'portal': 1}}
}


From bazulay at redhat.com  Tue Jan 24 13:47:42 2012
From: bazulay at redhat.com (Barak Azulay)
Date: Tue, 24 Jan 2012 15:47:42 +0200
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
Message-ID: <4F1EB67E.7050204@redhat.com>

I'm experiencing severe latency issues with gerrit.ovirt.org - am I the 
only one ?

Thanks
Barak Azulay


From kroberts at redhat.com  Tue Jan 24 13:58:10 2012
From: kroberts at redhat.com (Keith Robertson)
Date: Tue, 24 Jan 2012 08:58:10 -0500
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EB67E.7050204@redhat.com>
References: <4F1EB67E.7050204@redhat.com>
Message-ID: <4F1EB8F2.6020803@redhat.com>

On 01/24/2012 08:47 AM, Barak Azulay wrote:
> I'm experiencing severe latency issues with gerrit.ovirt.org - am I 
> the only one ?
>
> Thanks
> Barak Azulay
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
No latency issues here in Wake Forest, NC.
Cheers


From masayag at redhat.com  Tue Jan 24 14:18:53 2012
From: masayag at redhat.com (Moti Asayag)
Date: Tue, 24 Jan 2012 16:18:53 +0200
Subject: [Engine-devel] engine-core redundant instantiation of mappers
In-Reply-To: <4F1D7EEC.2060703@redhat.com>
References: <4F1D7EEC.2060703@redhat.com>
Message-ID: <4F1EBDCD.6010201@redhat.com>

On 01/23/2012 05:38 PM, Yair Zaslavsky wrote:
> Hi all,
> I am refactoring now DiskImageDAODbFacadeImpl.
> As part of my work, I'm also defining there a Spring-JDBC mapper ,
> similar to other ParameterizedRowMapper based mappers we have in code
> (i.e DbUserDAODbFacadeImpl has one)
> Looks like in our DAO getXXXX methods we instantiate these mapper
> objects (each method creates a new mapper object).
> Can anyone see a reason , in case a mapper object is stateless (and from
> what I see, they are) - why not to have a single instantiation per
> mapper type (i.e - Have a MapperUtils class, with static methods like
> MapperUtils.getDiskImageMapper() ?
> IMHO this can save us some unnecessary instanatiations and improve
> performance

+1 for the static suggestion.

Since each mapper is used by a specific DAO impl class (and sometimes
several mappers per DAO), but never shared between those DAOs, IMO they
should be remained in the scope of the DAO which uses them and be
defined inside the DAO class in a static block;

    static {
        mapper = new ParameterizedRowMapper<Entity>() {
            @Override
            public Entity mapRow(ResultSet rs, int rowNum){
            ...
            }
        };
    }

> 
> Yair
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From yzaslavs at redhat.com  Tue Jan 24 15:20:23 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Tue, 24 Jan 2012 17:20:23 +0200
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EB67E.7050204@redhat.com>
References: <4F1EB67E.7050204@redhat.com>
Message-ID: <4F1ECC37.1050208@redhat.com>

On 01/24/2012 03:47 PM, Barak Azulay wrote:
> I'm experiencing severe latency issues with gerrit.ovirt.org - am I the
> only one ?
Currently experiencing it as well (Jan 24th, 17:20, Israel time).

> 
> Thanks
> Barak Azulay
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From iheim at redhat.com  Tue Jan 24 16:28:45 2012
From: iheim at redhat.com (Itamar Heim)
Date: Tue, 24 Jan 2012 18:28:45 +0200
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1ECC37.1050208@redhat.com>
References: <4F1EB67E.7050204@redhat.com> <4F1ECC37.1050208@redhat.com>
Message-ID: <4F1EDC3D.80007@redhat.com>

On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
> On 01/24/2012 03:47 PM, Barak Azulay wrote:
>> I'm experiencing severe latency issues with gerrit.ovirt.org - am I the
>> only one ?
> Currently experiencing it as well (Jan 24th, 17:20, Israel time).

currently traveling in the states and no latency issues at all, so that 
rules out an issue with the server itself.


From dfediuck at redhat.com  Tue Jan 24 16:33:43 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Tue, 24 Jan 2012 18:33:43 +0200
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EDC3D.80007@redhat.com>
References: <4F1EB67E.7050204@redhat.com> <4F1ECC37.1050208@redhat.com>
	<4F1EDC3D.80007@redhat.com>
Message-ID: <4F1EDD67.8020702@redhat.com>

On 24/01/12 18:28, Itamar Heim wrote:
> On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
>> On 01/24/2012 03:47 PM, Barak Azulay wrote:
>>> I'm experiencing severe latency issues with gerrit.ovirt.org - am I the
>>> only one ?
>> Currently experiencing it as well (Jan 24th, 17:20, Israel time).
> 
> currently traveling in the states and no latency issues at all, so that
> rules out an issue with the server itself.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

I've been getting requests for help all day about it.
Most ppl in tlv simply can't work with it. Some patches are simply
not being pushed, and some people (/me included) were unable to login
for a long time.
This should be escalated somewhere and get fixed.

-- 
"Forty-two," said Deep Thought, with infinite majesty and calm.
--Douglas Adams, The Hitchhiker's Guide to the Galaxy


From iheim at redhat.com  Tue Jan 24 16:44:20 2012
From: iheim at redhat.com (Itamar Heim)
Date: Tue, 24 Jan 2012 18:44:20 +0200
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EDD67.8020702@redhat.com>
References: <4F1EB67E.7050204@redhat.com> <4F1ECC37.1050208@redhat.com>
	<4F1EDC3D.80007@redhat.com> <4F1EDD67.8020702@redhat.com>
Message-ID: <4F1EDFE4.4080104@redhat.com>

On 01/24/2012 06:33 PM, Doron Fediuck wrote:
> On 24/01/12 18:28, Itamar Heim wrote:
>> On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
>>> On 01/24/2012 03:47 PM, Barak Azulay wrote:
>>>> I'm experiencing severe latency issues with gerrit.ovirt.org - am I the
>>>> only one ?
>>> Currently experiencing it as well (Jan 24th, 17:20, Israel time).
>>
>> currently traveling in the states and no latency issues at all, so that
>> rules out an issue with the server itself.
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
> I've been getting requests for help all day about it.
> Most ppl in tlv simply can't work with it. Some patches are simply
> not being pushed, and some people (/me included) were unable to login
> for a long time.
> This should be escalated somewhere and get fixed.
>

performance from where i am is great and has no issues with fetch/clone 
commands.
still, to rule out any issues, i've restarted gerrit
has anyone tried not from tlv and has issues?


From ydary at redhat.com  Tue Jan 24 16:51:42 2012
From: ydary at redhat.com (Yaniv Dary)
Date: Tue, 24 Jan 2012 11:51:42 -0500 (EST)
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EDFE4.4080104@redhat.com>
Message-ID: <a0f17941-38b2-445e-95a0-fde8c56b597b@dhcp-1-61.tlv.redhat.com>



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Doron Fediuck" <dfediuck at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Tuesday, January 24, 2012 6:44:20 PM
> Subject: Re: [Engine-devel] latency issues with gerrit.ovirt.org
> 
> On 01/24/2012 06:33 PM, Doron Fediuck wrote:
> > On 24/01/12 18:28, Itamar Heim wrote:
> >> On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
> >>> On 01/24/2012 03:47 PM, Barak Azulay wrote:
> >>>> I'm experiencing severe latency issues with gerrit.ovirt.org -
> >>>> am I the
> >>>> only one ?
> >>> Currently experiencing it as well (Jan 24th, 17:20, Israel time).
> >>
> >> currently traveling in the states and no latency issues at all, so
> >> that
> >> rules out an issue with the server itself.
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >
> > I've been getting requests for help all day about it.
> > Most ppl in tlv simply can't work with it. Some patches are simply
> > not being pushed, and some people (/me included) were unable to
> > login
> > for a long time.
> > This should be escalated somewhere and get fixed.
> >
> 
> performance from where i am is great and has no issues with
> fetch/clone
> commands.
> still, to rule out any issues, i've restarted gerrit

It help for a few minutes and now nothing is working again.

> has anyone tried not from tlv and has issues?
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From jhernand at redhat.com  Tue Jan 24 17:03:31 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Tue, 24 Jan 2012 18:03:31 +0100
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EDFE4.4080104@redhat.com>
References: <4F1EB67E.7050204@redhat.com> <4F1ECC37.1050208@redhat.com>
	<4F1EDC3D.80007@redhat.com> <4F1EDD67.8020702@redhat.com>
	<4F1EDFE4.4080104@redhat.com>
Message-ID: <4F1EE463.7050004@redhat.com>

On 01/24/2012 05:44 PM, Itamar Heim wrote:
> On 01/24/2012 06:33 PM, Doron Fediuck wrote:
>> On 24/01/12 18:28, Itamar Heim wrote:
>>> On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
>>>> On 01/24/2012 03:47 PM, Barak Azulay wrote:
>>>>> I'm experiencing severe latency issues with gerrit.ovirt.org - am I the
>>>>> only one ?
>>>> Currently experiencing it as well (Jan 24th, 17:20, Israel time).
>>>
>>> currently traveling in the states and no latency issues at all, so that
>>> rules out an issue with the server itself.
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>> I've been getting requests for help all day about it.
>> Most ppl in tlv simply can't work with it. Some patches are simply
>> not being pushed, and some people (/me included) were unable to login
>> for a long time.
>> This should be escalated somewhere and get fixed.
>>
> 
> performance from where i am is great and has no issues with fetch/clone 
> commands.
> still, to rule out any issues, i've restarted gerrit
> has anyone tried not from tlv and has issues?

It works fine from Spain.

-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From lpeer at redhat.com  Tue Jan 24 17:43:39 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 24 Jan 2012 19:43:39 +0200
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>
References: <1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1EEDCB.4070000@redhat.com>

On 23/01/12 23:54, Saggi Mizrahi wrote:
> I have begun work at changing how API clients can control storage connections when interacting with VDSM.
> 
> Currently there are 2 API calls:
> connectStorageServer() - Will connect to the storage target if the host is not already connected to it.
> disconnectStorageServer() - Will disconnect from the storage target if the host is connected to it.
> 
> This API is very simple but is inappropriate when multiple clients and flows try to access the same storage.
> 
> This is currently solved by trying to synchronize things inside rhevm. This is hard and convoluted. It also brings out issues with other clients using the VDSM API.
> 
> Another problem is error recovery. Currently ovirt-engine(OE) has no way of monitoring the connections on all the hosts an if a connection disappears it's OE's responsibility to reconnect.
> 
> I suggest a different concept where VDSM 'manages' the connections. VDSM receives a manage request with the connection information and from that point forward VDSM will try to keep this connection alive. If the connection fails VDSM will automatically try and recover.
> 
> Every manage request will also have a connection ID(CID). This CID will be used when the same client asks to unamange the connection.
> When multiple requests for manage are received to the same connection they all have to have their own unique CID. By internally mapping CIDs to actual connections VDSM can properly disconnect when no CID is addressing the connection. This allows each client and even each flow to have it's own CID effectively eliminating connect\disconnect races.
> 
> The change from (dis)connect to (un)manage also changes the semantics of the calls significantly.
> Whereas connectStorageServer would have returned when the storage is either connected or failed to connect, manageStorageServer will return once VDSM registered the CID. This means that the connection might not be active immediately as the VDSM tries to connect. The connection might remain down for a long time if the storage target is down or is having issues.
> 
> This allows for VDSM to receive the manage request even if the storage is having issues and recover as soon as it's operational without user intervention.
> 
> In order for the client to query the current state of the connections I propose getStorageConnectionList(). This will return a mapping of CID to connection status. The status contains the connection info (excluding credentials), whether the connection is active, whether the connection is managed (unamanged connection are returned with transient IDs), and, if the connection is down, the last error information.
> 
> The same actual connection can return multiple times, once for each CID.
> 
> For cases where an operation requires a connection to be active a user can poll the status of the CID. The user can then choose to poll for a certain amount of time or until an error appears in the error field of the status. This will give you either a timeout or a "try once" semantic depending on the flows needs.
> 
> All connections that have been managed persist VDSM restart and will be managed until a corresponding unmanage command has been issued.
> 
> There is no concept of temporary connections as "temporary" is flow dependent and VDSM can't accommodate all interpretation of "temporary". An ad-hoc mechanism can be build using the CID field. For instance a client can manage a connection with "ENGINE_FLOW101_CON1". If the flow got interrupted the client can clean all IDs with certain flow IDs.
> 
> I think this API gives safety, robustness, and implementation freedom.
> 
> 
> Nitty Gritty:
> 
> manageStorageServer
> ===================
> Synopsis:
> manageStorageServer(uri, connectionID):
> 
> Parameters:
> uri - a uri pointing to a storage target (eg: nfs://server:export, iscsi://host/iqn;portal=1)
> connectionID - string with any char except "/".
> 
> Description:
> Tells VDSM to start managing the connection. From this moment on VDSM will try and have the connection available when needed. VDSM will monitor the connection and will automatically reconnect on failure.
> Returns:
> Success code if VDSM was able to manage the connection.
> It usually just verifies that the arguments are sane and that the CID is not already in use.
> This doesn't mean the host is connected.
> ----
> unmanageStorageServer
> =====================
> Synopsis:
> unmanageStorageServer(connectionID):
> 
> Parameters:
> connectionID - string with any char except "/".
> 
> Descriptions:
> Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.
> 
> Returns:
> Success code if VDSM was able to unmanage the connection.
> It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
> ----
> getStorageServerList
> ====================
> Synopsis:
> getStorageServerList()
> 
> Description:
> Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.
> 
> Results:VDSM was able to manage the connection.
> It usually just verifies that the arguments are sane and that the CID is not already in use.
> This doesn't mean the host is connected.
> ----
> unmanageStorageServer
> =====================
> Synopsis:
> unmanageStorageServer(connectionID):
> 
> Parameters:
> connectionID - string with any char except "/".
> 
> Descriptions:
> Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.
> 
> Returns:
> Success code if VDSM was able to unmanage the connection.
> It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
> ----
> getStorageServerList
> ====================
> Synopsis:
> getStorageServerList()
> 
> Description:
> Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.
> 
> Results:
> A mapping between CIDs and the status.
> example return value (Actual key names may differ)
> 
> {'conA': {'connected': True, 'managed': True, 'lastError': 0, 'connectionInfo': {
>     'remotePath': 'server:/export
>     'retrans': 3
>     'version': 4
>     }}
>  'iscsi_session_34': {'connected': False, 'managed': False, 'lastError': 339, 'connectionIfno': {
>     'hostname': 'dandylopn'
>     'portal': 1}}
> }
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel


Hi Saggi,

I see the added value in the above functionality and I think it is a
needed functionality in VDSM.

Your suggestion includes 2 concepts:
- Persist connection - auto-reconnect on failures
- Reference counting (with CID granularity)

Here are some comments:

* Assuming you meant that the new API will be a replacement to the
current API (based on previous chats we had on this topic) I think you
are missing a needed functionality to support non-persisted connection.
creating a storage domain is an example where it can be useful.
The flow includes connecting the host to the storage server creating
storage domain and disconnecting from the storage server.
Let's assume VDSM hangs while creating the storage domain any
unmanageStorageServer will fail, the engine rolls back and tries to
create the storage domain on another host, there is no reason for the
host to reconnect to this storage server.
In the above flow i would use non-persist connection if I had one.

* In the suggested solution the connect will not initiate an immediate
connect to the storage server instead it will register the connection as
handled connection and will actually generate the connect as part of the
managed connection mechanism.
I argue that this modeling is implementation driven which is wrong from
the user perspective. As a user I expect connect to actually initiate a
connect action and that the return value should indicate if the connect
succeeded, the way you modeled it the API will return true if you
succeeded 'registering' the connect.
You modeled the API to be asynchronous with no handler (task id) to
monitor the results of the action, which requires polling in the create
storage domain flow which I really don't like.
In addition you introduced a verb for monitoring the status of the
connections alone I would like to be able to monitor it as part of the
general host status and not have to poll on a new verb in addition to
the current one.


As part of solving the connection management flows in OE I am missing:

- A way to clear all managed connections.
use case: We move a host from one data center to another and we want the
host to clear all the managed connections.
we can ask for the list of managed connection and clear them but having
clearAll is much easier.

- Handling a list of Ids in each API verb

- A verb which handles create storage domain and encapsulates the
connect create and disconnect.


Thanks, Livnat


From iheim at redhat.com  Tue Jan 24 11:53:04 2012
From: iheim at redhat.com (Itamar Heim)
Date: Tue, 24 Jan 2012 13:53:04 +0200
Subject: [Engine-devel] ovirt core MOM
In-Reply-To: <af92aa7c-2ee0-45b5-bb9d-0ba7823a1a5d@zmail13.collab.prod.int.phx2.redhat.com>
References: <af92aa7c-2ee0-45b5-bb9d-0ba7823a1a5d@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1E9BA0.1050300@redhat.com>

On 01/23/2012 11:01 PM, Ayal Baron wrote:
>
>
> ----- Original Message -----
>>
>>
>> ----- Original Message -----
>>> From: "Ayal Baron"<abaron at redhat.com>
>>> To: "Itamar Heim"<iheim at redhat.com>
>>> Cc: engine-devel at ovirt.org, "Miki Kenneth"<mkenneth at redhat.com>
>>> Sent: Sunday, January 22, 2012 11:19:03 AM
>>> Subject: Re: [Engine-devel] ovirt core MOM
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> On 01/20/2012 11:42 PM, Miki Kenneth wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Itamar Heim"<iheim at redhat.com>
>>>>>> To: "Ayal Baron"<abaron at redhat.com>
>>>>>> Cc: engine-devel at ovirt.org
>>>>>> Sent: Friday, January 20, 2012 2:12:27 AM
>>>>>> Subject: Re: [Engine-devel] ovirt core MOM
>>>>>>
>>>>>> On 01/19/2012 11:58 AM, Ayal Baron wrote:
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> On 01/18/2012 05:53 PM, Livnat Peer wrote:
>>>>>>>>> Hi All,
>>>>>>>>>
>>>>>>>>> This is what we've discussed in the meeting today:
>>>>>>>>>
>>>>>>>>> Multiple storage domain:
>>>>>>>>> - Should have a single generic verb for removing a disk.
>>>>>>>>> - We block removing the last template disk - template is
>>>>>>>>> immutable.
>>>>>>>>
>>>>>>>> but it will be deleted when deleting the template, right?
>>>>>>>
>>>>>>> Of course.
>>>>>>> The point is that the template is an immutable object and
>>>>>>> should
>>>>>>> not change (until we support editing a template at which
>>>>>>> point
>>>>>>> the
>>>>>>> user would have to change the template to edit mode before
>>>>>>> being
>>>>>>> able to make such changes and maybe also be able to run it
>>>>>>> and
>>>>>>> make changes internally?).
>>>>>>
>>>>>> When i hear "edit a template" i don't expect replacing the
>>>>>> files.
>>>>>> I expect a new edition of disks appearing as a new version of
>>>>>> the
>>>>>> template. but they don't have to derive from same original
>>>>>> template.
>>>>>> say i want to create a "Fedora 16 template", then update it
>>>>>> every
>>>>>> month
>>>>>> with latest "yum update".
>>>>>> it doesn't matter if i use a VM from same template or just
>>>>>> create
>>>>>> a
>>>>>> new one.
>>>>>> then specify it is V2 of the "Fedora 16 template".
>>>>>> when someone creates a VM from this template, default version
>>>>>> would
>>>>>> be
>>>>>> latest (but we can let them choose specific older versions as
>>>>>> well)
>>>>> +1. Nicely put.
>>>>> And just to add another common use case is the pool usage.
>>>>> When we creating stateless VM pool from the template,
>>>>> it would be nice to be able to update the template to V2,
>>>>> and have all the newly created VMs dynamically based to the new
>>>>> template.
>>>>
>>>> that is indeed where i was going with it as well, but not as
>>>> trivial,
>>>> since need to wait for VMs to stop and return to pool and create
>>>> new
>>>> ones and remove old ones.
>>>> also, creating new ones may involve an admin action of first boot
>>>> +
>>>> take
>>>> of first snapshot
>>>>
>>>> (hence i stopped the previous description before this part, but
>>>> since
>>>> you opened the door...)
>>>
>>> Yes, but this all goes to template versioning (which is great and
>>> we
>>> need).
>>> For the user though, creating a new template version like you
>>> described would be a long and wasteful process, and is not what I'm
>>> talking about.
>>>
>>> Unless we support nested templates (second template would be a
>>> snapshot over the first one), then we're likely to require way too
>>> much space and creation process would be too slow (having to copy
>>> over all the bits).
>>> I think the pool example is an excellent example where I would not
>>> want to have 2 copies of the template where the only difference
>>> between them is a set of security patches I've applied to the new
>>> template.
>> Not sure I understand how you do that while vms are still running on
>> the original template?
>
> They either:
> 1. wouldn't be (if changes are in place)
> 2. if we support template over template (from snapshot) then no issue at all.
>     Once all VMs stop running on previous template we can live merge the 2.
>
>>>
>>> So the 2 options are for what I'm suggesting are:
>>> 1. decommission the old template by making in place changes
>>> 2. support template snapshots
>> Not sure how this will work and what use case it serves?
>
> number 1: changing the template for stateless pools.
> number 2: for anything you want including template versioning.
> Template versioning should have 2 flavours:
> 1. my golden image is outdated and I would like to decommission it and replace with a new one created from scratch (i.e. same name, new VMs would be derived from new template, no data dedup).
> 2. my golden image is outdated and I would like to update it internally - create a VM from it, make the changes, seal this VM as the new version of the template (not using the process we have today which copies all the data, just change it to be immutable).
>
> The latter requires supporting trees.

use case wise, #1 is easier, and covers both use cases - it only varies 
in amount of IO/Space, so when someone tackles this implementation wise, 
I'd vote for doing #1 first.


From bazulay at redhat.com  Tue Jan 24 18:13:13 2012
From: bazulay at redhat.com (Barak Azulay)
Date: Tue, 24 Jan 2012 20:13:13 +0200
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EDFE4.4080104@redhat.com>
References: <4F1EB67E.7050204@redhat.com> <4F1ECC37.1050208@redhat.com>
	<4F1EDC3D.80007@redhat.com> <4F1EDD67.8020702@redhat.com>
	<4F1EDFE4.4080104@redhat.com>
Message-ID: <4F1EF4B9.3010901@redhat.com>

On 01/24/2012 06:44 PM, Itamar Heim wrote:
> On 01/24/2012 06:33 PM, Doron Fediuck wrote:
>> On 24/01/12 18:28, Itamar Heim wrote:
>>> On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
>>>> On 01/24/2012 03:47 PM, Barak Azulay wrote:
>>>>> I'm experiencing severe latency issues with gerrit.ovirt.org - am I
>>>>> the
>>>>> only one ?
>>>> Currently experiencing it as well (Jan 24th, 17:20, Israel time).
>>>
>>> currently traveling in the states and no latency issues at all, so that
>>> rules out an issue with the server itself.
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>> I've been getting requests for help all day about it.
>> Most ppl in tlv simply can't work with it. Some patches are simply
>> not being pushed, and some people (/me included) were unable to login
>> for a long time.
>> This should be escalated somewhere and get fixed.
>>
>
> performance from where i am is great and has no issues with fetch/clone
> commands.
> still, to rule out any issues, i've restarted gerrit
> has anyone tried not from tlv and has issues?

still experiencing gerrit issues , the ui is so slow, and sometimes not 
responsive.

reviewing patches (in web gui) from TLV is problematic.

Barak




From iheim at redhat.com  Tue Jan 24 18:24:41 2012
From: iheim at redhat.com (Itamar Heim)
Date: Tue, 24 Jan 2012 20:24:41 +0200
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EF4B9.3010901@redhat.com>
References: <4F1EB67E.7050204@redhat.com> <4F1ECC37.1050208@redhat.com>
	<4F1EDC3D.80007@redhat.com> <4F1EDD67.8020702@redhat.com>
	<4F1EDFE4.4080104@redhat.com> <4F1EF4B9.3010901@redhat.com>
Message-ID: <4F1EF769.70908@redhat.com>

On 01/24/2012 08:13 PM, Barak Azulay wrote:
> On 01/24/2012 06:44 PM, Itamar Heim wrote:
>> On 01/24/2012 06:33 PM, Doron Fediuck wrote:
>>> On 24/01/12 18:28, Itamar Heim wrote:
>>>> On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
>>>>> On 01/24/2012 03:47 PM, Barak Azulay wrote:
>>>>>> I'm experiencing severe latency issues with gerrit.ovirt.org - am I
>>>>>> the
>>>>>> only one ?
>>>>> Currently experiencing it as well (Jan 24th, 17:20, Israel time).
>>>>
>>>> currently traveling in the states and no latency issues at all, so that
>>>> rules out an issue with the server itself.
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>> I've been getting requests for help all day about it.
>>> Most ppl in tlv simply can't work with it. Some patches are simply
>>> not being pushed, and some people (/me included) were unable to login
>>> for a long time.
>>> This should be escalated somewhere and get fixed.
>>>
>>
>> performance from where i am is great and has no issues with fetch/clone
>> commands.
>> still, to rule out any issues, i've restarted gerrit
>> has anyone tried not from tlv and has issues?
>
> still experiencing gerrit issues , the ui is so slow, and sometimes not
> responsive.
>
> reviewing patches (in web gui) from TLV is problematic.

1. this belongs in infra mailing list.
2. all reports indicate this is local to TLV users, I suggest analyzing 
the network there, etc.


From dougsland at redhat.com  Tue Jan 24 21:31:06 2012
From: dougsland at redhat.com (Douglas Landgraf)
Date: Tue, 24 Jan 2012 16:31:06 -0500
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1EF4B9.3010901@redhat.com>
References: <4F1EB67E.7050204@redhat.com>
	<4F1ECC37.1050208@redhat.com>	<4F1EDC3D.80007@redhat.com>
	<4F1EDD67.8020702@redhat.com>	<4F1EDFE4.4080104@redhat.com>
	<4F1EF4B9.3010901@redhat.com>
Message-ID: <4F1F231A.5070603@redhat.com>

On 01/24/2012 01:13 PM, Barak Azulay wrote:
> On 01/24/2012 06:44 PM, Itamar Heim wrote:
>> On 01/24/2012 06:33 PM, Doron Fediuck wrote:
>>> On 24/01/12 18:28, Itamar Heim wrote:
>>>> On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
>>>>> On 01/24/2012 03:47 PM, Barak Azulay wrote:
>>>>>> I'm experiencing severe latency issues with gerrit.ovirt.org - am I
>>>>>> the
>>>>>> only one ?
>>>>> Currently experiencing it as well (Jan 24th, 17:20, Israel time).
>>>>
>>>> currently traveling in the states and no latency issues at all, so 
>>>> that
>>>> rules out an issue with the server itself.
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>> I've been getting requests for help all day about it.
>>> Most ppl in tlv simply can't work with it. Some patches are simply
>>> not being pushed, and some people (/me included) were unable to login
>>> for a long time.
>>> This should be escalated somewhere and get fixed.
>>>
>>
>> performance from where i am is great and has no issues with fetch/clone
>> commands.
>> still, to rule out any issues, i've restarted gerrit
>> has anyone tried not from tlv and has issues?
>
> still experiencing gerrit issues , the ui is so slow, and sometimes 
> not responsive.
>
> reviewing patches (in web gui) from TLV is problematic.
>
Unfortunately, it seems a local (TLV) problem. From Brazil it's working 
just fine.

Cheers
Douglas


From sgordon at redhat.com  Tue Jan 24 19:53:47 2012
From: sgordon at redhat.com (Steve Gordon)
Date: Tue, 24 Jan 2012 14:53:47 -0500 (EST)
Subject: [Engine-devel] Call for oVirt subproject release note beats!
In-Reply-To: <8fa1a6b7-c39a-45a4-8cd6-9be6ac0d6faa@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <ba9c7a2c-7cba-4a70-9a81-4b0d3f137ae5@zmail15.collab.prod.int.phx2.redhat.com>

Hi all,

Assuming we are still on schedule for a release this is the last call for release note items. We will discuss further in the weekly sync up meeting tomorrow.

Steve

----- Original Message -----
> From: "Steve Gordon" <sgordon at redhat.com>
> To: arch at ovirt.org
> Sent: Monday, January 16, 2012 10:34:01 AM
> Subject: Re: Call for oVirt subproject release note beats!
> 
> Hi all,
> 
> As discussed in last week's sync-up meeting I emailed the
> contributors listed as oVirt subproject maintainers requesting
> release note beats. So far only the oVirt engine section of the
> release notes page has been populated (thanks Livnat!). The release
> notes page is here:
> 
>     http://www.ovirt.org/wiki/Release_Notes
> 
> Please take the time to update it with any features, resolved issues,
> or known issues you are aware of that you think should be noted for
> this first release of oVirt. Even one liners or a link to the
> relevant feature page/bugzilla would be great, I intend to edit this
> into a cohesive document before the actual release.
> 
> Thanks,
> 
> Steve
> 
> ----- Original Message -----
> > From: "Steve Gordon" <sgordon at redhat.com>
> > To: "Michael Pasternak" <mpastern at redhat.com>, "Einav Cohen"
> > <ecohen at redhat.com>, "Itamar Heim" <iheim at redhat.com>,
> > "Barak Azulay" <bazulay at redhat.com>, "Yaniv Dary"
> > <ydary at redhat.com>, "Ofer Schreiber" <oschreib at redhat.com>, "Alan
> > Pevec" <apevec at redhat.com>, "Livnat Peer" <lpeer at redhat.com>, "Ayal
> > Baron" <abaron at redhat.com>
> > Cc: "Karsten 'quaid' Wade" <kwade at redhat.com>, "Robyn Bergeron"
> > <rbergero at redhat.com>
> > Sent: Wednesday, January 11, 2012 11:51:09 AM
> > Subject: Call for oVirt subproject release note beats!
> > 
> > Hi all,
> > 
> > If you are getting this email you are listed as the first
> > maintainer
> > of an oVirt subproject at
> > http://www.ovirt.org/project/subprojects/.
> > As we ramp up towards our initial formal release of the oVirt
> > project we were hoping that you could provide a short run down of
> > what your subproject is, and a brief blurb on any noteworthy
> > features/resolved issues/known issues in it for this initial
> > release. If you have bugzilla #s to include even better!
> > 
> > This effort would assist us greatly in providing a release notes
> > document for this initial release. Please add your input to the
> > relevant section of this page on the wiki:
> > 
> > http://www.ovirt.org/wiki/Release_Notes
> > 
> > Note that the structure I have provided is just to be used as a
> > guide
> > for now, once we have see what we have we can discuss further and
> > reformat appropriately.
> > 
> > Thanks!
> > 
> > Steve
> 


From yzaslavs at redhat.com  Wed Jan 25 07:40:36 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Wed, 25 Jan 2012 09:40:36 +0200
Subject: [Engine-devel] data modelling in fixtures.xml
Message-ID: <4F1FB1F4.1090005@redhat.com>

Hi all,
When you introduce new business entities + DAOs or change some behaviour
of existing ones, besides providing tests, please try as much as
possible to insert reasonable test data to fixtures.xml, as we use the
same fixtures.xml file for all our DAO testing.

Kind regards,
Yair


From mkolesni at redhat.com  Wed Jan 25 07:58:19 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Wed, 25 Jan 2012 02:58:19 -0500 (EST)
Subject: [Engine-devel] data modelling in fixtures.xml
In-Reply-To: <4F1FB1F4.1090005@redhat.com>
Message-ID: <dc028898-144e-4a52-b611-98fb60fc9734@zmail14.collab.prod.int.phx2.redhat.com>


----- Original Message -----
> Hi all,
> When you introduce new business entities + DAOs or change some
> behaviour
> of existing ones, besides providing tests, please try as much as
> possible to insert reasonable test data to fixtures.xml, as we use
> the
> same fixtures.xml file for all our DAO testing.

Regarding that, perhaps we should use separate fixtures for different tests, so that they're lees dependant on each other (in terms of predefined data).

> 
> Kind regards,
> Yair
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From yzaslavs at redhat.com  Wed Jan 25 08:09:25 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Wed, 25 Jan 2012 10:09:25 +0200
Subject: [Engine-devel] data modelling in fixtures.xml
In-Reply-To: <dc028898-144e-4a52-b611-98fb60fc9734@zmail14.collab.prod.int.phx2.redhat.com>
References: <dc028898-144e-4a52-b611-98fb60fc9734@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1FB8B5.1050700@redhat.com>

On 01/25/2012 09:58 AM, Mike Kolesnik wrote:
> 
> ----- Original Message -----
>> Hi all,
>> When you introduce new business entities + DAOs or change some
>> behaviour
>> of existing ones, besides providing tests, please try as much as
>> possible to insert reasonable test data to fixtures.xml, as we use
>> the
>> same fixtures.xml file for all our DAO testing.
> 
> Regarding that, perhaps we should use separate fixtures for different tests, so that they're lees dependant on each other (in terms of predefined data).
I agree, this is definitely something we should check how to do using
DbUnit.

> 
>>
>> Kind regards,
>> Yair
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From dlaor at redhat.com  Wed Jan 25 09:09:02 2012
From: dlaor at redhat.com (Dor Laor)
Date: Wed, 25 Jan 2012 11:09:02 +0200
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <ae15611f-e655-45d0-8c5f-264daac1a1d7@zmail13.collab.prod.int.phx2.redhat.com>
References: <ae15611f-e655-45d0-8c5f-264daac1a1d7@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F1FC6AE.7050900@redhat.com>

On 01/22/2012 08:42 PM, Ayal Baron wrote:
>
>
> ----- Original Message -----
>>
>>
>> ----- Original Message -----
>>> From: "Ayal Baron"<abaron at redhat.com>
>>> To: "Oved Ourfalli"<ovedo at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Thursday, January 19, 2012 4:05:08 PM
>>> Subject: Re: [Engine-devel] VM Payload feature
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> Hey all,
>>>>
>>>> Continuing the discussion about Aeolus instance data injection to
>>>> a
>>>> VM
>>>> (http://lists.ovirt.org/pipermail/engine-devel/2012-January/000423.html)
>>>> we propose a new VM Payload feature.
>>>>
>>>> The following wiki page contains a description page of the
>>>> feature.
>>>> http://www.ovirt.org/wiki/Features/VMPayload
>>>>
>>>> Please read and review.
>>>> There are several approaches there, and we wish to head your
>>>> opinions
>>>> and thoughts about them.
>>>>
>>>> Once we agree on an approach, we will start designing.
>>>
>>> Permanent payload availability requires determining where the
>>> payload
>>> is stored.
>>> Makes sense to me to store it together with the VM disks on the
>>> storage domain, but that requires the small object store which will
>>> not be available in the coming version (payloads can be large and
>>> keeping them in the DB and passing over the net every time the VM
>>> is
>>> run doesn't make much sense).
>>>
>> I guess we can start with storing it in the database, with some size
>> limitation, and move it to the storage domain later on.
>
> If Aeolus and Deltacloud don't need the permanent payload feature then no need to store it at all and then just add this capability later on properly.

Currently it seems that there are too many options for it - floppy, cd, 
nfs and maybe more. It would be nice to have a single option that works 
for all cases. How about creating something like s3 compatible storage 
access that the guest can access? If you need boot time access then I'll 
recommend cdrom or plain virtio-blk.

>
>>
>>> Wrt availability, I don't see a reason to exclude attaching both a
>>> CD
>>> and a payload via another CD at the same time (i.e. multiple
>>> devices).
>>>
>>>>
>>>> Thank you,
>>>> Oved
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From iheim at redhat.com  Wed Jan 25 10:02:55 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 25 Jan 2012 12:02:55 +0200
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <4F1FC6AE.7050900@redhat.com>
References: <ae15611f-e655-45d0-8c5f-264daac1a1d7@zmail13.collab.prod.int.phx2.redhat.com>
	<4F1FC6AE.7050900@redhat.com>
Message-ID: <4F1FD34F.4040801@redhat.com>

On 01/25/2012 11:09 AM, Dor Laor wrote:
> On 01/22/2012 08:42 PM, Ayal Baron wrote:
...

>>>>> The following wiki page contains a description page of the
>>>>> feature.
>>>>> http://www.ovirt.org/wiki/Features/VMPayload
>>>>>
...
>
> Currently it seems that there are too many options for it - floppy, cd,
> nfs and maybe more. It would be nice to have a single option that works
> for all cases. How about creating something like s3 compatible storage
> access that the guest can access? If you need boot time access then I'll
> recommend cdrom or plain virtio-blk.

I think there are different use cases here:
1. floppy and iso cover the same use case, for similar needs (and behave 
the same). this would cover windows sysprep approach and basic 
attachment of files
2. http://192.169.192.168 - this would provide compatibility to cloud 
approaches iiuc
3. injecting into the file system - this covers various other needs, 
like injecting ssh key, and is relevant not only during bootstrap, 
should we want to allow editing a guest when it is down to troubleshoot 
issues.

so I think all have their place...



From ofrenkel at redhat.com  Wed Jan 25 10:50:22 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Wed, 25 Jan 2012 05:50:22 -0500 (EST)
Subject: [Engine-devel] latency issues with gerrit.ovirt.org
In-Reply-To: <4F1F231A.5070603@redhat.com>
Message-ID: <c17f9af4-238a-4a96-9517-911c2fa821d9@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Douglas Landgraf" <dougsland at redhat.com>
> To: "Barak Azulay" <bazulay at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Tuesday, January 24, 2012 11:31:06 PM
> Subject: Re: [Engine-devel] latency issues with gerrit.ovirt.org
> 
> On 01/24/2012 01:13 PM, Barak Azulay wrote:
> > On 01/24/2012 06:44 PM, Itamar Heim wrote:
> >> On 01/24/2012 06:33 PM, Doron Fediuck wrote:
> >>> On 24/01/12 18:28, Itamar Heim wrote:
> >>>> On 01/24/2012 05:20 PM, Yair Zaslavsky wrote:
> >>>>> On 01/24/2012 03:47 PM, Barak Azulay wrote:
> >>>>>> I'm experiencing severe latency issues with gerrit.ovirt.org -
> >>>>>> am I
> >>>>>> the
> >>>>>> only one ?
> >>>>> Currently experiencing it as well (Jan 24th, 17:20, Israel
> >>>>> time).
> >>>>
> >>>> currently traveling in the states and no latency issues at all,
> >>>> so
> >>>> that
> >>>> rules out an issue with the server itself.
> >>>> _______________________________________________
> >>>> Engine-devel mailing list
> >>>> Engine-devel at ovirt.org
> >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>
> >>> I've been getting requests for help all day about it.
> >>> Most ppl in tlv simply can't work with it. Some patches are
> >>> simply
> >>> not being pushed, and some people (/me included) were unable to
> >>> login
> >>> for a long time.
> >>> This should be escalated somewhere and get fixed.
> >>>
> >>
> >> performance from where i am is great and has no issues with
> >> fetch/clone
> >> commands.
> >> still, to rule out any issues, i've restarted gerrit
> >> has anyone tried not from tlv and has issues?
> >
> > still experiencing gerrit issues , the ui is so slow, and sometimes
> > not responsive.
> >
> > reviewing patches (in web gui) from TLV is problematic.
> >
> Unfortunately, it seems a local (TLV) problem. From Brazil it's
> working
> just fine.
> 
> Cheers
> Douglas
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 

i can even say the problem is in tlv office,
as it worked fine for me at home.
(while checking with Yair in the office, where it didn't work well)


From mlipchuk at redhat.com  Wed Jan 25 11:33:06 2012
From: mlipchuk at redhat.com (Maor)
Date: Wed, 25 Jan 2012 13:33:06 +0200
Subject: [Engine-devel] data modelling in fixtures.xml
In-Reply-To: <4F1FB1F4.1090005@redhat.com>
References: <4F1FB1F4.1090005@redhat.com>
Message-ID: <4F1FE872.5030508@redhat.com>

On 01/25/2012 09:40 AM, Yair Zaslavsky wrote:
> Hi all,
> When you introduce new business entities + DAOs or change some behaviour
> of existing ones, besides providing tests, please try as much as
> possible to insert reasonable test data to fixtures.xml, as we use the
> same fixtures.xml file for all our DAO testing.
Regarding inserting data to fixtures.xml,
I merged yesterday a helper class called FixturesTool.
It is a static reference file which should be correlated with the
fixtures.xml, and might help us, when referencing fixtures.xml data in
our tests.

When inserting new data to fixtures.xml, please also consider correlate
it with FixturesTool.

Thanks,
Maor
> 
> Kind regards,
> Yair
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From yzaslavs at redhat.com  Wed Jan 25 12:00:32 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Wed, 25 Jan 2012 14:00:32 +0200
Subject: [Engine-devel] data modelling in fixtures.xml
In-Reply-To: <4F1FE872.5030508@redhat.com>
References: <4F1FB1F4.1090005@redhat.com> <4F1FE872.5030508@redhat.com>
Message-ID: <4F1FEEE0.1040709@redhat.com>

On 01/25/2012 01:33 PM, Maor wrote:
> On 01/25/2012 09:40 AM, Yair Zaslavsky wrote:
>> Hi all,
>> When you introduce new business entities + DAOs or change some behaviour
>> of existing ones, besides providing tests, please try as much as
>> possible to insert reasonable test data to fixtures.xml, as we use the
>> same fixtures.xml file for all our DAO testing.
> Regarding inserting data to fixtures.xml,
> I merged yesterday a helper class called FixturesTool.
> It is a static reference file which should be correlated with the
> fixtures.xml, and might help us, when referencing fixtures.xml data in
> our tests.
> 
> When inserting new data to fixtures.xml, please also consider correlate
> it with FixturesTool.
> 
Indeed. At least as long as we have single file, it can help us with
data modelling.

> Thanks,
> Maor
>>
>> Kind regards,
>> Yair
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 



From agl at us.ibm.com  Wed Jan 25 14:20:59 2012
From: agl at us.ibm.com (Adam Litke)
Date: Wed, 25 Jan 2012 08:20:59 -0600
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>
References: <0a2b8e83-4b71-4c42-8bff-def24ba5dea8@zmail16.collab.prod.int.phx2.redhat.com>
	<1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <20120125142059.GB4837@us.ibm.com>

On Mon, Jan 23, 2012 at 04:54:10PM -0500, Saggi Mizrahi wrote:
> Nitty Gritty:

This seems like a good API but I have some suggestions with respect to API
naming:

> manageStorageServer
> ===================

Could we name this manageStorageConnection or manageStorageServerConnection?
Manage storage server is confusing because it implies you are managing the
server itself (ie. server configuration, NFS exports, reboot, etc).

> Synopsis:
> manageStorageServer(uri, connectionID):
> 
> Parameters:
> uri - a uri pointing to a storage target (eg: nfs://server:export, iscsi://host/iqn;portal=1)
> connectionID - string with any char except "/".
> 
> Description:
> Tells VDSM to start managing the connection. From this moment on VDSM will try and have the connection available when needed. VDSM will monitor the connection and will automatically reconnect on failure.
> Returns:
> Success code if VDSM was able to manage the connection.
> It usually just verifies that the arguments are sane and that the CID is not already in use.
> This doesn't mean the host is connected.
> ----
> unmanageStorageServer
> =====================

To match above: unmanageStorageConnection or unmanageStorageServerConnection

> Synopsis:
> unmanageStorageServer(connectionID):
> 
> Parameters:
> connectionID - string with any char except "/".
> 
> Descriptions:
> Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.
> 
> Returns:
> Success code if VDSM was able to unmanage the connection.
> It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
> ----
> getStorageServerList
> ====================

getStorageConnectionList or getStorageServerConnectionList

> Synopsis:
> getStorageServerList()
> 
> Description:
> Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.
> 
> Results:VDSM was able to manage the connection.
> It usually just verifies that the arguments are sane and that the CID is not already in use.
> This doesn't mean the host is connected.
> ----
> unmanageStorageServer
> =====================
> Synopsis:
> unmanageStorageServer(connectionID):
> 
> Parameters:
> connectionID - string with any char except "/".
> 
> Descriptions:
> Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.
> 
> Returns:
> Success code if VDSM was able to unmanage the connection.
> It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
> ----
> getStorageServerList
> ====================
> Synopsis:
> getStorageServerList()
> 
> Description:
> Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.
> 
> Results:
> A mapping between CIDs and the status.
> example return value (Actual key names may differ)
> 
> {'conA': {'connected': True, 'managed': True, 'lastError': 0, 'connectionInfo': {
>     'remotePath': 'server:/export
>     'retrans': 3
>     'version': 4
>     }}
>  'iscsi_session_34': {'connected': False, 'managed': False, 'lastError': 339, 'connectionIfno': {
>     'hostname': 'dandylopn'
>     'portal': 1}}
> }
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/vdsm-devel

-- 
Adam Litke <agl at us.ibm.com>
IBM Linux Technology Center



From smizrahi at redhat.com  Wed Jan 25 16:09:05 2012
From: smizrahi at redhat.com (Saggi Mizrahi)
Date: Wed, 25 Jan 2012 11:09:05 -0500 (EST)
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <4F1EEDCB.4070000@redhat.com>
Message-ID: <459a6105-a79e-4fb8-9a67-b75b8eb42367@zmail16.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Saggi Mizrahi" <smizrahi at redhat.com>
> Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
> Sent: Tuesday, January 24, 2012 12:43:39 PM
> Subject: Re: [Engine-devel] [RFC] New Connection Management API
> 
> On 23/01/12 23:54, Saggi Mizrahi wrote:
> > I have begun work at changing how API clients can control storage
> > connections when interacting with VDSM.
> > 
> > Currently there are 2 API calls:
> > connectStorageServer() - Will connect to the storage target if the
> > host is not already connected to it.
> > disconnectStorageServer() - Will disconnect from the storage target
> > if the host is connected to it.
> > 
> > This API is very simple but is inappropriate when multiple clients
> > and flows try to access the same storage.
> > 
> > This is currently solved by trying to synchronize things inside
> > rhevm. This is hard and convoluted. It also brings out issues with
> > other clients using the VDSM API.
> > 
> > Another problem is error recovery. Currently ovirt-engine(OE) has
> > no way of monitoring the connections on all the hosts an if a
> > connection disappears it's OE's responsibility to reconnect.
> > 
> > I suggest a different concept where VDSM 'manages' the connections.
> > VDSM receives a manage request with the connection information and
> > from that point forward VDSM will try to keep this connection
> > alive. If the connection fails VDSM will automatically try and
> > recover.
> > 
> > Every manage request will also have a connection ID(CID). This CID
> > will be used when the same client asks to unamange the connection.
> > When multiple requests for manage are received to the same
> > connection they all have to have their own unique CID. By
> > internally mapping CIDs to actual connections VDSM can properly
> > disconnect when no CID is addressing the connection. This allows
> > each client and even each flow to have it's own CID effectively
> > eliminating connect\disconnect races.
> > 
> > The change from (dis)connect to (un)manage also changes the
> > semantics of the calls significantly.
> > Whereas connectStorageServer would have returned when the storage
> > is either connected or failed to connect, manageStorageServer will
> > return once VDSM registered the CID. This means that the
> > connection might not be active immediately as the VDSM tries to
> > connect. The connection might remain down for a long time if the
> > storage target is down or is having issues.
> > 
> > This allows for VDSM to receive the manage request even if the
> > storage is having issues and recover as soon as it's operational
> > without user intervention.
> > 
> > In order for the client to query the current state of the
> > connections I propose getStorageConnectionList(). This will return
> > a mapping of CID to connection status. The status contains the
> > connection info (excluding credentials), whether the connection is
> > active, whether the connection is managed (unamanged connection
> > are returned with transient IDs), and, if the connection is down,
> > the last error information.
> > 
> > The same actual connection can return multiple times, once for each
> > CID.
> > 
> > For cases where an operation requires a connection to be active a
> > user can poll the status of the CID. The user can then choose to
> > poll for a certain amount of time or until an error appears in the
> > error field of the status. This will give you either a timeout or
> > a "try once" semantic depending on the flows needs.
> > 
> > All connections that have been managed persist VDSM restart and
> > will be managed until a corresponding unmanage command has been
> > issued.
> > 
> > There is no concept of temporary connections as "temporary" is flow
> > dependent and VDSM can't accommodate all interpretation of
> > "temporary". An ad-hoc mechanism can be build using the CID field.
> > For instance a client can manage a connection with
> > "ENGINE_FLOW101_CON1". If the flow got interrupted the client can
> > clean all IDs with certain flow IDs.
> > 
> > I think this API gives safety, robustness, and implementation
> > freedom.
> > 
> > 
> > Nitty Gritty:
> > 
> > manageStorageServer
> > ===================
> > Synopsis:
> > manageStorageServer(uri, connectionID):
> > 
> > Parameters:
> > uri - a uri pointing to a storage target (eg: nfs://server:export,
> > iscsi://host/iqn;portal=1)
> > connectionID - string with any char except "/".
> > 
> > Description:
> > Tells VDSM to start managing the connection. From this moment on
> > VDSM will try and have the connection available when needed. VDSM
> > will monitor the connection and will automatically reconnect on
> > failure.
> > Returns:
> > Success code if VDSM was able to manage the connection.
> > It usually just verifies that the arguments are sane and that the
> > CID is not already in use.
> > This doesn't mean the host is connected.
> > ----
> > unmanageStorageServer
> > =====================
> > Synopsis:
> > unmanageStorageServer(connectionID):
> > 
> > Parameters:
> > connectionID - string with any char except "/".
> > 
> > Descriptions:
> > Tells VDSM to stop managing the connection. VDSM will try and
> > disconnect for the storage target if this is the last CID
> > referencing the storage connection.
> > 
> > Returns:
> > Success code if VDSM was able to unmanage the connection.
> > It will return an error if the CID is not registered with VDSM.
> > Disconnect failures are not reported. Active unmanaged connections
> > can be tracked with getStorageServerList()
> > ----
> > getStorageServerList
> > ====================
> > Synopsis:
> > getStorageServerList()
> > 
> > Description:
> > Will return list of all managed and unmanaged connections.
> > Unmanaged connections have temporary IDs and are not guaranteed to
> > be consistent across calls.
> > 
> > Results:VDSM was able to manage the connection.
> > It usually just verifies that the arguments are sane and that the
> > CID is not already in use.
> > This doesn't mean the host is connected.
> > ----
> > unmanageStorageServer
> > =====================
> > Synopsis:
> > unmanageStorageServer(connectionID):
> > 
> > Parameters:
> > connectionID - string with any char except "/".
> > 
> > Descriptions:
> > Tells VDSM to stop managing the connection. VDSM will try and
> > disconnect for the storage target if this is the last CID
> > referencing the storage connection.
> > 
> > Returns:
> > Success code if VDSM was able to unmanage the connection.
> > It will return an error if the CID is not registered with VDSM.
> > Disconnect failures are not reported. Active unmanaged connections
> > can be tracked with getStorageServerList()
> > ----
> > getStorageServerList
> > ====================
> > Synopsis:
> > getStorageServerList()
> > 
> > Description:
> > Will return list of all managed and unmanaged connections.
> > Unmanaged connections have temporary IDs and are not guaranteed to
> > be consistent across calls.
> > 
> > Results:
> > A mapping between CIDs and the status.
> > example return value (Actual key names may differ)
> > 
> > {'conA': {'connected': True, 'managed': True, 'lastError': 0,
> > 'connectionInfo': {
> >     'remotePath': 'server:/export
> >     'retrans': 3
> >     'version': 4
> >     }}
> >  'iscsi_session_34': {'connected': False, 'managed': False,
> >  'lastError': 339, 'connectionIfno': {
> >     'hostname': 'dandylopn'
> >     'portal': 1}}
> > }
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 
> Hi Saggi,
> 
> I see the added value in the above functionality and I think it is a
> needed functionality in VDSM.
> 
> Your suggestion includes 2 concepts:
> - Persist connection - auto-reconnect on failures
> - Reference counting (with CID granularity)
It's not reference counting and the API user should not assume it is a reference count.
Each CID can only registered once.
Subsequent requests to register will fail if the CID is already registered.
There shouldn't be any assumptions between a manage call and how many physical connections are actually created.
Optimizations like internal multiplexing are implementation detail and might change.
> 
> Here are some comments:
> 
> * Assuming you meant that the new API will be a replacement to the
> current API (based on previous chats we had on this topic)
It is
> I think you
> are missing a needed functionality to support non-persisted
> connection.
the problem is with the term non-persisted. Everything is transient depending on the scale of time you consider "temporary". I leave the decision on what is temporary to the API user and give him the freedom to implement any connection lifecycle mechanism he chooses. I assume that all components, including VDSM, can crash in the middle of a flow and might want to either recover and continue or roll back. I leave it the the user to decide how to handle this as he is the one managing the flow and not VDSM.
> creating a storage domain is an example where it can be useful.
> The flow includes connecting the host to the storage server creating
> storage domain and disconnecting from the storage server.
I think you are confusing create storage domain flow in general and how create storage domain flow now in the RHEV GUI.
A flow can have multiple strategies waiting for connection availability:
* If it's a non interactive process I might not care if the actual connect takes 3 hours.
* On the other hand if it's an interactive connect I might only want to wait for 1 minute even if an actual connect request takes a lot more because of some problem.
* If I am testing the connection arguments I might want to wait until I see the connection succeed or lastError get a value no matter how long it takes.
* I might want to try as long as the error is not credential related (or other non transient issue).
* I might want to try until I see the connection active for X amount of time (To test for intermittent disconnects).

All of these can be accommodated by the suggested API.

> Let's assume VDSM hangs while creating the storage domain any
> unmanageStorageServer will fail, the engine rolls back and tries to
> create the storage domain on another host, there is no reason for the
> host to reconnect to this storage server.
That is true but there is no way for VDSM to know if the connection is deprecated or not. For all I know rhevm might be having issues and will continue on with the flow in a few minutes.
> In the above flow i would use non-persist connection if I had one.
Again, what does no-persist means.
> 
> * In the suggested solution the connect will not initiate an
> immediate
> connect to the storage server instead it will register the connection
> as
> handled connection and will actually generate the connect as part of
> the
> managed connection mechanism.
The mechanism guarantees maximum availability so it will immediately connect. The command might return before an actual connection succeeded as the Manage part is done.
> I argue that this modeling is implementation driven which is wrong
> from
> the user perspective.
VDSM is pretty low on the stack and has to accommodate many API users. I think it's wrong to model an API when you don't consider how things actually behave and try and glue stuff to appease a GUI flow. GUI flows change all the time, APIs don't so having a flexible API that supports multiple use patterns and does not enforce arbitrary limitations is better then one that is tightly coupled to 1 user flow.
> As a user I expect connect to actually initiate
> a
> connect action and that the return value should indicate if the
> connect
> succeeded, the way you modeled it the API will return true if you
> succeeded 'registering' the connect.
> You modeled the API to be asynchronous with no handler (task id) to
> monitor the results of the action, which requires polling
The API is not asynchronous it is perfectly synchronous. When manageStorageConnection() returns the connection is managed.
You will have maximum connection uptime. You will have to poll and check for the liveness of the connection before using it as some problems may occur preventing VDSM from supplying the connection at the moment.
> in the
> create
> storage domain flow which I really don't like.
> In addition you introduced a verb for monitoring the status of the
> connections alone I would like to be able to monitor it as part of
> the
> general host status and not have to poll on a new verb in addition to
> the current one.
> 
> 
> As part of solving the connection management flows in OE I am
> missing:
> 
> - A way to clear all managed connections.
> use case: We move a host from one data center to another and we want
> the
> host to clear all the managed connections.
> we can ask for the list of managed connection and clear them but
> having
> clearAll is much easier.
Nope, you should get all active connections. Cherry pick the ones you own using some ID scheme (RHEVM_FLOWID_CON?) and only clear you own connections. There might be other client using VDSM that you will forcibly disconnect.
> 
> - Handling a list of Ids in each API verb
Only getDeviceList will have a list of IDs handed to it. It makes no sense in other verbs.
> 
> - A verb which handles create storage domain and encapsulates the
> connect create and disconnect.
This is a hackish ad-hoc solution. Why not have one for the entire pool? Why not have one for a VM?
> 
> 
> Thanks, Livnat
> 


I will try and sum the points ups here:
manageConnection is not connectStorageServer. They are different. The latter means connect to the storage server, the former means manage it. They are both synchronous.

non-persistence makes no sense. Auto unmanage does. If anyone suggests a valid mechanism to auto clean CIDs that is correct and accommodates interactive and non interactive flows that I will be willing to accept it. Timeouts are never correct as no flow is really time capped and it will create more issues that it will solve.

Polling the CID to track connection availability is the correct way to go as what you really want to do is not connect to the storage bu rather have it available. Polling is just waiting until the connection is available or condition has been triggered. This give the flow manager freedom of what the condition is (see above).

Cleaning the connections, like closing FDs, freeing memory. and other resource management is a pain. I understand, and having a transaction like mechanism to lock resources to a flow will be great but this is outside the scope of this change.

VDSM being a tiny cog in the cluster can never have enough information to know when a flow started or finished. This is why I leave it to the management to manage these resources. I just prevent collisions (with the CIDs) and handle resource availability.

How to implement stuff:
I suggest this CID scheme:

For connections that persist across engine restarts.

OENGINE_<resource type>_<resource id>_CON<connection id>
EX:
OENGINE_DOMAIN_2131-321dsa-dsadsa-232_CON1

For connections that are managed for flows and do not might not persist engine restart

OENGINE_<engine isntanceid>_FLOW_<flow id>_CON<connection id>
EX:
OENGINE_4324-23423dfd-fsdfsd-21312_FLOW_1023_CON1

Note:
instance id is a uuid generate on each instance run to differentiate between running instances simply.



How to poll for connections: (in pythonic pseudo code)
---------------------------------------------------------
def pollConenctions(vdsm host, stringList CidList, func(void)bool stopContion, int interval):
    clist = CidList.copy()
    while (not stopCondition) and (len(clist) > 0):
       statuses = host.getStorageConnectionsStatuses()
       for id in statuses:
           if not id.startswith("OENGINE"):
               # This is not an engine connection, ignore
               continue

           # Check the scheme and see if it has an instance ID after the prefix or not
           if isPersistantConnection(id):
              continue

           instanceId, flowId, conId = parseCID(id)

           # Clean connections from past instances
           if instanceId != global_instance_id
              # Ignore errors here as some other thread may be clearing this ID as well
              # at any case VDSM is taking care of thread safety.
              host.unmanageStorageConnection(id)

           if id in CidList:
              if statuses[id].connected:
                  clist.remove(id)

       sleep(interval)
-------------------------------------------------
It's easy to see how you can modify this template to support multiple modes of tracking
* Pass a flow id instead of a CID list to track a flow
* Exit when at least X connections succeeded
* call getDeviceList after every successful connect and check if the lun you are looking for is available if it is continue and let the other connections complete at their own pace for multipathing.
* connect to multiple hosts and return once 1 host has connected successfuly
* you can also add an install id or a cluster id if you want to have multiple engines managing the same VDSM and not have them step on each others toes.

and much, much more.

Implementing this will give you everything you want with maximum correctness and flexibility. This will also make the transition to event driven communication with VDSM simpler.


From lpeer at redhat.com  Wed Jan 25 20:41:57 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 25 Jan 2012 22:41:57 +0200
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <459a6105-a79e-4fb8-9a67-b75b8eb42367@zmail16.collab.prod.int.phx2.redhat.com>
References: <459a6105-a79e-4fb8-9a67-b75b8eb42367@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F206915.7050902@redhat.com>


>> Hi Saggi,
>>
>> I see the added value in the above functionality and I think it is a
>> needed functionality in VDSM.
>>
>> Your suggestion includes 2 concepts:
>> - Persist connection - auto-reconnect on failures
>> - Reference counting (with CID granularity)
> It's not reference counting and the API user should not assume it is a reference count.
> Each CID can only registered once.

By reference counting with CID granularity I meant that as long as you
have more than one CID registered on a connection the connection will be
managed by the host.


> Subsequent requests to register will fail if the CID is already registered.
> There shouldn't be any assumptions between a manage call and how many physical connections are actually created.
> Optimizations like internal multiplexing are implementation detail and might change.


>>
>> Here are some comments:
>>
>> * Assuming you meant that the new API will be a replacement to the
>> current API (based on previous chats we had on this topic)
> It is
>> I think you
>> are missing a needed functionality to support non-persisted
>> connection.
> the problem is with the term non-persisted. Everything is transient depending on the scale of time you consider "temporary". I leave the decision on what is temporary to the API user and give him the freedom to implement any connection lifecycle mechanism he chooses. I assume that all components, including VDSM, can crash in the middle of a flow and might want to either recover and continue or roll back. I leave it the the user to decide how to handle this as he is the one managing the flow and not VDSM.

I would call connections that don't need to reconnect upon failure - non
persistent connection, it is not a function of time.

There are operations that upon failure can be done on another host and
there is no reason to reconnect to the storage target as it is not
interesting for the user any more.

>> creating a storage domain is an example where it can be useful.
>> The flow includes connecting the host to the storage server creating
>> storage domain and disconnecting from the storage server.

> I think you are confusing create storage domain flow in general and how create storage domain flow now in the RHEV GUI.
> A flow can have multiple strategies waiting for connection availability:
> * If it's a non interactive process I might not care if the actual connect takes 3 hours.
> * On the other hand if it's an interactive connect I might only want to wait for 1 minute even if an actual connect request takes a lot more because of some problem.
> * If I am testing the connection arguments I might want to wait until I see the connection succeed or lastError get a value no matter how long it takes.
> * I might want to try as long as the error is not credential related (or other non transient issue).
> * I might want to try until I see the connection active for X amount of time (To test for intermittent disconnects).
> 
> All of these can be accommodated by the suggested API.

That is great but i am not sure how is it related to non-persist connection.

> 
>> Let's assume VDSM hangs while creating the storage domain any
>> unmanageStorageServer will fail, the engine rolls back and tries to
>> create the storage domain on another host, there is no reason for the
>> host to reconnect to this storage server.
> That is true but there is no way for VDSM to know if the connection is deprecated or not. For all I know rhevm might be having issues and will continue on with the flow in a few minutes.

If there is an error VDSM doesn't need to reconnect the non-persist
connection, and it should be up to the VDSM user to ask for persist
connection or non-persist connection.


>> In the above flow i would use non-persist connection if I had one.
> Again, what does no-persist means.
>>
>> * In the suggested solution the connect will not initiate an
>> immediate
>> connect to the storage server instead it will register the connection
>> as
>> handled connection and will actually generate the connect as part of
>> the
>> managed connection mechanism.
> The mechanism guarantees maximum availability so it will immediately connect. The command might return before an actual connection succeeded as the Manage part is done.
>> I argue that this modeling is implementation driven which is wrong
>> from
>> the user perspective.
> VDSM is pretty low on the stack and has to accommodate many API users. I think it's wrong to model an API when you don't consider how things actually behave and try and glue stuff to appease a GUI flow. GUI flows change all the time, APIs don't so having a flexible API that supports multiple use patterns and does not enforce arbitrary limitations is better then one that is tightly coupled to 1 user flow.

I am not sure why do you think i am looking on the GUI flow, as I
actually was referring to the engine as the user of VDSM . The engine
has to support different clients, the UI is only one of them.

>> As a user I expect connect to actually initiate
>> a
>> connect action and that the return value should indicate if the
>> connect
>> succeeded, the way you modeled it the API will return true if you
>> succeeded 'registering' the connect.
>> You modeled the API to be asynchronous with no handler (task id) to
>> monitor the results of the action, which requires polling
> The API is not asynchronous it is perfectly synchronous. When manageStorageConnection() returns the connection is managed.
> You will have maximum connection uptime. You will have to poll and check for the liveness of the connection before using it as some problems may occur preventing VDSM from supplying the connection at the moment.
>> in the
>> create
>> storage domain flow which I really don't like.
>> In addition you introduced a verb for monitoring the status of the
>> connections alone I would like to be able to monitor it as part of
>> the
>> general host status and not have to poll on a new verb in addition to
>> the current one.
>>
>>
>> As part of solving the connection management flows in OE I am
>> missing:
>>
>> - A way to clear all managed connections.
>> use case: We move a host from one data center to another and we want
>> the
>> host to clear all the managed connections.
>> we can ask for the list of managed connection and clear them but
>> having
>> clearAll is much easier.
> Nope, you should get all active connections. Cherry pick the ones you own using some ID scheme (RHEVM_FLOWID_CON?) and only clear you own connections. There might be other client using VDSM that you will forcibly disconnect.

I hope that VDSM is going to serve many types of clients but clients
hybrid mode is the less interesting use case IMO.
How often will you have more than one virtualization manager manages the
same host? I think not a common use case, and if this is not the common
use case i expect the API to be more friendly to the single manager use
case.

moving the host from one data center to another is a clear use case
where clearAll API would be useful, and i am sure other clients will
find this API useful as well.

>>
>> - Handling a list of Ids in each API verb
> Only getDeviceList will have a list of IDs handed to it. It makes no sense in other verbs.

I disagree, if I need to connect a host to a storage domain I need to
execute number of API calls which is linear to the number of storage
servers i use for the storage domain, again not a friendly API.



>>
>> - A verb which handles create storage domain and encapsulates the
>> connect create and disconnect.
> This is a hackish ad-hoc solution. Why not have one for the entire pool? Why not have one for a VM?

I think we are going to remove pool in 4.0 so probably not, and for VM
well that's an interesting idea :)



>>
>>
>> Thanks, Livnat
>>
> 
> 
> I will try and sum the points ups here:
> manageConnection is not connectStorageServer. They are different. The latter means connect to the storage server, the former means manage it. They are both synchronous.
> 
> non-persistence makes no sense. Auto unmanage does. If anyone suggests a valid mechanism to auto clean CIDs that is correct and accommodates interactive and non interactive flows that I will be willing to accept it. Timeouts are never correct as no flow is really time capped and it will create more issues that it will solve.
> 

I am not sure what is the problem with the non-persistent mechanism i
suggested earlier.

> Polling the CID to track connection availability is the correct way to go as what you really want to do is not connect to the storage bu rather have it available. Polling is just waiting until the connection is available or condition has been triggered. This give the flow manager freedom of what the condition is (see above).
> 
> Cleaning the connections, like closing FDs, freeing memory. and other resource management is a pain. I understand, and having a transaction like mechanism to lock resources to a flow will be great but this is outside the scope of this change.
> 
> VDSM being a tiny cog in the cluster can never have enough information to know when a flow started or finished. This is why I leave it to the management to manage these resources. I just prevent collisions (with the CIDs) and handle resource availability.
> 
> How to implement stuff:
> I suggest this CID scheme:
> 
> For connections that persist across engine restarts.
> 
> OENGINE_<resource type>_<resource id>_CON<connection id>
> EX:
> OENGINE_DOMAIN_2131-321dsa-dsadsa-232_CON1
> 
> For connections that are managed for flows and do not might not persist engine restart
> 
> OENGINE_<engine isntanceid>_FLOW_<flow id>_CON<connection id>
> EX:
> OENGINE_4324-23423dfd-fsdfsd-21312_FLOW_1023_CON1
> 
> Note:
> instance id is a uuid generate on each instance run to differentiate between running instances simply.
> 
> 
> 
> How to poll for connections: (in pythonic pseudo code)
> ---------------------------------------------------------
> def pollConenctions(vdsm host, stringList CidList, func(void)bool stopContion, int interval):
>     clist = CidList.copy()
>     while (not stopCondition) and (len(clist) > 0):
>        statuses = host.getStorageConnectionsStatuses()
>        for id in statuses:
>            if not id.startswith("OENGINE"):
>                # This is not an engine connection, ignore
>                continue
> 
>            # Check the scheme and see if it has an instance ID after the prefix or not
>            if isPersistantConnection(id):
>               continue
> 
>            instanceId, flowId, conId = parseCID(id)
> 
>            # Clean connections from past instances
>            if instanceId != global_instance_id
>               # Ignore errors here as some other thread may be clearing this ID as well
>               # at any case VDSM is taking care of thread safety.
>               host.unmanageStorageConnection(id)
> 
>            if id in CidList:
>               if statuses[id].connected:
>                   clist.remove(id)
> 
>        sleep(interval)
> -------------------------------------------------

I would not use sleep, I would use a scheduler based monitoring and
release the thread in between cycles.

> It's easy to see how you can modify this template to support multiple modes of tracking
> * Pass a flow id instead of a CID list to track a flow
> * Exit when at least X connections succeeded
> * call getDeviceList after every successful connect and check if the lun you are looking for is available if it is continue and let the other connections complete at their own pace for multipathing.
> * connect to multiple hosts and return once 1 host has connected successfuly
> * you can also add an install id or a cluster id if you want to have multiple engines managing the same VDSM and not have them step on each others toes.
> 
> and much, much more.
> 
> Implementing this will give you everything you want with maximum correctness and flexibility. This will also make the transition to event driven communication with VDSM simpler.





From lpeer at redhat.com  Wed Jan 25 20:44:49 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 25 Jan 2012 22:44:49 +0200
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <20120125142059.GB4837@us.ibm.com>
References: <0a2b8e83-4b71-4c42-8bff-def24ba5dea8@zmail16.collab.prod.int.phx2.redhat.com>
	<1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>
	<20120125142059.GB4837@us.ibm.com>
Message-ID: <4F2069C1.8050901@redhat.com>

On 25/01/12 16:20, Adam Litke wrote:
> On Mon, Jan 23, 2012 at 04:54:10PM -0500, Saggi Mizrahi wrote:
>> Nitty Gritty:
> 
> This seems like a good API but I have some suggestions with respect to API
> naming:
> 
>> manageStorageServer
>> ===================
> 
> Could we name this manageStorageConnection or manageStorageServerConnection?
> Manage storage server is confusing because it implies you are managing the
> server itself (ie. server configuration, NFS exports, reboot, etc).
> 

+1

>> Synopsis:
>> manageStorageServer(uri, connectionID):
>>
>> Parameters:
>> uri - a uri pointing to a storage target (eg: nfs://server:export, iscsi://host/iqn;portal=1)
>> connectionID - string with any char except "/".
>>
>> Description:
>> Tells VDSM to start managing the connection. From this moment on VDSM will try and have the connection available when needed. VDSM will monitor the connection and will automatically reconnect on failure.
>> Returns:
>> Success code if VDSM was able to manage the connection.
>> It usually just verifies that the arguments are sane and that the CID is not already in use.
>> This doesn't mean the host is connected.
>> ----
>> unmanageStorageServer
>> =====================
> 
> To match above: unmanageStorageConnection or unmanageStorageServerConnection
> 

+1

>> Synopsis:
>> unmanageStorageServer(connectionID):
>>
>> Parameters:
>> connectionID - string with any char except "/".
>>
>> Descriptions:
>> Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.
>>
>> Returns:
>> Success code if VDSM was able to unmanage the connection.
>> It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
>> ----
>> getStorageServerList
>> ====================
> 
> getStorageConnectionList or getStorageServerConnectionList
> 

+1

>> Synopsis:
>> getStorageServerList()
>>
>> Description:
>> Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.
>>
>> Results:VDSM was able to manage the connection.
>> It usually just verifies that the arguments are sane and that the CID is not already in use.
>> This doesn't mean the host is connected.
>> ----
>> unmanageStorageServer
>> =====================
>> Synopsis:
>> unmanageStorageServer(connectionID):
>>
>> Parameters:
>> connectionID - string with any char except "/".
>>
>> Descriptions:
>> Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.
>>
>> Returns:
>> Success code if VDSM was able to unmanage the connection.
>> It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
>> ----
>> getStorageServerList
>> ====================
>> Synopsis:
>> getStorageServerList()
>>
>> Description:
>> Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.
>>
>> Results:
>> A mapping between CIDs and the status.
>> example return value (Actual key names may differ)
>>
>> {'conA': {'connected': True, 'managed': True, 'lastError': 0, 'connectionInfo': {
>>     'remotePath': 'server:/export
>>     'retrans': 3
>>     'version': 4
>>     }}
>>  'iscsi_session_34': {'connected': False, 'managed': False, 'lastError': 339, 'connectionIfno': {
>>     'hostname': 'dandylopn'
>>     'portal': 1}}
>> }
>> _______________________________________________
>> vdsm-devel mailing list
>> vdsm-devel at lists.fedorahosted.org
>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> 



From smizrahi at redhat.com  Wed Jan 25 21:35:13 2012
From: smizrahi at redhat.com (Saggi Mizrahi)
Date: Wed, 25 Jan 2012 16:35:13 -0500 (EST)
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <4F206915.7050902@redhat.com>
Message-ID: <c058f09d-b88a-46aa-8a00-70ee398ad2e0@zmail16.collab.prod.int.phx2.redhat.com>

<SNIP>
This is mail was getting way too long.

About the clear all verb.
No.
Just loop, find the connections YOU OWN and clean them. Even though you don't want to support multiple clients to VDSM API doesn't mean the engine shouldn't behave like a proper citizen.
It's the same reason why VDSM tries and not mess system resources it didn't initiate.

------------------------

As I see it the only point of conflict is the so called non-peristed connections.
I will call them transient connections from now on.

There are 2 user cases being discussed
1. Wait until a connection is made, if it fails don't retry and automatically unmanage.
2. If the called of the API forgets or fails to unmanage a connection.

Your suggestion as I understand it:
Transient connections are:
     - Connection that VDSM will only try to connect to once and will not reconnect to in case of disconnect.

My problem with this definition that it does not specify the "end of life" of the connection.
Meaning it solves only use case 1. If all is well, and it usually is, VDSM will not invoke a disconnect.
So the caller would have to call unmanage if the connection succeeded at the end of the flow.
Now, if you are already calling unmanage if connection succeeded you can just call it anyway.

instead of doing: (with your suggestion)
----------------
manage
wait until succeeds or lastError has value
try:
  do stuff
finally:
  unmanage

do: (with the canonical flow)
---
manage
try:
  wait until succeeds or lastError has value
  do stuff
finally:
  unmanage

This is simpler to do than having another connection type.

Now that we got that out of the way lets talk about the 2nd use case.
API client died in the middle of the operation and unmanage was never called.

Your suggested definition means that unless there was a problem with the connection VDSM will still have this connection active. The engine will have to clean it anyway.

The problem is, VDSM has no way of knowing that a client died, forgot or is thinking really hard and will continue on in about 2 minutes.

Connections that live until they die is a hard to define and work with lifecycle. Solving this problem is theoretically simple.

Have clients hold some sort of session token and force the client to update it at a specified interval. You could bind resources (like domains, VMs, connections) to that session token so when it expires VDSM auto cleans the resources.

This kind of mechanism is out of the scope of this API change. Further more I think that this mechanism should sit in the engine since the session might actually contain resources from multiple hosts and resources that are not managed by VDSM.

In GUI flows specifically the user might do actions that don't even touch the engine and forcing it to refresh the engine token is simpler then having it refresh the VDSM token.

I understand that engine currently has no way of tracking a user session. This, as I said, is also true in the case of VDSM. We can start and argue about which project should implement the session semantics. But as I see it it's not relevant to the connection management API.


From lutter at redhat.com  Wed Jan 25 22:47:25 2012
From: lutter at redhat.com (David Lutterkort)
Date: Wed, 25 Jan 2012 14:47:25 -0800
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <4F1FD34F.4040801@redhat.com>
References: <ae15611f-e655-45d0-8c5f-264daac1a1d7@zmail13.collab.prod.int.phx2.redhat.com>
	<4F1FC6AE.7050900@redhat.com> <4F1FD34F.4040801@redhat.com>
Message-ID: <1327531645.2366.6.camel@avon.watzmann.net>

On Wed, 2012-01-25 at 12:02 +0200, Itamar Heim wrote:
> On 01/25/2012 11:09 AM, Dor Laor wrote:
> > On 01/22/2012 08:42 PM, Ayal Baron wrote:
> ...
> 
> >>>>> The following wiki page contains a description page of the
> >>>>> feature.
> >>>>> http://www.ovirt.org/wiki/Features/VMPayload
> >>>>>
> ...
> >
> > Currently it seems that there are too many options for it - floppy, cd,
> > nfs and maybe more. It would be nice to have a single option that works
> > for all cases. How about creating something like s3 compatible storage
> > access that the guest can access? If you need boot time access then I'll
> > recommend cdrom or plain virtio-blk.

I agree with Dor that there seems to be a large number of options here.
>From the Aeolus and Deltacloud perspective, we only need something that
makes that information available fairly late during boot (certainly
until after file systems have been mounted, even after network start
isn't a deal killer)

The payload data that the VM sees should not change when the VM is
rebooted or stopped/started.

> I think there are different use cases here:
> 1. floppy and iso cover the same use case, for similar needs (and behave 
> the same). this would cover windows sysprep approach and basic 
> attachment of files

Just picking one or the other should be sufficient.

> 2. http://192.169.192.168 - this would provide compatibility to cloud 
> approaches iiuc

Except the address is 169.254.169.254 (link-local) ;)

> 3. injecting into the file system - this covers various other needs, 
> like injecting ssh key, and is relevant not only during bootstrap, 
> should we want to allow editing a guest when it is down to troubleshoot 
> issues.

You don't need that as a feature for troubleshooting; I've unmangled EBS
root volumes in AWS before simply by mounting the EBS disk from another
machine.

The thing I don't like about file injection is that it's inherently
fragile, since oVirt needs to understand the intricacies of disk layout,
volume manager (as in lvm), and filesystem. 

Even worse if it is exposed via the API so that you can provide target
paths - now you've tightly coupled your API user to the OS inside the
VM.

I would only entertain (3) if there is an absolutely compelling use case
to do it.

David




From iheim at redhat.com  Thu Jan 26 08:33:20 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 26 Jan 2012 10:33:20 +0200
Subject: [Engine-devel] VM Payload feature
In-Reply-To: <1327531645.2366.6.camel@avon.watzmann.net>
References: <ae15611f-e655-45d0-8c5f-264daac1a1d7@zmail13.collab.prod.int.phx2.redhat.com>
	<4F1FC6AE.7050900@redhat.com> <4F1FD34F.4040801@redhat.com>
	<1327531645.2366.6.camel@avon.watzmann.net>
Message-ID: <4F210FD0.9060904@redhat.com>

On 01/26/2012 12:47 AM, David Lutterkort wrote:
> On Wed, 2012-01-25 at 12:02 +0200, Itamar Heim wrote:
>> On 01/25/2012 11:09 AM, Dor Laor wrote:
>>> On 01/22/2012 08:42 PM, Ayal Baron wrote:
>> ...
>>
>>>>>>> The following wiki page contains a description page of the
>>>>>>> feature.
>>>>>>> http://www.ovirt.org/wiki/Features/VMPayload
>>>>>>>
>> ...
>>>
>>> Currently it seems that there are too many options for it - floppy, cd,
>>> nfs and maybe more. It would be nice to have a single option that works
>>> for all cases. How about creating something like s3 compatible storage
>>> access that the guest can access? If you need boot time access then I'll
>>> recommend cdrom or plain virtio-blk.
>
> I agree with Dor that there seems to be a large number of options here.
>  From the Aeolus and Deltacloud perspective, we only need something that
> makes that information available fairly late during boot (certainly
> until after file systems have been mounted, even after network start
> isn't a deal killer)
>
> The payload data that the VM sees should not change when the VM is
> rebooted or stopped/started.
>
>> I think there are different use cases here:
>> 1. floppy and iso cover the same use case, for similar needs (and behave
>> the same). this would cover windows sysprep approach and basic
>> attachment of files
>
> Just picking one or the other should be sufficient.
>
>> 2. http://192.169.192.168 - this would provide compatibility to cloud
>> approaches iiuc
>
> Except the address is 169.254.169.254 (link-local) ;)
>
>> 3. injecting into the file system - this covers various other needs,
>> like injecting ssh key, and is relevant not only during bootstrap,
>> should we want to allow editing a guest when it is down to troubleshoot
>> issues.
>
> You don't need that as a feature for troubleshooting; I've unmangled EBS
> root volumes in AWS before simply by mounting the EBS disk from another
> machine.
>
> The thing I don't like about file injection is that it's inherently
> fragile, since oVirt needs to understand the intricacies of disk layout,
> volume manager (as in lvm), and filesystem.

true, and Rich Jones abstracts this or us via libguestfs and virt-tools.

>
> Even worse if it is exposed via the API so that you can provide target
> paths - now you've tightly coupled your API user to the OS inside the
> VM.

especially if we'll expose editing the windows registry.
but there are merits to allow editing the OS aside of boot strapping.

>
> I would only entertain (3) if there is an absolutely compelling use case
> to do it.

we already discussed covering only #1 for now, and looking at #2 in the 
future.
the drive for #3 will be based on need, which could be based on other 
aspects than boot time pay load.



From oschreib at redhat.com  Thu Jan 26 09:47:17 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Thu, 26 Jan 2012 11:47:17 +0200
Subject: [Engine-devel] First release go/no go meeting on Monday (30.01)
Message-ID: <4F212125.1060207@redhat.com>

All,

As decided in the last oVirt sync meeting, we will have a go/no meeting 
about our first release in the upcoming Monday (30.01.12), 15:00 UTC.

I'm inviting you all to join this important meeting in the official 
#ovirt irc channel.

Thanks,

Ofer Schreiber
oVirt Release Manager


From lpeer at redhat.com  Thu Jan 26 10:22:42 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Thu, 26 Jan 2012 12:22:42 +0200
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <c058f09d-b88a-46aa-8a00-70ee398ad2e0@zmail16.collab.prod.int.phx2.redhat.com>
References: <c058f09d-b88a-46aa-8a00-70ee398ad2e0@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F212972.3000400@redhat.com>

On 25/01/12 23:35, Saggi Mizrahi wrote:
> <SNIP>
> This is mail was getting way too long.
> 
> About the clear all verb.
> No.
> Just loop, find the connections YOU OWN and clean them. Even though you don't want to support multiple clients to VDSM API doesn't mean the engine shouldn't behave like a proper citizen.
> It's the same reason why VDSM tries and not mess system resources it didn't initiate.


There is a big difference, VDSM living in hybrid mode with other
workload on the host is a valid use case, having more than one
concurrent manager for VDSM is not.
Generating a disconnect request for each connection does not seem like
the right API to me, again think on the simple flow of moving host from
one data center to another, the engine needs to disconnect tall storage
domains (each domain can have couple of connections associated with it).

I am giving example from the engine use cases as it is the main user of
VDSM ATM but I am sure it will be relevant to any other user of VDSM.

> 
> ------------------------
> 
> As I see it the only point of conflict is the so called non-peristed connections.
> I will call them transient connections from now on.
> 
> There are 2 user cases being discussed
> 1. Wait until a connection is made, if it fails don't retry and automatically unmanage.
> 2. If the called of the API forgets or fails to unmanage a connection.
> 

Actually I was not discussing #2 at all.

> Your suggestion as I understand it:
> Transient connections are:
>      - Connection that VDSM will only try to connect to once and will not reconnect to in case of disconnect.

yes

> 
> My problem with this definition that it does not specify the "end of life" of the connection.
> Meaning it solves only use case 1.

since this is the only use case i had in mind, it is what i was looking for.

> If all is well, and it usually is, VDSM will not invoke a disconnect.
> So the caller would have to call unmanage if the connection succeeded at the end of the flow.

agree.

> Now, if you are already calling unmanage if connection succeeded you can just call it anyway.

not exactly, an example I gave earlier on the thread was that VSDM hangs
or have other error and the engine can not initiate unmanaged, instead
let's assume the host is fenced (self-fence or external fence does not
matter), in this scenario the engine will not issue unmanage.

> 
> instead of doing: (with your suggestion)
> ----------------
> manage
> wait until succeeds or lastError has value
> try:
>   do stuff
> finally:
>   unmanage
> 
> do: (with the canonical flow)
> ---
> manage
> try:
>   wait until succeeds or lastError has value
>   do stuff
> finally:
>   unmanage
> 
> This is simpler to do than having another connection type.

You are assuming the engine can communicate with VDSM and there are
scenarios where it is not feasible.

> 
> Now that we got that out of the way lets talk about the 2nd use case.

Since I did not ask VDSM to clean after the (engine) user and you don't
want to do it I am not sure we need to discuss this.

If you insist we can start the discussion on who should implement the
cleanup mechanism but I'm afraid I have no strong arguments for VDSM to
do it, so I rather not go there ;)


You dropped from the discussion my request for supporting list of
connections for manage and unmanage verbs.

> API client died in the middle of the operation and unmanage was never called.
> 
> Your suggested definition means that unless there was a problem with the connection VDSM will still have this connection active. The engine will have to clean it anyway.
> 
> The problem is, VDSM has no way of knowing that a client died, forgot or is thinking really hard and will continue on in about 2 minutes.

> 
> Connections that live until they die is a hard to define and work with lifecycle. Solving this problem is theoretically simple.
> 
> Have clients hold some sort of session token and force the client to update it at a specified interval. You could bind resources (like domains, VMs, connections) to that session token so when it expires VDSM auto cleans the resources.
> 
> This kind of mechanism is out of the scope of this API change. Further more I think that this mechanism should sit in the engine since the session might actually contain resources from multiple hosts and resources that are not managed by VDSM.
> 
> In GUI flows specifically the user might do actions that don't even touch the engine and forcing it to refresh the engine token is simpler then having it refresh the VDSM token.
> 
> I understand that engine currently has no way of tracking a user session. This, as I said, is also true in the case of VDSM. We can start and argue about which project should implement the session semantics. But as I see it it's not relevant to the connection management API.



From agl at us.ibm.com  Thu Jan 26 14:01:08 2012
From: agl at us.ibm.com (Adam Litke)
Date: Thu, 26 Jan 2012 08:01:08 -0600
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <4F212972.3000400@redhat.com>
References: <c058f09d-b88a-46aa-8a00-70ee398ad2e0@zmail16.collab.prod.int.phx2.redhat.com>
	<4F212972.3000400@redhat.com>
Message-ID: <20120126140108.GE4837@us.ibm.com>

On Thu, Jan 26, 2012 at 12:22:42PM +0200, Livnat Peer wrote:
> On 25/01/12 23:35, Saggi Mizrahi wrote:
> > <SNIP>
> > This is mail was getting way too long.
> > 
> > About the clear all verb.
> > No.
> > Just loop, find the connections YOU OWN and clean them. Even though you don't want to support multiple clients to VDSM API doesn't mean the engine shouldn't behave like a proper citizen.
> > It's the same reason why VDSM tries and not mess system resources it didn't initiate.
> 
> 
> There is a big difference, VDSM living in hybrid mode with other
> workload on the host is a valid use case, having more than one
> concurrent manager for VDSM is not.
> Generating a disconnect request for each connection does not seem like
> the right API to me, again think on the simple flow of moving host from
> one data center to another, the engine needs to disconnect tall storage
> domains (each domain can have couple of connections associated with it).
> 
> I am giving example from the engine use cases as it is the main user of
> VDSM ATM but I am sure it will be relevant to any other user of VDSM.

I will speak up to represent other potential users of the VDSM API.  My vote is
with Saggi here to keep the API simple and have an unmanage call that operates
on a single connection only.  Every programming language has looping constructs
that make it easy to implement unmanageAll.  Why clog up vdsm's API with an
extra function just to avoid writing a 'for' loop?

> 
> > 
> > ------------------------
> > 
> > As I see it the only point of conflict is the so called non-peristed connections.
> > I will call them transient connections from now on.
> > 
> > There are 2 user cases being discussed
> > 1. Wait until a connection is made, if it fails don't retry and automatically unmanage.
> > 2. If the called of the API forgets or fails to unmanage a connection.
> > 
> 
> Actually I was not discussing #2 at all.
> 
> > Your suggestion as I understand it:
> > Transient connections are:
> >      - Connection that VDSM will only try to connect to once and will not reconnect to in case of disconnect.
> 
> yes
> 
> > 
> > My problem with this definition that it does not specify the "end of life" of the connection.
> > Meaning it solves only use case 1.
> 
> since this is the only use case i had in mind, it is what i was looking for.
> 
> > If all is well, and it usually is, VDSM will not invoke a disconnect.
> > So the caller would have to call unmanage if the connection succeeded at the end of the flow.
> 
> agree.
> 
> > Now, if you are already calling unmanage if connection succeeded you can just call it anyway.
> 
> not exactly, an example I gave earlier on the thread was that VSDM hangs
> or have other error and the engine can not initiate unmanaged, instead
> let's assume the host is fenced (self-fence or external fence does not
> matter), in this scenario the engine will not issue unmanage.
> 
> > 
> > instead of doing: (with your suggestion)
> > ----------------
> > manage
> > wait until succeeds or lastError has value
> > try:
> >   do stuff
> > finally:
> >   unmanage
> > 
> > do: (with the canonical flow)
> > ---
> > manage
> > try:
> >   wait until succeeds or lastError has value
> >   do stuff
> > finally:
> >   unmanage
> > 
> > This is simpler to do than having another connection type.
> 
> You are assuming the engine can communicate with VDSM and there are
> scenarios where it is not feasible.
> 
> > 
> > Now that we got that out of the way lets talk about the 2nd use case.
> 
> Since I did not ask VDSM to clean after the (engine) user and you don't
> want to do it I am not sure we need to discuss this.
> 
> If you insist we can start the discussion on who should implement the
> cleanup mechanism but I'm afraid I have no strong arguments for VDSM to
> do it, so I rather not go there ;)
> 
> 
> You dropped from the discussion my request for supporting list of
> connections for manage and unmanage verbs.
> 
> > API client died in the middle of the operation and unmanage was never called.
> > 
> > Your suggested definition means that unless there was a problem with the connection VDSM will still have this connection active. The engine will have to clean it anyway.
> > 
> > The problem is, VDSM has no way of knowing that a client died, forgot or is thinking really hard and will continue on in about 2 minutes.
> 
> > 
> > Connections that live until they die is a hard to define and work with lifecycle. Solving this problem is theoretically simple.
> > 
> > Have clients hold some sort of session token and force the client to update it at a specified interval. You could bind resources (like domains, VMs, connections) to that session token so when it expires VDSM auto cleans the resources.
> > 
> > This kind of mechanism is out of the scope of this API change. Further more I think that this mechanism should sit in the engine since the session might actually contain resources from multiple hosts and resources that are not managed by VDSM.
> > 
> > In GUI flows specifically the user might do actions that don't even touch the engine and forcing it to refresh the engine token is simpler then having it refresh the VDSM token.
> > 
> > I understand that engine currently has no way of tracking a user session. This, as I said, is also true in the case of VDSM. We can start and argue about which project should implement the session semantics. But as I see it it's not relevant to the connection management API.
> 
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/vdsm-devel

-- 
Adam Litke <agl at us.ibm.com>
IBM Linux Technology Center



From juan.hernandez at redhat.com  Thu Jan 26 14:21:15 2012
From: juan.hernandez at redhat.com (Juan Hernandez)
Date: Thu, 26 Jan 2012 15:21:15 +0100
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <20120126140108.GE4837@us.ibm.com>
References: <c058f09d-b88a-46aa-8a00-70ee398ad2e0@zmail16.collab.prod.int.phx2.redhat.com>
	<4F212972.3000400@redhat.com> <20120126140108.GE4837@us.ibm.com>
Message-ID: <4F21615B.9040106@redhat.com>

On 01/26/2012 03:01 PM, Adam Litke wrote:
> On Thu, Jan 26, 2012 at 12:22:42PM +0200, Livnat Peer wrote:
>> On 25/01/12 23:35, Saggi Mizrahi wrote:
>>> About the clear all verb.
>>> No.
>>> Just loop, find the connections YOU OWN and clean them. Even though you don't want to support multiple clients to VDSM API doesn't mean the engine shouldn't behave like a proper citizen.
>>> It's the same reason why VDSM tries and not mess system resources it didn't initiate.
>>
>>
>> There is a big difference, VDSM living in hybrid mode with other
>> workload on the host is a valid use case, having more than one
>> concurrent manager for VDSM is not.
>> Generating a disconnect request for each connection does not seem like
>> the right API to me, again think on the simple flow of moving host from
>> one data center to another, the engine needs to disconnect tall storage
>> domains (each domain can have couple of connections associated with it).
>>
>> I am giving example from the engine use cases as it is the main user of
>> VDSM ATM but I am sure it will be relevant to any other user of VDSM.
> 
> I will speak up to represent other potential users of the VDSM API.  My vote is
> with Saggi here to keep the API simple and have an unmanage call that operates
> on a single connection only.  Every programming language has looping constructs
> that make it easy to implement unmanageAll.  Why clog up vdsm's API with an
> extra function just to avoid writing a 'for' loop?

A very good reason to have that kind of bulk operations (in general,
maybe not in this case) is to reduce the number of network round trips
and thus improve performance. The loop is very easy to write, and very
expensive to execute.


From ykaul at redhat.com  Thu Jan 26 14:23:54 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Thu, 26 Jan 2012 16:23:54 +0200
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <4F21615B.9040106@redhat.com>
References: <c058f09d-b88a-46aa-8a00-70ee398ad2e0@zmail16.collab.prod.int.phx2.redhat.com>
	<4F212972.3000400@redhat.com> <20120126140108.GE4837@us.ibm.com>
	<4F21615B.9040106@redhat.com>
Message-ID: <4F2161FA.7010009@redhat.com>

On 01/26/2012 04:21 PM, Juan Hernandez wrote:
> On 01/26/2012 03:01 PM, Adam Litke wrote:
>> On Thu, Jan 26, 2012 at 12:22:42PM +0200, Livnat Peer wrote:
>>> On 25/01/12 23:35, Saggi Mizrahi wrote:
>>>> About the clear all verb.
>>>> No.
>>>> Just loop, find the connections YOU OWN and clean them. Even though you don't want to support multiple clients to VDSM API doesn't mean the engine shouldn't behave like a proper citizen.
>>>> It's the same reason why VDSM tries and not mess system resources it didn't initiate.
>>>
>>> There is a big difference, VDSM living in hybrid mode with other
>>> workload on the host is a valid use case, having more than one
>>> concurrent manager for VDSM is not.
>>> Generating a disconnect request for each connection does not seem like
>>> the right API to me, again think on the simple flow of moving host from
>>> one data center to another, the engine needs to disconnect tall storage
>>> domains (each domain can have couple of connections associated with it).
>>>
>>> I am giving example from the engine use cases as it is the main user of
>>> VDSM ATM but I am sure it will be relevant to any other user of VDSM.
>> I will speak up to represent other potential users of the VDSM API.  My vote is
>> with Saggi here to keep the API simple and have an unmanage call that operates
>> on a single connection only.  Every programming language has looping constructs
>> that make it easy to implement unmanageAll.  Why clog up vdsm's API with an
>> extra function just to avoid writing a 'for' loop?
> A very good reason to have that kind of bulk operations (in general,
> maybe not in this case) is to reduce the number of network round trips
> and thus improve performance. The loop is very easy to write, and very
> expensive to execute.

It's expensive mainly because we do not have a persistent connection 
between VDSM and Engine, and that it's not compressed (with TLS 
compression or internally).
Y.

> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From smizrahi at redhat.com  Thu Jan 26 15:00:57 2012
From: smizrahi at redhat.com (Saggi Mizrahi)
Date: Thu, 26 Jan 2012 10:00:57 -0500 (EST)
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <4F212972.3000400@redhat.com>
Message-ID: <9c7c59ea-7d35-4b70-9447-041b8e8c3246@zmail16.collab.prod.int.phx2.redhat.com>

<snip>
Again trying to sum up and address all comments

Clear all:
==========
My opinions is still to not implement it.
Even though it might generate a bit more traffic premature optimization is bad and there are other reasons we can improve VDSM command overhead without doing this.

In any case this argument is redundant because my intention is (as Litke pointed out) is to have a lean API.
and API call is something you have to support across versions, this call implemented in the engine is something that no one has to support and can change\evolve easily.

As a rule, if an API call C and be implemented by doing A + B then C is redundant.

List of connections as args:
============================
Sorry I forgot to respond about that. I'm not as strongly opposed to the idea as the other things you suggested. It'll just make implementing the persistence logic in VDSM significantly more complicated as I will have to commit multiple connection information to disk in an all or nothing mode. I can create a small sqlitedb to do that or do some directory tricks and exploit FS rename atomicity but I'd rather not.

The demands are not without base. I would like to keep the code simple under the hood in the price of a few more calls. You would like to make less calls and keep the code simpler on your side. There isn't a real way to settle this.
If anyone on the list as pros and cons for either way I'd be happy to hear them.
If no compelling arguments arise I will let Ayal call this one.

Transient connections:
======================
The problem you are describing as I understand it is that VDSM did not respond and not that the API client did not respond.
Again, this can happen for a number of reason, most of which VDSM might not be aware that there is actually a problem (network issues).

This relates to the EOL policy. I agree we have to find a good way to define an automatic EOL for resources. I have made my suggestion. Out of the scope of the API.

In the meantime cleaning stale connections is trivial and I have made it clear a previous email about how to go about it in a simple non intrusive way. Clean hosts on host connect, and on every poll if you find connections that you don't like. This should keep things squeaky clean.

----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Saggi Mizrahi" <smizrahi at redhat.com>
> Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
> Sent: Thursday, January 26, 2012 5:22:42 AM
> Subject: Re: [Engine-devel] [RFC] New Connection Management API
> 
> On 25/01/12 23:35, Saggi Mizrahi wrote:
> > <SNIP>
> > This is mail was getting way too long.
> > 
> > About the clear all verb.
> > No.
> > Just loop, find the connections YOU OWN and clean them. Even though
> > you don't want to support multiple clients to VDSM API doesn't
> > mean the engine shouldn't behave like a proper citizen.
> > It's the same reason why VDSM tries and not mess system resources
> > it didn't initiate.
> 
> 
> There is a big difference, VDSM living in hybrid mode with other
> workload on the host is a valid use case, having more than one
> concurrent manager for VDSM is not.
> Generating a disconnect request for each connection does not seem
> like
> the right API to me, again think on the simple flow of moving host
> from
> one data center to another, the engine needs to disconnect tall
> storage
> domains (each domain can have couple of connections associated with
> it).
> 
> I am giving example from the engine use cases as it is the main user
> of
> VDSM ATM but I am sure it will be relevant to any other user of VDSM.
> 
> > 
> > ------------------------
> > 
> > As I see it the only point of conflict is the so called
> > non-peristed connections.
> > I will call them transient connections from now on.
> > 
> > There are 2 user cases being discussed
> > 1. Wait until a connection is made, if it fails don't retry and
> > automatically unmanage.
> > 2. If the called of the API forgets or fails to unmanage a
> > connection.
> > 
> 
> Actually I was not discussing #2 at all.
> 
> > Your suggestion as I understand it:
> > Transient connections are:
> >      - Connection that VDSM will only try to connect to once and
> >      will not reconnect to in case of disconnect.
> 
> yes
> 
> > 
> > My problem with this definition that it does not specify the "end
> > of life" of the connection.
> > Meaning it solves only use case 1.
> 
> since this is the only use case i had in mind, it is what i was
> looking for.
> 
> > If all is well, and it usually is, VDSM will not invoke a
> > disconnect.
> > So the caller would have to call unmanage if the connection
> > succeeded at the end of the flow.
> 
> agree.
> 
> > Now, if you are already calling unmanage if connection succeeded
> > you can just call it anyway.
> 
> not exactly, an example I gave earlier on the thread was that VSDM
> hangs
> or have other error and the engine can not initiate unmanaged,
> instead
> let's assume the host is fenced (self-fence or external fence does
> not
> matter), in this scenario the engine will not issue unmanage.
> 
> > 
> > instead of doing: (with your suggestion)
> > ----------------
> > manage
> > wait until succeeds or lastError has value
> > try:
> >   do stuff
> > finally:
> >   unmanage
> > 
> > do: (with the canonical flow)
> > ---
> > manage
> > try:
> >   wait until succeeds or lastError has value
> >   do stuff
> > finally:
> >   unmanage
> > 
> > This is simpler to do than having another connection type.
> 
> You are assuming the engine can communicate with VDSM and there are
> scenarios where it is not feasible.
> 
> > 
> > Now that we got that out of the way lets talk about the 2nd use
> > case.
> 
> Since I did not ask VDSM to clean after the (engine) user and you
> don't
> want to do it I am not sure we need to discuss this.
> 
> If you insist we can start the discussion on who should implement the
> cleanup mechanism but I'm afraid I have no strong arguments for VDSM
> to
> do it, so I rather not go there ;)
> 
> 
> You dropped from the discussion my request for supporting list of
> connections for manage and unmanage verbs.
> 
> > API client died in the middle of the operation and unmanage was
> > never called.
> > 
> > Your suggested definition means that unless there was a problem
> > with the connection VDSM will still have this connection active.
> > The engine will have to clean it anyway.
> > 
> > The problem is, VDSM has no way of knowing that a client died,
> > forgot or is thinking really hard and will continue on in about 2
> > minutes.
> 
> > 
> > Connections that live until they die is a hard to define and work
> > with lifecycle. Solving this problem is theoretically simple.
> > 
> > Have clients hold some sort of session token and force the client
> > to update it at a specified interval. You could bind resources
> > (like domains, VMs, connections) to that session token so when it
> > expires VDSM auto cleans the resources.
> > 
> > This kind of mechanism is out of the scope of this API change.
> > Further more I think that this mechanism should sit in the engine
> > since the session might actually contain resources from multiple
> > hosts and resources that are not managed by VDSM.
> > 
> > In GUI flows specifically the user might do actions that don't even
> > touch the engine and forcing it to refresh the engine token is
> > simpler then having it refresh the VDSM token.
> > 
> > I understand that engine currently has no way of tracking a user
> > session. This, as I said, is also true in the case of VDSM. We can
> > start and argue about which project should implement the session
> > semantics. But as I see it it's not relevant to the connection
> > management API.
> 
> 


From agl at us.ibm.com  Thu Jan 26 18:58:40 2012
From: agl at us.ibm.com (Adam Litke)
Date: Thu, 26 Jan 2012 12:58:40 -0600
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <9c7c59ea-7d35-4b70-9447-041b8e8c3246@zmail16.collab.prod.int.phx2.redhat.com>
References: <4F212972.3000400@redhat.com>
	<9c7c59ea-7d35-4b70-9447-041b8e8c3246@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <20120126185840.GG4837@us.ibm.com>

On Thu, Jan 26, 2012 at 10:00:57AM -0500, Saggi Mizrahi wrote:
> <snip>
> Again trying to sum up and address all comments
> 
> Clear all:
> ==========
> My opinions is still to not implement it.
> Even though it might generate a bit more traffic premature optimization is bad and there are other reasons we can improve VDSM command overhead without doing this.
> 
> In any case this argument is redundant because my intention is (as Litke pointed out) is to have a lean API.
> and API call is something you have to support across versions, this call implemented in the engine is something that no one has to support and can change\evolve easily.
> 
> As a rule, if an API call C and be implemented by doing A + B then C is redundant.
> 
> List of connections as args:
> ============================
> Sorry I forgot to respond about that. I'm not as strongly opposed to the idea as the other things you suggested. It'll just make implementing the persistence logic in VDSM significantly more complicated as I will have to commit multiple connection information to disk in an all or nothing mode. I can create a small sqlitedb to do that or do some directory tricks and exploit FS rename atomicity but I'd rather not.

I would be strongly opposed to introducing a sqlite database into vdsm just to
enable "convenience mode" for this API.  Does the operation really need to be
atomic?  Why not just perform each connection sequentially and return a list of
statuses?  Is the only motivation for allowing a list of parameters to reduce
the number of API calls between engine and vdsm)?  If so, the same argument
Saggi makes above applies here.

> The demands are not without base. I would like to keep the code simple under the hood in the price of a few more calls. You would like to make less calls and keep the code simpler on your side. There isn't a real way to settle this.
> If anyone on the list as pros and cons for either way I'd be happy to hear them.
> If no compelling arguments arise I will let Ayal call this one.
> 
> Transient connections:
> ======================
> The problem you are describing as I understand it is that VDSM did not respond and not that the API client did not respond.
> Again, this can happen for a number of reason, most of which VDSM might not be aware that there is actually a problem (network issues).
> 
> This relates to the EOL policy. I agree we have to find a good way to define an automatic EOL for resources. I have made my suggestion. Out of the scope of the API.
> 
> In the meantime cleaning stale connections is trivial and I have made it clear a previous email about how to go about it in a simple non intrusive way. Clean hosts on host connect, and on every poll if you find connections that you don't like. This should keep things squeaky clean.
> 
> ----- Original Message -----
> > From: "Livnat Peer" <lpeer at redhat.com>
> > To: "Saggi Mizrahi" <smizrahi at redhat.com>
> > Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
> > Sent: Thursday, January 26, 2012 5:22:42 AM
> > Subject: Re: [Engine-devel] [RFC] New Connection Management API
> > 
> > On 25/01/12 23:35, Saggi Mizrahi wrote:
> > > <SNIP>
> > > This is mail was getting way too long.
> > > 
> > > About the clear all verb.
> > > No.
> > > Just loop, find the connections YOU OWN and clean them. Even though
> > > you don't want to support multiple clients to VDSM API doesn't
> > > mean the engine shouldn't behave like a proper citizen.
> > > It's the same reason why VDSM tries and not mess system resources
> > > it didn't initiate.
> > 
> > 
> > There is a big difference, VDSM living in hybrid mode with other
> > workload on the host is a valid use case, having more than one
> > concurrent manager for VDSM is not.
> > Generating a disconnect request for each connection does not seem
> > like
> > the right API to me, again think on the simple flow of moving host
> > from
> > one data center to another, the engine needs to disconnect tall
> > storage
> > domains (each domain can have couple of connections associated with
> > it).
> > 
> > I am giving example from the engine use cases as it is the main user
> > of
> > VDSM ATM but I am sure it will be relevant to any other user of VDSM.
> > 
> > > 
> > > ------------------------
> > > 
> > > As I see it the only point of conflict is the so called
> > > non-peristed connections.
> > > I will call them transient connections from now on.
> > > 
> > > There are 2 user cases being discussed
> > > 1. Wait until a connection is made, if it fails don't retry and
> > > automatically unmanage.
> > > 2. If the called of the API forgets or fails to unmanage a
> > > connection.
> > > 
> > 
> > Actually I was not discussing #2 at all.
> > 
> > > Your suggestion as I understand it:
> > > Transient connections are:
> > >      - Connection that VDSM will only try to connect to once and
> > >      will not reconnect to in case of disconnect.
> > 
> > yes
> > 
> > > 
> > > My problem with this definition that it does not specify the "end
> > > of life" of the connection.
> > > Meaning it solves only use case 1.
> > 
> > since this is the only use case i had in mind, it is what i was
> > looking for.
> > 
> > > If all is well, and it usually is, VDSM will not invoke a
> > > disconnect.
> > > So the caller would have to call unmanage if the connection
> > > succeeded at the end of the flow.
> > 
> > agree.
> > 
> > > Now, if you are already calling unmanage if connection succeeded
> > > you can just call it anyway.
> > 
> > not exactly, an example I gave earlier on the thread was that VSDM
> > hangs
> > or have other error and the engine can not initiate unmanaged,
> > instead
> > let's assume the host is fenced (self-fence or external fence does
> > not
> > matter), in this scenario the engine will not issue unmanage.
> > 
> > > 
> > > instead of doing: (with your suggestion)
> > > ----------------
> > > manage
> > > wait until succeeds or lastError has value
> > > try:
> > >   do stuff
> > > finally:
> > >   unmanage
> > > 
> > > do: (with the canonical flow)
> > > ---
> > > manage
> > > try:
> > >   wait until succeeds or lastError has value
> > >   do stuff
> > > finally:
> > >   unmanage
> > > 
> > > This is simpler to do than having another connection type.
> > 
> > You are assuming the engine can communicate with VDSM and there are
> > scenarios where it is not feasible.
> > 
> > > 
> > > Now that we got that out of the way lets talk about the 2nd use
> > > case.
> > 
> > Since I did not ask VDSM to clean after the (engine) user and you
> > don't
> > want to do it I am not sure we need to discuss this.
> > 
> > If you insist we can start the discussion on who should implement the
> > cleanup mechanism but I'm afraid I have no strong arguments for VDSM
> > to
> > do it, so I rather not go there ;)
> > 
> > 
> > You dropped from the discussion my request for supporting list of
> > connections for manage and unmanage verbs.
> > 
> > > API client died in the middle of the operation and unmanage was
> > > never called.
> > > 
> > > Your suggested definition means that unless there was a problem
> > > with the connection VDSM will still have this connection active.
> > > The engine will have to clean it anyway.
> > > 
> > > The problem is, VDSM has no way of knowing that a client died,
> > > forgot or is thinking really hard and will continue on in about 2
> > > minutes.
> > 
> > > 
> > > Connections that live until they die is a hard to define and work
> > > with lifecycle. Solving this problem is theoretically simple.
> > > 
> > > Have clients hold some sort of session token and force the client
> > > to update it at a specified interval. You could bind resources
> > > (like domains, VMs, connections) to that session token so when it
> > > expires VDSM auto cleans the resources.
> > > 
> > > This kind of mechanism is out of the scope of this API change.
> > > Further more I think that this mechanism should sit in the engine
> > > since the session might actually contain resources from multiple
> > > hosts and resources that are not managed by VDSM.
> > > 
> > > In GUI flows specifically the user might do actions that don't even
> > > touch the engine and forcing it to refresh the engine token is
> > > simpler then having it refresh the VDSM token.
> > > 
> > > I understand that engine currently has no way of tracking a user
> > > session. This, as I said, is also true in the case of VDSM. We can
> > > start and argue about which project should implement the session
> > > semantics. But as I see it it's not relevant to the connection
> > > management API.
> > 
> > 
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/vdsm-devel

-- 
Adam Litke <agl at us.ibm.com>
IBM Linux Technology Center



From smizrahi at redhat.com  Thu Jan 26 19:21:10 2012
From: smizrahi at redhat.com (Saggi Mizrahi)
Date: Thu, 26 Jan 2012 14:21:10 -0500 (EST)
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <20120126185840.GG4837@us.ibm.com>
Message-ID: <fea9a91c-72d6-43e8-b213-94767fa538e2@zmail16.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Adam Litke" <agl at us.ibm.com>
> To: "Saggi Mizrahi" <smizrahi at redhat.com>
> Cc: "Livnat Peer" <lpeer at redhat.com>, engine-devel at ovirt.org, vdsm-devel at lists.fedorahosted.org
> Sent: Thursday, January 26, 2012 1:58:40 PM
> Subject: Re: [vdsm] [Engine-devel] [RFC] New Connection Management API
> 
> On Thu, Jan 26, 2012 at 10:00:57AM -0500, Saggi Mizrahi wrote:
> > <snip>
> > Again trying to sum up and address all comments
> > 
> > Clear all:
> > ==========
> > My opinions is still to not implement it.
> > Even though it might generate a bit more traffic premature
> > optimization is bad and there are other reasons we can improve
> > VDSM command overhead without doing this.
> > 
> > In any case this argument is redundant because my intention is (as
> > Litke pointed out) is to have a lean API.
> > and API call is something you have to support across versions, this
> > call implemented in the engine is something that no one has to
> > support and can change\evolve easily.
> > 
> > As a rule, if an API call C and be implemented by doing A + B then
> > C is redundant.
> > 
> > List of connections as args:
> > ============================
> > Sorry I forgot to respond about that. I'm not as strongly opposed
> > to the idea as the other things you suggested. It'll just make
> > implementing the persistence logic in VDSM significantly more
> > complicated as I will have to commit multiple connection
> > information to disk in an all or nothing mode. I can create a
> > small sqlitedb to do that or do some directory tricks and exploit
> > FS rename atomicity but I'd rather not.
> 
> I would be strongly opposed to introducing a sqlite database into
> vdsm just to
> enable "convenience mode" for this API.  Does the operation really
> need to be
> atomic?  Why not just perform each connection sequentially and return
> a list of
> statuses? Is the only motivation for allowing a list of parameters
> to reduce
> the number of API calls between engine and vdsm)?  If so, the same
> argument
> Saggi makes above applies here.

I try and have VDSM expose APIs that are simple to predict. a command can either succeed or fail.
The problem is not actually validating the connections. The problem is that once I concluded that they are all OK I need to persist to disk the information that will allow me to reconnect if VDSM happens to crash. If I naively save them one by one I could get in a state where only some of the connections persisted before the operation failed. So I have to somehow put all this in a transaction.

I don't have to use sqlite. I could also put all the persistence information in a new dir for every call named <UUID>.tmp. Once I wrote everything down I rename the directory to just <UUID> and fsync it. This is guarantied by posix to be atomic. For unmanage, I move all the persistence information from the directories they sit in to a new dir named <UUID>. Rename it to a <UUDI>.tmp, fsync it and then remove it.

This all just looks like more trouble then it's worth to me.

> 
> > The demands are not without base. I would like to keep the code
> > simple under the hood in the price of a few more calls. You would
> > like to make less calls and keep the code simpler on your side.
> > There isn't a real way to settle this.
> > If anyone on the list as pros and cons for either way I'd be happy
> > to hear them.
> > If no compelling arguments arise I will let Ayal call this one.
> > 
> > Transient connections:
> > ======================
> > The problem you are describing as I understand it is that VDSM did
> > not respond and not that the API client did not respond.
> > Again, this can happen for a number of reason, most of which VDSM
> > might not be aware that there is actually a problem (network
> > issues).
> > 
> > This relates to the EOL policy. I agree we have to find a good way
> > to define an automatic EOL for resources. I have made my
> > suggestion. Out of the scope of the API.
> > 
> > In the meantime cleaning stale connections is trivial and I have
> > made it clear a previous email about how to go about it in a
> > simple non intrusive way. Clean hosts on host connect, and on
> > every poll if you find connections that you don't like. This
> > should keep things squeaky clean.
> > 
> > ----- Original Message -----
> > > From: "Livnat Peer" <lpeer at redhat.com>
> > > To: "Saggi Mizrahi" <smizrahi at redhat.com>
> > > Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
> > > Sent: Thursday, January 26, 2012 5:22:42 AM
> > > Subject: Re: [Engine-devel] [RFC] New Connection Management API
> > > 
> > > On 25/01/12 23:35, Saggi Mizrahi wrote:
> > > > <SNIP>
> > > > This is mail was getting way too long.
> > > > 
> > > > About the clear all verb.
> > > > No.
> > > > Just loop, find the connections YOU OWN and clean them. Even
> > > > though
> > > > you don't want to support multiple clients to VDSM API doesn't
> > > > mean the engine shouldn't behave like a proper citizen.
> > > > It's the same reason why VDSM tries and not mess system
> > > > resources
> > > > it didn't initiate.
> > > 
> > > 
> > > There is a big difference, VDSM living in hybrid mode with other
> > > workload on the host is a valid use case, having more than one
> > > concurrent manager for VDSM is not.
> > > Generating a disconnect request for each connection does not seem
> > > like
> > > the right API to me, again think on the simple flow of moving
> > > host
> > > from
> > > one data center to another, the engine needs to disconnect tall
> > > storage
> > > domains (each domain can have couple of connections associated
> > > with
> > > it).
> > > 
> > > I am giving example from the engine use cases as it is the main
> > > user
> > > of
> > > VDSM ATM but I am sure it will be relevant to any other user of
> > > VDSM.
> > > 
> > > > 
> > > > ------------------------
> > > > 
> > > > As I see it the only point of conflict is the so called
> > > > non-peristed connections.
> > > > I will call them transient connections from now on.
> > > > 
> > > > There are 2 user cases being discussed
> > > > 1. Wait until a connection is made, if it fails don't retry and
> > > > automatically unmanage.
> > > > 2. If the called of the API forgets or fails to unmanage a
> > > > connection.
> > > > 
> > > 
> > > Actually I was not discussing #2 at all.
> > > 
> > > > Your suggestion as I understand it:
> > > > Transient connections are:
> > > >      - Connection that VDSM will only try to connect to once
> > > >      and
> > > >      will not reconnect to in case of disconnect.
> > > 
> > > yes
> > > 
> > > > 
> > > > My problem with this definition that it does not specify the
> > > > "end
> > > > of life" of the connection.
> > > > Meaning it solves only use case 1.
> > > 
> > > since this is the only use case i had in mind, it is what i was
> > > looking for.
> > > 
> > > > If all is well, and it usually is, VDSM will not invoke a
> > > > disconnect.
> > > > So the caller would have to call unmanage if the connection
> > > > succeeded at the end of the flow.
> > > 
> > > agree.
> > > 
> > > > Now, if you are already calling unmanage if connection
> > > > succeeded
> > > > you can just call it anyway.
> > > 
> > > not exactly, an example I gave earlier on the thread was that
> > > VSDM
> > > hangs
> > > or have other error and the engine can not initiate unmanaged,
> > > instead
> > > let's assume the host is fenced (self-fence or external fence
> > > does
> > > not
> > > matter), in this scenario the engine will not issue unmanage.
> > > 
> > > > 
> > > > instead of doing: (with your suggestion)
> > > > ----------------
> > > > manage
> > > > wait until succeeds or lastError has value
> > > > try:
> > > >   do stuff
> > > > finally:
> > > >   unmanage
> > > > 
> > > > do: (with the canonical flow)
> > > > ---
> > > > manage
> > > > try:
> > > >   wait until succeeds or lastError has value
> > > >   do stuff
> > > > finally:
> > > >   unmanage
> > > > 
> > > > This is simpler to do than having another connection type.
> > > 
> > > You are assuming the engine can communicate with VDSM and there
> > > are
> > > scenarios where it is not feasible.
> > > 
> > > > 
> > > > Now that we got that out of the way lets talk about the 2nd use
> > > > case.
> > > 
> > > Since I did not ask VDSM to clean after the (engine) user and you
> > > don't
> > > want to do it I am not sure we need to discuss this.
> > > 
> > > If you insist we can start the discussion on who should implement
> > > the
> > > cleanup mechanism but I'm afraid I have no strong arguments for
> > > VDSM
> > > to
> > > do it, so I rather not go there ;)
> > > 
> > > 
> > > You dropped from the discussion my request for supporting list of
> > > connections for manage and unmanage verbs.
> > > 
> > > > API client died in the middle of the operation and unmanage was
> > > > never called.
> > > > 
> > > > Your suggested definition means that unless there was a problem
> > > > with the connection VDSM will still have this connection
> > > > active.
> > > > The engine will have to clean it anyway.
> > > > 
> > > > The problem is, VDSM has no way of knowing that a client died,
> > > > forgot or is thinking really hard and will continue on in about
> > > > 2
> > > > minutes.
> > > 
> > > > 
> > > > Connections that live until they die is a hard to define and
> > > > work
> > > > with lifecycle. Solving this problem is theoretically simple.
> > > > 
> > > > Have clients hold some sort of session token and force the
> > > > client
> > > > to update it at a specified interval. You could bind resources
> > > > (like domains, VMs, connections) to that session token so when
> > > > it
> > > > expires VDSM auto cleans the resources.
> > > > 
> > > > This kind of mechanism is out of the scope of this API change.
> > > > Further more I think that this mechanism should sit in the
> > > > engine
> > > > since the session might actually contain resources from
> > > > multiple
> > > > hosts and resources that are not managed by VDSM.
> > > > 
> > > > In GUI flows specifically the user might do actions that don't
> > > > even
> > > > touch the engine and forcing it to refresh the engine token is
> > > > simpler then having it refresh the VDSM token.
> > > > 
> > > > I understand that engine currently has no way of tracking a
> > > > user
> > > > session. This, as I said, is also true in the case of VDSM. We
> > > > can
> > > > start and argue about which project should implement the
> > > > session
> > > > semantics. But as I see it it's not relevant to the connection
> > > > management API.
> > > 
> > > 
> > _______________________________________________
> > vdsm-devel mailing list
> > vdsm-devel at lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/vdsm-devel
> 
> --
> Adam Litke <agl at us.ibm.com>
> IBM Linux Technology Center
> 
> 


From lpeer at redhat.com  Thu Jan 26 20:03:39 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Thu, 26 Jan 2012 22:03:39 +0200
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <9c7c59ea-7d35-4b70-9447-041b8e8c3246@zmail16.collab.prod.int.phx2.redhat.com>
References: <9c7c59ea-7d35-4b70-9447-041b8e8c3246@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F21B19B.40802@redhat.com>

On 26/01/12 17:00, Saggi Mizrahi wrote:
> <snip>
> Again trying to sum up and address all comments
> 
> Clear all:
> ==========
> My opinions is still to not implement it.
> Even though it might generate a bit more traffic premature optimization is bad and there are other reasons we can improve VDSM command overhead without doing this.
> 
> In any case this argument is redundant because my intention is (as Litke pointed out) is to have a lean API.
> and API call is something you have to support across versions, this call implemented in the engine is something that no one has to support and can change\evolve easily.
> 
> As a rule, if an API call C and be implemented by doing A + B then C is redundant.

I disagree with the above statement, exposing a bulk of operations in a
single API call is very common and not considered redundant.

> 
> List of connections as args:
> ============================
> Sorry I forgot to respond about that. I'm not as strongly opposed to the idea as the other things you suggested. It'll just make implementing the persistence logic in VDSM significantly more complicated as I will have to commit multiple connection information to disk in an all or nothing mode. I can create a small sqlitedb to do that or do some directory tricks and exploit FS rename atomicity but I'd rather not.
> 
> The demands are not without base. I would like to keep the code simple under the hood in the price of a few more calls. You would like to make less calls and keep the code simpler on your side. There isn't a real way to settle this.

It is not about keeping the code simple (writing a loop is simple as
well), it is about redundant round trips.

> If anyone on the list as pros and cons for either way I'd be happy to hear them.
> If no compelling arguments arise I will let Ayal call this one.
> 
> Transient connections:
> ======================
> The problem you are describing as I understand it is that VDSM did not respond and not that the API client did not respond.
> Again, this can happen for a number of reason, most of which VDSM might not be aware that there is actually a problem (network issues).
> 
> This relates to the EOL policy. I agree we have to find a good way to define an automatic EOL for resources. I have made my suggestion. Out of the scope of the API.
> 
> In the meantime cleaning stale connections is trivial and I have made it clear a previous email about how to go about it in a simple non intrusive way. Clean hosts on host connect, and on every poll if you find connections that you don't like. This should keep things squeaky clean.

I have no additional input on this.

I truly appreciate your effort for modeling clean and simple API, but at
the end of the day if the users of the API don't think it is clean and
simple you missed your goal.


> 
> ----- Original Message -----
>> From: "Livnat Peer" <lpeer at redhat.com>
>> To: "Saggi Mizrahi" <smizrahi at redhat.com>
>> Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
>> Sent: Thursday, January 26, 2012 5:22:42 AM
>> Subject: Re: [Engine-devel] [RFC] New Connection Management API
>>
>> On 25/01/12 23:35, Saggi Mizrahi wrote:
>>> <SNIP>
>>> This is mail was getting way too long.
>>>
>>> About the clear all verb.
>>> No.
>>> Just loop, find the connections YOU OWN and clean them. Even though
>>> you don't want to support multiple clients to VDSM API doesn't
>>> mean the engine shouldn't behave like a proper citizen.
>>> It's the same reason why VDSM tries and not mess system resources
>>> it didn't initiate.
>>
>>
>> There is a big difference, VDSM living in hybrid mode with other
>> workload on the host is a valid use case, having more than one
>> concurrent manager for VDSM is not.
>> Generating a disconnect request for each connection does not seem
>> like
>> the right API to me, again think on the simple flow of moving host
>> from
>> one data center to another, the engine needs to disconnect tall
>> storage
>> domains (each domain can have couple of connections associated with
>> it).
>>
>> I am giving example from the engine use cases as it is the main user
>> of
>> VDSM ATM but I am sure it will be relevant to any other user of VDSM.
>>
>>>
>>> ------------------------
>>>
>>> As I see it the only point of conflict is the so called
>>> non-peristed connections.
>>> I will call them transient connections from now on.
>>>
>>> There are 2 user cases being discussed
>>> 1. Wait until a connection is made, if it fails don't retry and
>>> automatically unmanage.
>>> 2. If the called of the API forgets or fails to unmanage a
>>> connection.
>>>
>>
>> Actually I was not discussing #2 at all.
>>
>>> Your suggestion as I understand it:
>>> Transient connections are:
>>>      - Connection that VDSM will only try to connect to once and
>>>      will not reconnect to in case of disconnect.
>>
>> yes
>>
>>>
>>> My problem with this definition that it does not specify the "end
>>> of life" of the connection.
>>> Meaning it solves only use case 1.
>>
>> since this is the only use case i had in mind, it is what i was
>> looking for.
>>
>>> If all is well, and it usually is, VDSM will not invoke a
>>> disconnect.
>>> So the caller would have to call unmanage if the connection
>>> succeeded at the end of the flow.
>>
>> agree.
>>
>>> Now, if you are already calling unmanage if connection succeeded
>>> you can just call it anyway.
>>
>> not exactly, an example I gave earlier on the thread was that VSDM
>> hangs
>> or have other error and the engine can not initiate unmanaged,
>> instead
>> let's assume the host is fenced (self-fence or external fence does
>> not
>> matter), in this scenario the engine will not issue unmanage.
>>
>>>
>>> instead of doing: (with your suggestion)
>>> ----------------
>>> manage
>>> wait until succeeds or lastError has value
>>> try:
>>>   do stuff
>>> finally:
>>>   unmanage
>>>
>>> do: (with the canonical flow)
>>> ---
>>> manage
>>> try:
>>>   wait until succeeds or lastError has value
>>>   do stuff
>>> finally:
>>>   unmanage
>>>
>>> This is simpler to do than having another connection type.
>>
>> You are assuming the engine can communicate with VDSM and there are
>> scenarios where it is not feasible.
>>
>>>
>>> Now that we got that out of the way lets talk about the 2nd use
>>> case.
>>
>> Since I did not ask VDSM to clean after the (engine) user and you
>> don't
>> want to do it I am not sure we need to discuss this.
>>
>> If you insist we can start the discussion on who should implement the
>> cleanup mechanism but I'm afraid I have no strong arguments for VDSM
>> to
>> do it, so I rather not go there ;)
>>
>>
>> You dropped from the discussion my request for supporting list of
>> connections for manage and unmanage verbs.
>>
>>> API client died in the middle of the operation and unmanage was
>>> never called.
>>>
>>> Your suggested definition means that unless there was a problem
>>> with the connection VDSM will still have this connection active.
>>> The engine will have to clean it anyway.
>>>
>>> The problem is, VDSM has no way of knowing that a client died,
>>> forgot or is thinking really hard and will continue on in about 2
>>> minutes.
>>
>>>
>>> Connections that live until they die is a hard to define and work
>>> with lifecycle. Solving this problem is theoretically simple.
>>>
>>> Have clients hold some sort of session token and force the client
>>> to update it at a specified interval. You could bind resources
>>> (like domains, VMs, connections) to that session token so when it
>>> expires VDSM auto cleans the resources.
>>>
>>> This kind of mechanism is out of the scope of this API change.
>>> Further more I think that this mechanism should sit in the engine
>>> since the session might actually contain resources from multiple
>>> hosts and resources that are not managed by VDSM.
>>>
>>> In GUI flows specifically the user might do actions that don't even
>>> touch the engine and forcing it to refresh the engine token is
>>> simpler then having it refresh the VDSM token.
>>>
>>> I understand that engine currently has no way of tracking a user
>>> session. This, as I said, is also true in the case of VDSM. We can
>>> start and argue about which project should implement the session
>>> semantics. But as I see it it's not relevant to the connection
>>> management API.
>>
>>



From lpeer at redhat.com  Thu Jan 26 20:16:32 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Thu, 26 Jan 2012 22:16:32 +0200
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <fea9a91c-72d6-43e8-b213-94767fa538e2@zmail16.collab.prod.int.phx2.redhat.com>
References: <fea9a91c-72d6-43e8-b213-94767fa538e2@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F21B4A0.8070901@redhat.com>

On 26/01/12 21:21, Saggi Mizrahi wrote:
> 
> 
> ----- Original Message -----
>> From: "Adam Litke" <agl at us.ibm.com>
>> To: "Saggi Mizrahi" <smizrahi at redhat.com>
>> Cc: "Livnat Peer" <lpeer at redhat.com>, engine-devel at ovirt.org, vdsm-devel at lists.fedorahosted.org
>> Sent: Thursday, January 26, 2012 1:58:40 PM
>> Subject: Re: [vdsm] [Engine-devel] [RFC] New Connection Management API
>>
>> On Thu, Jan 26, 2012 at 10:00:57AM -0500, Saggi Mizrahi wrote:
>>> <snip>
>>> Again trying to sum up and address all comments
>>>
>>> Clear all:
>>> ==========
>>> My opinions is still to not implement it.
>>> Even though it might generate a bit more traffic premature
>>> optimization is bad and there are other reasons we can improve
>>> VDSM command overhead without doing this.
>>>
>>> In any case this argument is redundant because my intention is (as
>>> Litke pointed out) is to have a lean API.
>>> and API call is something you have to support across versions, this
>>> call implemented in the engine is something that no one has to
>>> support and can change\evolve easily.
>>>
>>> As a rule, if an API call C and be implemented by doing A + B then
>>> C is redundant.
>>>
>>> List of connections as args:
>>> ============================
>>> Sorry I forgot to respond about that. I'm not as strongly opposed
>>> to the idea as the other things you suggested. It'll just make
>>> implementing the persistence logic in VDSM significantly more
>>> complicated as I will have to commit multiple connection
>>> information to disk in an all or nothing mode. I can create a
>>> small sqlitedb to do that or do some directory tricks and exploit
>>> FS rename atomicity but I'd rather not.
>>
>> I would be strongly opposed to introducing a sqlite database into
>> vdsm just to
>> enable "convenience mode" for this API.  Does the operation really
>> need to be
>> atomic?  Why not just perform each connection sequentially and return
>> a list of
>> statuses? Is the only motivation for allowing a list of parameters
>> to reduce
>> the number of API calls between engine and vdsm)?  If so, the same
>> argument
>> Saggi makes above applies here.
> 
> I try and have VDSM expose APIs that are simple to predict. a command can either succeed or fail.
> The problem is not actually validating the connections. The problem is that once I concluded that they are all OK I need to persist to disk the information that will allow me to reconnect if VDSM happens to crash. If I naively save them one by one I could get in a state where only some of the connections persisted before the operation failed. So I have to somehow put all this in a transaction.
> 
> I don't have to use sqlite. I could also put all the persistence information in a new dir for every call named <UUID>.tmp. Once I wrote everything down I rename the directory to just <UUID> and fsync it. This is guarantied by posix to be atomic. For unmanage, I move all the persistence information from the directories they sit in to a new dir named <UUID>. Rename it to a <UUDI>.tmp, fsync it and then remove it.
> 
> This all just looks like more trouble then it's worth to me.
> 


I agree with Adam, I don't think the operation should be atomic, having
only some of the connections persisted is a perfectly valid outcome if
the API returns a list of statuses.


>>
>>> The demands are not without base. I would like to keep the code
>>> simple under the hood in the price of a few more calls. You would
>>> like to make less calls and keep the code simpler on your side.
>>> There isn't a real way to settle this.
>>> If anyone on the list as pros and cons for either way I'd be happy
>>> to hear them.
>>> If no compelling arguments arise I will let Ayal call this one.
>>>
>>> Transient connections:
>>> ======================
>>> The problem you are describing as I understand it is that VDSM did
>>> not respond and not that the API client did not respond.
>>> Again, this can happen for a number of reason, most of which VDSM
>>> might not be aware that there is actually a problem (network
>>> issues).
>>>
>>> This relates to the EOL policy. I agree we have to find a good way
>>> to define an automatic EOL for resources. I have made my
>>> suggestion. Out of the scope of the API.
>>>
>>> In the meantime cleaning stale connections is trivial and I have
>>> made it clear a previous email about how to go about it in a
>>> simple non intrusive way. Clean hosts on host connect, and on
>>> every poll if you find connections that you don't like. This
>>> should keep things squeaky clean.
>>>
>>> ----- Original Message -----
>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>> To: "Saggi Mizrahi" <smizrahi at redhat.com>
>>>> Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
>>>> Sent: Thursday, January 26, 2012 5:22:42 AM
>>>> Subject: Re: [Engine-devel] [RFC] New Connection Management API
>>>>
>>>> On 25/01/12 23:35, Saggi Mizrahi wrote:
>>>>> <SNIP>
>>>>> This is mail was getting way too long.
>>>>>
>>>>> About the clear all verb.
>>>>> No.
>>>>> Just loop, find the connections YOU OWN and clean them. Even
>>>>> though
>>>>> you don't want to support multiple clients to VDSM API doesn't
>>>>> mean the engine shouldn't behave like a proper citizen.
>>>>> It's the same reason why VDSM tries and not mess system
>>>>> resources
>>>>> it didn't initiate.
>>>>
>>>>
>>>> There is a big difference, VDSM living in hybrid mode with other
>>>> workload on the host is a valid use case, having more than one
>>>> concurrent manager for VDSM is not.
>>>> Generating a disconnect request for each connection does not seem
>>>> like
>>>> the right API to me, again think on the simple flow of moving
>>>> host
>>>> from
>>>> one data center to another, the engine needs to disconnect tall
>>>> storage
>>>> domains (each domain can have couple of connections associated
>>>> with
>>>> it).
>>>>
>>>> I am giving example from the engine use cases as it is the main
>>>> user
>>>> of
>>>> VDSM ATM but I am sure it will be relevant to any other user of
>>>> VDSM.
>>>>
>>>>>
>>>>> ------------------------
>>>>>
>>>>> As I see it the only point of conflict is the so called
>>>>> non-peristed connections.
>>>>> I will call them transient connections from now on.
>>>>>
>>>>> There are 2 user cases being discussed
>>>>> 1. Wait until a connection is made, if it fails don't retry and
>>>>> automatically unmanage.
>>>>> 2. If the called of the API forgets or fails to unmanage a
>>>>> connection.
>>>>>
>>>>
>>>> Actually I was not discussing #2 at all.
>>>>
>>>>> Your suggestion as I understand it:
>>>>> Transient connections are:
>>>>>      - Connection that VDSM will only try to connect to once
>>>>>      and
>>>>>      will not reconnect to in case of disconnect.
>>>>
>>>> yes
>>>>
>>>>>
>>>>> My problem with this definition that it does not specify the
>>>>> "end
>>>>> of life" of the connection.
>>>>> Meaning it solves only use case 1.
>>>>
>>>> since this is the only use case i had in mind, it is what i was
>>>> looking for.
>>>>
>>>>> If all is well, and it usually is, VDSM will not invoke a
>>>>> disconnect.
>>>>> So the caller would have to call unmanage if the connection
>>>>> succeeded at the end of the flow.
>>>>
>>>> agree.
>>>>
>>>>> Now, if you are already calling unmanage if connection
>>>>> succeeded
>>>>> you can just call it anyway.
>>>>
>>>> not exactly, an example I gave earlier on the thread was that
>>>> VSDM
>>>> hangs
>>>> or have other error and the engine can not initiate unmanaged,
>>>> instead
>>>> let's assume the host is fenced (self-fence or external fence
>>>> does
>>>> not
>>>> matter), in this scenario the engine will not issue unmanage.
>>>>
>>>>>
>>>>> instead of doing: (with your suggestion)
>>>>> ----------------
>>>>> manage
>>>>> wait until succeeds or lastError has value
>>>>> try:
>>>>>   do stuff
>>>>> finally:
>>>>>   unmanage
>>>>>
>>>>> do: (with the canonical flow)
>>>>> ---
>>>>> manage
>>>>> try:
>>>>>   wait until succeeds or lastError has value
>>>>>   do stuff
>>>>> finally:
>>>>>   unmanage
>>>>>
>>>>> This is simpler to do than having another connection type.
>>>>
>>>> You are assuming the engine can communicate with VDSM and there
>>>> are
>>>> scenarios where it is not feasible.
>>>>
>>>>>
>>>>> Now that we got that out of the way lets talk about the 2nd use
>>>>> case.
>>>>
>>>> Since I did not ask VDSM to clean after the (engine) user and you
>>>> don't
>>>> want to do it I am not sure we need to discuss this.
>>>>
>>>> If you insist we can start the discussion on who should implement
>>>> the
>>>> cleanup mechanism but I'm afraid I have no strong arguments for
>>>> VDSM
>>>> to
>>>> do it, so I rather not go there ;)
>>>>
>>>>
>>>> You dropped from the discussion my request for supporting list of
>>>> connections for manage and unmanage verbs.
>>>>
>>>>> API client died in the middle of the operation and unmanage was
>>>>> never called.
>>>>>
>>>>> Your suggested definition means that unless there was a problem
>>>>> with the connection VDSM will still have this connection
>>>>> active.
>>>>> The engine will have to clean it anyway.
>>>>>
>>>>> The problem is, VDSM has no way of knowing that a client died,
>>>>> forgot or is thinking really hard and will continue on in about
>>>>> 2
>>>>> minutes.
>>>>
>>>>>
>>>>> Connections that live until they die is a hard to define and
>>>>> work
>>>>> with lifecycle. Solving this problem is theoretically simple.
>>>>>
>>>>> Have clients hold some sort of session token and force the
>>>>> client
>>>>> to update it at a specified interval. You could bind resources
>>>>> (like domains, VMs, connections) to that session token so when
>>>>> it
>>>>> expires VDSM auto cleans the resources.
>>>>>
>>>>> This kind of mechanism is out of the scope of this API change.
>>>>> Further more I think that this mechanism should sit in the
>>>>> engine
>>>>> since the session might actually contain resources from
>>>>> multiple
>>>>> hosts and resources that are not managed by VDSM.
>>>>>
>>>>> In GUI flows specifically the user might do actions that don't
>>>>> even
>>>>> touch the engine and forcing it to refresh the engine token is
>>>>> simpler then having it refresh the VDSM token.
>>>>>
>>>>> I understand that engine currently has no way of tracking a
>>>>> user
>>>>> session. This, as I said, is also true in the case of VDSM. We
>>>>> can
>>>>> start and argue about which project should implement the
>>>>> session
>>>>> semantics. But as I see it it's not relevant to the connection
>>>>> management API.
>>>>
>>>>
>>> _______________________________________________
>>> vdsm-devel mailing list
>>> vdsm-devel at lists.fedorahosted.org
>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>>
>> --
>> Adam Litke <agl at us.ibm.com>
>> IBM Linux Technology Center
>>
>>



From smizrahi at redhat.com  Thu Jan 26 21:32:20 2012
From: smizrahi at redhat.com (Saggi Mizrahi)
Date: Thu, 26 Jan 2012 16:32:20 -0500 (EST)
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <4F21B19B.40802@redhat.com>
Message-ID: <6132a6d3-cf5b-4480-b993-18da068bbee0@zmail16.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Saggi Mizrahi" <smizrahi at redhat.com>
> Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
> Sent: Thursday, January 26, 2012 3:03:39 PM
> Subject: Re: [Engine-devel] [RFC] New Connection Management API
> 
> On 26/01/12 17:00, Saggi Mizrahi wrote:
> > <snip>
> > Again trying to sum up and address all comments
> > 
> > Clear all:
> > ==========
> > My opinions is still to not implement it.
> > Even though it might generate a bit more traffic premature
> > optimization is bad and there are other reasons we can improve
> > VDSM command overhead without doing this.
> > 
> > In any case this argument is redundant because my intention is (as
> > Litke pointed out) is to have a lean API.
> > and API call is something you have to support across versions, this
> > call implemented in the engine is something that no one has to
> > support and can change\evolve easily.
> > 
> > As a rule, if an API call C and be implemented by doing A + B then
> > C is redundant.
> 
> I disagree with the above statement, exposing a bulk of operations in
> a
> single API call is very common and not considered redundant.
I agree that that APIs with those kind of calls exist but it doesn't mean they are not redundant.

re?dun?dant: adj. (of words or data) Able to be omitted without loss of meaning or function

This call can be omitted without loss of function.
API calls are a commitment for generations. Wrapping this in the clients doesn't.
To quot myself:
"API call is something you have to support across versions, this
call implemented in the engine is something that no one has to
support and can change\evolve easily." ~ Saggi Mizrahi, a few lines above

this API set will one day be considered stupid, obsolete and annoying.
That's just how life is. We'll find better ways of solving these problems.
When that moment comes I want to have as little functionality as possible I have to keep maintaining.
I doubt there is any way you can convince me otherwise.

Put yourself in my position and think if you would have made this sacrifice just to save someone a loop.

To sum up, I will not add any API calls I don't absolutely have to.

As to the amount of calls, this is not relevant to the clear all verb. This is addressed by the point right below this sentence.

> 
> > 
> > List of connections as args:
> > ============================
> > Sorry I forgot to respond about that. I'm not as strongly opposed
> > to the idea as the other things you suggested. It'll just make
> > implementing the persistence logic in VDSM significantly more
> > complicated as I will have to commit multiple connection
> > information to disk in an all or nothing mode. I can create a
> > small sqlitedb to do that or do some directory tricks and exploit
> > FS rename atomicity but I'd rather not.
> > 
> > The demands are not without base. I would like to keep the code
> > simple under the hood in the price of a few more calls. You would
> > like to make less calls and keep the code simpler on your side.
> > There isn't a real way to settle this.
> 
> It is not about keeping the code simple (writing a loop is simple as
> well), it is about redundant round trips.
As I said, I agree there is merit there.

I think that roundtrips is a general issue not specific to this call.
My opinion is that communication with VDSM should just use HTTP pipelining (http://en.wikipedia.org/wiki/HTTP_pipelining)
This will solve the problem globally instead of tacking it on to the API interface.

I generally prefer simplicity of the API and the implementation, and correctness over performance.

I laid out out what the change entails, multiple ways of solving this, and my personal perspective.
Unless someone on the list objects to either solution, Ayal will have final say on this matter.
He is more of a pragmatist than I (and doing what he says usually correlates with me getting my paycheck).

> 
> > If anyone on the list as pros and cons for either way I'd be happy
> > to hear them.
> > If no compelling arguments arise I will let Ayal call this one.
> > 
> > Transient connections:
> > ======================
> > The problem you are describing as I understand it is that VDSM did
> > not respond and not that the API client did not respond.
> > Again, this can happen for a number of reason, most of which VDSM
> > might not be aware that there is actually a problem (network
> > issues).
> > 
> > This relates to the EOL policy. I agree we have to find a good way
> > to define an automatic EOL for resources. I have made my
> > suggestion. Out of the scope of the API.
> > 
> > In the meantime cleaning stale connections is trivial and I have
> > made it clear a previous email about how to go about it in a
> > simple non intrusive way. Clean hosts on host connect, and on
> > every poll if you find connections that you don't like. This
> > should keep things squeaky clean.
> 
> I have no additional input on this.
> 
The only real legitimate reservation you still have with the API is transient connections. As I said, if you can find a way to define an End Of Life condition I will implement it. Promise, David star my heart and hope to die.

Just some pointers:
* It cannot be time as no flow is really restricted by time.
* It cannot be tied to specific calls as an argument (like createStorageDomain, connectStorageDomain, etc...) as this is arbitrary and solves problems in very specific flows.
* It cannot be tied to VDSM restart or host restart. This just makes no sense as there is never a connection between these and a flow.
* It cannot be tied to number of connection attempts. Flows should be able to decide weather they want to keep on trying or bail out. The EOL mechanism should not come in the expense of preventing legitimate EOL. VDSM\Client\ can't clean up because of some unrecoverable error.

I proposed my solution to the EOL issue. The only problem I see you having with it, is that it's not actually VDSM that has to implement it. You can either accept it or give a better one.

I know that garbage collection and other fun tricks made everyone spoiled rotten about managing resources.
But to create an automatic resource freeing mechanism (GC) you got to have a clear EOL condition.

Examples:
- Cleaning FDs is easy as processes have a very clear EOL.
- With memory it is a bit more complicated but this is why you got ref-counting, hard-refs and weak-refs, loop detection, scoped pointers, and a million other strategies.

All I need to create a collection mechanism is a clear condition where I am guaranteed that the resource is not in use.
Until I get that you will just have to manage the resources yourself.
This is not an API issue, this is a general problem when making software.

I don't go ask the lvm guys to automatically clear VGs if I fail while creating a block domain.
I understand and accept the fact that they are just too low on the stack to solve this for me.

> I truly appreciate your effort for modeling clean and simple API, but
> at
> the end of the day if the users of the API don't think it is clean
> and
> simple you missed your goal.

Simple is an amorphic concept that everyone have a personal definition for.
I don't care if this doesn't fit in to what you consider simple.
You are free to suggest improvements.
Hell, just write your dream API if it's better then this well use it.

> 
> 
> > 
> > ----- Original Message -----
> >> From: "Livnat Peer" <lpeer at redhat.com>
> >> To: "Saggi Mizrahi" <smizrahi at redhat.com>
> >> Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
> >> Sent: Thursday, January 26, 2012 5:22:42 AM
> >> Subject: Re: [Engine-devel] [RFC] New Connection Management API
> >>
> >> On 25/01/12 23:35, Saggi Mizrahi wrote:
> >>> <SNIP>
> >>> This is mail was getting way too long.
> >>>
> >>> About the clear all verb.
> >>> No.
> >>> Just loop, find the connections YOU OWN and clean them. Even
> >>> though
> >>> you don't want to support multiple clients to VDSM API doesn't
> >>> mean the engine shouldn't behave like a proper citizen.
> >>> It's the same reason why VDSM tries and not mess system resources
> >>> it didn't initiate.
> >>
> >>
> >> There is a big difference, VDSM living in hybrid mode with other
> >> workload on the host is a valid use case, having more than one
> >> concurrent manager for VDSM is not.
> >> Generating a disconnect request for each connection does not seem
> >> like
> >> the right API to me, again think on the simple flow of moving host
> >> from
> >> one data center to another, the engine needs to disconnect tall
> >> storage
> >> domains (each domain can have couple of connections associated
> >> with
> >> it).
> >>
> >> I am giving example from the engine use cases as it is the main
> >> user
> >> of
> >> VDSM ATM but I am sure it will be relevant to any other user of
> >> VDSM.
> >>
> >>>
> >>> ------------------------
> >>>
> >>> As I see it the only point of conflict is the so called
> >>> non-peristed connections.
> >>> I will call them transient connections from now on.
> >>>
> >>> There are 2 user cases being discussed
> >>> 1. Wait until a connection is made, if it fails don't retry and
> >>> automatically unmanage.
> >>> 2. If the called of the API forgets or fails to unmanage a
> >>> connection.
> >>>
> >>
> >> Actually I was not discussing #2 at all.
> >>
> >>> Your suggestion as I understand it:
> >>> Transient connections are:
> >>>      - Connection that VDSM will only try to connect to once and
> >>>      will not reconnect to in case of disconnect.
> >>
> >> yes
> >>
> >>>
> >>> My problem with this definition that it does not specify the "end
> >>> of life" of the connection.
> >>> Meaning it solves only use case 1.
> >>
> >> since this is the only use case i had in mind, it is what i was
> >> looking for.
> >>
> >>> If all is well, and it usually is, VDSM will not invoke a
> >>> disconnect.
> >>> So the caller would have to call unmanage if the connection
> >>> succeeded at the end of the flow.
> >>
> >> agree.
> >>
> >>> Now, if you are already calling unmanage if connection succeeded
> >>> you can just call it anyway.
> >>
> >> not exactly, an example I gave earlier on the thread was that VSDM
> >> hangs
> >> or have other error and the engine can not initiate unmanaged,
> >> instead
> >> let's assume the host is fenced (self-fence or external fence does
> >> not
> >> matter), in this scenario the engine will not issue unmanage.
> >>
> >>>
> >>> instead of doing: (with your suggestion)
> >>> ----------------
> >>> manage
> >>> wait until succeeds or lastError has value
> >>> try:
> >>>   do stuff
> >>> finally:
> >>>   unmanage
> >>>
> >>> do: (with the canonical flow)
> >>> ---
> >>> manage
> >>> try:
> >>>   wait until succeeds or lastError has value
> >>>   do stuff
> >>> finally:
> >>>   unmanage
> >>>
> >>> This is simpler to do than having another connection type.
> >>
> >> You are assuming the engine can communicate with VDSM and there
> >> are
> >> scenarios where it is not feasible.
> >>
> >>>
> >>> Now that we got that out of the way lets talk about the 2nd use
> >>> case.
> >>
> >> Since I did not ask VDSM to clean after the (engine) user and you
> >> don't
> >> want to do it I am not sure we need to discuss this.
> >>
> >> If you insist we can start the discussion on who should implement
> >> the
> >> cleanup mechanism but I'm afraid I have no strong arguments for
> >> VDSM
> >> to
> >> do it, so I rather not go there ;)
> >>
> >>
> >> You dropped from the discussion my request for supporting list of
> >> connections for manage and unmanage verbs.
> >>
> >>> API client died in the middle of the operation and unmanage was
> >>> never called.
> >>>
> >>> Your suggested definition means that unless there was a problem
> >>> with the connection VDSM will still have this connection active.
> >>> The engine will have to clean it anyway.
> >>>
> >>> The problem is, VDSM has no way of knowing that a client died,
> >>> forgot or is thinking really hard and will continue on in about 2
> >>> minutes.
> >>
> >>>
> >>> Connections that live until they die is a hard to define and work
> >>> with lifecycle. Solving this problem is theoretically simple.
> >>>
> >>> Have clients hold some sort of session token and force the client
> >>> to update it at a specified interval. You could bind resources
> >>> (like domains, VMs, connections) to that session token so when it
> >>> expires VDSM auto cleans the resources.
> >>>
> >>> This kind of mechanism is out of the scope of this API change.
> >>> Further more I think that this mechanism should sit in the engine
> >>> since the session might actually contain resources from multiple
> >>> hosts and resources that are not managed by VDSM.
> >>>
> >>> In GUI flows specifically the user might do actions that don't
> >>> even
> >>> touch the engine and forcing it to refresh the engine token is
> >>> simpler then having it refresh the VDSM token.
> >>>
> >>> I understand that engine currently has no way of tracking a user
> >>> session. This, as I said, is also true in the case of VDSM. We
> >>> can
> >>> start and argue about which project should implement the session
> >>> semantics. But as I see it it's not relevant to the connection
> >>> management API.
> >>
> >>
> 
> 


From smizrahi at redhat.com  Thu Jan 26 21:42:50 2012
From: smizrahi at redhat.com (Saggi Mizrahi)
Date: Thu, 26 Jan 2012 16:42:50 -0500 (EST)
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <4F21B4A0.8070901@redhat.com>
Message-ID: <9c119395-58f8-4b15-9a82-501d2db6bdcc@zmail16.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Saggi Mizrahi" <smizrahi at redhat.com>
> Cc: "Adam Litke" <agl at us.ibm.com>, engine-devel at ovirt.org, vdsm-devel at lists.fedorahosted.org
> Sent: Thursday, January 26, 2012 3:16:32 PM
> Subject: Re: [vdsm] [Engine-devel] [RFC] New Connection Management API
> 
> On 26/01/12 21:21, Saggi Mizrahi wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Adam Litke" <agl at us.ibm.com>
> >> To: "Saggi Mizrahi" <smizrahi at redhat.com>
> >> Cc: "Livnat Peer" <lpeer at redhat.com>, engine-devel at ovirt.org,
> >> vdsm-devel at lists.fedorahosted.org
> >> Sent: Thursday, January 26, 2012 1:58:40 PM
> >> Subject: Re: [vdsm] [Engine-devel] [RFC] New Connection Management
> >> API
> >>
> >> On Thu, Jan 26, 2012 at 10:00:57AM -0500, Saggi Mizrahi wrote:
> >>> <snip>
> >>> Again trying to sum up and address all comments
> >>>
> >>> Clear all:
> >>> ==========
> >>> My opinions is still to not implement it.
> >>> Even though it might generate a bit more traffic premature
> >>> optimization is bad and there are other reasons we can improve
> >>> VDSM command overhead without doing this.
> >>>
> >>> In any case this argument is redundant because my intention is
> >>> (as
> >>> Litke pointed out) is to have a lean API.
> >>> and API call is something you have to support across versions,
> >>> this
> >>> call implemented in the engine is something that no one has to
> >>> support and can change\evolve easily.
> >>>
> >>> As a rule, if an API call C and be implemented by doing A + B
> >>> then
> >>> C is redundant.
> >>>
> >>> List of connections as args:
> >>> ============================
> >>> Sorry I forgot to respond about that. I'm not as strongly opposed
> >>> to the idea as the other things you suggested. It'll just make
> >>> implementing the persistence logic in VDSM significantly more
> >>> complicated as I will have to commit multiple connection
> >>> information to disk in an all or nothing mode. I can create a
> >>> small sqlitedb to do that or do some directory tricks and exploit
> >>> FS rename atomicity but I'd rather not.
> >>
> >> I would be strongly opposed to introducing a sqlite database into
> >> vdsm just to
> >> enable "convenience mode" for this API.  Does the operation really
> >> need to be
> >> atomic?  Why not just perform each connection sequentially and
> >> return
> >> a list of
> >> statuses? Is the only motivation for allowing a list of parameters
> >> to reduce
> >> the number of API calls between engine and vdsm)?  If so, the same
> >> argument
> >> Saggi makes above applies here.
> > 
> > I try and have VDSM expose APIs that are simple to predict. a
> > command can either succeed or fail.
> > The problem is not actually validating the connections. The problem
> > is that once I concluded that they are all OK I need to persist to
> > disk the information that will allow me to reconnect if VDSM
> > happens to crash. If I naively save them one by one I could get in
> > a state where only some of the connections persisted before the
> > operation failed. So I have to somehow put all this in a
> > transaction.
> > 
> > I don't have to use sqlite. I could also put all the persistence
> > information in a new dir for every call named <UUID>.tmp. Once I
> > wrote everything down I rename the directory to just <UUID> and
> > fsync it. This is guarantied by posix to be atomic. For unmanage,
> > I move all the persistence information from the directories they
> > sit in to a new dir named <UUID>. Rename it to a <UUDI>.tmp, fsync
> > it and then remove it.
> > 
> > This all just looks like more trouble then it's worth to me.
> > 
> 
> 
> I agree with Adam, I don't think the operation should be atomic,
> having
> only some of the connections persisted is a perfectly valid outcome
> if
> the API returns a list of statuses.
What if it doesn't return at all?
The only reasons that something will fail manage is if the URI is broken so I assume 99% of the issued manage commands will succeed.
My problem is with VDSM crashing mid operation. The operation will appear to fail but when VDSM returns some of the connections persisted so it will reconnect. because the client's manage call failed it doesn't expect the CIDs to be in the list. This will cause ambiguity when finding an already registered CID at runtime.

> 
> 
> >>
> >>> The demands are not without base. I would like to keep the code
> >>> simple under the hood in the price of a few more calls. You would
> >>> like to make less calls and keep the code simpler on your side.
> >>> There isn't a real way to settle this.
> >>> If anyone on the list as pros and cons for either way I'd be
> >>> happy
> >>> to hear them.
> >>> If no compelling arguments arise I will let Ayal call this one.
> >>>
> >>> Transient connections:
> >>> ======================
> >>> The problem you are describing as I understand it is that VDSM
> >>> did
> >>> not respond and not that the API client did not respond.
> >>> Again, this can happen for a number of reason, most of which VDSM
> >>> might not be aware that there is actually a problem (network
> >>> issues).
> >>>
> >>> This relates to the EOL policy. I agree we have to find a good
> >>> way
> >>> to define an automatic EOL for resources. I have made my
> >>> suggestion. Out of the scope of the API.
> >>>
> >>> In the meantime cleaning stale connections is trivial and I have
> >>> made it clear a previous email about how to go about it in a
> >>> simple non intrusive way. Clean hosts on host connect, and on
> >>> every poll if you find connections that you don't like. This
> >>> should keep things squeaky clean.
> >>>
> >>> ----- Original Message -----
> >>>> From: "Livnat Peer" <lpeer at redhat.com>
> >>>> To: "Saggi Mizrahi" <smizrahi at redhat.com>
> >>>> Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
> >>>> Sent: Thursday, January 26, 2012 5:22:42 AM
> >>>> Subject: Re: [Engine-devel] [RFC] New Connection Management API
> >>>>
> >>>> On 25/01/12 23:35, Saggi Mizrahi wrote:
> >>>>> <SNIP>
> >>>>> This is mail was getting way too long.
> >>>>>
> >>>>> About the clear all verb.
> >>>>> No.
> >>>>> Just loop, find the connections YOU OWN and clean them. Even
> >>>>> though
> >>>>> you don't want to support multiple clients to VDSM API doesn't
> >>>>> mean the engine shouldn't behave like a proper citizen.
> >>>>> It's the same reason why VDSM tries and not mess system
> >>>>> resources
> >>>>> it didn't initiate.
> >>>>
> >>>>
> >>>> There is a big difference, VDSM living in hybrid mode with other
> >>>> workload on the host is a valid use case, having more than one
> >>>> concurrent manager for VDSM is not.
> >>>> Generating a disconnect request for each connection does not
> >>>> seem
> >>>> like
> >>>> the right API to me, again think on the simple flow of moving
> >>>> host
> >>>> from
> >>>> one data center to another, the engine needs to disconnect tall
> >>>> storage
> >>>> domains (each domain can have couple of connections associated
> >>>> with
> >>>> it).
> >>>>
> >>>> I am giving example from the engine use cases as it is the main
> >>>> user
> >>>> of
> >>>> VDSM ATM but I am sure it will be relevant to any other user of
> >>>> VDSM.
> >>>>
> >>>>>
> >>>>> ------------------------
> >>>>>
> >>>>> As I see it the only point of conflict is the so called
> >>>>> non-peristed connections.
> >>>>> I will call them transient connections from now on.
> >>>>>
> >>>>> There are 2 user cases being discussed
> >>>>> 1. Wait until a connection is made, if it fails don't retry and
> >>>>> automatically unmanage.
> >>>>> 2. If the called of the API forgets or fails to unmanage a
> >>>>> connection.
> >>>>>
> >>>>
> >>>> Actually I was not discussing #2 at all.
> >>>>
> >>>>> Your suggestion as I understand it:
> >>>>> Transient connections are:
> >>>>>      - Connection that VDSM will only try to connect to once
> >>>>>      and
> >>>>>      will not reconnect to in case of disconnect.
> >>>>
> >>>> yes
> >>>>
> >>>>>
> >>>>> My problem with this definition that it does not specify the
> >>>>> "end
> >>>>> of life" of the connection.
> >>>>> Meaning it solves only use case 1.
> >>>>
> >>>> since this is the only use case i had in mind, it is what i was
> >>>> looking for.
> >>>>
> >>>>> If all is well, and it usually is, VDSM will not invoke a
> >>>>> disconnect.
> >>>>> So the caller would have to call unmanage if the connection
> >>>>> succeeded at the end of the flow.
> >>>>
> >>>> agree.
> >>>>
> >>>>> Now, if you are already calling unmanage if connection
> >>>>> succeeded
> >>>>> you can just call it anyway.
> >>>>
> >>>> not exactly, an example I gave earlier on the thread was that
> >>>> VSDM
> >>>> hangs
> >>>> or have other error and the engine can not initiate unmanaged,
> >>>> instead
> >>>> let's assume the host is fenced (self-fence or external fence
> >>>> does
> >>>> not
> >>>> matter), in this scenario the engine will not issue unmanage.
> >>>>
> >>>>>
> >>>>> instead of doing: (with your suggestion)
> >>>>> ----------------
> >>>>> manage
> >>>>> wait until succeeds or lastError has value
> >>>>> try:
> >>>>>   do stuff
> >>>>> finally:
> >>>>>   unmanage
> >>>>>
> >>>>> do: (with the canonical flow)
> >>>>> ---
> >>>>> manage
> >>>>> try:
> >>>>>   wait until succeeds or lastError has value
> >>>>>   do stuff
> >>>>> finally:
> >>>>>   unmanage
> >>>>>
> >>>>> This is simpler to do than having another connection type.
> >>>>
> >>>> You are assuming the engine can communicate with VDSM and there
> >>>> are
> >>>> scenarios where it is not feasible.
> >>>>
> >>>>>
> >>>>> Now that we got that out of the way lets talk about the 2nd use
> >>>>> case.
> >>>>
> >>>> Since I did not ask VDSM to clean after the (engine) user and
> >>>> you
> >>>> don't
> >>>> want to do it I am not sure we need to discuss this.
> >>>>
> >>>> If you insist we can start the discussion on who should
> >>>> implement
> >>>> the
> >>>> cleanup mechanism but I'm afraid I have no strong arguments for
> >>>> VDSM
> >>>> to
> >>>> do it, so I rather not go there ;)
> >>>>
> >>>>
> >>>> You dropped from the discussion my request for supporting list
> >>>> of
> >>>> connections for manage and unmanage verbs.
> >>>>
> >>>>> API client died in the middle of the operation and unmanage was
> >>>>> never called.
> >>>>>
> >>>>> Your suggested definition means that unless there was a problem
> >>>>> with the connection VDSM will still have this connection
> >>>>> active.
> >>>>> The engine will have to clean it anyway.
> >>>>>
> >>>>> The problem is, VDSM has no way of knowing that a client died,
> >>>>> forgot or is thinking really hard and will continue on in about
> >>>>> 2
> >>>>> minutes.
> >>>>
> >>>>>
> >>>>> Connections that live until they die is a hard to define and
> >>>>> work
> >>>>> with lifecycle. Solving this problem is theoretically simple.
> >>>>>
> >>>>> Have clients hold some sort of session token and force the
> >>>>> client
> >>>>> to update it at a specified interval. You could bind resources
> >>>>> (like domains, VMs, connections) to that session token so when
> >>>>> it
> >>>>> expires VDSM auto cleans the resources.
> >>>>>
> >>>>> This kind of mechanism is out of the scope of this API change.
> >>>>> Further more I think that this mechanism should sit in the
> >>>>> engine
> >>>>> since the session might actually contain resources from
> >>>>> multiple
> >>>>> hosts and resources that are not managed by VDSM.
> >>>>>
> >>>>> In GUI flows specifically the user might do actions that don't
> >>>>> even
> >>>>> touch the engine and forcing it to refresh the engine token is
> >>>>> simpler then having it refresh the VDSM token.
> >>>>>
> >>>>> I understand that engine currently has no way of tracking a
> >>>>> user
> >>>>> session. This, as I said, is also true in the case of VDSM. We
> >>>>> can
> >>>>> start and argue about which project should implement the
> >>>>> session
> >>>>> semantics. But as I see it it's not relevant to the connection
> >>>>> management API.
> >>>>
> >>>>
> >>> _______________________________________________
> >>> vdsm-devel mailing list
> >>> vdsm-devel at lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> >>
> >> --
> >> Adam Litke <agl at us.ibm.com>
> >> IBM Linux Technology Center
> >>
> >>
> 
> 


From jchoate at redhat.com  Fri Jan 27 13:37:44 2012
From: jchoate at redhat.com (Jon Choate)
Date: Fri, 27 Jan 2012 08:37:44 -0500 (EST)
Subject: [Engine-devel] First release go/no go meeting on Monday (30.01)
In-Reply-To: <4F212125.1060207@redhat.com>
Message-ID: <b9545daf-8d94-4487-8274-b36b1ecb606c@zmail15.collab.prod.int.phx2.redhat.com>

Will there be a formal release vote (3 +1's) in this meeting? 

Also,I don't see a link on the site for a release candidate. 
Is there one available so the release can be evaluated/tested before deciding on the go/no-go?

----- Original Message -----
> From: "Ofer Schreiber" <oschreib at redhat.com>
> To: board at ovirt.org, arch at ovirt.org, engine-devel at ovirt.org
> Sent: Thursday, January 26, 2012 4:47:17 AM
> Subject: [Engine-devel] First release go/no go meeting on Monday (30.01)
> 
> All,
> 
> As decided in the last oVirt sync meeting, we will have a go/no
> meeting
> about our first release in the upcoming Monday (30.01.12), 15:00 UTC.
> 
> I'm inviting you all to join this important meeting in the official
> #ovirt irc channel.
> 
> Thanks,
> 
> Ofer Schreiber
> oVirt Release Manager
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From iheim at redhat.com  Sat Jan 28 02:50:29 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sat, 28 Jan 2012 04:50:29 +0200
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>
References: <1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F236275.1050509@redhat.com>

top posting since there was a long thread on this anyway.
some questions/comments:

1. about the CIDs - it sounds like the engine needs to persist this 
info, so it can resume normally in case of a failure/restart (this is 
different than today, when the persisted info is the connection details, 
rather than some generated identifier)?

2. sounds like the engine needs to block in certain cases after a 
manageConnection to make sure it is there and alive before doing an 
operation.
this means now engine has to check a host has all relevant connections 
online before choosing it as a target for live migration even for a 
regular VM (all disks on a storage domain).
worse/uglier (well, imho), in case of a disk based on a direct LUN, the 
engine needs to actively connect the target host, poll till it's up, and 
only then live migrate (would be much nicer if vdsm migration protocol 
would have taken care of this manageConnection call (preserving the CID?)

3. in unmanageStorageServer(connectionID) below you finish with
"Returns:
Success code if VDSM was able to unmanage the connection.
It will return an error if the CID is not registered with VDSM. 
Disconnect failures are not reported. Active unmanaged connections can 
be tracked with getStorageServerList()"

it is not clear if vdsm will retry to disconnect, and how races between 
those retries and new manage connection requests will be handled.
if the connection only becomes unmanaged, there is no way to track and 
clean it up (engine is not supposed to touch the unmanaged connections)

4. I don't think we handle this today, but while we are planning for the 
future - what if the host needs one of the connections to exist 
regardless of engine for another need (say it does boot from network 
from same iscsi target - this is an unmanaged connection which you will 
disconnect based on the CID refcount concept).
i.e., what happens if the host has an unmanaged connection, which 
becomes a managed one.
solving this probably means when adding a connection, need to add an 
unmanaged_existed_before CID for refcount?


On 01/23/2012 11:54 PM, Saggi Mizrahi wrote:
> I have begun work at changing how API clients can control storage connections when interacting with VDSM.
>
> Currently there are 2 API calls:
> connectStorageServer() - Will connect to the storage target if the host is not already connected to it.
> disconnectStorageServer() - Will disconnect from the storage target if the host is connected to it.
>
> This API is very simple but is inappropriate when multiple clients and flows try to access the same storage.
>
> This is currently solved by trying to synchronize things inside rhevm. This is hard and convoluted. It also brings out issues with other clients using the VDSM API.
>
> Another problem is error recovery. Currently ovirt-engine(OE) has no way of monitoring the connections on all the hosts an if a connection disappears it's OE's responsibility to reconnect.
>
> I suggest a different concept where VDSM 'manages' the connections. VDSM receives a manage request with the connection information and from that point forward VDSM will try to keep this connection alive. If the connection fails VDSM will automatically try and recover.
>
> Every manage request will also have a connection ID(CID). This CID will be used when the same client asks to unamange the connection.
> When multiple requests for manage are received to the same connection they all have to have their own unique CID. By internally mapping CIDs to actual connections VDSM can properly disconnect when no CID is addressing the connection. This allows each client and even each flow to have it's own CID effectively eliminating connect\disconnect races.
>
> The change from (dis)connect to (un)manage also changes the semantics of the calls significantly.
> Whereas connectStorageServer would have returned when the storage is either connected or failed to connect, manageStorageServer will return once VDSM registered the CID. This means that the connection might not be active immediately as the VDSM tries to connect. The connection might remain down for a long time if the storage target is down or is having issues.
>
> This allows for VDSM to receive the manage request even if the storage is having issues and recover as soon as it's operational without user intervention.
>
> In order for the client to query the current state of the connections I propose getStorageConnectionList(). This will return a mapping of CID to connection status. The status contains the connection info (excluding credentials), whether the connection is active, whether the connection is managed (unamanged connection are returned with transient IDs), and, if the connection is down, the last error information.
>
> The same actual connection can return multiple times, once for each CID.
>
> For cases where an operation requires a connection to be active a user can poll the status of the CID. The user can then choose to poll for a certain amount of time or until an error appears in the error field of the status. This will give you either a timeout or a "try once" semantic depending on the flows needs.
>
> All connections that have been managed persist VDSM restart and will be managed until a corresponding unmanage command has been issued.
>
> There is no concept of temporary connections as "temporary" is flow dependent and VDSM can't accommodate all interpretation of "temporary". An ad-hoc mechanism can be build using the CID field. For instance a client can manage a connection with "ENGINE_FLOW101_CON1". If the flow got interrupted the client can clean all IDs with certain flow IDs.
>
> I think this API gives safety, robustness, and implementation freedom.
>
>
> Nitty Gritty:
>
> manageStorageServer
> ===================
> Synopsis:
> manageStorageServer(uri, connectionID):
>
> Parameters:
> uri - a uri pointing to a storage target (eg: nfs://server:export, iscsi://host/iqn;portal=1)
> connectionID - string with any char except "/".
>
> Description:
> Tells VDSM to start managing the connection. From this moment on VDSM will try and have the connection available when needed. VDSM will monitor the connection and will automatically reconnect on failure.
> Returns:
> Success code if VDSM was able to manage the connection.
> It usually just verifies that the arguments are sane and that the CID is not already in use.
> This doesn't mean the host is connected.
> ----
> unmanageStorageServer
> =====================
> Synopsis:
> unmanageStorageServer(connectionID):
>
> Parameters:
> connectionID - string with any char except "/".
>
> Descriptions:
> Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.
>
> Returns:
> Success code if VDSM was able to unmanage the connection.
> It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
> ----
> getStorageServerList
> ====================
> Synopsis:
> getStorageServerList()
>
> Description:
> Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.
>
> Results:VDSM was able to manage the connection.
> It usually just verifies that the arguments are sane and that the CID is not already in use.
> This doesn't mean the host is connected.
> ----
> unmanageStorageServer
> =====================
> Synopsis:
> unmanageStorageServer(connectionID):
>
> Parameters:
> connectionID - string with any char except "/".
>
> Descriptions:
> Tells VDSM to stop managing the connection. VDSM will try and disconnect for the storage target if this is the last CID referencing the storage connection.
>
> Returns:
> Success code if VDSM was able to unmanage the connection.
> It will return an error if the CID is not registered with VDSM. Disconnect failures are not reported. Active unmanaged connections can be tracked with getStorageServerList()
> ----
> getStorageServerList
> ====================
> Synopsis:
> getStorageServerList()
>
> Description:
> Will return list of all managed and unmanaged connections. Unmanaged connections have temporary IDs and are not guaranteed to be consistent across calls.
>
> Results:
> A mapping between CIDs and the status.
> example return value (Actual key names may differ)
>
> {'conA': {'connected': True, 'managed': True, 'lastError': 0, 'connectionInfo': {
>      'remotePath': 'server:/export
>      'retrans': 3
>      'version': 4
>      }}
>   'iscsi_session_34': {'connected': False, 'managed': False, 'lastError': 339, 'connectionIfno': {
>      'hostname': 'dandylopn'
>      'portal': 1}}
> }
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From lpeer at redhat.com  Sat Jan 28 14:25:11 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sat, 28 Jan 2012 16:25:11 +0200
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <9c119395-58f8-4b15-9a82-501d2db6bdcc@zmail16.collab.prod.int.phx2.redhat.com>
References: <9c119395-58f8-4b15-9a82-501d2db6bdcc@zmail16.collab.prod.int.phx2.redhat.com>
Message-ID: <4F240547.2000402@redhat.com>

On 26/01/12 23:42, Saggi Mizrahi wrote:
> 
> 
> ----- Original Message -----
>> From: "Livnat Peer" <lpeer at redhat.com>
>> To: "Saggi Mizrahi" <smizrahi at redhat.com>
>> Cc: "Adam Litke" <agl at us.ibm.com>, engine-devel at ovirt.org, vdsm-devel at lists.fedorahosted.org
>> Sent: Thursday, January 26, 2012 3:16:32 PM
>> Subject: Re: [vdsm] [Engine-devel] [RFC] New Connection Management API
>>
>> On 26/01/12 21:21, Saggi Mizrahi wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Adam Litke" <agl at us.ibm.com>
>>>> To: "Saggi Mizrahi" <smizrahi at redhat.com>
>>>> Cc: "Livnat Peer" <lpeer at redhat.com>, engine-devel at ovirt.org,
>>>> vdsm-devel at lists.fedorahosted.org
>>>> Sent: Thursday, January 26, 2012 1:58:40 PM
>>>> Subject: Re: [vdsm] [Engine-devel] [RFC] New Connection Management
>>>> API
>>>>
>>>> On Thu, Jan 26, 2012 at 10:00:57AM -0500, Saggi Mizrahi wrote:
>>>>> <snip>
>>>>> Again trying to sum up and address all comments
>>>>>
>>>>> Clear all:
>>>>> ==========
>>>>> My opinions is still to not implement it.
>>>>> Even though it might generate a bit more traffic premature
>>>>> optimization is bad and there are other reasons we can improve
>>>>> VDSM command overhead without doing this.
>>>>>
>>>>> In any case this argument is redundant because my intention is
>>>>> (as
>>>>> Litke pointed out) is to have a lean API.
>>>>> and API call is something you have to support across versions,
>>>>> this
>>>>> call implemented in the engine is something that no one has to
>>>>> support and can change\evolve easily.
>>>>>
>>>>> As a rule, if an API call C and be implemented by doing A + B
>>>>> then
>>>>> C is redundant.
>>>>>
>>>>> List of connections as args:
>>>>> ============================
>>>>> Sorry I forgot to respond about that. I'm not as strongly opposed
>>>>> to the idea as the other things you suggested. It'll just make
>>>>> implementing the persistence logic in VDSM significantly more
>>>>> complicated as I will have to commit multiple connection
>>>>> information to disk in an all or nothing mode. I can create a
>>>>> small sqlitedb to do that or do some directory tricks and exploit
>>>>> FS rename atomicity but I'd rather not.
>>>>
>>>> I would be strongly opposed to introducing a sqlite database into
>>>> vdsm just to
>>>> enable "convenience mode" for this API.  Does the operation really
>>>> need to be
>>>> atomic?  Why not just perform each connection sequentially and
>>>> return
>>>> a list of
>>>> statuses? Is the only motivation for allowing a list of parameters
>>>> to reduce
>>>> the number of API calls between engine and vdsm)?  If so, the same
>>>> argument
>>>> Saggi makes above applies here.
>>>
>>> I try and have VDSM expose APIs that are simple to predict. a
>>> command can either succeed or fail.
>>> The problem is not actually validating the connections. The problem
>>> is that once I concluded that they are all OK I need to persist to
>>> disk the information that will allow me to reconnect if VDSM
>>> happens to crash. If I naively save them one by one I could get in
>>> a state where only some of the connections persisted before the
>>> operation failed. So I have to somehow put all this in a
>>> transaction.
>>>
>>> I don't have to use sqlite. I could also put all the persistence
>>> information in a new dir for every call named <UUID>.tmp. Once I
>>> wrote everything down I rename the directory to just <UUID> and
>>> fsync it. This is guarantied by posix to be atomic. For unmanage,
>>> I move all the persistence information from the directories they
>>> sit in to a new dir named <UUID>. Rename it to a <UUDI>.tmp, fsync
>>> it and then remove it.
>>>
>>> This all just looks like more trouble then it's worth to me.
>>>
>>
>>
>> I agree with Adam, I don't think the operation should be atomic,
>> having
>> only some of the connections persisted is a perfectly valid outcome
>> if
>> the API returns a list of statuses.
> What if it doesn't return at all?
> The only reasons that something will fail manage is if the URI is broken so I assume 99% of the issued manage commands will succeed.
> My problem is with VDSM crashing mid operation. The operation will appear to fail but when VDSM returns some of the connections persisted so it will reconnect. because the client's manage call failed it doesn't expect the CIDs to be in the list. This will cause ambiguity when finding an already registered CID at runtime.
> 

I think that if VDSM did not return at all, it is reasonable expectation
to use the status verb for finding the connections status (managed or not).

general comment - It would help if manage verb returns a dedicated error
code which indicates that the CID is already managed.



>>
>>
>>>>
>>>>> The demands are not without base. I would like to keep the code
>>>>> simple under the hood in the price of a few more calls. You would
>>>>> like to make less calls and keep the code simpler on your side.
>>>>> There isn't a real way to settle this.
>>>>> If anyone on the list as pros and cons for either way I'd be
>>>>> happy
>>>>> to hear them.
>>>>> If no compelling arguments arise I will let Ayal call this one.
>>>>>
>>>>> Transient connections:
>>>>> ======================
>>>>> The problem you are describing as I understand it is that VDSM
>>>>> did
>>>>> not respond and not that the API client did not respond.
>>>>> Again, this can happen for a number of reason, most of which VDSM
>>>>> might not be aware that there is actually a problem (network
>>>>> issues).
>>>>>
>>>>> This relates to the EOL policy. I agree we have to find a good
>>>>> way
>>>>> to define an automatic EOL for resources. I have made my
>>>>> suggestion. Out of the scope of the API.
>>>>>
>>>>> In the meantime cleaning stale connections is trivial and I have
>>>>> made it clear a previous email about how to go about it in a
>>>>> simple non intrusive way. Clean hosts on host connect, and on
>>>>> every poll if you find connections that you don't like. This
>>>>> should keep things squeaky clean.
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>>>> To: "Saggi Mizrahi" <smizrahi at redhat.com>
>>>>>> Cc: vdsm-devel at lists.fedorahosted.org, engine-devel at ovirt.org
>>>>>> Sent: Thursday, January 26, 2012 5:22:42 AM
>>>>>> Subject: Re: [Engine-devel] [RFC] New Connection Management API
>>>>>>
>>>>>> On 25/01/12 23:35, Saggi Mizrahi wrote:
>>>>>>> <SNIP>
>>>>>>> This is mail was getting way too long.
>>>>>>>
>>>>>>> About the clear all verb.
>>>>>>> No.
>>>>>>> Just loop, find the connections YOU OWN and clean them. Even
>>>>>>> though
>>>>>>> you don't want to support multiple clients to VDSM API doesn't
>>>>>>> mean the engine shouldn't behave like a proper citizen.
>>>>>>> It's the same reason why VDSM tries and not mess system
>>>>>>> resources
>>>>>>> it didn't initiate.
>>>>>>
>>>>>>
>>>>>> There is a big difference, VDSM living in hybrid mode with other
>>>>>> workload on the host is a valid use case, having more than one
>>>>>> concurrent manager for VDSM is not.
>>>>>> Generating a disconnect request for each connection does not
>>>>>> seem
>>>>>> like
>>>>>> the right API to me, again think on the simple flow of moving
>>>>>> host
>>>>>> from
>>>>>> one data center to another, the engine needs to disconnect tall
>>>>>> storage
>>>>>> domains (each domain can have couple of connections associated
>>>>>> with
>>>>>> it).
>>>>>>
>>>>>> I am giving example from the engine use cases as it is the main
>>>>>> user
>>>>>> of
>>>>>> VDSM ATM but I am sure it will be relevant to any other user of
>>>>>> VDSM.
>>>>>>
>>>>>>>
>>>>>>> ------------------------
>>>>>>>
>>>>>>> As I see it the only point of conflict is the so called
>>>>>>> non-peristed connections.
>>>>>>> I will call them transient connections from now on.
>>>>>>>
>>>>>>> There are 2 user cases being discussed
>>>>>>> 1. Wait until a connection is made, if it fails don't retry and
>>>>>>> automatically unmanage.
>>>>>>> 2. If the called of the API forgets or fails to unmanage a
>>>>>>> connection.
>>>>>>>
>>>>>>
>>>>>> Actually I was not discussing #2 at all.
>>>>>>
>>>>>>> Your suggestion as I understand it:
>>>>>>> Transient connections are:
>>>>>>>      - Connection that VDSM will only try to connect to once
>>>>>>>      and
>>>>>>>      will not reconnect to in case of disconnect.
>>>>>>
>>>>>> yes
>>>>>>
>>>>>>>
>>>>>>> My problem with this definition that it does not specify the
>>>>>>> "end
>>>>>>> of life" of the connection.
>>>>>>> Meaning it solves only use case 1.
>>>>>>
>>>>>> since this is the only use case i had in mind, it is what i was
>>>>>> looking for.
>>>>>>
>>>>>>> If all is well, and it usually is, VDSM will not invoke a
>>>>>>> disconnect.
>>>>>>> So the caller would have to call unmanage if the connection
>>>>>>> succeeded at the end of the flow.
>>>>>>
>>>>>> agree.
>>>>>>
>>>>>>> Now, if you are already calling unmanage if connection
>>>>>>> succeeded
>>>>>>> you can just call it anyway.
>>>>>>
>>>>>> not exactly, an example I gave earlier on the thread was that
>>>>>> VSDM
>>>>>> hangs
>>>>>> or have other error and the engine can not initiate unmanaged,
>>>>>> instead
>>>>>> let's assume the host is fenced (self-fence or external fence
>>>>>> does
>>>>>> not
>>>>>> matter), in this scenario the engine will not issue unmanage.
>>>>>>
>>>>>>>
>>>>>>> instead of doing: (with your suggestion)
>>>>>>> ----------------
>>>>>>> manage
>>>>>>> wait until succeeds or lastError has value
>>>>>>> try:
>>>>>>>   do stuff
>>>>>>> finally:
>>>>>>>   unmanage
>>>>>>>
>>>>>>> do: (with the canonical flow)
>>>>>>> ---
>>>>>>> manage
>>>>>>> try:
>>>>>>>   wait until succeeds or lastError has value
>>>>>>>   do stuff
>>>>>>> finally:
>>>>>>>   unmanage
>>>>>>>
>>>>>>> This is simpler to do than having another connection type.
>>>>>>
>>>>>> You are assuming the engine can communicate with VDSM and there
>>>>>> are
>>>>>> scenarios where it is not feasible.
>>>>>>
>>>>>>>
>>>>>>> Now that we got that out of the way lets talk about the 2nd use
>>>>>>> case.
>>>>>>
>>>>>> Since I did not ask VDSM to clean after the (engine) user and
>>>>>> you
>>>>>> don't
>>>>>> want to do it I am not sure we need to discuss this.
>>>>>>
>>>>>> If you insist we can start the discussion on who should
>>>>>> implement
>>>>>> the
>>>>>> cleanup mechanism but I'm afraid I have no strong arguments for
>>>>>> VDSM
>>>>>> to
>>>>>> do it, so I rather not go there ;)
>>>>>>
>>>>>>
>>>>>> You dropped from the discussion my request for supporting list
>>>>>> of
>>>>>> connections for manage and unmanage verbs.
>>>>>>
>>>>>>> API client died in the middle of the operation and unmanage was
>>>>>>> never called.
>>>>>>>
>>>>>>> Your suggested definition means that unless there was a problem
>>>>>>> with the connection VDSM will still have this connection
>>>>>>> active.
>>>>>>> The engine will have to clean it anyway.
>>>>>>>
>>>>>>> The problem is, VDSM has no way of knowing that a client died,
>>>>>>> forgot or is thinking really hard and will continue on in about
>>>>>>> 2
>>>>>>> minutes.
>>>>>>
>>>>>>>
>>>>>>> Connections that live until they die is a hard to define and
>>>>>>> work
>>>>>>> with lifecycle. Solving this problem is theoretically simple.
>>>>>>>
>>>>>>> Have clients hold some sort of session token and force the
>>>>>>> client
>>>>>>> to update it at a specified interval. You could bind resources
>>>>>>> (like domains, VMs, connections) to that session token so when
>>>>>>> it
>>>>>>> expires VDSM auto cleans the resources.
>>>>>>>
>>>>>>> This kind of mechanism is out of the scope of this API change.
>>>>>>> Further more I think that this mechanism should sit in the
>>>>>>> engine
>>>>>>> since the session might actually contain resources from
>>>>>>> multiple
>>>>>>> hosts and resources that are not managed by VDSM.
>>>>>>>
>>>>>>> In GUI flows specifically the user might do actions that don't
>>>>>>> even
>>>>>>> touch the engine and forcing it to refresh the engine token is
>>>>>>> simpler then having it refresh the VDSM token.
>>>>>>>
>>>>>>> I understand that engine currently has no way of tracking a
>>>>>>> user
>>>>>>> session. This, as I said, is also true in the case of VDSM. We
>>>>>>> can
>>>>>>> start and argue about which project should implement the
>>>>>>> session
>>>>>>> semantics. But as I see it it's not relevant to the connection
>>>>>>> management API.
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> vdsm-devel mailing list
>>>>> vdsm-devel at lists.fedorahosted.org
>>>>> https://fedorahosted.org/mailman/listinfo/vdsm-devel
>>>>
>>>> --
>>>> Adam Litke <agl at us.ibm.com>
>>>> IBM Linux Technology Center
>>>>
>>>>
>>
>>



From lpeer at redhat.com  Sat Jan 28 17:09:06 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Sat, 28 Jan 2012 19:09:06 +0200
Subject: [Engine-devel] [RFC] New Connection Management API
In-Reply-To: <4F236275.1050509@redhat.com>
References: <1af53449-c304-4153-90a5-ef56c6efa1fa@zmail16.collab.prod.int.phx2.redhat.com>
	<4F236275.1050509@redhat.com>
Message-ID: <4F242BB2.9020709@redhat.com>

On 28/01/12 04:50, Itamar Heim wrote:
> top posting since there was a long thread on this anyway.
> some questions/comments:
> 
> 1. about the CIDs - it sounds like the engine needs to persist this
> info, so it can resume normally in case of a failure/restart (this is
> different than today, when the persisted info is the connection details,
> rather than some generated identifier)?

This info should be persisted in the engine, in addition to the
connection details.

> 
> 2. sounds like the engine needs to block in certain cases after a
> manageConnection to make sure it is there and alive before doing an
> operation.
> this means now engine has to check a host has all relevant connections
> online before choosing it as a target for live migration even for a
> regular VM (all disks on a storage domain).

With the current flow it is not needed for 'regular VM'.
The engine currently do not monitor the storage domain's connections on
a periodic basis because the storage domain status represents the
availability of the domain.


> worse/uglier (well, imho), in case of a disk based on a direct LUN, the
> engine needs to actively connect the target host, poll till it's up, and
> only then live migrate (would be much nicer if vdsm migration protocol
> would have taken care of this manageConnection call (preserving the CID?)
> 
> 3. in unmanageStorageServer(connectionID) below you finish with
> "Returns:
> Success code if VDSM was able to unmanage the connection.
> It will return an error if the CID is not registered with VDSM.
> Disconnect failures are not reported. Active unmanaged connections can
> be tracked with getStorageServerList()"
> 
> it is not clear if vdsm will retry to disconnect, and how races between
> those retries and new manage connection requests will be handled.
> if the connection only becomes unmanaged, there is no way to track and
> clean it up (engine is not supposed to touch the unmanaged connections)
> 
> 4. I don't think we handle this today, but while we are planning for the
> future - what if the host needs one of the connections to exist
> regardless of engine for another need (say it does boot from network
> from same iscsi target - this is an unmanaged connection which you will
> disconnect based on the CID refcount concept).
> i.e., what happens if the host has an unmanaged connection, which
> becomes a managed one.
> solving this probably means when adding a connection, need to add an
> unmanaged_existed_before CID for refcount?
> 
> 
> On 01/23/2012 11:54 PM, Saggi Mizrahi wrote:
>> I have begun work at changing how API clients can control storage
>> connections when interacting with VDSM.
>>
>> Currently there are 2 API calls:
>> connectStorageServer() - Will connect to the storage target if the
>> host is not already connected to it.
>> disconnectStorageServer() - Will disconnect from the storage target if
>> the host is connected to it.
>>
>> This API is very simple but is inappropriate when multiple clients and
>> flows try to access the same storage.
>>
>> This is currently solved by trying to synchronize things inside rhevm.
>> This is hard and convoluted. It also brings out issues with other
>> clients using the VDSM API.
>>
>> Another problem is error recovery. Currently ovirt-engine(OE) has no
>> way of monitoring the connections on all the hosts an if a connection
>> disappears it's OE's responsibility to reconnect.
>>
>> I suggest a different concept where VDSM 'manages' the connections.
>> VDSM receives a manage request with the connection information and
>> from that point forward VDSM will try to keep this connection alive.
>> If the connection fails VDSM will automatically try and recover.
>>
>> Every manage request will also have a connection ID(CID). This CID
>> will be used when the same client asks to unamange the connection.
>> When multiple requests for manage are received to the same connection
>> they all have to have their own unique CID. By internally mapping CIDs
>> to actual connections VDSM can properly disconnect when no CID is
>> addressing the connection. This allows each client and even each flow
>> to have it's own CID effectively eliminating connect\disconnect races.
>>
>> The change from (dis)connect to (un)manage also changes the semantics
>> of the calls significantly.
>> Whereas connectStorageServer would have returned when the storage is
>> either connected or failed to connect, manageStorageServer will return
>> once VDSM registered the CID. This means that the connection might not
>> be active immediately as the VDSM tries to connect. The connection
>> might remain down for a long time if the storage target is down or is
>> having issues.
>>
>> This allows for VDSM to receive the manage request even if the storage
>> is having issues and recover as soon as it's operational without user
>> intervention.
>>
>> In order for the client to query the current state of the connections
>> I propose getStorageConnectionList(). This will return a mapping of
>> CID to connection status. The status contains the connection info
>> (excluding credentials), whether the connection is active, whether the
>> connection is managed (unamanged connection are returned with
>> transient IDs), and, if the connection is down, the last error
>> information.
>>
>> The same actual connection can return multiple times, once for each CID.
>>
>> For cases where an operation requires a connection to be active a user
>> can poll the status of the CID. The user can then choose to poll for a
>> certain amount of time or until an error appears in the error field of
>> the status. This will give you either a timeout or a "try once"
>> semantic depending on the flows needs.
>>
>> All connections that have been managed persist VDSM restart and will
>> be managed until a corresponding unmanage command has been issued.
>>
>> There is no concept of temporary connections as "temporary" is flow
>> dependent and VDSM can't accommodate all interpretation of
>> "temporary". An ad-hoc mechanism can be build using the CID field. For
>> instance a client can manage a connection with "ENGINE_FLOW101_CON1".
>> If the flow got interrupted the client can clean all IDs with certain
>> flow IDs.
>>
>> I think this API gives safety, robustness, and implementation freedom.
>>
>>
>> Nitty Gritty:
>>
>> manageStorageServer
>> ===================
>> Synopsis:
>> manageStorageServer(uri, connectionID):
>>
>> Parameters:
>> uri - a uri pointing to a storage target (eg: nfs://server:export,
>> iscsi://host/iqn;portal=1)
>> connectionID - string with any char except "/".
>>
>> Description:
>> Tells VDSM to start managing the connection. From this moment on VDSM
>> will try and have the connection available when needed. VDSM will
>> monitor the connection and will automatically reconnect on failure.
>> Returns:
>> Success code if VDSM was able to manage the connection.
>> It usually just verifies that the arguments are sane and that the CID
>> is not already in use.
>> This doesn't mean the host is connected.
>> ----
>> unmanageStorageServer
>> =====================
>> Synopsis:
>> unmanageStorageServer(connectionID):
>>
>> Parameters:
>> connectionID - string with any char except "/".
>>
>> Descriptions:
>> Tells VDSM to stop managing the connection. VDSM will try and
>> disconnect for the storage target if this is the last CID referencing
>> the storage connection.
>>
>> Returns:
>> Success code if VDSM was able to unmanage the connection.
>> It will return an error if the CID is not registered with VDSM.
>> Disconnect failures are not reported. Active unmanaged connections can
>> be tracked with getStorageServerList()
>> ----
>> getStorageServerList
>> ====================
>> Synopsis:
>> getStorageServerList()
>>
>> Description:
>> Will return list of all managed and unmanaged connections. Unmanaged
>> connections have temporary IDs and are not guaranteed to be consistent
>> across calls.
>>
>> Results:VDSM was able to manage the connection.
>> It usually just verifies that the arguments are sane and that the CID
>> is not already in use.
>> This doesn't mean the host is connected.
>> ----
>> unmanageStorageServer
>> =====================
>> Synopsis:
>> unmanageStorageServer(connectionID):
>>
>> Parameters:
>> connectionID - string with any char except "/".
>>
>> Descriptions:
>> Tells VDSM to stop managing the connection. VDSM will try and
>> disconnect for the storage target if this is the last CID referencing
>> the storage connection.
>>
>> Returns:
>> Success code if VDSM was able to unmanage the connection.
>> It will return an error if the CID is not registered with VDSM.
>> Disconnect failures are not reported. Active unmanaged connections can
>> be tracked with getStorageServerList()
>> ----
>> getStorageServerList
>> ====================
>> Synopsis:
>> getStorageServerList()
>>
>> Description:
>> Will return list of all managed and unmanaged connections. Unmanaged
>> connections have temporary IDs and are not guaranteed to be consistent
>> across calls.
>>
>> Results:
>> A mapping between CIDs and the status.
>> example return value (Actual key names may differ)
>>
>> {'conA': {'connected': True, 'managed': True, 'lastError': 0,
>> 'connectionInfo': {
>>      'remotePath': 'server:/export
>>      'retrans': 3
>>      'version': 4
>>      }}
>>   'iscsi_session_34': {'connected': False, 'managed': False,
>> 'lastError': 339, 'connectionIfno': {
>>      'hostname': 'dandylopn'
>>      'portal': 1}}
>> }
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From abaron at redhat.com  Sun Jan 29 08:49:15 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 29 Jan 2012 03:49:15 -0500 (EST)
Subject: [Engine-devel] [vdsm]  [RFC] New Connection Management API
In-Reply-To: <4F236275.1050509@redhat.com>
Message-ID: <f914ba20-c130-459a-8f23-71ba844632ed@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> top posting since there was a long thread on this anyway.
> some questions/comments:
> 
> 1. about the CIDs - it sounds like the engine needs to persist this
> info, so it can resume normally in case of a failure/restart (this is
> different than today, when the persisted info is the connection
> details,
> rather than some generated identifier)?

It doesn't have to.
engine can have 2 types of CIDs:
1. for engine internal flows:
- This would mostly be around a storage domain so the CID can start with the sd name and whenever engine wants to disconnect the connections of an sd.
Several use cases for this -
a. when moving a host between DCs, and don't tell me engine can't run operations on a host in 'maintenance' mode, either solve this silliness or allow moving a host directly between DCs without it being in maintenance).
b. when removing a connection from the storage domain definition
c. when removing the storage domain from the db
Anyway, in the above cases you have 2 options:
i. just getStorageConnectionList and disconnect anything that starts with this domain name
ii. If the name is constant (e.g. SD_NAME-CONN-IQN) then you can always 'build it'

2. for UI generated flows
I see no reason to persist anything here, just need a connection cleanup flow to get rid of irrelevant connections (you would need this anyway).

> 
> 2. sounds like the engine needs to block in certain cases after a
> manageConnection to make sure it is there and alive before doing an
> operation.

I don't see how this is different from today.

> this means now engine has to check a host has all relevant
> connections
> online before choosing it as a target for live migration even for a
> regular VM (all disks on a storage domain).
> worse/uglier (well, imho), in case of a disk based on a direct LUN,
> the
> engine needs to actively connect the target host, poll till it's up,
> and
> only then live migrate (would be much nicer if vdsm migration
> protocol
> would have taken care of this manageConnection call (preserving the
> CID?)

Today all hosts are connected to all the storage connections (that are needed for storage domains) beforehand and when a VM is migrated the connection is assumed to be on the target host.  What is the difference?
You preconnect to make sure the host is a valid target for migration.
If you keep a set of hosts always connected you can better ensure SLAs.

> 
> 3. in unmanageStorageServer(connectionID) below you finish with
> "Returns:
> Success code if VDSM was able to unmanage the connection.
> It will return an error if the CID is not registered with VDSM.
> Disconnect failures are not reported. Active unmanaged connections
> can
> be tracked with getStorageServerList()"
> 
> it is not clear if vdsm will retry to disconnect, and how races
> between
> those retries and new manage connection requests will be handled.
> if the connection only becomes unmanaged, there is no way to track
> and
> clean it up (engine is not supposed to touch the unmanaged
> connections)

Basically disconnect should never fail, if it does then there is a bug somewhere as it does not require access to a remote host.
If and when this happens, qe will open a bug.
Unless we have a ton of manage ops and then unmanage ops which fail this will not be an issue at all.

> 
> 4. I don't think we handle this today, but while we are planning for
> the
> future - what if the host needs one of the connections to exist
> regardless of engine for another need (say it does boot from network
> from same iscsi target - this is an unmanaged connection which you
> will
> disconnect based on the CID refcount concept).
> i.e., what happens if the host has an unmanaged connection, which
> becomes a managed one.
> solving this probably means when adding a connection, need to add an
> unmanaged_existed_before CID for refcount?

So what happens if the order is reveresed? i.e. manage is called by ovirt and then the other application calls connect underneath? vdsm would have no idea this happened and would disconnect when unmanage arrives.
The solution is simple, either separate connections (hybrid mode doesn't mean we share connections) or the other application has to be aware it is running in hybrid mode and then go through vdsm for connecting (no app is hybrid aware today).

> 
> 
> On 01/23/2012 11:54 PM, Saggi Mizrahi wrote:
> > I have begun work at changing how API clients can control storage
> > connections when interacting with VDSM.
> >
> > Currently there are 2 API calls:
> > connectStorageServer() - Will connect to the storage target if the
> > host is not already connected to it.
> > disconnectStorageServer() - Will disconnect from the storage target
> > if the host is connected to it.
> >
> > This API is very simple but is inappropriate when multiple clients
> > and flows try to access the same storage.
> >
> > This is currently solved by trying to synchronize things inside
> > rhevm. This is hard and convoluted. It also brings out issues with
> > other clients using the VDSM API.
> >
> > Another problem is error recovery. Currently ovirt-engine(OE) has
> > no way of monitoring the connections on all the hosts an if a
> > connection disappears it's OE's responsibility to reconnect.
> >
> > I suggest a different concept where VDSM 'manages' the connections.
> > VDSM receives a manage request with the connection information and
> > from that point forward VDSM will try to keep this connection
> > alive. If the connection fails VDSM will automatically try and
> > recover.
> >
> > Every manage request will also have a connection ID(CID). This CID
> > will be used when the same client asks to unamange the connection.
> > When multiple requests for manage are received to the same
> > connection they all have to have their own unique CID. By
> > internally mapping CIDs to actual connections VDSM can properly
> > disconnect when no CID is addressing the connection. This allows
> > each client and even each flow to have it's own CID effectively
> > eliminating connect\disconnect races.
> >
> > The change from (dis)connect to (un)manage also changes the
> > semantics of the calls significantly.
> > Whereas connectStorageServer would have returned when the storage
> > is either connected or failed to connect, manageStorageServer will
> > return once VDSM registered the CID. This means that the
> > connection might not be active immediately as the VDSM tries to
> > connect. The connection might remain down for a long time if the
> > storage target is down or is having issues.
> >
> > This allows for VDSM to receive the manage request even if the
> > storage is having issues and recover as soon as it's operational
> > without user intervention.
> >
> > In order for the client to query the current state of the
> > connections I propose getStorageConnectionList(). This will return
> > a mapping of CID to connection status. The status contains the
> > connection info (excluding credentials), whether the connection is
> > active, whether the connection is managed (unamanged connection
> > are returned with transient IDs), and, if the connection is down,
> > the last error information.
> >
> > The same actual connection can return multiple times, once for each
> > CID.
> >
> > For cases where an operation requires a connection to be active a
> > user can poll the status of the CID. The user can then choose to
> > poll for a certain amount of time or until an error appears in the
> > error field of the status. This will give you either a timeout or
> > a "try once" semantic depending on the flows needs.
> >
> > All connections that have been managed persist VDSM restart and
> > will be managed until a corresponding unmanage command has been
> > issued.
> >
> > There is no concept of temporary connections as "temporary" is flow
> > dependent and VDSM can't accommodate all interpretation of
> > "temporary". An ad-hoc mechanism can be build using the CID field.
> > For instance a client can manage a connection with
> > "ENGINE_FLOW101_CON1". If the flow got interrupted the client can
> > clean all IDs with certain flow IDs.
> >
> > I think this API gives safety, robustness, and implementation
> > freedom.
> >
> >
> > Nitty Gritty:
> >
> > manageStorageServer
> > ===================
> > Synopsis:
> > manageStorageServer(uri, connectionID):
> >
> > Parameters:
> > uri - a uri pointing to a storage target (eg: nfs://server:export,
> > iscsi://host/iqn;portal=1)
> > connectionID - string with any char except "/".
> >
> > Description:
> > Tells VDSM to start managing the connection. From this moment on
> > VDSM will try and have the connection available when needed. VDSM
> > will monitor the connection and will automatically reconnect on
> > failure.
> > Returns:
> > Success code if VDSM was able to manage the connection.
> > It usually just verifies that the arguments are sane and that the
> > CID is not already in use.
> > This doesn't mean the host is connected.
> > ----
> > unmanageStorageServer
> > =====================
> > Synopsis:
> > unmanageStorageServer(connectionID):
> >
> > Parameters:
> > connectionID - string with any char except "/".
> >
> > Descriptions:
> > Tells VDSM to stop managing the connection. VDSM will try and
> > disconnect for the storage target if this is the last CID
> > referencing the storage connection.
> >
> > Returns:
> > Success code if VDSM was able to unmanage the connection.
> > It will return an error if the CID is not registered with VDSM.
> > Disconnect failures are not reported. Active unmanaged connections
> > can be tracked with getStorageServerList()
> > ----
> > getStorageServerList
> > ====================
> > Synopsis:
> > getStorageServerList()
> >
> > Description:
> > Will return list of all managed and unmanaged connections.
> > Unmanaged connections have temporary IDs and are not guaranteed to
> > be consistent across calls.
> >
> > Results:VDSM was able to manage the connection.
> > It usually just verifies that the arguments are sane and that the
> > CID is not already in use.
> > This doesn't mean the host is connected.
> > ----
> > unmanageStorageServer
> > =====================
> > Synopsis:
> > unmanageStorageServer(connectionID):
> >
> > Parameters:
> > connectionID - string with any char except "/".
> >
> > Descriptions:
> > Tells VDSM to stop managing the connection. VDSM will try and
> > disconnect for the storage target if this is the last CID
> > referencing the storage connection.
> >
> > Returns:
> > Success code if VDSM was able to unmanage the connection.
> > It will return an error if the CID is not registered with VDSM.
> > Disconnect failures are not reported. Active unmanaged connections
> > can be tracked with getStorageServerList()
> > ----
> > getStorageServerList
> > ====================
> > Synopsis:
> > getStorageServerList()
> >
> > Description:
> > Will return list of all managed and unmanaged connections.
> > Unmanaged connections have temporary IDs and are not guaranteed to
> > be consistent across calls.
> >
> > Results:
> > A mapping between CIDs and the status.
> > example return value (Actual key names may differ)
> >
> > {'conA': {'connected': True, 'managed': True, 'lastError': 0,
> > 'connectionInfo': {
> >      'remotePath': 'server:/export
> >      'retrans': 3
> >      'version': 4
> >      }}
> >   'iscsi_session_34': {'connected': False, 'managed': False,
> >   'lastError': 339, 'connectionIfno': {
> >      'hostname': 'dandylopn'
> >      'portal': 1}}
> > }
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/vdsm-devel
> 


From oschreib at redhat.com  Sun Jan 29 08:53:46 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Sun, 29 Jan 2012 10:53:46 +0200
Subject: [Engine-devel] First release go/no go meeting on Monday (30.01)
In-Reply-To: <b9545daf-8d94-4487-8274-b36b1ecb606c@zmail15.collab.prod.int.phx2.redhat.com>
References: <b9545daf-8d94-4487-8274-b36b1ecb606c@zmail15.collab.prod.int.phx2.redhat.com>
Message-ID: <4F25091A.6070101@redhat.com>

On 01/27/2012 03:37 PM, Jon Choate wrote:
> Will there be a formal release vote (3 +1's) in this meeting?
>
> Also,I don't see a link on the site for a release candidate.
> Is there one available so the release can be evaluated/tested before deciding on the go/no-go?

Since this is our first release, we're still trying to produce a formal 
release process.

Although we didn't talk about a formal release vote, that sounds like 
the general intention of the go/no go meeting.

I'll send a separate mail about  a formal RC.

>
> ----- Original Message -----
>> From: "Ofer Schreiber"<oschreib at redhat.com>
>> To: board at ovirt.org, arch at ovirt.org, engine-devel at ovirt.org
>> Sent: Thursday, January 26, 2012 4:47:17 AM
>> Subject: [Engine-devel] First release go/no go meeting on Monday (30.01)
>>
>> All,
>>
>> As decided in the last oVirt sync meeting, we will have a go/no
>> meeting
>> about our first release in the upcoming Monday (30.01.12), 15:00 UTC.
>>
>> I'm inviting you all to join this important meeting in the official
>> #ovirt irc channel.
>>
>> Thanks,
>>
>> Ofer Schreiber
>> oVirt Release Manager
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From oschreib at redhat.com  Sun Jan 29 09:25:44 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Sun, 29 Jan 2012 11:25:44 +0200
Subject: [Engine-devel] New oVirt-engine RPMs available - Release Candidates
Message-ID: <4F251098.6080503@redhat.com>

Hi all,

We've just finished uploading a new set of oVirt-engine rpms into
ovirt.org.

These rpms, versioned 3.0.0_0001-1.4, are considered as release candidates for our first release.

In order to install the RPMs, please follow the instructions described in
http://www.ovirt.org/wiki/Installing_ovirt_from_rpm

Feedback, comments and bug reports are always welcome.

The oVirt-Engine team



From gchaplik at redhat.com  Sun Jan 29 14:06:21 2012
From: gchaplik at redhat.com (Gilad Chaplik)
Date: Sun, 29 Jan 2012 09:06:21 -0500 (EST)
Subject: [Engine-devel] better gwt compile times
In-Reply-To: <46b1211f-228b-4d81-91fb-8135b3ef3871@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <273a3ea5-7e59-45f2-8248-9a33e18bbf86@zmail14.collab.prod.int.phx2.redhat.com>

Use draftCompile flag: Enable faster, but less-optimized, compilations.

draft compile only for gecko:
-Dgwt.userAgent=gecko1_8 -Dgwt.draftCompile=true, Total time: 2:10.928s (3:05s without it)
Saves a cool one minute (more time to be stuck in traffic)

*obviously gwt developers should test their code fully compiled (for all browsers and without drafCompile flag)

Thanks,
Gilad.


From acathrow at redhat.com  Mon Jan 30 03:10:49 2012
From: acathrow at redhat.com (Andrew Cathrow)
Date: Sun, 29 Jan 2012 22:10:49 -0500 (EST)
Subject: [Engine-devel] New oVirt-engine RPMs available - Release
	Candidates
In-Reply-To: <4F251098.6080503@redhat.com>
Message-ID: <2d5c7bd5-c6bf-409f-908d-91bacdf20121@zmail07.collab.prod.int.phx2.redhat.com>


----- Original Message -----
> From: "Ofer Schreiber" <oschreib at redhat.com>
> To: users at ovirt.org, engine-devel at ovirt.org
> Sent: Sunday, January 29, 2012 4:25:44 AM
> Subject: [Engine-devel] New oVirt-engine RPMs available - Release Candidates
> 
> Hi all,
> 
> We've just finished uploading a new set of oVirt-engine rpms into
> ovirt.org.
> 
> These rpms, versioned 3.0.0_0001-1.4, are considered as release
> candidates for our first release.
> 
> In order to install the RPMs, please follow the instructions
> described in
> http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
> 
> Feedback, comments and bug reports are always welcome.

I'm still working through testing and will file some bugs when I've gotten through some downstream stuff but here's a few issues :

1) engine-setup

After the setup I get the following warning

" * There is less than 4 GB available free memory on the Host.
It is  recommended to have at least 4 GB available memory to run the RHEV Manager."

4GB is what we're using for RHEV but with the reduced overhead of AS7 -vs- EAP5 is that still the right number

But we obviously need to change the "RHEV Manager" reference.

2) engine-config

There's a number of text labels that need to be fixed eg.
oVirtISOsRepositoryPath: "The RHEV-H installation files path" (Value Type: String)

Also we need to fix some of the longer names eg. 
VdcVersion: "oVirt Enterprise Virtualization Engine Manager Version" (Value Type: String)

3) webadmin
Logo in WebAdmin is Red Hat Shadowman, that needs to be updated.
We can't go out with "Browser not supported" messages like this. The browsers installed on f16 - our target platform can't give this error message.


4) VDS Bootstrap

If you check the  box for iptables then it breaks the system with an invalid rule

Looking at the error from my system
"Jan 29 21:38:50 host1 iptables.init[1637]: iptables: Applying firewall rules: iptables-restore v1.4.12: physdev: option "--physdev-is-bridged" cannot be inverted."

The problem is that we are using 
"-A FORWARD -m physdev ! --physdev-is-bridged -j REJECT --reject-with icmp-host-prohibited"

-vs-

"-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT"


On the point of iptables, why do we open libvirt tls port and the snmp port.


After I fixed iptables I still had a problem, the host was showing as non-operational with the error message "NETWORK_UNREACHABLE" (there's a literal missing there)

Looking at the host it appears that the management bridge wasn't created.

The bootstrap log is attached. It shows the operation failing but still marked the bootstrap as successful.
(also in the log we have rhn/satellite references that need to be removed)

I've not looked at the reason for the failure yet, but an empty line at the end of my ifcfg-em1 file looks suspicious.







> 
> The oVirt-Engine team
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: vds_bootstrap.847324.log
Type: text/x-log
Size: 30099 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120129/e1fdc513/attachment.bin>

From jl.alarcon at near.es  Sun Jan 29 23:28:11 2012
From: jl.alarcon at near.es (=?iso-8859-1?Q?Juan_Luis_Alarcon_Ma=F1as?=)
Date: Sun, 29 Jan 2012 23:28:11 +0000
Subject: [Engine-devel] [Users] New oVirt-engine RPMs available -
	Release Candidates
In-Reply-To: <4F251098.6080503@redhat.com>
References: <4F251098.6080503@redhat.com>
Message-ID: <0u8abes4uqd65um4ytfqduap.1327879682868@email.android.com>

I did an engine installation and i think the memory overcommit % ( cluster properties ) for servers and desktops loads are exchanged, at least the descriptions are.


Ofer Schreiber escribi?:
Hi all,

We've just finished uploading a new set of oVirt-engine rpms into
ovirt.org.

These rpms, versioned 3.0.0_0001-1.4, are considered as release candidates for our first release.

In order to install the RPMs, please follow the instructions described in
http://www.ovirt.org/wiki/Installing_ovirt_from_rpm

Feedback, comments and bug reports are always welcome.

The oVirt-Engine team

_______________________________________________
Users mailing list
Users at ovirt.org
http://lists.ovirt.org/mailman/listinfo/users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120129/06192afd/attachment.html>

From ovedo at redhat.com  Mon Jan 30 07:45:47 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Mon, 30 Jan 2012 02:45:47 -0500 (EST)
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <f3c991fe-0246-4188-b94c-32602174e9e2@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <f1eb8c27-4d72-407b-ac0a-972290ac8e99@zmail02.collab.prod.int.phx2.redhat.com>

Hey all,

I've wrote a wiki page on the oVirt development process.
It contains mostly information on the patch review process, patch submission and some git guidelines.

http://www.ovirt.org/wiki/DevProcess

I added a link to it from the main wiki page.

Your comments are welcome.

Thank you,
Oved


From masayag at redhat.com  Mon Jan 30 08:30:01 2012
From: masayag at redhat.com (Moti Asayag)
Date: Mon, 30 Jan 2012 10:30:01 +0200
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <f1eb8c27-4d72-407b-ac0a-972290ac8e99@zmail02.collab.prod.int.phx2.redhat.com>
References: <f1eb8c27-4d72-407b-ac0a-972290ac8e99@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <4F265509.1030807@redhat.com>

We should add some guideline for the review process as well to ease the
process for the reviewers.

When pushing a newer version of a patch to gerrit we can distinguish
between a patch which was already reviewed to non-reviewed patch:

1. A reviewed patch: the previous patch-set contains reviewer comments,
therefore the expected change should be described there, as part of the
conversation around a specific piece of code (either as a reply or using
the 'Done' button).

2. A new reviewed patch: A general message should be added on the patch
level describing what have changed since previous patch-set.

Thoughts ?

On 01/30/2012 09:45 AM, Oved Ourfalli wrote:
> Hey all,
> 
> I've wrote a wiki page on the oVirt development process.
> It contains mostly information on the patch review process, patch submission and some git guidelines.
> 
> http://www.ovirt.org/wiki/DevProcess
> 
> I added a link to it from the main wiki page.
> 
> Your comments are welcome.
> 
> Thank you,
> Oved
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From ovedo at redhat.com  Mon Jan 30 09:27:27 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Mon, 30 Jan 2012 04:27:27 -0500 (EST)
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <4F265509.1030807@redhat.com>
Message-ID: <2d44e58c-f593-4e53-bc6b-e17a5c383cb6@zmail02.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Moti Asayag" <masayag at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Monday, January 30, 2012 10:30:01 AM
> Subject: Re: [Engine-devel] Development process wiki page
> 
> We should add some guideline for the review process as well to ease
> the
> process for the reviewers.
> 
> When pushing a newer version of a patch to gerrit we can distinguish
> between a patch which was already reviewed to non-reviewed patch:
> 
> 1. A reviewed patch: the previous patch-set contains reviewer
> comments,
> therefore the expected change should be described there, as part of
> the
> conversation around a specific piece of code (either as a reply or
> using
> the 'Done' button).
> 
> 2. A new reviewed patch: A general message should be added on the
> patch
> level describing what have changed since previous patch-set.
> 
> Thoughts ?
Sounds good to me.
In case a newer patch version is needed due to gerrit asking you to fetch+rebase, it adds such a general comment message on its own, so in that case one won't need to do it explicitly.

I'll update the oVirt gerrit wiki page with these details (will wait a while for other people to comment...).

> 
> On 01/30/2012 09:45 AM, Oved Ourfalli wrote:
> > Hey all,
> > 
> > I've wrote a wiki page on the oVirt development process.
> > It contains mostly information on the patch review process, patch
> > submission and some git guidelines.
> > 
> > http://www.ovirt.org/wiki/DevProcess
> > 
> > I added a link to it from the main wiki page.
> > 
> > Your comments are welcome.
> > 
> > Thank you,
> > Oved
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From amureini at redhat.com  Mon Jan 30 10:21:35 2012
From: amureini at redhat.com (Allon Mureinik)
Date: Mon, 30 Jan 2012 05:21:35 -0500 (EST)
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <f1eb8c27-4d72-407b-ac0a-972290ac8e99@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <2ddf34d8-8a63-4810-bcf4-83704b5846c8@zmail17.collab.prod.int.phx2.redhat.com>

Not sure if this is the place for it, but I think we should have some guideline regarding what is an 
acceptable change, e.g. - new logic is tested with a unit test, a cobertura report can prove that no 
branches were missed except catching and rethrowing, every public method is javadoced, etc.

----- Original Message -----
> From: "Oved Ourfalli" <ovedo at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Monday, January 30, 2012 9:45:47 AM
> Subject: [Engine-devel] Development process wiki page
> 
> Hey all,
> 
> I've wrote a wiki page on the oVirt development process.
> It contains mostly information on the patch review process, patch
> submission and some git guidelines.
> 
> http://www.ovirt.org/wiki/DevProcess
> 
> I added a link to it from the main wiki page.
> 
> Your comments are welcome.
> 
> Thank you,
> Oved
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From oschreib at redhat.com  Mon Jan 30 11:07:46 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Mon, 30 Jan 2012 13:07:46 +0200
Subject: [Engine-devel] New oVirt-engine RPMs available - Release
	Candidates
In-Reply-To: <2d5c7bd5-c6bf-409f-908d-91bacdf20121@zmail07.collab.prod.int.phx2.redhat.com>
References: <2d5c7bd5-c6bf-409f-908d-91bacdf20121@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <4F267A02.6050003@redhat.com>

On 01/30/2012 05:10 AM, Andrew Cathrow wrote:
> ----- Original Message -----
>> From: "Ofer Schreiber"<oschreib at redhat.com>
>> To: users at ovirt.org, engine-devel at ovirt.org
>> Sent: Sunday, January 29, 2012 4:25:44 AM
>> Subject: [Engine-devel] New oVirt-engine RPMs available - Release Candidates
>>
>> Hi all,
>>
>> We've just finished uploading a new set of oVirt-engine rpms into
>> ovirt.org.
>>
>> These rpms, versioned 3.0.0_0001-1.4, are considered as release
>> candidates for our first release.
>>
>> In order to install the RPMs, please follow the instructions
>> described in
>> http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
>>
>> Feedback, comments and bug reports are always welcome.
> I'm still working through testing and will file some bugs when I've gotten through some downstream stuff but here's a few issues :
>
> 1) engine-setup
>
> After the setup I get the following warning
>
> " * There is less than 4 GB available free memory on the Host.
> It is  recommended to have at least 4 GB available memory to run the RHEV Manager."
>
> 4GB is what we're using for RHEV but with the reduced overhead of AS7 -vs- EAP5 is that still the right number
>
> But we obviously need to change the "RHEV Manager" reference.

Text issue already fixed in main branch.
About the memory concerns, you're right, we should redefine the memory 
requirements.

>
> 2) engine-config
>
> There's a number of text labels that need to be fixed eg.
> oVirtISOsRepositoryPath: "The RHEV-H installation files path" (Value Type: String)
>
> Also we need to fix some of the longer names eg.
> VdcVersion: "oVirt Enterprise Virtualization Engine Manager Version" (Value Type: String)
>
> 3) webadmin
> Logo in WebAdmin is Red Hat Shadowman, that needs to be updated.
> We can't go out with "Browser not supported" messages like this. The browsers installed on f16 - our target platform can't give this error message.
>
>
> 4) VDS Bootstrap
>
> If you check the  box for iptables then it breaks the system with an invalid rule
>
> Looking at the error from my system
> "Jan 29 21:38:50 host1 iptables.init[1637]: iptables: Applying firewall rules: iptables-restore v1.4.12: physdev: option "--physdev-is-bridged" cannot be inverted."
>
> The problem is that we are using
> "-A FORWARD -m physdev ! --physdev-is-bridged -j REJECT --reject-with icmp-host-prohibited"
>
> -vs-
>
> "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT"
>
>
> On the point of iptables, why do we open libvirt tls port and the snmp port.
>
>
> After I fixed iptables I still had a problem, the host was showing as non-operational with the error message "NETWORK_UNREACHABLE" (there's a literal missing there)
>
> Looking at the host it appears that the management bridge wasn't created.
>
> The bootstrap log is attached. It shows the operation failing but still marked the bootstrap as successful.
> (also in the log we have rhn/satellite references that need to be removed)
>
> I've not looked at the reason for the failure yet, but an empty line at the end of my ifcfg-em1 file looks suspicious.
>
>
>
>
>
>
>
>> The oVirt-Engine team
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From danken at redhat.com  Mon Jan 30 11:58:07 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Mon, 30 Jan 2012 13:58:07 +0200
Subject: [Engine-devel] New oVirt-engine RPMs available - Release Candidates
In-Reply-To: <4F2678E8.5060008@redhat.com>
References: <4F2678E8.5060008@redhat.com>
Message-ID: <20120130115801.GF18371@redhat.com>

<snip>

> 
> 4) VDS Bootstrap
> 
> If you check the  box for iptables then it breaks the system with an invalid rule
> 
> Looking at the error from my system
> "Jan 29 21:38:50 host1 iptables.init[1637]: iptables: Applying firewall rules: iptables-restore v1.4.12: physdev: option "--physdev-is-bridged" cannot be inverted."
> 
> The problem is that we are using
> "-A FORWARD -m physdev ! --physdev-is-bridged -j REJECT --reject-with icmp-host-prohibited"
> 
> -vs-
> 
> "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT"
> 
> 
> On the point of iptables, why do we open libvirt tls port and the snmp port.

This as these iptables rules are kept within Engine, I'll keep that
question for them.

> 
> After I fixed iptables I still had a problem, the host was showing as non-operational with the error message "NETWORK_UNREACHABLE" (there's a literal missing there)
> 
> Looking at the host it appears that the management bridge wasn't created.
> 
> The bootstrap log is attached. It shows the operation failing but still marked the bootstrap as successful.
> (also in the log we have rhn/satellite references that need to be removed)
> 
> I've not looked at the reason for the failure yet, but an empty line at the end of my ifcfg-em1 file looks suspicious.

You are correct (too bad I've noticed your suspicion only after reading
the log)

> Sun, 29 Jan 2012 21:30:05 DEBUG    makeBridge found the following bridge paramaters: ['ONBOOT=yes', 'BOOTPROTO=none', 'IPADDR=172.16.31.230', 'DNS1=172.16.31.4', 'NM_CONTROLLED=no', 'NETMASK=255.255.255.0', 'DNS2=172.16.31.1', 'GATEWAY=172.16.31.1', '']
> Sun, 29 Jan 2012 21:30:05 DEBUG    ['/usr/share/vdsm/addNetwork', 'ovirtmgmt', '', '', 'em1', 'ONBOOT=yes', 'BOOTPROTO=none', 'IPADDR=172.16.31.230', 'DNS1=172.16.31.4', 'NM_CONTROLLED=no', 'NETMASK=255.255.255.0', 'DNS2=172.16.31.1', 'GATEWAY=172.16.31.1', '', 'blockingdhcp=true', 'skipLibvirt=True']
> Sun, 29 Jan 2012 21:30:05 DEBUG    
> Sun, 29 Jan 2012 21:30:05 DEBUG    Traceback (most recent call last):
>   File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
>     "__main__", fname, loader, pkg_name)
>   File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
>     exec code in run_globals
>   File "/usr/share/vdsm/configNetwork.py", line 924, in <module>
>     main()
>   File "/usr/share/vdsm/configNetwork.py", line 890, in main
>     kwargs = _parseKwargs(sys.argv[3:])
>   File "/usr/share/vdsm/configNetwork.py", line 876, in _parseKwargs
>     return dict(arg.split('=', 1) for arg in args)
> ValueError: dictionary update sequence element #11 has length 1; 2 is required

addNetwork script breaks down in tears if it sees the empty arg '',
which is passed to it by makeBridge. makeBridge should become more
robust - but until then, please del lines with no key=value form from
ifcfg.

Dan.


From acathrow at redhat.com  Mon Jan 30 12:39:06 2012
From: acathrow at redhat.com (Andrew Cathrow)
Date: Mon, 30 Jan 2012 07:39:06 -0500 (EST)
Subject: [Engine-devel] New oVirt-engine RPMs available - Release
	Candidates
In-Reply-To: <20120130115801.GF18371@redhat.com>
Message-ID: <20bec93e-c509-4a76-a632-50318a88958b@zmail07.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Dan Kenigsberg" <danken at redhat.com>
> To: "Ofer Schreiber" <oschreib at redhat.com>, "Andrew Cathrow" <acathrow at redhat.com>
> Cc: engine-devel at ovirt.org, dougsland at redhat.com
> Sent: Monday, January 30, 2012 6:58:07 AM
> Subject: [Engine-devel] New oVirt-engine RPMs available - Release Candidates
> 
> <snip>
> 
> > 
> > 4) VDS Bootstrap
> > 
> > If you check the  box for iptables then it breaks the system with
> > an invalid rule
> > 
> > Looking at the error from my system
> > "Jan 29 21:38:50 host1 iptables.init[1637]: iptables: Applying
> > firewall rules: iptables-restore v1.4.12: physdev: option
> > "--physdev-is-bridged" cannot be inverted."
> > 
> > The problem is that we are using
> > "-A FORWARD -m physdev ! --physdev-is-bridged -j REJECT
> > --reject-with icmp-host-prohibited"
> > 
> > -vs-
> > 
> > "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT"
> > 
> > 
> > On the point of iptables, why do we open libvirt tls port and the
> > snmp port.
> 
> This as these iptables rules are kept within Engine, I'll keep that
> question for them.
> 
> > 
> > After I fixed iptables I still had a problem, the host was showing
> > as non-operational with the error message "NETWORK_UNREACHABLE"
> > (there's a literal missing there)
> > 
> > Looking at the host it appears that the management bridge wasn't
> > created.
> > 
> > The bootstrap log is attached. It shows the operation failing but
> > still marked the bootstrap as successful.
> > (also in the log we have rhn/satellite references that need to be
> > removed)
> > 
> > I've not looked at the reason for the failure yet, but an empty
> > line at the end of my ifcfg-em1 file looks suspicious.
> 
> You are correct (too bad I've noticed your suspicion only after
> reading
> the log)
> 
> > Sun, 29 Jan 2012 21:30:05 DEBUG    makeBridge found the following
> > bridge paramaters: ['ONBOOT=yes', 'BOOTPROTO=none',
> > 'IPADDR=172.16.31.230', 'DNS1=172.16.31.4', 'NM_CONTROLLED=no',
> > 'NETMASK=255.255.255.0', 'DNS2=172.16.31.1',
> > 'GATEWAY=172.16.31.1', '']
> > Sun, 29 Jan 2012 21:30:05 DEBUG    ['/usr/share/vdsm/addNetwork',
> > 'ovirtmgmt', '', '', 'em1', 'ONBOOT=yes', 'BOOTPROTO=none',
> > 'IPADDR=172.16.31.230', 'DNS1=172.16.31.4', 'NM_CONTROLLED=no',
> > 'NETMASK=255.255.255.0', 'DNS2=172.16.31.1',
> > 'GATEWAY=172.16.31.1', '', 'blockingdhcp=true',
> > 'skipLibvirt=True']
> > Sun, 29 Jan 2012 21:30:05 DEBUG
> > Sun, 29 Jan 2012 21:30:05 DEBUG    Traceback (most recent call
> > last):
> >   File "/usr/lib64/python2.7/runpy.py", line 162, in
> >   _run_module_as_main
> >     "__main__", fname, loader, pkg_name)
> >   File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
> >     exec code in run_globals
> >   File "/usr/share/vdsm/configNetwork.py", line 924, in <module>
> >     main()
> >   File "/usr/share/vdsm/configNetwork.py", line 890, in main
> >     kwargs = _parseKwargs(sys.argv[3:])
> >   File "/usr/share/vdsm/configNetwork.py", line 876, in
> >   _parseKwargs
> >     return dict(arg.split('=', 1) for arg in args)
> > ValueError: dictionary update sequence element #11 has length 1; 2
> > is required
> 
> addNetwork script breaks down in tears if it sees the empty arg '',
> which is passed to it by makeBridge. makeBridge should become more
> robust - but until then, please del lines with no key=value form from
> ifcfg.

Yeah, I did that to work around it, funnily enough vdsm adds emtpy lines when it makes it's config file!
> 
> Dan.
> 


From lpeer at redhat.com  Mon Jan 30 13:59:53 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Mon, 30 Jan 2012 15:59:53 +0200
Subject: [Engine-devel] [Users] New oVirt-engine RPMs available -
	Release Candidates
In-Reply-To: <0u8abes4uqd65um4ytfqduap.1327879682868@email.android.com>
References: <4F251098.6080503@redhat.com>
	<0u8abes4uqd65um4ytfqduap.1327879682868@email.android.com>
Message-ID: <4F26A259.1070200@redhat.com>

On 30/01/12 01:28, Juan Luis Alarcon Ma?as wrote:
> I did an engine installation and i think the memory overcommit % ( cluster properties ) for servers and desktops loads are exchanged, at least the descriptions are.
> 

The fix is already pushed:

http://gerrit.ovirt.org/#change,1340

Thanks for the input.


> Ofer Schreiber escribi?:
> Hi all,
> 
> We've just finished uploading a new set of oVirt-engine rpms into
> ovirt.org.
> 
> These rpms, versioned 3.0.0_0001-1.4, are considered as release
> candidates for our first release.
> 
> In order to install the RPMs, please follow the instructions described in
> http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
> 
> Feedback, comments and bug reports are always welcome.
> 
> The oVirt-Engine team
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
> 
> 
> 
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users



From mgoldboi at redhat.com  Mon Jan 30 14:46:21 2012
From: mgoldboi at redhat.com (Moran Goldboim)
Date: Mon, 30 Jan 2012 16:46:21 +0200
Subject: [Engine-devel] oVirt release go/nogo meeting
Message-ID: <4F26AD3D.70308@redhat.com>

towards today meeting, listed are the problematic bugs from QE 
perspective, most in new state, several are blockers and the rest needs 
to be discussed and fixed with high priority for next version, filtered 
by component

VDSM:

	773371 <https://bugzilla.redhat.com/show_bug.cgi?id=773371> 	high 
2012-01-11 	urgent 	Linux 	fsimonce at redhat.com 	NEW 	--- 	vdsm: when 
installing vdsm manually in the host and then installing host with 
web-admin /etc/libvirt/qemu.conf is using spice_tls=1 which causes vm's 
to fail to run with cert error 	dron at redhat.com 	
	vdsm

	781970 <https://bugzilla.redhat.com/show_bug.cgi?id=781970> 	urgent 
2012-01-16 	urgent 	Linux 	danken at redhat.com 	NEW 	--- 	[vdsm] Migration 
fails due to changes in xmlrpclib in Python 2.7 	jlibosva at redhat.com 	
	vdsm

	784324 <https://bugzilla.redhat.com/show_bug.cgi?id=784324> 	high 	Tue 
10:21 	high 	Linux 	extras-orphan at fedoraproject... 	NEW 	--- 	[qemu-kvm] 
guest crash when connection to storage is blocked on host machine 
dron at redhat.com 	
	kvm

	785557 <https://bugzilla.redhat.com/show_bug.cgi?id=785557> 	high 	Sun 
09:51 	unspecified 	All 	dougsland at redhat.com 	NEW 	--- 
[vdsm][deployUtil] host installation adds interface configured with 
NM_CONTROLLED="yes" to a bridge which is unsupported. 	dnaori at redhat.com 	
	vdsm

	785749 <https://bugzilla.redhat.com/show_bug.cgi?id=785749> 
unspecified 	09:27:13 	unspecified 	Unspecified 	danken at redhat.com 
NEW 	--- 	[ovirt] [vdsm] deadlock after prepareForShutdown when 
connection to storage is blocked with VMs running 	hateya at redhat.com 	
	vdsm



Engine:

	782432 <https://bugzilla.redhat.com/show_bug.cgi?id=782432> 	urgent 
2012-01-17 	urgent 	Linux 	lpeer at redhat.com 	NEW 	--- 
ovirt-engine-core: extend fails when there is more than once host in 
cluster 	dron at redhat.com 	
	ovirt-engine-core

	783083 <https://bugzilla.redhat.com/show_bug.cgi?id=783083> 	high 
2012-01-19 	unspecified 	Linux 	lpeer at redhat.com 	NEW 	--- 	NPE during 
SD removal 	jlibosva at redhat.com 	
	ovirt-engine-core

	783662 <https://bugzilla.redhat.com/show_bug.cgi?id=783662> 	urgent 
2012-01-21 	unspecified 	Linux 	oourfali at redhat.com 	MODIFIED 	--- 	IPA 
- IPA does not perform login with UPN 	pstehlik at redhat.com 	
	ovirt-engine-core

	783789 <https://bugzilla.redhat.com/show_bug.cgi?id=783789> 	high 
2012-01-22 	unspecified 	Linux 	lpeer at redhat.com 	NEW 	--- 
[ovirt][engine][core] - exception when trying to fence a SPM host that 
moved to non responsive 	yeylon at redhat.com 	
	ovirt-engine-core

	784900 <https://bugzilla.redhat.com/show_bug.cgi?id=784900> 	high 	Thu 
10:35 	high 	Unspecified 	lpeer at redhat.com 	NEW 	--- 	[ovirt] 
[engine-core] null pointer exception when merging snapshot and engine 
gets restarted - VM stuck in unknown state 	hateya at redhat.com 	
	ovirt-engine-core

	785671 <https://bugzilla.redhat.com/show_bug.cgi?id=785671> 	urgent 
04:55:48 	urgent 	Linux 	lpeer at redhat.com 	NEW 	--- 	ovirt-engine-core: 
after trying to add a snapshot while a snapshot is created and getting 
an error vm CreateAllSnapshotsFromVmCommand will fail to aquire lock 
forever 	dron at redhat.com 	
	ovirt-engine-core


webadmin:
785565 <https://bugzilla.redhat.com/show_bug.cgi?id=785565> 	high 	Sun 
10:52 	high 	Linux 	ecohen at redhat.com 	NEW 	--- 	[ovirt] [webadmin] 
refresh problem after discover & login to target - content of 
getDeviceList is not presented 	dron at redhat.com 	
	ovirt-engine-webadmin


ovirt-node:

	785728 <https://bugzilla.redhat.com/show_bug.cgi?id=785728> 	high 
08:36:17 	unspecified 	Unspecified 	jboggs at redhat.com 	NEW 	--- 
[installation] ovirt node installation fails 	dnaori at redhat.com


https://bugzilla.redhat.com/buglist.cgi?quicksearch=773371+781970+784324+78557+785749+782432+783083+783662+783789+784900+785671+785565+785728

Moran.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120130/2643ad43/attachment.html>

From danken at redhat.com  Mon Jan 30 15:30:12 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Mon, 30 Jan 2012 17:30:12 +0200
Subject: [Engine-devel] New oVirt-engine RPMs available - Release
 Candidates
In-Reply-To: <20bec93e-c509-4a76-a632-50318a88958b@zmail07.collab.prod.int.phx2.redhat.com>
References: <20120130115801.GF18371@redhat.com>
	<20bec93e-c509-4a76-a632-50318a88958b@zmail07.collab.prod.int.phx2.redhat.com>
Message-ID: <20120130153010.GB32517@redhat.com>

On Mon, Jan 30, 2012 at 07:39:06AM -0500, Andrew Cathrow wrote:
> 
> 
> ----- Original Message -----
> > From: "Dan Kenigsberg" <danken at redhat.com>
> > To: "Ofer Schreiber" <oschreib at redhat.com>, "Andrew Cathrow" <acathrow at redhat.com>
> > Cc: engine-devel at ovirt.org, dougsland at redhat.com
> > Sent: Monday, January 30, 2012 6:58:07 AM
> > Subject: [Engine-devel] New oVirt-engine RPMs available - Release Candidates
> > 
> > <snip>
> > 
> > > 
> > > 4) VDS Bootstrap
> > > 
> > > If you check the  box for iptables then it breaks the system with
> > > an invalid rule
> > > 
> > > Looking at the error from my system
> > > "Jan 29 21:38:50 host1 iptables.init[1637]: iptables: Applying
> > > firewall rules: iptables-restore v1.4.12: physdev: option
> > > "--physdev-is-bridged" cannot be inverted."
> > > 
> > > The problem is that we are using
> > > "-A FORWARD -m physdev ! --physdev-is-bridged -j REJECT
> > > --reject-with icmp-host-prohibited"
> > > 
> > > -vs-
> > > 
> > > "-I FORWARD -m physdev --physdev-is-bridged -j ACCEPT"
> > > 
> > > 
> > > On the point of iptables, why do we open libvirt tls port and the
> > > snmp port.
> > 
> > This as these iptables rules are kept within Engine, I'll keep that
> > question for them.
> > 
> > > 
> > > After I fixed iptables I still had a problem, the host was showing
> > > as non-operational with the error message "NETWORK_UNREACHABLE"
> > > (there's a literal missing there)
> > > 
> > > Looking at the host it appears that the management bridge wasn't
> > > created.
> > > 
> > > The bootstrap log is attached. It shows the operation failing but
> > > still marked the bootstrap as successful.
> > > (also in the log we have rhn/satellite references that need to be
> > > removed)
> > > 
> > > I've not looked at the reason for the failure yet, but an empty
> > > line at the end of my ifcfg-em1 file looks suspicious.
> > 
> > You are correct (too bad I've noticed your suspicion only after
> > reading
> > the log)
> > 
> > > Sun, 29 Jan 2012 21:30:05 DEBUG    makeBridge found the following
> > > bridge paramaters: ['ONBOOT=yes', 'BOOTPROTO=none',
> > > 'IPADDR=172.16.31.230', 'DNS1=172.16.31.4', 'NM_CONTROLLED=no',
> > > 'NETMASK=255.255.255.0', 'DNS2=172.16.31.1',
> > > 'GATEWAY=172.16.31.1', '']
> > > Sun, 29 Jan 2012 21:30:05 DEBUG    ['/usr/share/vdsm/addNetwork',
> > > 'ovirtmgmt', '', '', 'em1', 'ONBOOT=yes', 'BOOTPROTO=none',
> > > 'IPADDR=172.16.31.230', 'DNS1=172.16.31.4', 'NM_CONTROLLED=no',
> > > 'NETMASK=255.255.255.0', 'DNS2=172.16.31.1',
> > > 'GATEWAY=172.16.31.1', '', 'blockingdhcp=true',
> > > 'skipLibvirt=True']
> > > Sun, 29 Jan 2012 21:30:05 DEBUG
> > > Sun, 29 Jan 2012 21:30:05 DEBUG    Traceback (most recent call
> > > last):
> > >   File "/usr/lib64/python2.7/runpy.py", line 162, in
> > >   _run_module_as_main
> > >     "__main__", fname, loader, pkg_name)
> > >   File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
> > >     exec code in run_globals
> > >   File "/usr/share/vdsm/configNetwork.py", line 924, in <module>
> > >     main()
> > >   File "/usr/share/vdsm/configNetwork.py", line 890, in main
> > >     kwargs = _parseKwargs(sys.argv[3:])
> > >   File "/usr/share/vdsm/configNetwork.py", line 876, in
> > >   _parseKwargs
> > >     return dict(arg.split('=', 1) for arg in args)
> > > ValueError: dictionary update sequence element #11 has length 1; 2
> > > is required
> > 
> > addNetwork script breaks down in tears if it sees the empty arg '',
> > which is passed to it by makeBridge. makeBridge should become more
> > robust - but until then, please del lines with no key=value form from
> > ifcfg.
> 
> Yeah, I did that to work around it, funnily enough vdsm adds emtpy lines when it makes it's config file!

I'm not laughing.



From mburns at redhat.com  Mon Jan 30 21:00:47 2012
From: mburns at redhat.com (Mike Burns)
Date: Mon, 30 Jan 2012 16:00:47 -0500
Subject: [Engine-devel] New oVirt-engine RPMs available - Release
 Candidates
In-Reply-To: <4F251098.6080503@redhat.com>
References: <4F251098.6080503@redhat.com>
Message-ID: <1327957247.2417.58.camel@beelzebub.mburnsfire.net>

On Sun, 2012-01-29 at 11:25 +0200, Ofer Schreiber wrote:
> Hi all,
> 
> We've just finished uploading a new set of oVirt-engine rpms into
> ovirt.org.
> 
> These rpms, versioned 3.0.0_0001-1.4, are considered as release candidates for our first release.
> 
> In order to install the RPMs, please follow the instructions described in
> http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
> 
> Feedback, comments and bug reports are always welcome.
> 
> The oVirt-Engine team
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel


I just noticed that a fix [1] to a bug [2] required for ovirt-node to
register itself to the engine is not in the RC.  Can we get this pushed
prior to release?

[1] http://gerrit.ovirt.org/#change,1120
[2] https://bugzilla.redhat.com/show_bug.cgi?id=782663

Thanks

Mike



From emesika at redhat.com  Tue Jan 31 01:03:38 2012
From: emesika at redhat.com (Eli Mesika)
Date: Mon, 30 Jan 2012 20:03:38 -0500 (EST)
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <f1eb8c27-4d72-407b-ac0a-972290ac8e99@zmail02.collab.prod.int.phx2.redhat.com>
Message-ID: <e5563867-a84b-48cd-90e2-f436aa7e1f8c@zmail13.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Oved Ourfalli" <ovedo at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Monday, January 30, 2012 9:45:47 AM
> Subject: [Engine-devel] Development process wiki page
> 
> Hey all,
> 
> I've wrote a wiki page on the oVirt development process.
> It contains mostly information on the patch review process, patch
> submission and some git guidelines.
> 
> http://www.ovirt.org/wiki/DevProcess
> 
> I added a link to it from the main wiki page.
> 
> Your comments are welcome.

my 2 cents : (obvious , but should be in the wiki)

1) Regarding patch separation, still each commit should not break the build
2) When having more than one patch set due to comments and applied fixes add to the commit message V[n] : <changes description> where [n] is the patch set number 


> 
> Thank you,
> Oved
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From oschreib at redhat.com  Tue Jan 31 09:20:04 2012
From: oschreib at redhat.com (Ofer Schreiber)
Date: Tue, 31 Jan 2012 11:20:04 +0200
Subject: [Engine-devel] New oVirt-engine RPMs available - Release
	Candidates
In-Reply-To: <1327957247.2417.58.camel@beelzebub.mburnsfire.net>
References: <4F251098.6080503@redhat.com>
	<1327957247.2417.58.camel@beelzebub.mburnsfire.net>
Message-ID: <4F27B244.4070906@redhat.com>

On 01/30/2012 11:00 PM, Mike Burns wrote:
> On Sun, 2012-01-29 at 11:25 +0200, Ofer Schreiber wrote:
>> Hi all,
>>
>> We've just finished uploading a new set of oVirt-engine rpms into
>> ovirt.org.
>>
>> These rpms, versioned 3.0.0_0001-1.4, are considered as release candidates for our first release.
>>
>> In order to install the RPMs, please follow the instructions described in
>> http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
>>
>> Feedback, comments and bug reports are always welcome.
>>
>> The oVirt-Engine team
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
> I just noticed that a fix [1] to a bug [2] required for ovirt-node to
> register itself to the engine is not in the RC.  Can we get this pushed
> prior to release?
>
> [1] http://gerrit.ovirt.org/#change,1120
> [2] https://bugzilla.redhat.com/show_bug.cgi?id=782663
>
> Thanks
>
> Mike
>

Just to understand the impact- All nodes cannot be registered?
Roy- could you please push this fix?

Added to first release blockers atm.





From dfediuck at redhat.com  Tue Jan 31 09:35:26 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Tue, 31 Jan 2012 11:35:26 +0200
Subject: [Engine-devel] New oVirt-engine RPMs available - Release
	Candidates
In-Reply-To: <4F27B244.4070906@redhat.com>
References: <4F251098.6080503@redhat.com>
	<1327957247.2417.58.camel@beelzebub.mburnsfire.net>
	<4F27B244.4070906@redhat.com>
Message-ID: <4F27B5DE.1080701@redhat.com>

On 31/01/12 11:20, Ofer Schreiber wrote:
> On 01/30/2012 11:00 PM, Mike Burns wrote:
>> On Sun, 2012-01-29 at 11:25 +0200, Ofer Schreiber wrote:
>>> Hi all,
>>>
>>> We've just finished uploading a new set of oVirt-engine rpms into
>>> ovirt.org.
>>>
>>> These rpms, versioned 3.0.0_0001-1.4, are considered as release candidates for our first release.
>>>
>>> In order to install the RPMs, please follow the instructions described in
>>> http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
>>>
>>> Feedback, comments and bug reports are always welcome.
>>>
>>> The oVirt-Engine team
>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>> I just noticed that a fix [1] to a bug [2] required for ovirt-node to
>> register itself to the engine is not in the RC.  Can we get this pushed
>> prior to release?
>>
>> [1] http://gerrit.ovirt.org/#change,1120
>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=782663
>>
>> Thanks
>>
>> Mike
>>
> 
> Just to understand the impact- All nodes cannot be registered?
> Roy- could you please push this fix?
> 
Since this changes the URI, it needs basic integration with ovirt node.
Before pushing it, this should be verified with current upstream ovirt-node.
Mike & Roy- did someone try to use it with latest ovirt-node?

> Added to first release blockers atm.
> 


-- 

/d

Why doesn't DOS ever say "EXCELLENT command or filename!"


From iheim at redhat.com  Tue Jan 31 09:58:53 2012
From: iheim at redhat.com (Itamar Heim)
Date: Tue, 31 Jan 2012 11:58:53 +0200
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <e5563867-a84b-48cd-90e2-f436aa7e1f8c@zmail13.collab.prod.int.phx2.redhat.com>
References: <e5563867-a84b-48cd-90e2-f436aa7e1f8c@zmail13.collab.prod.int.phx2.redhat.com>
Message-ID: <4F27BB5D.2050401@redhat.com>

On 01/31/2012 03:03 AM, Eli Mesika wrote:
>
>
> ----- Original Message -----
>> From: "Oved Ourfalli"<ovedo at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Monday, January 30, 2012 9:45:47 AM
>> Subject: [Engine-devel] Development process wiki page
>>
>> Hey all,
>>
>> I've wrote a wiki page on the oVirt development process.
>> It contains mostly information on the patch review process, patch
>> submission and some git guidelines.
>>
>> http://www.ovirt.org/wiki/DevProcess
>>
>> I added a link to it from the main wiki page.
>>
>> Your comments are welcome.
>
> my 2 cents : (obvious , but should be in the wiki)
>
> 1) Regarding patch separation, still each commit should not break the build
> 2) When having more than one patch set due to comments and applied fixes add to the commit message V[n] :<changes description>  where [n] is the patch set number

I agree we need this, but the problem is we are lacking the [0/nnn] 
cover letter email you get for such commentary when using gerrit.
and there is a concept of the commit itself should not include the 
history of the patch being reviewed.
so we need to adapt this to gerrit.
maybe a convention that when uploading a new patch set, contributor also 
adds a review comment of the changes from previous version (similar to 
the cover letter).
I wonder if the git/gerrit utility some adopted from openstack can help 
with such a step to allow adding a comment when pushing the patch for 
review.


From mkolesni at redhat.com  Tue Jan 31 10:02:25 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Tue, 31 Jan 2012 05:02:25 -0500 (EST)
Subject: [Engine-devel] Simplifying our POJOs
In-Reply-To: <508cf1c3-2f24-4780-b5f8-2634ef65c96f@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <fc563815-f97c-4cc4-8d6f-0b9a3cd0bbc3@zmail14.collab.prod.int.phx2.redhat.com>

Hi, 

Today many POJO s are used throughout the system to convey data: 

    ? Parameters - To send data to commands. 
    ? Business Entities - To transfer data in the parameters & to/from the DB. 

These POJOs are (usually) very verbose and full of boilerplate code . 

This, in turn, reduces their readability and maintainability for a couple of reasons (that I can think of): 

    ? It's hard to know what does what: 


        ? Who participates in equals/hashCode? 
        ? What fields are printed in toString? 
    ? 
Consistency is problematic: 

        ? A field may be part of equals but not hashCode, or vice versa. 
        ? This breaks the Object.hashCode() contract! 
    ? 
Adding/Removing fields take more time since you need to synchronize the change to all boilerplate methods. 

        ? Again, we're facing the consistency problem. 
    ? 
These simple classes tend to be very long and not very readable.     ? Boilerplate code makes it harder to find out which methods don't behave the default way. 
    ? Javadoc, if existent, is usually meaningless (but you might see some banal documentation that doesn't add any real value). 
    ? Our existing classes are not up to standard! 

So what can be done to remedy the situation? 

We could, of course, try to simplify the classes as much as we can and maybe address some of the issues. 
This won't alleviate the boilerplate code problem altogether, though. 

We could write annotations to do some of the things for us automatically. 
The easiest approach would be runtime-based, and would hinder performance. 
This also means we need to maintain this "infrastructure" and all the implications of such a decision. 


Luckily, there is a much easier solution: Someone else already did it! 

Check out Project Lombok: http://projectlombok.org 
What Lombok gives us, among some other things, is a way to greatly simplify our POJOs by using annotations to get the boilerplate code automatically generated. 
This means we get the benefit of annotations which would simplify the code a whole lot, while not imposing a performance cost (since the boilerplate code is generated during compilation). 
However, it's also possible to create the methods yourself if you want them to behave differently. 
Outside the POJO itself, you would see it as you would always see it. 

So what are the downsides to this approach? 

    ? First of all, Lombok provides also some other capabilities which I'm not sure are required/wanted at this time. 


        ? That's why I propose we use it for commons project, and make use of it's POJO-related annotations ONLY. 
    ? 
There might be a problem debugging the code since it's auto-generated. 

        ? I think this is rather negligible, since usually you don't debug POJOs anyway. 
    ? 
There might be a problem if the auto-generated code throws an Exception. 

        ? As before, I'm rather sure this is an edge-case which we usually won't hit (if at all). 

Even given these possible downsides, I think that we would benefit greatly if we would introduce this library. 

If you have any questions, you're welcome to study out the project site which has very thorough documentation: http://projectlombok.org 

Your thoughts on the matter? 


Regards, 
Mike 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120131/c1dd3e36/attachment.html>

From mkolesni at redhat.com  Tue Jan 31 10:04:29 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Tue, 31 Jan 2012 05:04:29 -0500 (EST)
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <4F27BB5D.2050401@redhat.com>
Message-ID: <6d5d3742-15be-4dfc-b733-6c4b9e86507c@zmail14.collab.prod.int.phx2.redhat.com>


----- Original Message -----
> On 01/31/2012 03:03 AM, Eli Mesika wrote:
> >
> >
> > ----- Original Message -----
> >> From: "Oved Ourfalli"<ovedo at redhat.com>
> >> To: engine-devel at ovirt.org
> >> Sent: Monday, January 30, 2012 9:45:47 AM
> >> Subject: [Engine-devel] Development process wiki page
> >>
> >> Hey all,
> >>
> >> I've wrote a wiki page on the oVirt development process.
> >> It contains mostly information on the patch review process, patch
> >> submission and some git guidelines.
> >>
> >> http://www.ovirt.org/wiki/DevProcess
> >>
> >> I added a link to it from the main wiki page.
> >>
> >> Your comments are welcome.
> >
> > my 2 cents : (obvious , but should be in the wiki)
> >
> > 1) Regarding patch separation, still each commit should not break
> > the build
> > 2) When having more than one patch set due to comments and applied
> > fixes add to the commit message V[n] :<changes description>  where
> > [n] is the patch set number
> 
> I agree we need this, but the problem is we are lacking the [0/nnn]
> cover letter email you get for such commentary when using gerrit.
> and there is a concept of the commit itself should not include the
> history of the patch being reviewed.
> so we need to adapt this to gerrit.
> maybe a convention that when uploading a new patch set, contributor
> also
> adds a review comment of the changes from previous version (similar
> to
> the cover letter).

I think if the commit history already exists in gerrit, and the commit itself is pushed through gerrit, then this will be redundant.

> I wonder if the git/gerrit utility some adopted from openstack can
> help
> with such a step to allow adding a comment when pushing the patch for
> review.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mlipchuk at redhat.com  Tue Jan 31 10:07:12 2012
From: mlipchuk at redhat.com (Maor)
Date: Tue, 31 Jan 2012 12:07:12 +0200
Subject: [Engine-devel] Quota feature design changes
Message-ID: <4F27BD50.70104@redhat.com>

Hi all,

Few changes were made in the quota design,
Main issues that were changed :
1.Quota usages will be calculated and fetched using DB function instead
using persistency.
Using memory should improve performance and prevent integrity issues.
2.Default unlimited quota will be assigned with resources only when DC
is at disable mode, in any other DC mode, user should not notice it as a
consumable quota or consume new resources from it.

You can view quota changes in the following design wiki:
	http://www.ovirt.org/wiki/Features/Design/Quota

Thanks,
Maor


From lpeer at redhat.com  Tue Jan 31 10:39:35 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 31 Jan 2012 12:39:35 +0200
Subject: [Engine-devel] Simplifying our POJOs
In-Reply-To: <fc563815-f97c-4cc4-8d6f-0b9a3cd0bbc3@zmail14.collab.prod.int.phx2.redhat.com>
References: <fc563815-f97c-4cc4-8d6f-0b9a3cd0bbc3@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F27C4E7.3050600@redhat.com>

On 31/01/12 12:02, Mike Kolesnik wrote:
> Hi,
> 
> Today many POJO <http://en.wikipedia.org/wiki/Plain_Old_Java_Object>s
> are used throughout the system to convey data:
> 
>   *   Parameters - To send data to commands.
>   *   Business Entities - To transfer data in the parameters & to/from
>     the DB.
> 
> These POJOs are (usually) very verbose and full of boilerplate code
> <http://en.wikipedia.org/wiki/Boilerplate_code>.
> 
> This, in turn, reduces their readability and maintainability for a
> couple of reasons (that I can think of):
> 
>   * It's hard to know what does what:
>       o Who participates in equals/hashCode?
>       o What fields are printed in toString? 
>   * Consistency is problematic:
>       o A field may be part of equals but not hashCode, or vice versa.
>       o This breaks the Object.hashCode()
>         <http://docs.oracle.com/javase/6/docs/api/java/lang/Object.html#hashCode%28%29>
>         contract!
>   * Adding/Removing fields take more time since you need to synchronize
>     the change to all boilerplate methods.
>       o Again, we're facing the consistency problem.
>   * These simple classes tend to be very long and not very readable.
>   * Boilerplate code makes it harder to find out which methods *don't*
>     behave the default way.
>   * Javadoc, if existent, is usually meaningless (but you might see some
>     banal documentation that doesn't add any real value).
>   * Our existing classes are not up to standard!
> 
> 
> So what can be done to remedy the situation?
> 
> We could, of course, try to simplify the classes as much as we can and
> maybe address some of the issues.
> This won't alleviate the boilerplate code problem altogether, though.
> 
> We could write annotations to do some of the things for us automatically.
> The easiest approach would be runtime-based, and would hinder performance.
> This also means we need to maintain this "infrastructure" and all the
> implications of such a decision.
> 
> 
> Luckily, there is a much easier solution: Someone else already did it!
> 
> Check out Project Lombok: http://projectlombok.org
> What Lombok gives us, among some other things, is a way to greatly
> simplify our POJOs by using annotations to get the boilerplate code
> automatically generated.
> This means we get the benefit of annotations which would simplify the
> code a whole lot, while not imposing a performance cost (since the
> boilerplate code is generated during compilation).
> However, it's also possible to create the methods yourself if you want
> them to behave differently.
> Outside the POJO itself, you would see it as you would always see it.
> 
> So what are the downsides to this approach?
> 
>   * First of all, Lombok provides also some other capabilities which I'm
>     not sure are required/wanted at this time.
>       o That's why I propose we use it for commons project, and make use
>         of it's POJO-related annotations ONLY.
>   * There might be a problem debugging the code since it's auto-generated.
>       o I think this is rather negligible, since usually you don't debug
>         POJOs anyway.
>   * There might be a problem if the auto-generated code throws an Exception.
>       o As before, I'm rather sure this is an edge-case which we usually
>         won't hit (if at all).
> 
> 
> Even given these possible downsides, I think that we would benefit
> greatly if we would introduce this library.
> 
> If you have any questions, you're welcome to study out the project site
> which has very thorough documentation: http://projectlombok.org
> 
> Your thoughts on the matter?
> 

- I think an example of before/after pojo would help demonstrating how
good the framework is.

- Would it work when adding JPA annotations?

> Regards,
> Mike
> 
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From dfediuck at redhat.com  Tue Jan 31 10:45:37 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Tue, 31 Jan 2012 12:45:37 +0200
Subject: [Engine-devel] Simplifying our POJOs
In-Reply-To: <4F27C4E7.3050600@redhat.com>
References: <fc563815-f97c-4cc4-8d6f-0b9a3cd0bbc3@zmail14.collab.prod.int.phx2.redhat.com>
	<4F27C4E7.3050600@redhat.com>
Message-ID: <4F27C651.7070508@redhat.com>

On 31/01/12 12:39, Livnat Peer wrote:
> On 31/01/12 12:02, Mike Kolesnik wrote:
>> Hi,
>>
>> Today many POJO <http://en.wikipedia.org/wiki/Plain_Old_Java_Object>s
>> are used throughout the system to convey data:
>>
>>   *   Parameters - To send data to commands.
>>   *   Business Entities - To transfer data in the parameters & to/from
>>     the DB.
>>
>> These POJOs are (usually) very verbose and full of boilerplate code
>> <http://en.wikipedia.org/wiki/Boilerplate_code>.
>>
>> This, in turn, reduces their readability and maintainability for a
>> couple of reasons (that I can think of):
>>
>>   * It's hard to know what does what:
>>       o Who participates in equals/hashCode?
>>       o What fields are printed in toString? 
>>   * Consistency is problematic:
>>       o A field may be part of equals but not hashCode, or vice versa.
>>       o This breaks the Object.hashCode()
>>         <http://docs.oracle.com/javase/6/docs/api/java/lang/Object.html#hashCode%28%29>
>>         contract!
>>   * Adding/Removing fields take more time since you need to synchronize
>>     the change to all boilerplate methods.
>>       o Again, we're facing the consistency problem.
>>   * These simple classes tend to be very long and not very readable.
>>   * Boilerplate code makes it harder to find out which methods *don't*
>>     behave the default way.
>>   * Javadoc, if existent, is usually meaningless (but you might see some
>>     banal documentation that doesn't add any real value).
>>   * Our existing classes are not up to standard!
>>
>>
>> So what can be done to remedy the situation?
>>
>> We could, of course, try to simplify the classes as much as we can and
>> maybe address some of the issues.
>> This won't alleviate the boilerplate code problem altogether, though.
>>
>> We could write annotations to do some of the things for us automatically.
>> The easiest approach would be runtime-based, and would hinder performance.
>> This also means we need to maintain this "infrastructure" and all the
>> implications of such a decision.
>>
>>
>> Luckily, there is a much easier solution: Someone else already did it!
>>
>> Check out Project Lombok: http://projectlombok.org
>> What Lombok gives us, among some other things, is a way to greatly
>> simplify our POJOs by using annotations to get the boilerplate code
>> automatically generated.
>> This means we get the benefit of annotations which would simplify the
>> code a whole lot, while not imposing a performance cost (since the
>> boilerplate code is generated during compilation).
>> However, it's also possible to create the methods yourself if you want
>> them to behave differently.
>> Outside the POJO itself, you would see it as you would always see it.
>>
>> So what are the downsides to this approach?
>>
>>   * First of all, Lombok provides also some other capabilities which I'm
>>     not sure are required/wanted at this time.
>>       o That's why I propose we use it for commons project, and make use
>>         of it's POJO-related annotations ONLY.
>>   * There might be a problem debugging the code since it's auto-generated.
>>       o I think this is rather negligible, since usually you don't debug
>>         POJOs anyway.
>>   * There might be a problem if the auto-generated code throws an Exception.
>>       o As before, I'm rather sure this is an edge-case which we usually
>>         won't hit (if at all).
>>
>>
>> Even given these possible downsides, I think that we would benefit
>> greatly if we would introduce this library.
>>
>> If you have any questions, you're welcome to study out the project site
>> which has very thorough documentation: http://projectlombok.org
>>
>> Your thoughts on the matter?
>>
> 
> - I think an example of before/after pojo would help demonstrating how
> good the framework is.
> 
> - Would it work when adding JPA annotations?
> 
>> Regards,
>> Mike
>>

Watching the demo it looks like we'll get less code, which in many cases is a good thing.
What I'm concerned about is traceability; or- how can we track issues coming from the field
when function calls and line numbers in the stack trace will not match the code we know.

-- 

/d

"Willyoupleasehelpmefixmykeyboard?Thespacebarisbroken!"


From iheim at redhat.com  Tue Jan 31 11:31:49 2012
From: iheim at redhat.com (Itamar Heim)
Date: Tue, 31 Jan 2012 13:31:49 +0200
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <6d5d3742-15be-4dfc-b733-6c4b9e86507c@zmail14.collab.prod.int.phx2.redhat.com>
References: <6d5d3742-15be-4dfc-b733-6c4b9e86507c@zmail14.collab.prod.int.phx2.redhat.com>
Message-ID: <4F27D125.20204@redhat.com>

On 01/31/2012 12:04 PM, Mike Kolesnik wrote:
>
> ----- Original Message -----
>> On 01/31/2012 03:03 AM, Eli Mesika wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Oved Ourfalli"<ovedo at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Sent: Monday, January 30, 2012 9:45:47 AM
>>>> Subject: [Engine-devel] Development process wiki page
>>>>
>>>> Hey all,
>>>>
>>>> I've wrote a wiki page on the oVirt development process.
>>>> It contains mostly information on the patch review process, patch
>>>> submission and some git guidelines.
>>>>
>>>> http://www.ovirt.org/wiki/DevProcess
>>>>
>>>> I added a link to it from the main wiki page.
>>>>
>>>> Your comments are welcome.
>>>
>>> my 2 cents : (obvious , but should be in the wiki)
>>>
>>> 1) Regarding patch separation, still each commit should not break
>>> the build
>>> 2) When having more than one patch set due to comments and applied
>>> fixes add to the commit message V[n] :<changes description>   where
>>> [n] is the patch set number
>>
>> I agree we need this, but the problem is we are lacking the [0/nnn]
>> cover letter email you get for such commentary when using gerrit.
>> and there is a concept of the commit itself should not include the
>> history of the patch being reviewed.
>> so we need to adapt this to gerrit.
>> maybe a convention that when uploading a new patch set, contributor
>> also
>> adds a review comment of the changes from previous version (similar
>> to
>> the cover letter).
>
> I think if the commit history already exists in gerrit, and the commit itself is pushed through gerrit, then this will be redundant.

this is not about the patch history, it is about providing reviewers a 
focus point as to what changed in this new version of the patch set.

>
>> I wonder if the git/gerrit utility some adopted from openstack can
>> help
>> with such a step to allow adding a comment when pushing the patch for
>> review.
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>



From mkolesni at redhat.com  Tue Jan 31 12:14:03 2012
From: mkolesni at redhat.com (Mike Kolesnik)
Date: Tue, 31 Jan 2012 07:14:03 -0500 (EST)
Subject: [Engine-devel] Simplifying our POJOs
In-Reply-To: <4F27C651.7070508@redhat.com>
Message-ID: <8ef1b30e-641b-4e6f-bd39-175bf0363c69@zmail14.collab.prod.int.phx2.redhat.com>


----- Original Message -----
> On 31/01/12 12:39, Livnat Peer wrote:
> > On 31/01/12 12:02, Mike Kolesnik wrote:
> >> Hi,
> >>
> >> Today many POJO
> >> <http://en.wikipedia.org/wiki/Plain_Old_Java_Object>s
> >> are used throughout the system to convey data:
> >>
> >>   *   Parameters - To send data to commands.
> >>   *   Business Entities - To transfer data in the parameters &
> >>   to/from
> >>     the DB.
> >>
> >> These POJOs are (usually) very verbose and full of boilerplate
> >> code
> >> <http://en.wikipedia.org/wiki/Boilerplate_code>.
> >>
> >> This, in turn, reduces their readability and maintainability for a
> >> couple of reasons (that I can think of):
> >>
> >>   * It's hard to know what does what:
> >>       o Who participates in equals/hashCode?
> >>       o What fields are printed in toString?
> >>   * Consistency is problematic:
> >>       o A field may be part of equals but not hashCode, or vice
> >>       versa.
> >>       o This breaks the Object.hashCode()
> >>         <http://docs.oracle.com/javase/6/docs/api/java/lang/Object.html#hashCode%28%29>
> >>         contract!
> >>   * Adding/Removing fields take more time since you need to
> >>   synchronize
> >>     the change to all boilerplate methods.
> >>       o Again, we're facing the consistency problem.
> >>   * These simple classes tend to be very long and not very
> >>   readable.
> >>   * Boilerplate code makes it harder to find out which methods
> >>   *don't*
> >>     behave the default way.
> >>   * Javadoc, if existent, is usually meaningless (but you might
> >>   see some
> >>     banal documentation that doesn't add any real value).
> >>   * Our existing classes are not up to standard!
> >>
> >>
> >> So what can be done to remedy the situation?
> >>
> >> We could, of course, try to simplify the classes as much as we can
> >> and
> >> maybe address some of the issues.
> >> This won't alleviate the boilerplate code problem altogether,
> >> though.
> >>
> >> We could write annotations to do some of the things for us
> >> automatically.
> >> The easiest approach would be runtime-based, and would hinder
> >> performance.
> >> This also means we need to maintain this "infrastructure" and all
> >> the
> >> implications of such a decision.
> >>
> >>
> >> Luckily, there is a much easier solution: Someone else already did
> >> it!
> >>
> >> Check out Project Lombok: http://projectlombok.org

If you're interested in just the effect and don't/can't watch the 3min demo on the site, I attached examples of a class before & after.

> >> What Lombok gives us, among some other things, is a way to greatly
> >> simplify our POJOs by using annotations to get the boilerplate
> >> code
> >> automatically generated.
> >> This means we get the benefit of annotations which would simplify
> >> the
> >> code a whole lot, while not imposing a performance cost (since the
> >> boilerplate code is generated during compilation).
> >> However, it's also possible to create the methods yourself if you
> >> want
> >> them to behave differently.
> >> Outside the POJO itself, you would see it as you would always see
> >> it.
> >>
> >> So what are the downsides to this approach?
> >>
> >>   * First of all, Lombok provides also some other capabilities
> >>   which I'm
> >>     not sure are required/wanted at this time.
> >>       o That's why I propose we use it for commons project, and
> >>       make use
> >>         of it's POJO-related annotations ONLY.
> >>   * There might be a problem debugging the code since it's
> >>   auto-generated.
> >>       o I think this is rather negligible, since usually you don't
> >>       debug
> >>         POJOs anyway.
> >>   * There might be a problem if the auto-generated code throws an
> >>   Exception.
> >>       o As before, I'm rather sure this is an edge-case which we
> >>       usually
> >>         won't hit (if at all).
> >>
> >>
> >> Even given these possible downsides, I think that we would benefit
> >> greatly if we would introduce this library.
> >>
> >> If you have any questions, you're welcome to study out the project
> >> site
> >> which has very thorough documentation: http://projectlombok.org
> >>
> >> Your thoughts on the matter?
> >>
> > 
> > - I think an example of before/after pojo would help demonstrating
> > how
> > good the framework is.
> > 
> > - Would it work when adding JPA annotations?
> > 
> >> Regards,
> >> Mike
> >>
> 
> Watching the demo it looks like we'll get less code, which in many
> cases is a good thing.
> What I'm concerned about is traceability; or- how can we track issues
> coming from the field
> when function calls and line numbers in the stack trace will not
> match the code we know.

As I said these are edge cases which although are important, I'm not sure if we will ever hit them.
Also bear in mind that you can still write your own code in the POJO, which will take precedence over the auto generated code, and I believe that for that code the lin numbers will be in-sync.

> 
> --
> 
> /d
> 
> "Willyoupleasehelpmefixmykeyboard?Thespacebarisbroken!"
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: SimplePerson.java
Type: text/x-java
Size: 90 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120131/15aa2202/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: BolierplatePerson.java
Type: text/x-java
Size: 1018 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120131/15aa2202/attachment-0001.bin>

From mburns at redhat.com  Tue Jan 31 12:49:54 2012
From: mburns at redhat.com (Mike Burns)
Date: Tue, 31 Jan 2012 07:49:54 -0500
Subject: [Engine-devel] [Users] New oVirt-engine RPMs available -
 Release Candidates
In-Reply-To: <4F27B5DE.1080701@redhat.com>
References: <4F251098.6080503@redhat.com>
	<1327957247.2417.58.camel@beelzebub.mburnsfire.net>
	<4F27B244.4070906@redhat.com> <4F27B5DE.1080701@redhat.com>
Message-ID: <1328014194.2417.71.camel@beelzebub.mburnsfire.net>

On Tue, 2012-01-31 at 11:35 +0200, Doron Fediuck wrote:
> On 31/01/12 11:20, Ofer Schreiber wrote:
> > On 01/30/2012 11:00 PM, Mike Burns wrote:
> >> On Sun, 2012-01-29 at 11:25 +0200, Ofer Schreiber wrote:
> >>> Hi all,
> >>>
> >>> We've just finished uploading a new set of oVirt-engine rpms into
> >>> ovirt.org.
> >>>
> >>> These rpms, versioned 3.0.0_0001-1.4, are considered as release candidates for our first release.
> >>>
> >>> In order to install the RPMs, please follow the instructions described in
> >>> http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
> >>>
> >>> Feedback, comments and bug reports are always welcome.
> >>>
> >>> The oVirt-Engine team
> >>>
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> >> I just noticed that a fix [1] to a bug [2] required for ovirt-node to
> >> register itself to the engine is not in the RC.  Can we get this pushed
> >> prior to release?
> >>
> >> [1] http://gerrit.ovirt.org/#change,1120
> >> [2] https://bugzilla.redhat.com/show_bug.cgi?id=782663
> >>
> >> Thanks
> >>
> >> Mike
> >>
> > 
> > Just to understand the impact- All nodes cannot be registered?

The add host flow works (mostly [1]).  But the logical flow when
configuring ovirt-node is to configure the hostname/ip address of the
engine and then approve it from the engine interface.  This flow doesn't
work.

> > Roy- could you please push this fix?
> > 
> Since this changes the URI, it needs basic integration with ovirt node.
> Before pushing it, this should be verified with current upstream ovirt-node.
> Mike & Roy- did someone try to use it with latest ovirt-node?

At the moment, we don't have a host setup where we can build and deploy
the engine from the git repo. If you can share one (off-list if it's
internal only), then it would be much appreciated.   If not, then I'll
try to get something up today.

Mike
> 
> > Added to first release blockers atm.
> > 
> 
> 

Mike

[1] https://bugzilla.redhat.com/show_bug.cgi?id=785956



From sgordon at redhat.com  Tue Jan 31 14:45:26 2012
From: sgordon at redhat.com (Steve Gordon)
Date: Tue, 31 Jan 2012 09:45:26 -0500 (EST)
Subject: [Engine-devel] [Users] New oVirt-engine RPMs available -
	Release Candidates
In-Reply-To: <4F251098.6080503@redhat.com>
Message-ID: <d273135c-c0b5-4505-9250-01a200c2f99c@zmail15.collab.prod.int.phx2.redhat.com>



----- Original Message -----
> From: "Ofer Schreiber" <oschreib at redhat.com>
> To: users at ovirt.org, engine-devel at ovirt.org
> Sent: Sunday, January 29, 2012 4:25:44 AM
> Subject: [Users] New oVirt-engine RPMs available - Release Candidates
> 
> Hi all,
> 
> We've just finished uploading a new set of oVirt-engine rpms into
> ovirt.org.
> 
> These rpms, versioned 3.0.0_0001-1.4, are considered as release
> candidates for our first release.
> 
> In order to install the RPMs, please follow the instructions
> described in
> http://www.ovirt.org/wiki/Installing_ovirt_from_rpm
> 
> Feedback, comments and bug reports are always welcome.
> 
> The oVirt-Engine team
> 

Given we are nearing a stable release, is it possible to get an updated repo file put up that takes into account both nightly and stable releases (unless of course you want a separate file for each)? I had previously proposed something like this, which defaults to pointing at the stable repository but --enablerepo=ovirt-nightly can be used to get nightly builds:

    http://www.ovirt.org/wiki/Yum_repo_file

If we took this approach then it probably could/should go in http://www.ovirt.org/releases/ rather than the current location under nightly. Just my 2c.

Thanks,

STeve


From yzaslavs at redhat.com  Tue Jan 31 18:40:38 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Tue, 31 Jan 2012 20:40:38 +0200
Subject: [Engine-devel] Simplifying our POJOs
In-Reply-To: <4F27C651.7070508@redhat.com>
References: <fc563815-f97c-4cc4-8d6f-0b9a3cd0bbc3@zmail14.collab.prod.int.phx2.redhat.com>	<4F27C4E7.3050600@redhat.com>
	<4F27C651.7070508@redhat.com>
Message-ID: <4F2835A6.9020707@redhat.com>

On 01/31/2012 12:45 PM, Doron Fediuck wrote:
> On 31/01/12 12:39, Livnat Peer wrote:
>> On 31/01/12 12:02, Mike Kolesnik wrote:
>>> Hi,
>>>
>>> Today many POJO <http://en.wikipedia.org/wiki/Plain_Old_Java_Object>s
>>> are used throughout the system to convey data:
>>>
>>>   *   Parameters - To send data to commands.
>>>   *   Business Entities - To transfer data in the parameters & to/from
>>>     the DB.
>>>
>>> These POJOs are (usually) very verbose and full of boilerplate code
>>> <http://en.wikipedia.org/wiki/Boilerplate_code>.
>>>
>>> This, in turn, reduces their readability and maintainability for a
>>> couple of reasons (that I can think of):
>>>
>>>   * It's hard to know what does what:
>>>       o Who participates in equals/hashCode?
>>>       o What fields are printed in toString? 
>>>   * Consistency is problematic:
>>>       o A field may be part of equals but not hashCode, or vice versa.
>>>       o This breaks the Object.hashCode()
>>>         <http://docs.oracle.com/javase/6/docs/api/java/lang/Object.html#hashCode%28%29>
>>>         contract!
>>>   * Adding/Removing fields take more time since you need to synchronize
>>>     the change to all boilerplate methods.
>>>       o Again, we're facing the consistency problem.
>>>   * These simple classes tend to be very long and not very readable.
>>>   * Boilerplate code makes it harder to find out which methods *don't*
>>>     behave the default way.
>>>   * Javadoc, if existent, is usually meaningless (but you might see some
>>>     banal documentation that doesn't add any real value).
>>>   * Our existing classes are not up to standard!
>>>
>>>
>>> So what can be done to remedy the situation?
>>>
>>> We could, of course, try to simplify the classes as much as we can and
>>> maybe address some of the issues.
>>> This won't alleviate the boilerplate code problem altogether, though.
>>>
>>> We could write annotations to do some of the things for us automatically.
>>> The easiest approach would be runtime-based, and would hinder performance.
>>> This also means we need to maintain this "infrastructure" and all the
>>> implications of such a decision.
>>>
>>>
>>> Luckily, there is a much easier solution: Someone else already did it!
>>>
>>> Check out Project Lombok: http://projectlombok.org
>>> What Lombok gives us, among some other things, is a way to greatly
>>> simplify our POJOs by using annotations to get the boilerplate code
>>> automatically generated.
>>> This means we get the benefit of annotations which would simplify the
>>> code a whole lot, while not imposing a performance cost (since the
>>> boilerplate code is generated during compilation).
>>> However, it's also possible to create the methods yourself if you want
>>> them to behave differently.
>>> Outside the POJO itself, you would see it as you would always see it.
>>>
>>> So what are the downsides to this approach?
>>>
>>>   * First of all, Lombok provides also some other capabilities which I'm
>>>     not sure are required/wanted at this time.
>>>       o That's why I propose we use it for commons project, and make use
>>>         of it's POJO-related annotations ONLY.
>>>   * There might be a problem debugging the code since it's auto-generated.
>>>       o I think this is rather negligible, since usually you don't debug
>>>         POJOs anyway.
>>>   * There might be a problem if the auto-generated code throws an Exception.
>>>       o As before, I'm rather sure this is an edge-case which we usually
>>>         won't hit (if at all).
>>>
>>>
>>> Even given these possible downsides, I think that we would benefit
>>> greatly if we would introduce this library.
>>>
>>> If you have any questions, you're welcome to study out the project site
>>> which has very thorough documentation: http://projectlombok.org
>>>
>>> Your thoughts on the matter?
>>>
>>
>> - I think an example of before/after pojo would help demonstrating how
>> good the framework is.
>>
>> - Would it work when adding JPA annotations?
I suspect that yes (needs to be checked)
Will it work with GWT (if we create new business entity that needs to be
exposed to GWT guys) ?
>>
>>> Regards,
>>> Mike
>>>
> 
> Watching the demo it looks like we'll get less code, which in many cases is a good thing.
> What I'm concerned about is traceability; or- how can we track issues coming from the field
> when function calls and line numbers in the stack trace will not match the code we know.
> 



From masayag at redhat.com  Tue Jan 31 22:56:29 2012
From: masayag at redhat.com (Moti Asayag)
Date: Wed, 01 Feb 2012 00:56:29 +0200
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <4F27D125.20204@redhat.com>
References: <6d5d3742-15be-4dfc-b733-6c4b9e86507c@zmail14.collab.prod.int.phx2.redhat.com>
	<4F27D125.20204@redhat.com>
Message-ID: <4F28719D.2050208@redhat.com>

On 01/31/2012 01:31 PM, Itamar Heim wrote:
> On 01/31/2012 12:04 PM, Mike Kolesnik wrote:
>>
>> ----- Original Message -----
>>> On 01/31/2012 03:03 AM, Eli Mesika wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Oved Ourfalli"<ovedo at redhat.com>
>>>>> To: engine-devel at ovirt.org
>>>>> Sent: Monday, January 30, 2012 9:45:47 AM
>>>>> Subject: [Engine-devel] Development process wiki page
>>>>>
>>>>> Hey all,
>>>>>
>>>>> I've wrote a wiki page on the oVirt development process.
>>>>> It contains mostly information on the patch review process, patch
>>>>> submission and some git guidelines.
>>>>>
>>>>> http://www.ovirt.org/wiki/DevProcess
>>>>>
>>>>> I added a link to it from the main wiki page.
>>>>>
>>>>> Your comments are welcome.
>>>>
>>>> my 2 cents : (obvious , but should be in the wiki)
>>>>
>>>> 1) Regarding patch separation, still each commit should not break
>>>> the build
>>>> 2) When having more than one patch set due to comments and applied
>>>> fixes add to the commit message V[n] :<changes description>   where
>>>> [n] is the patch set number
>>>
>>> I agree we need this, but the problem is we are lacking the [0/nnn]
>>> cover letter email you get for such commentary when using gerrit.
>>> and there is a concept of the commit itself should not include the
>>> history of the patch being reviewed.
>>> so we need to adapt this to gerrit.
>>> maybe a convention that when uploading a new patch set, contributor
>>> also
>>> adds a review comment of the changes from previous version (similar
>>> to
>>> the cover letter).
>>
>> I think if the commit history already exists in gerrit, and the commit
>> itself is pushed through gerrit, then this will be redundant.
> 
> this is not about the patch history, it is about providing reviewers a
> focus point as to what changed in this new version of the patch set.
> 

However for the +2 the entire patch should be reviewed again as there
might be changes after rebasing the code (even for a new file, e.g. db
upgrade scripts with already used version in the file name).

Following the small patches convention should ease the review, combining
with the short message after the push.

However many times the newer version of a patch is just a rebase, so it
seems redundant adding just the "patch was rebased" message.

>>
>>> I wonder if the git/gerrit utility some adopted from openstack can
>>> help
>>> with such a step to allow adding a comment when pushing the patch for
>>> review.
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From iheim at redhat.com  Tue Jan 31 23:04:33 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 01 Feb 2012 01:04:33 +0200
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <4F28719D.2050208@redhat.com>
References: <6d5d3742-15be-4dfc-b733-6c4b9e86507c@zmail14.collab.prod.int.phx2.redhat.com>
	<4F27D125.20204@redhat.com> <4F28719D.2050208@redhat.com>
Message-ID: <4F287381.6040001@redhat.com>

On 02/01/2012 12:56 AM, Moti Asayag wrote:
> On 01/31/2012 01:31 PM, Itamar Heim wrote:
>> On 01/31/2012 12:04 PM, Mike Kolesnik wrote:
>>>
>>> ----- Original Message -----
>>>> On 01/31/2012 03:03 AM, Eli Mesika wrote:
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Oved Ourfalli"<ovedo at redhat.com>
>>>>>> To: engine-devel at ovirt.org
>>>>>> Sent: Monday, January 30, 2012 9:45:47 AM
>>>>>> Subject: [Engine-devel] Development process wiki page
>>>>>>
>>>>>> Hey all,
>>>>>>
>>>>>> I've wrote a wiki page on the oVirt development process.
>>>>>> It contains mostly information on the patch review process, patch
>>>>>> submission and some git guidelines.
>>>>>>
>>>>>> http://www.ovirt.org/wiki/DevProcess
>>>>>>
>>>>>> I added a link to it from the main wiki page.
>>>>>>
>>>>>> Your comments are welcome.
>>>>>
>>>>> my 2 cents : (obvious , but should be in the wiki)
>>>>>
>>>>> 1) Regarding patch separation, still each commit should not break
>>>>> the build
>>>>> 2) When having more than one patch set due to comments and applied
>>>>> fixes add to the commit message V[n] :<changes description>    where
>>>>> [n] is the patch set number
>>>>
>>>> I agree we need this, but the problem is we are lacking the [0/nnn]
>>>> cover letter email you get for such commentary when using gerrit.
>>>> and there is a concept of the commit itself should not include the
>>>> history of the patch being reviewed.
>>>> so we need to adapt this to gerrit.
>>>> maybe a convention that when uploading a new patch set, contributor
>>>> also
>>>> adds a review comment of the changes from previous version (similar
>>>> to
>>>> the cover letter).
>>>
>>> I think if the commit history already exists in gerrit, and the commit
>>> itself is pushed through gerrit, then this will be redundant.
>>
>> this is not about the patch history, it is about providing reviewers a
>> focus point as to what changed in this new version of the patch set.
>>
>
> However for the +2 the entire patch should be reviewed again as there
> might be changes after rebasing the code (even for a new file, e.g. db
> upgrade scripts with already used version in the file name).
>
> Following the small patches convention should ease the review, combining
> with the short message after the push.
>
> However many times the newer version of a patch is just a rebase, so it
> seems redundant adding just the "patch was rebased" message.

why - wouldn't it save a lot of time for the reviewers to know this is 
the only change in this patchset, rather than trying to see there are no 
changes in it?
I agree message shouldn't be in the commit, but a review comment by the 
patch submitter would go a long way helping reviewers.


From masayag at redhat.com  Tue Jan 31 23:25:36 2012
From: masayag at redhat.com (Moti Asayag)
Date: Wed, 01 Feb 2012 01:25:36 +0200
Subject: [Engine-devel] Development process wiki page
In-Reply-To: <4F287381.6040001@redhat.com>
References: <6d5d3742-15be-4dfc-b733-6c4b9e86507c@zmail14.collab.prod.int.phx2.redhat.com>
	<4F27D125.20204@redhat.com> <4F28719D.2050208@redhat.com>
	<4F287381.6040001@redhat.com>
Message-ID: <4F287870.9050800@redhat.com>

On 02/01/2012 01:04 AM, Itamar Heim wrote:
> On 02/01/2012 12:56 AM, Moti Asayag wrote:
>> On 01/31/2012 01:31 PM, Itamar Heim wrote:
>>> On 01/31/2012 12:04 PM, Mike Kolesnik wrote:
>>>>
>>>> ----- Original Message -----
>>>>> On 01/31/2012 03:03 AM, Eli Mesika wrote:
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Oved Ourfalli"<ovedo at redhat.com>
>>>>>>> To: engine-devel at ovirt.org
>>>>>>> Sent: Monday, January 30, 2012 9:45:47 AM
>>>>>>> Subject: [Engine-devel] Development process wiki page
>>>>>>>
>>>>>>> Hey all,
>>>>>>>
>>>>>>> I've wrote a wiki page on the oVirt development process.
>>>>>>> It contains mostly information on the patch review process, patch
>>>>>>> submission and some git guidelines.
>>>>>>>
>>>>>>> http://www.ovirt.org/wiki/DevProcess
>>>>>>>
>>>>>>> I added a link to it from the main wiki page.
>>>>>>>
>>>>>>> Your comments are welcome.
>>>>>>
>>>>>> my 2 cents : (obvious , but should be in the wiki)
>>>>>>
>>>>>> 1) Regarding patch separation, still each commit should not break
>>>>>> the build
>>>>>> 2) When having more than one patch set due to comments and applied
>>>>>> fixes add to the commit message V[n] :<changes description>    where
>>>>>> [n] is the patch set number
>>>>>
>>>>> I agree we need this, but the problem is we are lacking the [0/nnn]
>>>>> cover letter email you get for such commentary when using gerrit.
>>>>> and there is a concept of the commit itself should not include the
>>>>> history of the patch being reviewed.
>>>>> so we need to adapt this to gerrit.
>>>>> maybe a convention that when uploading a new patch set, contributor
>>>>> also
>>>>> adds a review comment of the changes from previous version (similar
>>>>> to
>>>>> the cover letter).
>>>>
>>>> I think if the commit history already exists in gerrit, and the commit
>>>> itself is pushed through gerrit, then this will be redundant.
>>>
>>> this is not about the patch history, it is about providing reviewers a
>>> focus point as to what changed in this new version of the patch set.
>>>
>>
>> However for the +2 the entire patch should be reviewed again as there
>> might be changes after rebasing the code (even for a new file, e.g. db
>> upgrade scripts with already used version in the file name).
>>
>> Following the small patches convention should ease the review, combining
>> with the short message after the push.
>>
>> However many times the newer version of a patch is just a rebase, so it
>> seems redundant adding just the "patch was rebased" message.
> 
> why - wouldn't it save a lot of time for the reviewers to know this is
> the only change in this patchset, rather than trying to see there are no
> changes in it?
> I agree message shouldn't be in the commit, but a review comment by the
> patch submitter would go a long way helping reviewers.

If those patches are about to be submitted, there is a need to go all
over them. If it is 'review-in-process' it will surely save time.

Just thinking on multi-patchset (series of dependent patches) where each
push to gerrit for review requires going over all of the patches.