From ovedo at redhat.com  Sun Sep  2 07:08:36 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Sun, 2 Sep 2012 03:08:36 -0400 (EDT)
Subject: [Engine-devel] network subnet
In-Reply-To: <571855213.4031461.1346357490996.JavaMail.root@redhat.com>
Message-ID: <1094820802.7482459.1346569716272.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Alon Bar-Lev" <alonbl at redhat.com>
> To: "Livnat Peer" <lpeer at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Thursday, August 30, 2012 11:11:30 PM
> Subject: Re: [Engine-devel] network subnet
> 
> 
> 
> ----- Original Message -----
> > From: "Livnat Peer" <lpeer at redhat.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Thursday, August 30, 2012 10:16:05 PM
> > Subject: Re: [Engine-devel] network subnet
> > 
> > On 30/08/12 21:39, Alon Bar-Lev wrote:
> > > 
> > > 
> > > ----- Original Message -----
> > >> From: "Livnat Peer" <lpeer at redhat.com>
> > >> To: engine-devel at ovirt.org
> > >> Sent: Thursday, August 30, 2012 3:22:29 PM
> > >> Subject: [Engine-devel] network subnet
> > >>
> > >> Hi All,
> > >>
> > >> Today when a user wants to define a network subnet mask, he does
> > >> it
> > >> when
> > >> he attaches the network to a host NIC.
> > >>
> > >> I was wondering if there is a reason not to define the network
> > >> subnet
> > >> on
> > >> the logical network entity (Data center level).
> > >>
> > >> Thanks, Livnat
> > > 
> > > Hello,
> > > 
> > > I am sorry, maybe I do not understand... the IP scheme enforces
> > > the
> > > use of address mask in order to properly route packets.
> > 
> > of course. My proposal is related to our user usage of the system.
> > Today
> > ovirt user, who wants to define a network subnet, has to type the
> > subnet
> > per host (per network), I think the user should only define it once
> > on
> > the logical network entity in the Data Center.
> > Propagating the value to all hosts is needed but it should be our
> > internal implementation detail.

I guess that would be solved by using host-network profiles/templates.
The user will define it once, and apply each host with the relevant profile.

Hope we will get to that sometime in the near future. It will surely save a lot of time and effort when trying to set up new environments.

> > 
> > > 
> > > Network mask is used in any case, I guess it can be dropped from
> > > configuration in favour of using the address class as mask, is
> > > that what you suggest?
> > > 
> > 
> > No, hope the above paragraph made it more clear.
> > 
> 
> Hello,
> 
> Then you assume that a logical network, which is actually layer 2
> network in our implementation, has layer 3 characteristics, right?
> 
> In our current implementation "data center logical network" is pure
> layer 2 segment aka layer 2 broadcast domain.
> 
> One can use the same logical network for multiple layer 3 segments,
> which is totally valid and consistent with standard physical layer 2
> setup.
> 
> Unless I am missing something crucial, I would suggest to keep the
> consistent physical->virtual mapping, unless we emulate layer 3
> switching. Layer 2 does not have layer 3 characteristics.
> 
> Regards,
> Alon.
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From irenab at mellanox.com  Sun Sep  2 12:33:38 2012
From: irenab at mellanox.com (Irena Berezovsky)
Date: Sun, 2 Sep 2012 12:33:38 +0000
Subject: [Engine-devel] VM migration state in after_vm_destroy hook
Message-ID: <9D25E123B44F4A4291F4B5C13DA94E772B3C426F@MTRDAG01.mtl.com>


Hi,
Is there any way to know in context of VDSM 'after_vm_destroy' hook if VM is terminated (globally in the cluster) or migrated to other Host?
Is there some Environment parameter or other indication?

Thanks a lot,
Irena
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120902/68b8b173/attachment.html>

From ovedo at redhat.com  Sun Sep  2 13:08:51 2012
From: ovedo at redhat.com (Oved Ourfalli)
Date: Sun, 2 Sep 2012 09:08:51 -0400 (EDT)
Subject: [Engine-devel] VM migration state in after_vm_destroy hook
In-Reply-To: <9D25E123B44F4A4291F4B5C13DA94E772B3C426F@MTRDAG01.mtl.com>
Message-ID: <425527357.7523973.1346591331601.JavaMail.root@redhat.com>

Not a hook expert, but I looked at the code, and besides after_vm_destroy we also have:
before_vm_migrate_source
after_vm_migrate_source
before_vm_migrate_destination
after_vm_migrate_destination 

So, I guess you can put the relevant code in the after_vm_migrate_source for the migration use-case, and after_vm_destroy for the destroy use-case.

Oved

----- Original Message -----
> From: "Irena Berezovsky" <irenab at mellanox.com>
> To: engine-devel at ovirt.org
> Sent: Sunday, September 2, 2012 3:33:38 PM
> Subject: [Engine-devel] VM migration state in after_vm_destroy hook
> 
> 
> 
> 
> 
> 
> 
> Hi,
> 
> Is there any way to know in context of VDSM ?after_vm_destroy? hook
> if VM is terminated (globally in the cluster) or migrated to other
> Host?
> 
> Is there some Environment parameter or other indication?
> 
> 
> 
> Thanks a lot,
> 
> Irena
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From thomas at parker.iz34.com  Sun Sep  2 13:20:04 2012
From: thomas at parker.iz34.com (thomas at parker.iz34.com)
Date: Sun, 2 Sep 2012 15:20:04 +0200
Subject: [Engine-devel] Change in ovirt-engine[master]: webadmin: Power
	management	confirmation fix (#848133)
In-Reply-To: <201209021316.q82DGCQA028831@gerrit.ovirt.org>
References: <201209021316.q82DGCQA028831@gerrit.ovirt.org>
Message-ID: <201209021316.q82DGCQA028831@gerrit.ovirt.org>

Alexey Chub has submitted this change and it was merged.

Change subject: webadmin: "Power management" confirmation fix (#848133)
.....................................................................


webadmin: "Power management" confirmation fix (#848133)

Change-Id: I7636e84ffc66ae6fda3d3bc5f5c48d9e1449da6a
Signed-off-by: Alexey Chub <achub at redhat.com>
---
M frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/clusters/ClusterGuideModel.java
M frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/datacenters/DataCenterGuideModel.java
M frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/hosts/HostListModel.java
M frontend/webadmin/modules/uicompat/src/main/java/org/ovirt/engine/ui/uicompat/Constants.java
4 files changed, 27 insertions(+), 20 deletions(-)

Approvals:
  Alexey Chub: Verified; Looks good to me, approved


--
To view, visit http://gerrit.ovirt.org/7670
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I7636e84ffc66ae6fda3d3bc5f5c48d9e1449da6a
Gerrit-PatchSet: 2
Gerrit-Project: ovirt-engine
Gerrit-Branch: master
Gerrit-Owner: Alexey Chub <achub at redhat.com>
Gerrit-Reviewer: Alexey Chub <achub at redhat.com>
Gerrit-Reviewer: Gilad Chaplik <gchaplik at redhat.com>
_______________________________________________
Engine-commits mailing list
Engine-commits at ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-commits
-------------- next part --------------
A non-text attachment was scrubbed...
Name: originalBody.txt
Type: application/octet-stream
Size: 1545 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120902/0957c0b2/attachment.obj>

From dfediuck at redhat.com  Sun Sep  2 14:21:47 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Sun, 2 Sep 2012 10:21:47 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <1750220219.3758540.1346415154344.JavaMail.root@redhat.com>
Message-ID: <1890405098.4216852.1346595707950.JavaMail.root@redhat.com>

----- Original Message -----
> From: "Andrew Cathrow" <acathrow at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "David Ja?a" <djasa at redhat.com>, engine-devel at ovirt.org
> Sent: Friday, August 31, 2012 3:12:34 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> 
> 
> ----- Original Message -----
> > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > To: "David Ja?a" <djasa at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Friday, August 31, 2012 5:09:47 AM
> > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "David Ja?a" <djasa at redhat.com>
> > > To: engine-devel at ovirt.org
> > > Sent: Friday, August 31, 2012 11:57:11 AM
> > > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > 
> > > Alon Bar-Lev p??e v ?t 30. 08. 2012 v 14:40 -0400:
> > > > 
> > > > ----- Original Message -----
> > > > > From: "Andrew Cathrow" <acathrow at redhat.com>
> > > > > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > > > > Cc: "Shireesh Anjal" <sanjal at redhat.com>,
> > > > > engine-devel at ovirt.org,
> > > > > "Selvasundaram" <sesubram at redhat.com>
> > > > > Sent: Thursday, August 30, 2012 9:37:59 PM
> > > > > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > > > 
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > > > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > > > > > To: "Selvasundaram" <sesubram at redhat.com>
> > > > > > Cc: "Shireesh Anjal" <sanjal at redhat.com>,
> > > > > > engine-devel at ovirt.org
> > > > > > Sent: Thursday, August 30, 2012 2:35:16 PM
> > > > > > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > ----- Original Message -----
> > > > > > > From: "Selvasundaram" <sesubram at redhat.com>
> > > > > > > To: engine-devel at ovirt.org
> > > > > > > Cc: "Shireesh Anjal" <sanjal at redhat.com>
> > > > > > > Sent: Thursday, August 30, 2012 4:30:16 PM
> > > > > > > Subject: [Engine-devel] Gluster IPTable configuration
> > > > > > > 
> > > > > > > 
> > > > > > > Hi,
> > > > > > > 
> > > > > > > I want to add gluster specific IPTable configuration in
> > > > > > > addition
> > > > > > > to
> > > > > > > the ovirt IPTable configuration (if it is gluster node).
> > > > > > > 
> > > > > > > There are two approaches,
> > > > > > > 1. Having one more gluster specific IP table config in db
> > > > > > > and
> > > > > > > merge
> > > > > > > with ovirt IPTable config (merging NOT appending)
> > > > > > > [I have the patch engine: Gluster specific firewall
> > > > > > > configurations
> > > > > > > #7244]
> > > > > > > 2. Having two different IP Table config (ovirt and
> > > > > > > ovirt+gluster)
> > > > > > > and
> > > > > > > use either one.
> > > > > > > 
> > > > > > > Please provide your suggestions or improvements on this.
> > > > > > > 
> > > > > > 
> > > > > > Hello all,
> > > > > > 
> > > > > > The mentioned patch[1], adds hard coded gluster code into
> > > > > > the
> > > > > > bootstrap code, manipulate the firewall configuration to be
> > > > > > gluster
> > > > > > specific. It hardcoded search for "reject", insert before
> > > > > > some
> > > > > > other
> > > > > > rules.
> > > > > > 
> > > > > > I believe this hardcode approach is obsolete now that we
> > > > > > have
> > > > > > proper
> > > > > > tools for templates.
> > > > > > 
> > > > > > A more robust solution would be defining generic profiles,
> > > > > > each
> > > > > > profile as a template, each template can refer to different
> > > > > > profiles, and assign profile to a node.
> > > > > > 
> > > > > > This way the implementation is not gluster [or any]
> > > > > > specific
> > > > > > and
> > > > > > can
> > > > > > be reused for more setups, code is cleaner.
> > > > > 
> > > > > 
> > > > > or create custom chains ?
> > > > 
> > > > Can you please elaborate what is custom chains?
> > > > Thanks!
> > > 
> > > iptables -N my_new_chain
> > > iptables -A my_new_chain <rule_1>
> > > iptables -A my_new_chain ...
> > > iptables -A my_new_chain <rule_n>
> > > 
> > > # if this <rule> is matched, packet goes through rules in
> > > my_new_chain
> > > iptables -A INPUT <rule> -j my_new_chain
> > > 
> > 
> > Hello,
> > 
> > How does this solve the original issue?
> 
> It makes it easier for customers who are adding their own IPTables
> configuration - when we do rhev-h plugins it'll make things easier.
> This way we don't wipe out the original rules we just add our own
> chain.
> 
> 

Current default (virt) iptables configuration is kept in the engine's DB.
So...

1. We can update the DB value to include Gluster configuration.
or
2. Have an additional DB value for virt+Gluster.

So (1) will work, but when gluster is not relevant we have redundant ports.
But my only reservation from (2) is that it's not scalable for additional
services.

Either way, I agree with Alon that melding the configurations at runtime
is unexpected, and shouldn't be used.

After some additional thought... we can have (1), and make all gluster
entries commented out (let's say with #GLSTR) by default, so it'll be
enabled only when gluster is being used. This will allow introducing future
configurations as well without the need for additional DB values.

Sample snip of DB value:
==================================================================================
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
# vdsm
-A INPUT -p tcp --dport 54321 -j ACCEPT
# libvirt tls
-A INPUT -p tcp --dport 16514 -j ACCEPT
# SSH
-A INPUT -p tcp --dport 22 -j ACCEPT
# guest consoles
-A INPUT -p tcp -m multiport --dports 5634:6166 -j ACCEPT
# gluster p1
#GLSTR-A INPUT -p tcp --dport p1 -j ACCEPT
# gluster p2-p6
#GLSTR-A INPUT -p tcp --dport p2:p8 -j ACCEPT
# Reject any other input traffic
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -m physdev ! --physdev-is-bridged -j REJECT --reject-with icmp-host-prohibited
COMMIT
==================================================================================

So as you can see, this is something we can handle in run-time. Also it's easy
to test, so predictable, and finally- human beings can look and understand what
it means.


> > The need to provide different rules to different hosts by software
> > installed on destination?
> > 
> > Standard host needs iptables X.
> > Gluster host needs iptables X+Y.
> > XXX host needs iptables X+Z.
> > Maintainer of Gluster knows what Y is.
> > Maintainer of XXX knows what Z is.
> > 
> > If we merge all to one entry product comes with default X.
> > User override X to A.
> > New version of product comes with default Y.
> > Upgrade options:
> > 1. System continues to use A.
> > 2. Some AI to upgrade and create A'.
> > 3. Revert to Y, dropping user's customization.
> > 
> > Or we can maintain one large table with complete configuration and
> > conditionals.
> > 
> > Alon.


From iheim at redhat.com  Sun Sep  2 20:51:40 2012
From: iheim at redhat.com (Itamar Heim)
Date: Sun, 02 Sep 2012 23:51:40 +0300
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <1890405098.4216852.1346595707950.JavaMail.root@redhat.com>
References: <1890405098.4216852.1346595707950.JavaMail.root@redhat.com>
Message-ID: <5043C6DC.1000705@redhat.com>

On 09/02/2012 05:21 PM, Doron Fediuck wrote:
> ----- Original Message -----
>> From: "Andrew Cathrow" <acathrow at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>> Cc: "David Ja?a" <djasa at redhat.com>, engine-devel at ovirt.org
>> Sent: Friday, August 31, 2012 3:12:34 PM
>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>
>>
>>
>> ----- Original Message -----
>>> From: "Alon Bar-Lev" <alonbl at redhat.com>
>>> To: "David Ja?a" <djasa at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Friday, August 31, 2012 5:09:47 AM
>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: "David Ja?a" <djasa at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Sent: Friday, August 31, 2012 11:57:11 AM
>>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>>>
>>>> Alon Bar-Lev p??e v ?t 30. 08. 2012 v 14:40 -0400:
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Andrew Cathrow" <acathrow at redhat.com>
>>>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>,
>>>>>> engine-devel at ovirt.org,
>>>>>> "Selvasundaram" <sesubram at redhat.com>
>>>>>> Sent: Thursday, August 30, 2012 9:37:59 PM
>>>>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Alon Bar-Lev" <alonbl at redhat.com>
>>>>>>> To: "Selvasundaram" <sesubram at redhat.com>
>>>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>,
>>>>>>> engine-devel at ovirt.org
>>>>>>> Sent: Thursday, August 30, 2012 2:35:16 PM
>>>>>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Selvasundaram" <sesubram at redhat.com>
>>>>>>>> To: engine-devel at ovirt.org
>>>>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
>>>>>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
>>>>>>>> Subject: [Engine-devel] Gluster IPTable configuration
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I want to add gluster specific IPTable configuration in
>>>>>>>> addition
>>>>>>>> to
>>>>>>>> the ovirt IPTable configuration (if it is gluster node).
>>>>>>>>
>>>>>>>> There are two approaches,
>>>>>>>> 1. Having one more gluster specific IP table config in db
>>>>>>>> and
>>>>>>>> merge
>>>>>>>> with ovirt IPTable config (merging NOT appending)
>>>>>>>> [I have the patch engine: Gluster specific firewall
>>>>>>>> configurations
>>>>>>>> #7244]
>>>>>>>> 2. Having two different IP Table config (ovirt and
>>>>>>>> ovirt+gluster)
>>>>>>>> and
>>>>>>>> use either one.
>>>>>>>>
>>>>>>>> Please provide your suggestions or improvements on this.
>>>>>>>>
>>>>>>>
>>>>>>> Hello all,
>>>>>>>
>>>>>>> The mentioned patch[1], adds hard coded gluster code into
>>>>>>> the
>>>>>>> bootstrap code, manipulate the firewall configuration to be
>>>>>>> gluster
>>>>>>> specific. It hardcoded search for "reject", insert before
>>>>>>> some
>>>>>>> other
>>>>>>> rules.
>>>>>>>
>>>>>>> I believe this hardcode approach is obsolete now that we
>>>>>>> have
>>>>>>> proper
>>>>>>> tools for templates.
>>>>>>>
>>>>>>> A more robust solution would be defining generic profiles,
>>>>>>> each
>>>>>>> profile as a template, each template can refer to different
>>>>>>> profiles, and assign profile to a node.
>>>>>>>
>>>>>>> This way the implementation is not gluster [or any]
>>>>>>> specific
>>>>>>> and
>>>>>>> can
>>>>>>> be reused for more setups, code is cleaner.
>>>>>>
>>>>>>
>>>>>> or create custom chains ?
>>>>>
>>>>> Can you please elaborate what is custom chains?
>>>>> Thanks!
>>>>
>>>> iptables -N my_new_chain
>>>> iptables -A my_new_chain <rule_1>
>>>> iptables -A my_new_chain ...
>>>> iptables -A my_new_chain <rule_n>
>>>>
>>>> # if this <rule> is matched, packet goes through rules in
>>>> my_new_chain
>>>> iptables -A INPUT <rule> -j my_new_chain
>>>>
>>>
>>> Hello,
>>>
>>> How does this solve the original issue?
>>
>> It makes it easier for customers who are adding their own IPTables
>> configuration - when we do rhev-h plugins it'll make things easier.
>> This way we don't wipe out the original rules we just add our own
>> chain.
>>
>>
>
> Current default (virt) iptables configuration is kept in the engine's DB.
> So...
>
> 1. We can update the DB value to include Gluster configuration.
> or
> 2. Have an additional DB value for virt+Gluster.

3. gluster only, not virt.

>
> So (1) will work, but when gluster is not relevant we have redundant ports.
> But my only reservation from (2) is that it's not scalable for additional
> services.
>
> Either way, I agree with Alon that melding the configurations at runtime
> is unexpected, and shouldn't be used.
>
> After some additional thought... we can have (1), and make all gluster
> entries commented out (let's say with #GLSTR) by default, so it'll be
> enabled only when gluster is being used. This will allow introducing future
> configurations as well without the need for additional DB values.

why not use the chains approach, and have a chain per service?

>
> Sample snip of DB value:
> ==================================================================================
> :INPUT ACCEPT [0:0]
> :FORWARD ACCEPT [0:0]
> :OUTPUT ACCEPT [0:0]
> -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
> -A INPUT -p icmp -j ACCEPT
> -A INPUT -i lo -j ACCEPT
> # vdsm
> -A INPUT -p tcp --dport 54321 -j ACCEPT
> # libvirt tls
> -A INPUT -p tcp --dport 16514 -j ACCEPT
> # SSH
> -A INPUT -p tcp --dport 22 -j ACCEPT
> # guest consoles
> -A INPUT -p tcp -m multiport --dports 5634:6166 -j ACCEPT
> # gluster p1
> #GLSTR-A INPUT -p tcp --dport p1 -j ACCEPT
> # gluster p2-p6
> #GLSTR-A INPUT -p tcp --dport p2:p8 -j ACCEPT
> # Reject any other input traffic
> -A INPUT -j REJECT --reject-with icmp-host-prohibited
> -A FORWARD -m physdev ! --physdev-is-bridged -j REJECT --reject-with icmp-host-prohibited
> COMMIT
> ==================================================================================
>
> So as you can see, this is something we can handle in run-time. Also it's easy
> to test, so predictable, and finally- human beings can look and understand what
> it means.
>
>
>>> The need to provide different rules to different hosts by software
>>> installed on destination?
>>>
>>> Standard host needs iptables X.
>>> Gluster host needs iptables X+Y.
>>> XXX host needs iptables X+Z.
>>> Maintainer of Gluster knows what Y is.
>>> Maintainer of XXX knows what Z is.
>>>
>>> If we merge all to one entry product comes with default X.
>>> User override X to A.
>>> New version of product comes with default Y.
>>> Upgrade options:
>>> 1. System continues to use A.
>>> 2. Some AI to upgrade and create A'.
>>> 3. Revert to Y, dropping user's customization.
>>>
>>> Or we can maintain one large table with complete configuration and
>>> conditionals.
>>>
>>> Alon.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>




From dfediuck at redhat.com  Mon Sep  3 06:09:04 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Mon, 3 Sep 2012 02:09:04 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <5043C6DC.1000705@redhat.com>
Message-ID: <1305562668.4309974.1346652544333.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Doron Fediuck" <dfediuck at redhat.com>
> Cc: "David Ja?a" <djasa at redhat.com>, engine-devel at ovirt.org
> Sent: Sunday, September 2, 2012 11:51:40 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> On 09/02/2012 05:21 PM, Doron Fediuck wrote:
> > ----- Original Message -----
> >> From: "Andrew Cathrow" <acathrow at redhat.com>
> >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >> Cc: "David Ja?a" <djasa at redhat.com>, engine-devel at ovirt.org
> >> Sent: Friday, August 31, 2012 3:12:34 PM
> >> Subject: Re: [Engine-devel] Gluster IPTable configuration
> >>
> >>
> >>
> >> ----- Original Message -----
> >>> From: "Alon Bar-Lev" <alonbl at redhat.com>
> >>> To: "David Ja?a" <djasa at redhat.com>
> >>> Cc: engine-devel at ovirt.org
> >>> Sent: Friday, August 31, 2012 5:09:47 AM
> >>> Subject: Re: [Engine-devel] Gluster IPTable configuration
> >>>
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "David Ja?a" <djasa at redhat.com>
> >>>> To: engine-devel at ovirt.org
> >>>> Sent: Friday, August 31, 2012 11:57:11 AM
> >>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
> >>>>
> >>>> Alon Bar-Lev p??e v ?t 30. 08. 2012 v 14:40 -0400:
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "Andrew Cathrow" <acathrow at redhat.com>
> >>>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>,
> >>>>>> engine-devel at ovirt.org,
> >>>>>> "Selvasundaram" <sesubram at redhat.com>
> >>>>>> Sent: Thursday, August 30, 2012 9:37:59 PM
> >>>>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ----- Original Message -----
> >>>>>>> From: "Alon Bar-Lev" <alonbl at redhat.com>
> >>>>>>> To: "Selvasundaram" <sesubram at redhat.com>
> >>>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>,
> >>>>>>> engine-devel at ovirt.org
> >>>>>>> Sent: Thursday, August 30, 2012 2:35:16 PM
> >>>>>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> ----- Original Message -----
> >>>>>>>> From: "Selvasundaram" <sesubram at redhat.com>
> >>>>>>>> To: engine-devel at ovirt.org
> >>>>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
> >>>>>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
> >>>>>>>> Subject: [Engine-devel] Gluster IPTable configuration
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> I want to add gluster specific IPTable configuration in
> >>>>>>>> addition
> >>>>>>>> to
> >>>>>>>> the ovirt IPTable configuration (if it is gluster node).
> >>>>>>>>
> >>>>>>>> There are two approaches,
> >>>>>>>> 1. Having one more gluster specific IP table config in db
> >>>>>>>> and
> >>>>>>>> merge
> >>>>>>>> with ovirt IPTable config (merging NOT appending)
> >>>>>>>> [I have the patch engine: Gluster specific firewall
> >>>>>>>> configurations
> >>>>>>>> #7244]
> >>>>>>>> 2. Having two different IP Table config (ovirt and
> >>>>>>>> ovirt+gluster)
> >>>>>>>> and
> >>>>>>>> use either one.
> >>>>>>>>
> >>>>>>>> Please provide your suggestions or improvements on this.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Hello all,
> >>>>>>>
> >>>>>>> The mentioned patch[1], adds hard coded gluster code into
> >>>>>>> the
> >>>>>>> bootstrap code, manipulate the firewall configuration to be
> >>>>>>> gluster
> >>>>>>> specific. It hardcoded search for "reject", insert before
> >>>>>>> some
> >>>>>>> other
> >>>>>>> rules.
> >>>>>>>
> >>>>>>> I believe this hardcode approach is obsolete now that we
> >>>>>>> have
> >>>>>>> proper
> >>>>>>> tools for templates.
> >>>>>>>
> >>>>>>> A more robust solution would be defining generic profiles,
> >>>>>>> each
> >>>>>>> profile as a template, each template can refer to different
> >>>>>>> profiles, and assign profile to a node.
> >>>>>>>
> >>>>>>> This way the implementation is not gluster [or any]
> >>>>>>> specific
> >>>>>>> and
> >>>>>>> can
> >>>>>>> be reused for more setups, code is cleaner.
> >>>>>>
> >>>>>>
> >>>>>> or create custom chains ?
> >>>>>
> >>>>> Can you please elaborate what is custom chains?
> >>>>> Thanks!
> >>>>
> >>>> iptables -N my_new_chain
> >>>> iptables -A my_new_chain <rule_1>
> >>>> iptables -A my_new_chain ...
> >>>> iptables -A my_new_chain <rule_n>
> >>>>
> >>>> # if this <rule> is matched, packet goes through rules in
> >>>> my_new_chain
> >>>> iptables -A INPUT <rule> -j my_new_chain
> >>>>
> >>>
> >>> Hello,
> >>>
> >>> How does this solve the original issue?
> >>
> >> It makes it easier for customers who are adding their own IPTables
> >> configuration - when we do rhev-h plugins it'll make things
> >> easier.
> >> This way we don't wipe out the original rules we just add our own
> >> chain.
> >>
> >>
> >
> > Current default (virt) iptables configuration is kept in the
> > engine's DB.
> > So...
> >
> > 1. We can update the DB value to include Gluster configuration.
> > or
> > 2. Have an additional DB value for virt+Gluster.
> 
> 3. gluster only, not virt.
> 
> >
> > So (1) will work, but when gluster is not relevant we have
> > redundant ports.
> > But my only reservation from (2) is that it's not scalable for
> > additional
> > services.
> >
> > Either way, I agree with Alon that melding the configurations at
> > runtime
> > is unexpected, and shouldn't be used.
> >
> > After some additional thought... we can have (1), and make all
> > gluster
> > entries commented out (let's say with #GLSTR) by default, so it'll
> > be
> > enabled only when gluster is being used. This will allow
> > introducing future
> > configurations as well without the need for additional DB values.
> 
> why not use the chains approach, and have a chain per service?
> 

Since you wish to avoid collisions.
So for gluster only, have a VIRT prefix as well.

> >
> > Sample snip of DB value:
> > ==================================================================================
> > :INPUT ACCEPT [0:0]
> > :FORWARD ACCEPT [0:0]
> > :OUTPUT ACCEPT [0:0]
> > -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
> > -A INPUT -p icmp -j ACCEPT
> > -A INPUT -i lo -j ACCEPT
> > # vdsm
> > -A INPUT -p tcp --dport 54321 -j ACCEPT
> > # libvirt tls
> > -A INPUT -p tcp --dport 16514 -j ACCEPT
> > # SSH
> > -A INPUT -p tcp --dport 22 -j ACCEPT
> > # guest consoles
> > -A INPUT -p tcp -m multiport --dports 5634:6166 -j ACCEPT
> > # gluster p1
> > #GLSTR-A INPUT -p tcp --dport p1 -j ACCEPT
> > # gluster p2-p6
> > #GLSTR-A INPUT -p tcp --dport p2:p8 -j ACCEPT
> > # Reject any other input traffic
> > -A INPUT -j REJECT --reject-with icmp-host-prohibited
> > -A FORWARD -m physdev ! --physdev-is-bridged -j REJECT
> > --reject-with icmp-host-prohibited
> > COMMIT
> > ==================================================================================
> >
> > So as you can see, this is something we can handle in run-time.
> > Also it's easy
> > to test, so predictable, and finally- human beings can look and
> > understand what
> > it means.
> >
> >
> >>> The need to provide different rules to different hosts by
> >>> software
> >>> installed on destination?
> >>>
> >>> Standard host needs iptables X.
> >>> Gluster host needs iptables X+Y.
> >>> XXX host needs iptables X+Z.
> >>> Maintainer of Gluster knows what Y is.
> >>> Maintainer of XXX knows what Z is.
> >>>
> >>> If we merge all to one entry product comes with default X.
> >>> User override X to A.
> >>> New version of product comes with default Y.
> >>> Upgrade options:
> >>> 1. System continues to use A.
> >>> 2. Some AI to upgrade and create A'.
> >>> 3. Revert to Y, dropping user's customization.
> >>>
> >>> Or we can maintain one large table with complete configuration
> >>> and
> >>> conditionals.
> >>>
> >>> Alon.
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> >
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From alonbl at redhat.com  Mon Sep  3 09:51:44 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 3 Sep 2012 05:51:44 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <1305562668.4309974.1346652544333.JavaMail.root@redhat.com>
Message-ID: <776170247.4369474.1346665904154.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Doron Fediuck" <dfediuck at redhat.com>
> To: "Itamar Heim" <iheim at redhat.com>
> Cc: "David Ja?a" <djasa at redhat.com>, engine-devel at ovirt.org
> Sent: Monday, September 3, 2012 9:09:04 AM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> > 
> > why not use the chains approach, and have a chain per service?
> > 
> 
> Since you wish to avoid collisions.
> So for gluster only, have a VIRT prefix as well.

If an implementation may separate between the WHAT and the HOW, it may be easier to be maintained.

---
WHAT

Merge several iptables rules into one node iptables.

HOW

Use templates to build string, send string as a file in remote.
---

As you can see the HOW (which is the actual implementation) knows nothing about iptables. So it is simple and can be reused. The whole logic of WHAT is put into the metadata, where humans may customized without touching the code, even when iptables get messy and complex.

An example of WHAT and HOW that are not separated is the authentication/authorization (Kerberos/LDAP) implementation, where both WHAT and HOW are inter-connected, the cost of adding a new environment in this case is huge.

Doron suggested to use comments or some signature within the iptables configuration, this is what templates are all about, however, instead of re-inventing the wheel, a standard text based templates engine can be used.

The template (the WHAT) may use custom chains, regular chains, it is not important as implementation (the HOW) is not looking into the content.

Alon.


From djasa at redhat.com  Mon Sep  3 11:42:56 2012
From: djasa at redhat.com (David =?UTF-8?Q?Ja=C5=A1a?=)
Date: Mon, 03 Sep 2012 13:42:56 +0200
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <776170247.4369474.1346665904154.JavaMail.root@redhat.com>
References: <776170247.4369474.1346665904154.JavaMail.root@redhat.com>
Message-ID: <1346672576.15842.126.camel@dhcp-29-7.brq.redhat.com>

what about using netcf for the configuration similarly as libvirt does?
http://libvirt.org/formatnwfilter.html

IMHO it should solve the problem temporarily before firewalld matures.

David

PS: please keep me out of CC, I'm more than happy when I watch these
discussions via list


Alon Bar-Lev p??e v Po 03. 09. 2012 v 05:51 -0400:
> 
> ----- Original Message -----
> > From: "Doron Fediuck" <dfediuck at redhat.com>
> > To: "Itamar Heim" <iheim at redhat.com>
> > Cc: "David Ja?a" <djasa at redhat.com>, engine-devel at ovirt.org
> > Sent: Monday, September 3, 2012 9:09:04 AM
> > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > 
> > > 
> > > why not use the chains approach, and have a chain per service?
> > > 
> > 
> > Since you wish to avoid collisions.
> > So for gluster only, have a VIRT prefix as well.
> 
> If an implementation may separate between the WHAT and the HOW, it may be easier to be maintained.
> 
> ---
> WHAT
> 
> Merge several iptables rules into one node iptables.
> 
> HOW
> 
> Use templates to build string, send string as a file in remote.
> ---
> 
> As you can see the HOW (which is the actual implementation) knows nothing about iptables. So it is simple and can be reused. The whole logic of WHAT is put into the metadata, where humans may customized without touching the code, even when iptables get messy and complex.
> 
> An example of WHAT and HOW that are not separated is the authentication/authorization (Kerberos/LDAP) implementation, where both WHAT and HOW are inter-connected, the cost of adding a new environment in this case is huge.
> 
> Doron suggested to use comments or some signature within the iptables configuration, this is what templates are all about, however, instead of re-inventing the wheel, a standard text based templates engine can be used.
> 
> The template (the WHAT) may use custom chains, regular chains, it is not important as implementation (the HOW) is not looking into the content.
> 
> Alon.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

-- 

David Ja?a, RHCE

SPICE QE based in Brno
GPG Key:     22C33E24 
Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24





From sanjal at redhat.com  Mon Sep  3 12:42:17 2012
From: sanjal at redhat.com (Shireesh Anjal)
Date: Mon, 03 Sep 2012 18:12:17 +0530
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <278376844.4015237.1346351716576.JavaMail.root@redhat.com>
References: <278376844.4015237.1346351716576.JavaMail.root@redhat.com>
Message-ID: <5044A5A9.1090008@redhat.com>

On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
>
> ----- Original Message -----
>> From: "Selvasundaram" <sesubram at redhat.com>
>> To: engine-devel at ovirt.org
>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
>> Sent: Thursday, August 30, 2012 4:30:16 PM
>> Subject: [Engine-devel] Gluster IPTable configuration
>>
>>
>> Hi,
>>
>> I want to add gluster specific IPTable configuration in addition to
>> the ovirt IPTable configuration (if it is gluster node).
>>
>> There are two approaches,
>> 1. Having one more gluster specific IP table config in db and merge
>> with ovirt IPTable config (merging NOT appending)
>> [I have the patch engine: Gluster specific firewall configurations
>> #7244]
>> 2. Having two different IP Table config (ovirt and ovirt+gluster) and
>> use either one.
>>
>> Please provide your suggestions or improvements on this.
>>
> Hello all,
>
> The mentioned patch[1], adds hard coded gluster code into the bootstrap code, manipulate the firewall configuration to be gluster specific. It hardcoded search for "reject", insert before some other rules.
>
> I believe this hardcode approach is obsolete now that we have proper tools for templates.
>
> A more robust solution would be defining generic profiles, each profile as a template, each template can refer to different profiles, and assign profile to a node.
>
> This way the implementation is not gluster [or any] specific and can be reused for more setups, code is cleaner.
>
> Example:
>
> BASIC.PRE
>      :INPUT ACCEPT [0:0]
>      :FORWARD ACCEPT [0:0]
>      :OUTPUT ACCEPT [0:0]
> BASIC.IN
>      accept ...
>      accept ...
> BASIC.POST
>      reject ...
>      reject ...
>
> BASIC
>      ${BASIC.PRE}
>      ${BASIC.IN}
>      ${BASIC.POST}
>
> GLUSTER
>      ${BASIC.PRE}
>      ${BASIC.IN}
>      accept ...
>      ${BASIC.POST}
>      reject ...

I like the separation of PRE/IN/POST rules here. However I think it is 
better to keep the service specific rules in separate configurations. 
Currently, whole iptables rules script is kept in the vdc option 
"IPTablesConfig". How about changing this as follows?

- Split the current config into three: IPTablesConfig.PRE, 
IPTablesConfig.VIRT and IPTablesConfig.POST
- Let services like Gluster add their own vdc options e.g. 
IPTablesConfig.GLUSTER
- When assembling the full script in VdsInstaller,
   - Take IPTablesConfig.PRE
   - Append it with IPTablesConfig.<service> for every service to be 
enabled on the host/cluster
   - Append it with IPTablesConfig.POST

Thoughts?

>
> Regards,
> Alon Bar-Lev
>
> [1] http://gerrit.ovirt.org/#/c/7244/
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From alonbl at redhat.com  Mon Sep  3 12:52:37 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 3 Sep 2012 08:52:37 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <5044A5A9.1090008@redhat.com>
Message-ID: <992596012.4387628.1346676757949.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Shireesh Anjal" <sanjal at redhat.com>
> To: engine-devel at ovirt.org
> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram" <sesubram at redhat.com>
> Sent: Monday, September 3, 2012 3:42:17 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
> >
> > ----- Original Message -----
> >> From: "Selvasundaram" <sesubram at redhat.com>
> >> To: engine-devel at ovirt.org
> >> Cc: "Shireesh Anjal" <sanjal at redhat.com>
> >> Sent: Thursday, August 30, 2012 4:30:16 PM
> >> Subject: [Engine-devel] Gluster IPTable configuration
> >>
> >>
> >> Hi,
> >>
> >> I want to add gluster specific IPTable configuration in addition
> >> to
> >> the ovirt IPTable configuration (if it is gluster node).
> >>
> >> There are two approaches,
> >> 1. Having one more gluster specific IP table config in db and
> >> merge
> >> with ovirt IPTable config (merging NOT appending)
> >> [I have the patch engine: Gluster specific firewall configurations
> >> #7244]
> >> 2. Having two different IP Table config (ovirt and ovirt+gluster)
> >> and
> >> use either one.
> >>
> >> Please provide your suggestions or improvements on this.
> >>
> > Hello all,
> >
> > The mentioned patch[1], adds hard coded gluster code into the
> > bootstrap code, manipulate the firewall configuration to be
> > gluster specific. It hardcoded search for "reject", insert before
> > some other rules.
> >
> > I believe this hardcode approach is obsolete now that we have
> > proper tools for templates.
> >
> > A more robust solution would be defining generic profiles, each
> > profile as a template, each template can refer to different
> > profiles, and assign profile to a node.
> >
> > This way the implementation is not gluster [or any] specific and
> > can be reused for more setups, code is cleaner.
> >
> > Example:
> >
> > BASIC.PRE
> >      :INPUT ACCEPT [0:0]
> >      :FORWARD ACCEPT [0:0]
> >      :OUTPUT ACCEPT [0:0]
> > BASIC.IN
> >      accept ...
> >      accept ...
> > BASIC.POST
> >      reject ...
> >      reject ...
> >
> > BASIC
> >      ${BASIC.PRE}
> >      ${BASIC.IN}
> >      ${BASIC.POST}
> >
> > GLUSTER
> >      ${BASIC.PRE}
> >      ${BASIC.IN}
> >      accept ...
> >      ${BASIC.POST}
> >      reject ...
> 
> I like the separation of PRE/IN/POST rules here. However I think it
> is
> better to keep the service specific rules in separate configurations.
> Currently, whole iptables rules script is kept in the vdc option
> "IPTablesConfig". How about changing this as follows?
> 
> - Split the current config into three: IPTablesConfig.PRE,
> IPTablesConfig.VIRT and IPTablesConfig.POST
> - Let services like Gluster add their own vdc options e.g.
> IPTablesConfig.GLUSTER
> - When assembling the full script in VdsInstaller,
>    - Take IPTablesConfig.PRE
>    - Append it with IPTablesConfig.<service> for every service to be
> enabled on the host/cluster
>    - Append it with IPTablesConfig.POST
> 
> Thoughts?

This is a simple approach that will work for current implementation and configuration.

However, it will effect all nodes, with or without gluster.

I think that while we at it we should consider how to effect only appropriate subset of nodes.

Alon.


From sanjal at redhat.com  Mon Sep  3 13:00:14 2012
From: sanjal at redhat.com (Shireesh Anjal)
Date: Mon, 03 Sep 2012 18:30:14 +0530
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <992596012.4387628.1346676757949.JavaMail.root@redhat.com>
References: <992596012.4387628.1346676757949.JavaMail.root@redhat.com>
Message-ID: <5044A9DE.3090501@redhat.com>

On Monday 03 September 2012 06:22 PM, Alon Bar-Lev wrote:
>
> ----- Original Message -----
>> From: "Shireesh Anjal" <sanjal at redhat.com>
>> To: engine-devel at ovirt.org
>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram" <sesubram at redhat.com>
>> Sent: Monday, September 3, 2012 3:42:17 PM
>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>
>> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
>>> ----- Original Message -----
>>>> From: "Selvasundaram" <sesubram at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
>>>> Subject: [Engine-devel] Gluster IPTable configuration
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I want to add gluster specific IPTable configuration in addition
>>>> to
>>>> the ovirt IPTable configuration (if it is gluster node).
>>>>
>>>> There are two approaches,
>>>> 1. Having one more gluster specific IP table config in db and
>>>> merge
>>>> with ovirt IPTable config (merging NOT appending)
>>>> [I have the patch engine: Gluster specific firewall configurations
>>>> #7244]
>>>> 2. Having two different IP Table config (ovirt and ovirt+gluster)
>>>> and
>>>> use either one.
>>>>
>>>> Please provide your suggestions or improvements on this.
>>>>
>>> Hello all,
>>>
>>> The mentioned patch[1], adds hard coded gluster code into the
>>> bootstrap code, manipulate the firewall configuration to be
>>> gluster specific. It hardcoded search for "reject", insert before
>>> some other rules.
>>>
>>> I believe this hardcode approach is obsolete now that we have
>>> proper tools for templates.
>>>
>>> A more robust solution would be defining generic profiles, each
>>> profile as a template, each template can refer to different
>>> profiles, and assign profile to a node.
>>>
>>> This way the implementation is not gluster [or any] specific and
>>> can be reused for more setups, code is cleaner.
>>>
>>> Example:
>>>
>>> BASIC.PRE
>>>       :INPUT ACCEPT [0:0]
>>>       :FORWARD ACCEPT [0:0]
>>>       :OUTPUT ACCEPT [0:0]
>>> BASIC.IN
>>>       accept ...
>>>       accept ...
>>> BASIC.POST
>>>       reject ...
>>>       reject ...
>>>
>>> BASIC
>>>       ${BASIC.PRE}
>>>       ${BASIC.IN}
>>>       ${BASIC.POST}
>>>
>>> GLUSTER
>>>       ${BASIC.PRE}
>>>       ${BASIC.IN}
>>>       accept ...
>>>       ${BASIC.POST}
>>>       reject ...
>> I like the separation of PRE/IN/POST rules here. However I think it
>> is
>> better to keep the service specific rules in separate configurations.
>> Currently, whole iptables rules script is kept in the vdc option
>> "IPTablesConfig". How about changing this as follows?
>>
>> - Split the current config into three: IPTablesConfig.PRE,
>> IPTablesConfig.VIRT and IPTablesConfig.POST
>> - Let services like Gluster add their own vdc options e.g.
>> IPTablesConfig.GLUSTER
>> - When assembling the full script in VdsInstaller,
>>     - Take IPTablesConfig.PRE
>>     - Append it with IPTablesConfig.<service> for every service to be
>> enabled on the host/cluster
>>     - Append it with IPTablesConfig.POST
>>
>> Thoughts?
> This is a simple approach that will work for current implementation and configuration.
>
> However, it will effect all nodes, with or without gluster.

I don't get the concern here. Could you please elaborate?

>
> I think that while we at it we should consider how to effect only appropriate subset of nodes.
>
> Alon.



From alonbl at redhat.com  Mon Sep  3 13:11:57 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 3 Sep 2012 09:11:57 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <5044A9DE.3090501@redhat.com>
Message-ID: <34986317.4390291.1346677917107.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Shireesh Anjal" <sanjal at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "Selvasundaram" <sesubram at redhat.com>, engine-devel at ovirt.org
> Sent: Monday, September 3, 2012 4:00:14 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> On Monday 03 September 2012 06:22 PM, Alon Bar-Lev wrote:
> >
> > ----- Original Message -----
> >> From: "Shireesh Anjal" <sanjal at redhat.com>
> >> To: engine-devel at ovirt.org
> >> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram"
> >> <sesubram at redhat.com>
> >> Sent: Monday, September 3, 2012 3:42:17 PM
> >> Subject: Re: [Engine-devel] Gluster IPTable configuration
> >>
> >> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
> >>> ----- Original Message -----
> >>>> From: "Selvasundaram" <sesubram at redhat.com>
> >>>> To: engine-devel at ovirt.org
> >>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
> >>>> Sent: Thursday, August 30, 2012 4:30:16 PM
> >>>> Subject: [Engine-devel] Gluster IPTable configuration
> >>>>
> >>>>
> >>>> Hi,
> >>>>
> >>>> I want to add gluster specific IPTable configuration in addition
> >>>> to
> >>>> the ovirt IPTable configuration (if it is gluster node).
> >>>>
> >>>> There are two approaches,
> >>>> 1. Having one more gluster specific IP table config in db and
> >>>> merge
> >>>> with ovirt IPTable config (merging NOT appending)
> >>>> [I have the patch engine: Gluster specific firewall
> >>>> configurations
> >>>> #7244]
> >>>> 2. Having two different IP Table config (ovirt and
> >>>> ovirt+gluster)
> >>>> and
> >>>> use either one.
> >>>>
> >>>> Please provide your suggestions or improvements on this.
> >>>>
> >>> Hello all,
> >>>
> >>> The mentioned patch[1], adds hard coded gluster code into the
> >>> bootstrap code, manipulate the firewall configuration to be
> >>> gluster specific. It hardcoded search for "reject", insert before
> >>> some other rules.
> >>>
> >>> I believe this hardcode approach is obsolete now that we have
> >>> proper tools for templates.
> >>>
> >>> A more robust solution would be defining generic profiles, each
> >>> profile as a template, each template can refer to different
> >>> profiles, and assign profile to a node.
> >>>
> >>> This way the implementation is not gluster [or any] specific and
> >>> can be reused for more setups, code is cleaner.
> >>>
> >>> Example:
> >>>
> >>> BASIC.PRE
> >>>       :INPUT ACCEPT [0:0]
> >>>       :FORWARD ACCEPT [0:0]
> >>>       :OUTPUT ACCEPT [0:0]
> >>> BASIC.IN
> >>>       accept ...
> >>>       accept ...
> >>> BASIC.POST
> >>>       reject ...
> >>>       reject ...
> >>>
> >>> BASIC
> >>>       ${BASIC.PRE}
> >>>       ${BASIC.IN}
> >>>       ${BASIC.POST}
> >>>
> >>> GLUSTER
> >>>       ${BASIC.PRE}
> >>>       ${BASIC.IN}
> >>>       accept ...
> >>>       ${BASIC.POST}
> >>>       reject ...
> >> I like the separation of PRE/IN/POST rules here. However I think
> >> it
> >> is
> >> better to keep the service specific rules in separate
> >> configurations.
> >> Currently, whole iptables rules script is kept in the vdc option
> >> "IPTablesConfig". How about changing this as follows?
> >>
> >> - Split the current config into three: IPTablesConfig.PRE,
> >> IPTablesConfig.VIRT and IPTablesConfig.POST
> >> - Let services like Gluster add their own vdc options e.g.
> >> IPTablesConfig.GLUSTER
> >> - When assembling the full script in VdsInstaller,
> >>     - Take IPTablesConfig.PRE
> >>     - Append it with IPTablesConfig.<service> for every service to
> >>     be
> >> enabled on the host/cluster
> >>     - Append it with IPTablesConfig.POST
> >>
> >> Thoughts?
> > This is a simple approach that will work for current implementation
> > and configuration.
> >
> > However, it will effect all nodes, with or without gluster.
> 
> I don't get the concern here. Could you please elaborate?

If we have 500 nodes, out of them 200 gluster, why do I need to distribute gluster specific rules to all 500?

Alon.


From sanjal at redhat.com  Mon Sep  3 13:21:58 2012
From: sanjal at redhat.com (Shireesh Anjal)
Date: Mon, 03 Sep 2012 18:51:58 +0530
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <34986317.4390291.1346677917107.JavaMail.root@redhat.com>
References: <34986317.4390291.1346677917107.JavaMail.root@redhat.com>
Message-ID: <5044AEF6.7090309@redhat.com>

On Monday 03 September 2012 06:41 PM, Alon Bar-Lev wrote:
>
> ----- Original Message -----
>> From: "Shireesh Anjal" <sanjal at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>> Cc: "Selvasundaram" <sesubram at redhat.com>, engine-devel at ovirt.org
>> Sent: Monday, September 3, 2012 4:00:14 PM
>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>
>> On Monday 03 September 2012 06:22 PM, Alon Bar-Lev wrote:
>>> ----- Original Message -----
>>>> From: "Shireesh Anjal" <sanjal at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram"
>>>> <sesubram at redhat.com>
>>>> Sent: Monday, September 3, 2012 3:42:17 PM
>>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>>>
>>>> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
>>>>> ----- Original Message -----
>>>>>> From: "Selvasundaram" <sesubram at redhat.com>
>>>>>> To: engine-devel at ovirt.org
>>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
>>>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
>>>>>> Subject: [Engine-devel] Gluster IPTable configuration
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I want to add gluster specific IPTable configuration in addition
>>>>>> to
>>>>>> the ovirt IPTable configuration (if it is gluster node).
>>>>>>
>>>>>> There are two approaches,
>>>>>> 1. Having one more gluster specific IP table config in db and
>>>>>> merge
>>>>>> with ovirt IPTable config (merging NOT appending)
>>>>>> [I have the patch engine: Gluster specific firewall
>>>>>> configurations
>>>>>> #7244]
>>>>>> 2. Having two different IP Table config (ovirt and
>>>>>> ovirt+gluster)
>>>>>> and
>>>>>> use either one.
>>>>>>
>>>>>> Please provide your suggestions or improvements on this.
>>>>>>
>>>>> Hello all,
>>>>>
>>>>> The mentioned patch[1], adds hard coded gluster code into the
>>>>> bootstrap code, manipulate the firewall configuration to be
>>>>> gluster specific. It hardcoded search for "reject", insert before
>>>>> some other rules.
>>>>>
>>>>> I believe this hardcode approach is obsolete now that we have
>>>>> proper tools for templates.
>>>>>
>>>>> A more robust solution would be defining generic profiles, each
>>>>> profile as a template, each template can refer to different
>>>>> profiles, and assign profile to a node.
>>>>>
>>>>> This way the implementation is not gluster [or any] specific and
>>>>> can be reused for more setups, code is cleaner.
>>>>>
>>>>> Example:
>>>>>
>>>>> BASIC.PRE
>>>>>        :INPUT ACCEPT [0:0]
>>>>>        :FORWARD ACCEPT [0:0]
>>>>>        :OUTPUT ACCEPT [0:0]
>>>>> BASIC.IN
>>>>>        accept ...
>>>>>        accept ...
>>>>> BASIC.POST
>>>>>        reject ...
>>>>>        reject ...
>>>>>
>>>>> BASIC
>>>>>        ${BASIC.PRE}
>>>>>        ${BASIC.IN}
>>>>>        ${BASIC.POST}
>>>>>
>>>>> GLUSTER
>>>>>        ${BASIC.PRE}
>>>>>        ${BASIC.IN}
>>>>>        accept ...
>>>>>        ${BASIC.POST}
>>>>>        reject ...
>>>> I like the separation of PRE/IN/POST rules here. However I think
>>>> it
>>>> is
>>>> better to keep the service specific rules in separate
>>>> configurations.
>>>> Currently, whole iptables rules script is kept in the vdc option
>>>> "IPTablesConfig". How about changing this as follows?
>>>>
>>>> - Split the current config into three: IPTablesConfig.PRE,
>>>> IPTablesConfig.VIRT and IPTablesConfig.POST
>>>> - Let services like Gluster add their own vdc options e.g.
>>>> IPTablesConfig.GLUSTER
>>>> - When assembling the full script in VdsInstaller,
>>>>      - Take IPTablesConfig.PRE
>>>>      - Append it with IPTablesConfig.<service> for every service to
>>>>      be
>>>> enabled on the host/cluster
>>>>      - Append it with IPTablesConfig.POST
>>>>
>>>> Thoughts?
>>> This is a simple approach that will work for current implementation
>>> and configuration.
>>>
>>> However, it will effect all nodes, with or without gluster.
>> I don't get the concern here. Could you please elaborate?
> If we have 500 nodes, out of them 200 gluster, why do I need to distribute gluster specific rules to all 500?

When I say "Append it with IPTablesConfig.<service> for every service to be enabled on the host/cluster", I mean every service that "needs to be enabled" on the host. In today's scenario, where virt and gluster are the only two services, it will look something like:

  - If cluster supports virt, append IPTablesConfig.VIRT
  - If cluster supports gluster, append IPTablesConfig.GLUSTER

So it will be driven by the cluster in which the host is being added. And gluster rules will be sent to only the hosts of clusters that have gluster enabled.

>
> Alon.



From alonbl at redhat.com  Mon Sep  3 13:50:52 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 3 Sep 2012 09:50:52 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <5044AEF6.7090309@redhat.com>
Message-ID: <1943701531.4395068.1346680252910.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Shireesh Anjal" <sanjal at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "Selvasundaram" <sesubram at redhat.com>, engine-devel at ovirt.org
> Sent: Monday, September 3, 2012 4:21:58 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> On Monday 03 September 2012 06:41 PM, Alon Bar-Lev wrote:
> >
> > ----- Original Message -----
> >> From: "Shireesh Anjal" <sanjal at redhat.com>
> >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >> Cc: "Selvasundaram" <sesubram at redhat.com>, engine-devel at ovirt.org
> >> Sent: Monday, September 3, 2012 4:00:14 PM
> >> Subject: Re: [Engine-devel] Gluster IPTable configuration
> >>
> >> On Monday 03 September 2012 06:22 PM, Alon Bar-Lev wrote:
> >>> ----- Original Message -----
> >>>> From: "Shireesh Anjal" <sanjal at redhat.com>
> >>>> To: engine-devel at ovirt.org
> >>>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram"
> >>>> <sesubram at redhat.com>
> >>>> Sent: Monday, September 3, 2012 3:42:17 PM
> >>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
> >>>>
> >>>> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
> >>>>> ----- Original Message -----
> >>>>>> From: "Selvasundaram" <sesubram at redhat.com>
> >>>>>> To: engine-devel at ovirt.org
> >>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
> >>>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
> >>>>>> Subject: [Engine-devel] Gluster IPTable configuration
> >>>>>>
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I want to add gluster specific IPTable configuration in
> >>>>>> addition
> >>>>>> to
> >>>>>> the ovirt IPTable configuration (if it is gluster node).
> >>>>>>
> >>>>>> There are two approaches,
> >>>>>> 1. Having one more gluster specific IP table config in db and
> >>>>>> merge
> >>>>>> with ovirt IPTable config (merging NOT appending)
> >>>>>> [I have the patch engine: Gluster specific firewall
> >>>>>> configurations
> >>>>>> #7244]
> >>>>>> 2. Having two different IP Table config (ovirt and
> >>>>>> ovirt+gluster)
> >>>>>> and
> >>>>>> use either one.
> >>>>>>
> >>>>>> Please provide your suggestions or improvements on this.
> >>>>>>
> >>>>> Hello all,
> >>>>>
> >>>>> The mentioned patch[1], adds hard coded gluster code into the
> >>>>> bootstrap code, manipulate the firewall configuration to be
> >>>>> gluster specific. It hardcoded search for "reject", insert
> >>>>> before
> >>>>> some other rules.
> >>>>>
> >>>>> I believe this hardcode approach is obsolete now that we have
> >>>>> proper tools for templates.
> >>>>>
> >>>>> A more robust solution would be defining generic profiles, each
> >>>>> profile as a template, each template can refer to different
> >>>>> profiles, and assign profile to a node.
> >>>>>
> >>>>> This way the implementation is not gluster [or any] specific
> >>>>> and
> >>>>> can be reused for more setups, code is cleaner.
> >>>>>
> >>>>> Example:
> >>>>>
> >>>>> BASIC.PRE
> >>>>>        :INPUT ACCEPT [0:0]
> >>>>>        :FORWARD ACCEPT [0:0]
> >>>>>        :OUTPUT ACCEPT [0:0]
> >>>>> BASIC.IN
> >>>>>        accept ...
> >>>>>        accept ...
> >>>>> BASIC.POST
> >>>>>        reject ...
> >>>>>        reject ...
> >>>>>
> >>>>> BASIC
> >>>>>        ${BASIC.PRE}
> >>>>>        ${BASIC.IN}
> >>>>>        ${BASIC.POST}
> >>>>>
> >>>>> GLUSTER
> >>>>>        ${BASIC.PRE}
> >>>>>        ${BASIC.IN}
> >>>>>        accept ...
> >>>>>        ${BASIC.POST}
> >>>>>        reject ...
> >>>> I like the separation of PRE/IN/POST rules here. However I think
> >>>> it
> >>>> is
> >>>> better to keep the service specific rules in separate
> >>>> configurations.
> >>>> Currently, whole iptables rules script is kept in the vdc option
> >>>> "IPTablesConfig". How about changing this as follows?
> >>>>
> >>>> - Split the current config into three: IPTablesConfig.PRE,
> >>>> IPTablesConfig.VIRT and IPTablesConfig.POST
> >>>> - Let services like Gluster add their own vdc options e.g.
> >>>> IPTablesConfig.GLUSTER
> >>>> - When assembling the full script in VdsInstaller,
> >>>>      - Take IPTablesConfig.PRE
> >>>>      - Append it with IPTablesConfig.<service> for every service
> >>>>      to
> >>>>      be
> >>>> enabled on the host/cluster
> >>>>      - Append it with IPTablesConfig.POST
> >>>>
> >>>> Thoughts?
> >>> This is a simple approach that will work for current
> >>> implementation
> >>> and configuration.
> >>>
> >>> However, it will effect all nodes, with or without gluster.
> >> I don't get the concern here. Could you please elaborate?
> > If we have 500 nodes, out of them 200 gluster, why do I need to
> > distribute gluster specific rules to all 500?
> 
> When I say "Append it with IPTablesConfig.<service> for every service
> to be enabled on the host/cluster", I mean every service that "needs
> to be enabled" on the host. In today's scenario, where virt and
> gluster are the only two services, it will look something like:
> 
>   - If cluster supports virt, append IPTablesConfig.VIRT
>   - If cluster supports gluster, append IPTablesConfig.GLUSTER
> 
> So it will be driven by the cluster in which the host is being added.
> And gluster rules will be sent to only the hosts of clusters that
> have gluster enabled.

OK... this is why I prefer templates. Much more simple and generic. No need to have custom application logic within application.

Alon


From sanjal at redhat.com  Mon Sep  3 14:34:41 2012
From: sanjal at redhat.com (Shireesh Anjal)
Date: Mon, 03 Sep 2012 20:04:41 +0530
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <1943701531.4395068.1346680252910.JavaMail.root@redhat.com>
References: <1943701531.4395068.1346680252910.JavaMail.root@redhat.com>
Message-ID: <5044C001.9080106@redhat.com>

On Monday 03 September 2012 07:20 PM, Alon Bar-Lev wrote:
>
> ----- Original Message -----
>> From: "Shireesh Anjal" <sanjal at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>> Cc: "Selvasundaram" <sesubram at redhat.com>, engine-devel at ovirt.org
>> Sent: Monday, September 3, 2012 4:21:58 PM
>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>
>> On Monday 03 September 2012 06:41 PM, Alon Bar-Lev wrote:
>>> ----- Original Message -----
>>>> From: "Shireesh Anjal" <sanjal at redhat.com>
>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>> Cc: "Selvasundaram" <sesubram at redhat.com>, engine-devel at ovirt.org
>>>> Sent: Monday, September 3, 2012 4:00:14 PM
>>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>>>
>>>> On Monday 03 September 2012 06:22 PM, Alon Bar-Lev wrote:
>>>>> ----- Original Message -----
>>>>>> From: "Shireesh Anjal" <sanjal at redhat.com>
>>>>>> To: engine-devel at ovirt.org
>>>>>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram"
>>>>>> <sesubram at redhat.com>
>>>>>> Sent: Monday, September 3, 2012 3:42:17 PM
>>>>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
>>>>>>
>>>>>> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Selvasundaram" <sesubram at redhat.com>
>>>>>>>> To: engine-devel at ovirt.org
>>>>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
>>>>>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
>>>>>>>> Subject: [Engine-devel] Gluster IPTable configuration
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I want to add gluster specific IPTable configuration in
>>>>>>>> addition
>>>>>>>> to
>>>>>>>> the ovirt IPTable configuration (if it is gluster node).
>>>>>>>>
>>>>>>>> There are two approaches,
>>>>>>>> 1. Having one more gluster specific IP table config in db and
>>>>>>>> merge
>>>>>>>> with ovirt IPTable config (merging NOT appending)
>>>>>>>> [I have the patch engine: Gluster specific firewall
>>>>>>>> configurations
>>>>>>>> #7244]
>>>>>>>> 2. Having two different IP Table config (ovirt and
>>>>>>>> ovirt+gluster)
>>>>>>>> and
>>>>>>>> use either one.
>>>>>>>>
>>>>>>>> Please provide your suggestions or improvements on this.
>>>>>>>>
>>>>>>> Hello all,
>>>>>>>
>>>>>>> The mentioned patch[1], adds hard coded gluster code into the
>>>>>>> bootstrap code, manipulate the firewall configuration to be
>>>>>>> gluster specific. It hardcoded search for "reject", insert
>>>>>>> before
>>>>>>> some other rules.
>>>>>>>
>>>>>>> I believe this hardcode approach is obsolete now that we have
>>>>>>> proper tools for templates.
>>>>>>>
>>>>>>> A more robust solution would be defining generic profiles, each
>>>>>>> profile as a template, each template can refer to different
>>>>>>> profiles, and assign profile to a node.
>>>>>>>
>>>>>>> This way the implementation is not gluster [or any] specific
>>>>>>> and
>>>>>>> can be reused for more setups, code is cleaner.
>>>>>>>
>>>>>>> Example:
>>>>>>>
>>>>>>> BASIC.PRE
>>>>>>>         :INPUT ACCEPT [0:0]
>>>>>>>         :FORWARD ACCEPT [0:0]
>>>>>>>         :OUTPUT ACCEPT [0:0]
>>>>>>> BASIC.IN
>>>>>>>         accept ...
>>>>>>>         accept ...
>>>>>>> BASIC.POST
>>>>>>>         reject ...
>>>>>>>         reject ...
>>>>>>>
>>>>>>> BASIC
>>>>>>>         ${BASIC.PRE}
>>>>>>>         ${BASIC.IN}
>>>>>>>         ${BASIC.POST}
>>>>>>>
>>>>>>> GLUSTER
>>>>>>>         ${BASIC.PRE}
>>>>>>>         ${BASIC.IN}
>>>>>>>         accept ...
>>>>>>>         ${BASIC.POST}
>>>>>>>         reject ...
>>>>>> I like the separation of PRE/IN/POST rules here. However I think
>>>>>> it
>>>>>> is
>>>>>> better to keep the service specific rules in separate
>>>>>> configurations.
>>>>>> Currently, whole iptables rules script is kept in the vdc option
>>>>>> "IPTablesConfig". How about changing this as follows?
>>>>>>
>>>>>> - Split the current config into three: IPTablesConfig.PRE,
>>>>>> IPTablesConfig.VIRT and IPTablesConfig.POST
>>>>>> - Let services like Gluster add their own vdc options e.g.
>>>>>> IPTablesConfig.GLUSTER
>>>>>> - When assembling the full script in VdsInstaller,
>>>>>>       - Take IPTablesConfig.PRE
>>>>>>       - Append it with IPTablesConfig.<service> for every service
>>>>>>       to
>>>>>>       be
>>>>>> enabled on the host/cluster
>>>>>>       - Append it with IPTablesConfig.POST
>>>>>>
>>>>>> Thoughts?
>>>>> This is a simple approach that will work for current
>>>>> implementation
>>>>> and configuration.
>>>>>
>>>>> However, it will effect all nodes, with or without gluster.
>>>> I don't get the concern here. Could you please elaborate?
>>> If we have 500 nodes, out of them 200 gluster, why do I need to
>>> distribute gluster specific rules to all 500?
>> When I say "Append it with IPTablesConfig.<service> for every service
>> to be enabled on the host/cluster", I mean every service that "needs
>> to be enabled" on the host. In today's scenario, where virt and
>> gluster are the only two services, it will look something like:
>>
>>    - If cluster supports virt, append IPTablesConfig.VIRT
>>    - If cluster supports gluster, append IPTablesConfig.GLUSTER
>>
>> So it will be driven by the cluster in which the host is being added.
>> And gluster rules will be sent to only the hosts of clusters that
>> have gluster enabled.
> OK... this is why I prefer templates. Much more simple and generic. No need to have custom application logic within application.

I don't see what is so complex about the above approach, but do agree 
that it is not completely generic. A new service will require a new if 
condition in the VdsInstaller.

In order to understand the templates approach better, can you please 
point me to some existing code in oVirt that uses such an approach? I 
see an example of the template text above, but I'm not sure how it can 
be used from the code.

>
> Alon
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From acathrow at redhat.com  Mon Sep  3 20:05:20 2012
From: acathrow at redhat.com (Andrew Cathrow)
Date: Mon, 3 Sep 2012 16:05:20 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <1943701531.4395068.1346680252910.JavaMail.root@redhat.com>
Message-ID: <379791011.4807536.1346702720899.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Alon Bar-Lev" <alonbl at redhat.com>
> To: "Shireesh Anjal" <sanjal at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Monday, September 3, 2012 9:50:52 AM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> 
> 
> ----- Original Message -----
> > From: "Shireesh Anjal" <sanjal at redhat.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: "Selvasundaram" <sesubram at redhat.com>, engine-devel at ovirt.org
> > Sent: Monday, September 3, 2012 4:21:58 PM
> > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > 
> > On Monday 03 September 2012 06:41 PM, Alon Bar-Lev wrote:
> > >
> > > ----- Original Message -----
> > >> From: "Shireesh Anjal" <sanjal at redhat.com>
> > >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> > >> Cc: "Selvasundaram" <sesubram at redhat.com>,
> > >> engine-devel at ovirt.org
> > >> Sent: Monday, September 3, 2012 4:00:14 PM
> > >> Subject: Re: [Engine-devel] Gluster IPTable configuration
> > >>
> > >> On Monday 03 September 2012 06:22 PM, Alon Bar-Lev wrote:
> > >>> ----- Original Message -----
> > >>>> From: "Shireesh Anjal" <sanjal at redhat.com>
> > >>>> To: engine-devel at ovirt.org
> > >>>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram"
> > >>>> <sesubram at redhat.com>
> > >>>> Sent: Monday, September 3, 2012 3:42:17 PM
> > >>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
> > >>>>
> > >>>> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
> > >>>>> ----- Original Message -----
> > >>>>>> From: "Selvasundaram" <sesubram at redhat.com>
> > >>>>>> To: engine-devel at ovirt.org
> > >>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
> > >>>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
> > >>>>>> Subject: [Engine-devel] Gluster IPTable configuration
> > >>>>>>
> > >>>>>>
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> I want to add gluster specific IPTable configuration in
> > >>>>>> addition
> > >>>>>> to
> > >>>>>> the ovirt IPTable configuration (if it is gluster node).
> > >>>>>>
> > >>>>>> There are two approaches,
> > >>>>>> 1. Having one more gluster specific IP table config in db
> > >>>>>> and
> > >>>>>> merge
> > >>>>>> with ovirt IPTable config (merging NOT appending)
> > >>>>>> [I have the patch engine: Gluster specific firewall
> > >>>>>> configurations
> > >>>>>> #7244]
> > >>>>>> 2. Having two different IP Table config (ovirt and
> > >>>>>> ovirt+gluster)
> > >>>>>> and
> > >>>>>> use either one.
> > >>>>>>
> > >>>>>> Please provide your suggestions or improvements on this.
> > >>>>>>
> > >>>>> Hello all,
> > >>>>>
> > >>>>> The mentioned patch[1], adds hard coded gluster code into the
> > >>>>> bootstrap code, manipulate the firewall configuration to be
> > >>>>> gluster specific. It hardcoded search for "reject", insert
> > >>>>> before
> > >>>>> some other rules.
> > >>>>>
> > >>>>> I believe this hardcode approach is obsolete now that we have
> > >>>>> proper tools for templates.
> > >>>>>
> > >>>>> A more robust solution would be defining generic profiles,
> > >>>>> each
> > >>>>> profile as a template, each template can refer to different
> > >>>>> profiles, and assign profile to a node.
> > >>>>>
> > >>>>> This way the implementation is not gluster [or any] specific
> > >>>>> and
> > >>>>> can be reused for more setups, code is cleaner.
> > >>>>>
> > >>>>> Example:
> > >>>>>
> > >>>>> BASIC.PRE
> > >>>>>        :INPUT ACCEPT [0:0]
> > >>>>>        :FORWARD ACCEPT [0:0]
> > >>>>>        :OUTPUT ACCEPT [0:0]
> > >>>>> BASIC.IN
> > >>>>>        accept ...
> > >>>>>        accept ...
> > >>>>> BASIC.POST
> > >>>>>        reject ...
> > >>>>>        reject ...
> > >>>>>
> > >>>>> BASIC
> > >>>>>        ${BASIC.PRE}
> > >>>>>        ${BASIC.IN}
> > >>>>>        ${BASIC.POST}
> > >>>>>
> > >>>>> GLUSTER
> > >>>>>        ${BASIC.PRE}
> > >>>>>        ${BASIC.IN}
> > >>>>>        accept ...
> > >>>>>        ${BASIC.POST}
> > >>>>>        reject ...
> > >>>> I like the separation of PRE/IN/POST rules here. However I
> > >>>> think
> > >>>> it
> > >>>> is
> > >>>> better to keep the service specific rules in separate
> > >>>> configurations.
> > >>>> Currently, whole iptables rules script is kept in the vdc
> > >>>> option
> > >>>> "IPTablesConfig". How about changing this as follows?
> > >>>>
> > >>>> - Split the current config into three: IPTablesConfig.PRE,
> > >>>> IPTablesConfig.VIRT and IPTablesConfig.POST
> > >>>> - Let services like Gluster add their own vdc options e.g.
> > >>>> IPTablesConfig.GLUSTER
> > >>>> - When assembling the full script in VdsInstaller,
> > >>>>      - Take IPTablesConfig.PRE
> > >>>>      - Append it with IPTablesConfig.<service> for every
> > >>>>      service
> > >>>>      to
> > >>>>      be
> > >>>> enabled on the host/cluster
> > >>>>      - Append it with IPTablesConfig.POST
> > >>>>
> > >>>> Thoughts?
> > >>> This is a simple approach that will work for current
> > >>> implementation
> > >>> and configuration.
> > >>>
> > >>> However, it will effect all nodes, with or without gluster.
> > >> I don't get the concern here. Could you please elaborate?
> > > If we have 500 nodes, out of them 200 gluster, why do I need to
> > > distribute gluster specific rules to all 500?
> > 
> > When I say "Append it with IPTablesConfig.<service> for every
> > service
> > to be enabled on the host/cluster", I mean every service that
> > "needs
> > to be enabled" on the host. In today's scenario, where virt and
> > gluster are the only two services, it will look something like:
> > 
> >   - If cluster supports virt, append IPTablesConfig.VIRT
> >   - If cluster supports gluster, append IPTablesConfig.GLUSTER
> > 
> > So it will be driven by the cluster in which the host is being
> > added.
> > And gluster rules will be sent to only the hosts of clusters that
> > have gluster enabled.
> 
> OK... this is why I prefer templates. Much more simple and generic.
> No need to have custom application logic within application.


Do we want to use a template that's so tightly tied into iptables - does it lock is into an implementation in the config files? Wouldn't we rather have the rules in the config file (eg. tcp.port 80, udp.port xyz, etc) rather than be so closely tied to an implementation.
How will things look when we move to firewalld[1],  or some other system, rather than creating and managing our own iptables rules?

What happened to the custom chain discussion, did I miss that or was it silently dropped?


[1] https://fedoraproject.org/wiki/FirewallD



> 
> Alon
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From alonbl at redhat.com  Mon Sep  3 20:50:00 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 3 Sep 2012 16:50:00 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <379791011.4807536.1346702720899.JavaMail.root@redhat.com>
Message-ID: <1615906426.4421296.1346705400892.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Andrew Cathrow" <acathrow at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: engine-devel at ovirt.org, "Shireesh Anjal" <sanjal at redhat.com>
> Sent: Monday, September 3, 2012 11:05:20 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> 
> 
> ----- Original Message -----
> > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > To: "Shireesh Anjal" <sanjal at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Monday, September 3, 2012 9:50:52 AM
> > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Shireesh Anjal" <sanjal at redhat.com>
> > > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > > Cc: "Selvasundaram" <sesubram at redhat.com>, engine-devel at ovirt.org
> > > Sent: Monday, September 3, 2012 4:21:58 PM
> > > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > 
> > > On Monday 03 September 2012 06:41 PM, Alon Bar-Lev wrote:
> > > >
> > > > ----- Original Message -----
> > > >> From: "Shireesh Anjal" <sanjal at redhat.com>
> > > >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> > > >> Cc: "Selvasundaram" <sesubram at redhat.com>,
> > > >> engine-devel at ovirt.org
> > > >> Sent: Monday, September 3, 2012 4:00:14 PM
> > > >> Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > >>
> > > >> On Monday 03 September 2012 06:22 PM, Alon Bar-Lev wrote:
> > > >>> ----- Original Message -----
> > > >>>> From: "Shireesh Anjal" <sanjal at redhat.com>
> > > >>>> To: engine-devel at ovirt.org
> > > >>>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram"
> > > >>>> <sesubram at redhat.com>
> > > >>>> Sent: Monday, September 3, 2012 3:42:17 PM
> > > >>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > >>>>
> > > >>>> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
> > > >>>>> ----- Original Message -----
> > > >>>>>> From: "Selvasundaram" <sesubram at redhat.com>
> > > >>>>>> To: engine-devel at ovirt.org
> > > >>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
> > > >>>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
> > > >>>>>> Subject: [Engine-devel] Gluster IPTable configuration
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Hi,
> > > >>>>>>
> > > >>>>>> I want to add gluster specific IPTable configuration in
> > > >>>>>> addition
> > > >>>>>> to
> > > >>>>>> the ovirt IPTable configuration (if it is gluster node).
> > > >>>>>>
> > > >>>>>> There are two approaches,
> > > >>>>>> 1. Having one more gluster specific IP table config in db
> > > >>>>>> and
> > > >>>>>> merge
> > > >>>>>> with ovirt IPTable config (merging NOT appending)
> > > >>>>>> [I have the patch engine: Gluster specific firewall
> > > >>>>>> configurations
> > > >>>>>> #7244]
> > > >>>>>> 2. Having two different IP Table config (ovirt and
> > > >>>>>> ovirt+gluster)
> > > >>>>>> and
> > > >>>>>> use either one.
> > > >>>>>>
> > > >>>>>> Please provide your suggestions or improvements on this.
> > > >>>>>>
> > > >>>>> Hello all,
> > > >>>>>
> > > >>>>> The mentioned patch[1], adds hard coded gluster code into
> > > >>>>> the
> > > >>>>> bootstrap code, manipulate the firewall configuration to be
> > > >>>>> gluster specific. It hardcoded search for "reject", insert
> > > >>>>> before
> > > >>>>> some other rules.
> > > >>>>>
> > > >>>>> I believe this hardcode approach is obsolete now that we
> > > >>>>> have
> > > >>>>> proper tools for templates.
> > > >>>>>
> > > >>>>> A more robust solution would be defining generic profiles,
> > > >>>>> each
> > > >>>>> profile as a template, each template can refer to different
> > > >>>>> profiles, and assign profile to a node.
> > > >>>>>
> > > >>>>> This way the implementation is not gluster [or any]
> > > >>>>> specific
> > > >>>>> and
> > > >>>>> can be reused for more setups, code is cleaner.
> > > >>>>>
> > > >>>>> Example:
> > > >>>>>
> > > >>>>> BASIC.PRE
> > > >>>>>        :INPUT ACCEPT [0:0]
> > > >>>>>        :FORWARD ACCEPT [0:0]
> > > >>>>>        :OUTPUT ACCEPT [0:0]
> > > >>>>> BASIC.IN
> > > >>>>>        accept ...
> > > >>>>>        accept ...
> > > >>>>> BASIC.POST
> > > >>>>>        reject ...
> > > >>>>>        reject ...
> > > >>>>>
> > > >>>>> BASIC
> > > >>>>>        ${BASIC.PRE}
> > > >>>>>        ${BASIC.IN}
> > > >>>>>        ${BASIC.POST}
> > > >>>>>
> > > >>>>> GLUSTER
> > > >>>>>        ${BASIC.PRE}
> > > >>>>>        ${BASIC.IN}
> > > >>>>>        accept ...
> > > >>>>>        ${BASIC.POST}
> > > >>>>>        reject ...
> > > >>>> I like the separation of PRE/IN/POST rules here. However I
> > > >>>> think
> > > >>>> it
> > > >>>> is
> > > >>>> better to keep the service specific rules in separate
> > > >>>> configurations.
> > > >>>> Currently, whole iptables rules script is kept in the vdc
> > > >>>> option
> > > >>>> "IPTablesConfig". How about changing this as follows?
> > > >>>>
> > > >>>> - Split the current config into three: IPTablesConfig.PRE,
> > > >>>> IPTablesConfig.VIRT and IPTablesConfig.POST
> > > >>>> - Let services like Gluster add their own vdc options e.g.
> > > >>>> IPTablesConfig.GLUSTER
> > > >>>> - When assembling the full script in VdsInstaller,
> > > >>>>      - Take IPTablesConfig.PRE
> > > >>>>      - Append it with IPTablesConfig.<service> for every
> > > >>>>      service
> > > >>>>      to
> > > >>>>      be
> > > >>>> enabled on the host/cluster
> > > >>>>      - Append it with IPTablesConfig.POST
> > > >>>>
> > > >>>> Thoughts?
> > > >>> This is a simple approach that will work for current
> > > >>> implementation
> > > >>> and configuration.
> > > >>>
> > > >>> However, it will effect all nodes, with or without gluster.
> > > >> I don't get the concern here. Could you please elaborate?
> > > > If we have 500 nodes, out of them 200 gluster, why do I need to
> > > > distribute gluster specific rules to all 500?
> > > 
> > > When I say "Append it with IPTablesConfig.<service> for every
> > > service
> > > to be enabled on the host/cluster", I mean every service that
> > > "needs
> > > to be enabled" on the host. In today's scenario, where virt and
> > > gluster are the only two services, it will look something like:
> > > 
> > >   - If cluster supports virt, append IPTablesConfig.VIRT
> > >   - If cluster supports gluster, append IPTablesConfig.GLUSTER
> > > 
> > > So it will be driven by the cluster in which the host is being
> > > added.
> > > And gluster rules will be sent to only the hosts of clusters that
> > > have gluster enabled.
> > 
> > OK... this is why I prefer templates. Much more simple and generic.
> > No need to have custom application logic within application.
> 
> 
> Do we want to use a template that's so tightly tied into iptables -
> does it lock is into an implementation in the config files? Wouldn't
> we rather have the rules in the config file (eg. tcp.port 80,
> udp.port xyz, etc) rather than be so closely tied to an
> implementation.

You refer to a model similar to libvirt[1] referred by David - abstract anything.

This model will not work well without templates, as one probably need to construct the abstract rule base similar manner by merging of hunks of configuration.

You can take the result in whatever language and either convert it to another language (iptables) or to dbus calls (firewalld).

My main questions are:
a. Do we want to invest NOW in generic language to define firewall rules?
b. Is such decision is required at this point?

There are so many projects out there that are specialized in this particular domain. I, personally, find firehol[2] very useful for my configurations. But I don't know many that are intermediate between generic rules into an interface other than iptables (eg. firewalld).

So we can have dependency of firewalld at node side, and feed it with whatever it accepts, however, having firewalld as dependency will make the cost of porting to other distributions much higher.

> How will things look when we move to firewalld[1],  or some other
> system, rather than creating and managing our own iptables rules?
> 
> What happened to the custom chain discussion, did I miss that or was
> it silently dropped?

As I did not fully understand your suggestion asked, but not answered.

How do you use custom chains in order to meet the requirement of providing different firewall configurations to different hosts?

Regards,
Alon

[1] http://libvirt.org/formatnwfilter.html
[2] http://firehol.sourceforge.net/


From acathrow at redhat.com  Mon Sep  3 20:57:57 2012
From: acathrow at redhat.com (Andrew Cathrow)
Date: Mon, 3 Sep 2012 16:57:57 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <1615906426.4421296.1346705400892.JavaMail.root@redhat.com>
Message-ID: <1782554206.4811021.1346705877535.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Alon Bar-Lev" <alonbl at redhat.com>
> To: "Andrew Cathrow" <acathrow at redhat.com>
> Cc: engine-devel at ovirt.org, "Shireesh Anjal" <sanjal at redhat.com>
> Sent: Monday, September 3, 2012 4:50:00 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> 
> 
> ----- Original Message -----
> > From: "Andrew Cathrow" <acathrow at redhat.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: engine-devel at ovirt.org, "Shireesh Anjal" <sanjal at redhat.com>
> > Sent: Monday, September 3, 2012 11:05:20 PM
> > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > > To: "Shireesh Anjal" <sanjal at redhat.com>
> > > Cc: engine-devel at ovirt.org
> > > Sent: Monday, September 3, 2012 9:50:52 AM
> > > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > 
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Shireesh Anjal" <sanjal at redhat.com>
> > > > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > > > Cc: "Selvasundaram" <sesubram at redhat.com>,
> > > > engine-devel at ovirt.org
> > > > Sent: Monday, September 3, 2012 4:21:58 PM
> > > > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > > 
> > > > On Monday 03 September 2012 06:41 PM, Alon Bar-Lev wrote:
> > > > >
> > > > > ----- Original Message -----
> > > > >> From: "Shireesh Anjal" <sanjal at redhat.com>
> > > > >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> > > > >> Cc: "Selvasundaram" <sesubram at redhat.com>,
> > > > >> engine-devel at ovirt.org
> > > > >> Sent: Monday, September 3, 2012 4:00:14 PM
> > > > >> Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > > >>
> > > > >> On Monday 03 September 2012 06:22 PM, Alon Bar-Lev wrote:
> > > > >>> ----- Original Message -----
> > > > >>>> From: "Shireesh Anjal" <sanjal at redhat.com>
> > > > >>>> To: engine-devel at ovirt.org
> > > > >>>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Selvasundaram"
> > > > >>>> <sesubram at redhat.com>
> > > > >>>> Sent: Monday, September 3, 2012 3:42:17 PM
> > > > >>>> Subject: Re: [Engine-devel] Gluster IPTable configuration
> > > > >>>>
> > > > >>>> On Friday 31 August 2012 12:05 AM, Alon Bar-Lev wrote:
> > > > >>>>> ----- Original Message -----
> > > > >>>>>> From: "Selvasundaram" <sesubram at redhat.com>
> > > > >>>>>> To: engine-devel at ovirt.org
> > > > >>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>
> > > > >>>>>> Sent: Thursday, August 30, 2012 4:30:16 PM
> > > > >>>>>> Subject: [Engine-devel] Gluster IPTable configuration
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> Hi,
> > > > >>>>>>
> > > > >>>>>> I want to add gluster specific IPTable configuration in
> > > > >>>>>> addition
> > > > >>>>>> to
> > > > >>>>>> the ovirt IPTable configuration (if it is gluster node).
> > > > >>>>>>
> > > > >>>>>> There are two approaches,
> > > > >>>>>> 1. Having one more gluster specific IP table config in
> > > > >>>>>> db
> > > > >>>>>> and
> > > > >>>>>> merge
> > > > >>>>>> with ovirt IPTable config (merging NOT appending)
> > > > >>>>>> [I have the patch engine: Gluster specific firewall
> > > > >>>>>> configurations
> > > > >>>>>> #7244]
> > > > >>>>>> 2. Having two different IP Table config (ovirt and
> > > > >>>>>> ovirt+gluster)
> > > > >>>>>> and
> > > > >>>>>> use either one.
> > > > >>>>>>
> > > > >>>>>> Please provide your suggestions or improvements on this.
> > > > >>>>>>
> > > > >>>>> Hello all,
> > > > >>>>>
> > > > >>>>> The mentioned patch[1], adds hard coded gluster code into
> > > > >>>>> the
> > > > >>>>> bootstrap code, manipulate the firewall configuration to
> > > > >>>>> be
> > > > >>>>> gluster specific. It hardcoded search for "reject",
> > > > >>>>> insert
> > > > >>>>> before
> > > > >>>>> some other rules.
> > > > >>>>>
> > > > >>>>> I believe this hardcode approach is obsolete now that we
> > > > >>>>> have
> > > > >>>>> proper tools for templates.
> > > > >>>>>
> > > > >>>>> A more robust solution would be defining generic
> > > > >>>>> profiles,
> > > > >>>>> each
> > > > >>>>> profile as a template, each template can refer to
> > > > >>>>> different
> > > > >>>>> profiles, and assign profile to a node.
> > > > >>>>>
> > > > >>>>> This way the implementation is not gluster [or any]
> > > > >>>>> specific
> > > > >>>>> and
> > > > >>>>> can be reused for more setups, code is cleaner.
> > > > >>>>>
> > > > >>>>> Example:
> > > > >>>>>
> > > > >>>>> BASIC.PRE
> > > > >>>>>        :INPUT ACCEPT [0:0]
> > > > >>>>>        :FORWARD ACCEPT [0:0]
> > > > >>>>>        :OUTPUT ACCEPT [0:0]
> > > > >>>>> BASIC.IN
> > > > >>>>>        accept ...
> > > > >>>>>        accept ...
> > > > >>>>> BASIC.POST
> > > > >>>>>        reject ...
> > > > >>>>>        reject ...
> > > > >>>>>
> > > > >>>>> BASIC
> > > > >>>>>        ${BASIC.PRE}
> > > > >>>>>        ${BASIC.IN}
> > > > >>>>>        ${BASIC.POST}
> > > > >>>>>
> > > > >>>>> GLUSTER
> > > > >>>>>        ${BASIC.PRE}
> > > > >>>>>        ${BASIC.IN}
> > > > >>>>>        accept ...
> > > > >>>>>        ${BASIC.POST}
> > > > >>>>>        reject ...
> > > > >>>> I like the separation of PRE/IN/POST rules here. However I
> > > > >>>> think
> > > > >>>> it
> > > > >>>> is
> > > > >>>> better to keep the service specific rules in separate
> > > > >>>> configurations.
> > > > >>>> Currently, whole iptables rules script is kept in the vdc
> > > > >>>> option
> > > > >>>> "IPTablesConfig". How about changing this as follows?
> > > > >>>>
> > > > >>>> - Split the current config into three: IPTablesConfig.PRE,
> > > > >>>> IPTablesConfig.VIRT and IPTablesConfig.POST
> > > > >>>> - Let services like Gluster add their own vdc options e.g.
> > > > >>>> IPTablesConfig.GLUSTER
> > > > >>>> - When assembling the full script in VdsInstaller,
> > > > >>>>      - Take IPTablesConfig.PRE
> > > > >>>>      - Append it with IPTablesConfig.<service> for every
> > > > >>>>      service
> > > > >>>>      to
> > > > >>>>      be
> > > > >>>> enabled on the host/cluster
> > > > >>>>      - Append it with IPTablesConfig.POST
> > > > >>>>
> > > > >>>> Thoughts?
> > > > >>> This is a simple approach that will work for current
> > > > >>> implementation
> > > > >>> and configuration.
> > > > >>>
> > > > >>> However, it will effect all nodes, with or without gluster.
> > > > >> I don't get the concern here. Could you please elaborate?
> > > > > If we have 500 nodes, out of them 200 gluster, why do I need
> > > > > to
> > > > > distribute gluster specific rules to all 500?
> > > > 
> > > > When I say "Append it with IPTablesConfig.<service> for every
> > > > service
> > > > to be enabled on the host/cluster", I mean every service that
> > > > "needs
> > > > to be enabled" on the host. In today's scenario, where virt and
> > > > gluster are the only two services, it will look something like:
> > > > 
> > > >   - If cluster supports virt, append IPTablesConfig.VIRT
> > > >   - If cluster supports gluster, append IPTablesConfig.GLUSTER
> > > > 
> > > > So it will be driven by the cluster in which the host is being
> > > > added.
> > > > And gluster rules will be sent to only the hosts of clusters
> > > > that
> > > > have gluster enabled.
> > > 
> > > OK... this is why I prefer templates. Much more simple and
> > > generic.
> > > No need to have custom application logic within application.
> > 
> > 
> > Do we want to use a template that's so tightly tied into iptables -
> > does it lock is into an implementation in the config files?
> > Wouldn't
> > we rather have the rules in the config file (eg. tcp.port 80,
> > udp.port xyz, etc) rather than be so closely tied to an
> > implementation.
> 
> You refer to a model similar to libvirt[1] referred by David -
> abstract anything.
> 
> This model will not work well without templates, as one probably need
> to construct the abstract rule base similar manner by merging of
> hunks of configuration.
> 
> You can take the result in whatever language and either convert it to
> another language (iptables) or to dbus calls (firewalld).
> 
> My main questions are:
> a. Do we want to invest NOW in generic language to define firewall
> rules?
> b. Is such decision is required at this point?
> 
> There are so many projects out there that are specialized in this
> particular domain. I, personally, find firehol[2] very useful for my
> configurations. But I don't know many that are intermediate between
> generic rules into an interface other than iptables (eg. firewalld).
> 
> So we can have dependency of firewalld at node side, and feed it with
> whatever it accepts, however, having firewalld as dependency will
> make the cost of porting to other distributions much higher.
> 
> > How will things look when we move to firewalld[1],  or some other
> > system, rather than creating and managing our own iptables rules?
> > 
> > What happened to the custom chain discussion, did I miss that or
> > was
> > it silently dropped?
> 
> As I did not fully understand your suggestion asked, but not
> answered.
> 
> How do you use custom chains in order to meet the requirement of
> providing different firewall configurations to different hosts?

Right now we just overwrite the existing iptables configuration with our own, so if a user already added a rule to their host - eg. for a monitoring agent the we stomp over it.
Adding our rules as a custom chain means that we don't need to overwrite existing rules we can extend them.

The advantage of using something like firewalld to add rules is that we don't have to worry about parsing existing rules, etc we can let that service handle it.

We need to handle firewall rules for ovirt-node plugins  (eg. open port XYZ for monitoring agent ABC) - I'm not sure if this has been discussed yet, adding Mike ....



> 
> Regards,
> Alon
> 
> [1] http://libvirt.org/formatnwfilter.html
> [2] http://firehol.sourceforge.net/
> 


From alonbl at redhat.com  Mon Sep  3 21:09:34 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 3 Sep 2012 17:09:34 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <1782554206.4811021.1346705877535.JavaMail.root@redhat.com>
Message-ID: <811147153.4421845.1346706574096.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Andrew Cathrow" <acathrow at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: engine-devel at ovirt.org, "Shireesh Anjal" <sanjal at redhat.com>, "Mike Burns" <mburns at redhat.com>
> Sent: Monday, September 3, 2012 11:57:57 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration

<snip>

> Right now we just overwrite the existing iptables configuration with
> our own, so if a user already added a rule to their host - eg. for a
> monitoring agent the we stomp over it.
> Adding our rules as a custom chain means that we don't need to

Here I lost you... :)

I thought ovirt-engine is the master and ovirt-hypervisor is a slave.

This derives that all management activities of slave is done by master...

There should be no setting at slave that master is not aware of.

This also enables you to duplicate hipervisor, recover configuration or push mass configuration change.

In your above case, this rule for monitoring agent may be added at master repository and pushed to slaves belongs to specific group, just like the gluster case.

The template mechanism is what enable you to create a custom configuration per environment.

Using push and not re-integrate derives much simpler and deterministic implementation.

But maybe I did not understand some of the fundamental concepts of the architecture.

Regards,
Alon.


From acathrow at redhat.com  Mon Sep  3 21:21:11 2012
From: acathrow at redhat.com (Andrew Cathrow)
Date: Mon, 3 Sep 2012 17:21:11 -0400 (EDT)
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <811147153.4421845.1346706574096.JavaMail.root@redhat.com>
Message-ID: <2000454800.4812856.1346707271710.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Alon Bar-Lev" <alonbl at redhat.com>
> To: "Andrew Cathrow" <acathrow at redhat.com>
> Cc: engine-devel at ovirt.org, "Shireesh Anjal" <sanjal at redhat.com>, "Mike Burns" <mburns at redhat.com>
> Sent: Monday, September 3, 2012 5:09:34 PM
> Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> 
> 
> ----- Original Message -----
> > From: "Andrew Cathrow" <acathrow at redhat.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: engine-devel at ovirt.org, "Shireesh Anjal" <sanjal at redhat.com>,
> > "Mike Burns" <mburns at redhat.com>
> > Sent: Monday, September 3, 2012 11:57:57 PM
> > Subject: Re: [Engine-devel] Gluster IPTable configuration
> 
> <snip>
> 
> > Right now we just overwrite the existing iptables configuration
> > with
> > our own, so if a user already added a rule to their host - eg. for
> > a
> > monitoring agent the we stomp over it.
> > Adding our rules as a custom chain means that we don't need to
> 
> Here I lost you... :)
> 
> I thought ovirt-engine is the master and ovirt-hypervisor is a slave.
> 
> This derives that all management activities of slave is done by
> master...
> 

Let's say I use nagios for my host monitoring.
I setup a rhel/fedora/*EL host using my standard corporate and include port 5667/5666 for nagios.
ovirt engine connects to it and blocks nagios.

While it would be great to have all firewall rules (and other settings) managed from ovirt-engine we are a long way away from that.
Adding rules rather than overwriting iptables config would allow us not to stomp on the user's existing settings.


> There should be no setting at slave that master is not aware of.
> 
> This also enables you to duplicate hipervisor, recover configuration
> or push mass configuration change.

> 
> In your above case, this rule for monitoring agent may be added at
> master repository and pushed to slaves belongs to specific group,
> just like the gluster case.

yes, but in the 24 months between now and when we get to implement that feature ......

> 
> The template mechanism is what enable you to create a custom
> configuration per environment.

> 
> Using push and not re-integrate derives much simpler and
> deterministic implementation.
> 
> But maybe I did not understand some of the fundamental concepts of
> the architecture.
> 
> Regards,
> Alon.
> 


From djasa at redhat.com  Tue Sep  4 09:06:08 2012
From: djasa at redhat.com (David =?UTF-8?Q?Ja=C5=A1a?=)
Date: Tue, 04 Sep 2012 11:06:08 +0200
Subject: [Engine-devel] Gluster IPTable configuration
In-Reply-To: <2000454800.4812856.1346707271710.JavaMail.root@redhat.com>
References: <2000454800.4812856.1346707271710.JavaMail.root@redhat.com>
Message-ID: <1346749568.15842.143.camel@dhcp-29-7.brq.redhat.com>

Andrew Cathrow p??e v Po 03. 09. 2012 v 17:21 -0400:
> 
> ----- Original Message -----
> > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > To: "Andrew Cathrow" <acathrow at redhat.com>
> > Cc: engine-devel at ovirt.org, "Shireesh Anjal" <sanjal at redhat.com>, "Mike Burns" <mburns at redhat.com>
> > Sent: Monday, September 3, 2012 5:09:34 PM
> > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Andrew Cathrow" <acathrow at redhat.com>
> > > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > > Cc: engine-devel at ovirt.org, "Shireesh Anjal" <sanjal at redhat.com>,
> > > "Mike Burns" <mburns at redhat.com>
> > > Sent: Monday, September 3, 2012 11:57:57 PM
> > > Subject: Re: [Engine-devel] Gluster IPTable configuration
> > 
> > <snip>
> > 
> > > Right now we just overwrite the existing iptables configuration
> > > with
> > > our own, so if a user already added a rule to their host - eg. for
> > > a
> > > monitoring agent the we stomp over it.
> > > Adding our rules as a custom chain means that we don't need to
> > 
> > Here I lost you... :)
> > 
> > I thought ovirt-engine is the master and ovirt-hypervisor is a slave.
> > 
> > This derives that all management activities of slave is done by
> > master...
> > 
> 
> Let's say I use nagios for my host monitoring.
> I setup a rhel/fedora/*EL host using my standard corporate and include port 5667/5666 for nagios.
> ovirt engine connects to it and blocks nagios.
> 
> While it would be great to have all firewall rules (and other settings) managed from ovirt-engine we are a long way away from that.
> Adding rules rather than overwriting iptables config would allow us not to stomp on the user's existing settings.

This sounds like you want precise feature set of firewalld, just faster.

David

> 
> 
> > There should be no setting at slave that master is not aware of.
> > 
> > This also enables you to duplicate hipervisor, recover configuration
> > or push mass configuration change.
> 
> > 
> > In your above case, this rule for monitoring agent may be added at
> > master repository and pushed to slaves belongs to specific group,
> > just like the gluster case.
> 
> yes, but in the 24 months between now and when we get to implement that feature ......
> 
> > 
> > The template mechanism is what enable you to create a custom
> > configuration per environment.
> 
> > 
> > Using push and not re-integrate derives much simpler and
> > deterministic implementation.
> > 
> > But maybe I did not understand some of the fundamental concepts of
> > the architecture.
> > 
> > Regards,
> > Alon.
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

-- 

David Ja?a, RHCE

SPICE QE based in Brno
GPG Key:     22C33E24 
Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24





From vszocs at redhat.com  Tue Sep  4 12:04:50 2012
From: vszocs at redhat.com (Vojtech Szocs)
Date: Tue, 4 Sep 2012 08:04:50 -0400 (EDT)
Subject: [Engine-devel] Update on UI Plugins: PoC patch revision 4
In-Reply-To: <503FAFC2.2060504@redhat.com>
Message-ID: <627709453.17137271.1346760290602.JavaMail.root@redhat.com>

Hi Juan, thanks for your comments :)

Server-side components of UI plugin infrastructure (such as PluginSourcePageServlet) indeed need some more work, I agree with your points.

I was thinking that PluginSourcePageServlet and FileServlet are quite similar in their purpose, serving static resources from local filesystem. FileServlet is intended for general use, with 'file' parameter configured as servlet init-param. For example, FileServlet could be used to serve static resources from /usr/share/ovirt-engine/ui-plugins:

<servlet>
  <servlet-name>pluginResourceServlet</servlet-name>
  <servlet-class>org.ovirt.engine.core.FileServlet</servlet-class>
  <init-param>
    <param-name>file</param-name>
    <param-value>/usr/share/ovirt-engine/ui-plugins</param-value>
  </init-param>
</servlet>
<servlet-mapping>
  <servlet-name>pluginResourceServlet</servlet-name>
  <url-pattern>/plugins/*</url-pattern>
</servlet-mapping>

Assuming following directory convention for UI plugin descriptors and actual plugin resources:
/usr/share/ovirt-engine/ui-plugins/foo.json -> Plugin descriptor
/usr/share/ovirt-engine/ui-plugins/foo/start.html -> Plugin host page
/usr/share/ovirt-engine/ui-plugins/foo/foo.js -> Actual plugin code (referenced by plugin host page)

Such servlet could be used to map "http(s)://<EngineHost>:8700/plugins/foo/start.html" to /usr/share/ovirt-engine/ui-plugins/foo/start.html
(note that FileServlet is in root WAR context)

The purpose of PluginSourcePageServlet is very similar, but in terms of FileServlet, the 'file' parameter is not static (defined in web.xml as init-param), but depends on plugin meta-data (defined in foo.json) for each plugin.

PluginSourcePageServlet was meant to map "http(s)://<EngineHost>:8700/webadmin/webadmin/plugin/foo/start.html" to /custom/plugin/base/directory/start.html
(note that PluginSourcePageServlet is in WebAdmin WAR context)

Juan - do you think we could modify/reuse FileServlet for serving UI plugin static resources? As mentioned above, the only difference is that the 'file' parameter (base directory) would be potentially different for each plugin. Please let me know what you think about it.

Thanks,
Vojtech


----- Original Message -----
From: "Juan Hernandez" <jhernand at redhat.com>
To: "Vojtech Szocs" <vszocs at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Thursday, August 30, 2012 8:24:02 PM
Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4

Nice work Vojtech, just some comments about the PluginSourcePageServlet:

* You can avoid the hardcoded plugin code location with something like this:

  import org.ovirt.engine.core.utils.LocalConfig;

  File dataDir = LocalConfig.getInstance().getUsrDir();
  File pluginCodeLocation = new File(etcDir, "ui-plugins");

That will result in /usr/share/ovirt-engine/ui-plugins or whatever
directory is configured in the ENGINE_USR parameter in the
/etc/sysconfig/ovirt-engine file.

* It is very important to check the sanity of the value of the "plugin"
parameter, otherwise an attacker could send you a name with backpaths,
and that can result in accessing an unexpected file. In this particular
case you are adding the ".js" extension, so it won't probably result in
accessing dangerous files, but anyhow it is a good practice. I would
recommend to do something like this:

  String pluginName = request.getParameter("plugin");
  if (pluginName != null || !isSane(pluginName)) {
      ...
  }

The "isSane" method can do something similar to the "isSane" method in
the "FileServlet" class (I think you already mentioned this at some
point), maybe even forbid slashes as well.

* When copying the plugin file to the generated page you can avoid the
extra Buffered reader/writer as you are already using your own buffer in
the "copyChars" method (which is very good practice).

For the output you can directly use "response.getWriter()" instead of
"response.getOutputStream()", that is already buffered by the container.

On 08/30/2012 05:39 PM, Vojtech Szocs wrote:
> 
> 
> Hello everyone, 
> 
> as a follow-up to my last email on improving plugin API, here comes the latest revision of UI Plugins proof-of-concept patch (please find it attached). 
> 
> This patch is focused on improving JavaScript plugin API, along with important changes and improvements made to plugin infrastructure ( PluginManager ). Let's walk through the changes step by step. 
> 
> 
> 
> Improved plugin API, taking some inspiration from jQuery 
> 
> Following is a sample plugin code that uses new plugin API: 
> 
> var myPlugin = pluginApi('myPlugin'); // Obtain plugin API instance for 'myPlugin' 
> var myPluginConfig = myPlugin.configObject(); // Obtain plugin-specific configuration 
> 
> // Register event handler functions to be invoked by WebAdmin 
> // Note: all functions are optional, the plugin only defines functions for events it wants to handle 
> myPlugin.register({ 
> UiInit: function() { 
> var testUrl = 'http://www.example.com/' + myPluginConfig.foo; // Assume plugin configuration has 'foo' attribute 
> myPlugin.ui.addMainTab('Custom Tab', 'custom-tab', testUrl); // Invoke some operation using plugin API 
> } 
> }); 
> 
> myPlugin.ready(); // Event handler functions are registered, we are now ready to get initialized (UiInit) 
> 
> 
> 
> UI plugin life-cycle, enforced by plugin infrastructure 
> 
> The PluginState enumeration lists possible states of a plugin during its runtime: 
> 
>     * DEFINED : This is the initial state for all plugins. Plugin meta-data has been read by PluginManager and the corresponding iframe element has been created for the plugin. Note that at this point, the iframe element is not attached to DOM yet. 
>     * LOADING : The iframe element for the plugin has been attached to DOM, which causes plugin host page (previously known as plugin source page) to be fetched asynchronously in the background. We are now waiting for plugin to report in as ready. In practice, due to JavaScript runtime being single-threaded, WebAdmin startup logic will continue to execute until the JavaScript runtime is "idle" (browser event loop returns), and at this point JavaScript plugin code gets invoked through the plugin host page. 
>     * READY : The plugin has indicated that it is ready for use. We assume the plugin has already registered its event handler object (object containing various event handler functions to be called by WebAdmin) at this point. We can now proceed with plugin initialization. 
>     * INITIALIZED : The plugin has been initialized by calling UiInit function on its event handler object. We can now call other event handler functions, the plugin is now initialized and in use. 
> 
> 
> Note on plugin initialization: the UiInit function will be called just once during the lifetime of the plugin, after the plugin reports in as ready AND WebAdmin enters the state that allows plugins to be invoked (entering main section for logged-in users), and before other event handler functions are invoked by the plugin infrastructure. 
> 
> 
> 
> 
> Plugin meta-data is now passed to client using different format 
> 
> 
> Previously, plugin meta-data was embedded into WebAdmin host page as a simple JavaScript object, like so: 
> 
> 
> var pluginDefinitions = { myPlugin: "<URL>", anotherPlugin: "<URL>" } 
> 
> 
> 
> Now, plugin meta-data is embedded into WebAdmin host page as a JavaScript array, like so: 
> 
> 
> 
> var pluginDefinitions = [ 
> { name: "myPlugin", url: "<URL>", config: { "foo": 1, "bar": "whatever" } }, 
> { name: "anotherPlugin", url: "<URL>" } 
> 
> ]; 
> 
> 
> As you can see, pluginDefinitions is now an array of JavaScript objects, with each object representing plugin meta-data. The "name" and "url" attributes are mandatory (we need to check them when loading plugin descriptors). "config" is the plugin configuration (JSON) object, obtained by merging default plugin configuration (defined in plugin descriptor) with custom plugin configuration (defined in external plugin configuration file). Note that the "config" attribute is optional. 
> 
> 
> 
> In terms of Java classes, pluginDefinitions is mapped to PluginDefinitions overlay type, and each meta-data object within the array is mapped to PluginMetaData overlay type. 
> 
> 
> 
> 
> 
> Note on using assert statements in client code: you might notice that I'm using a lot of assert statements in Plugin class. This is to ensure consistency and guard against corrupted state during development. In GWT, assert statements work in a different way than in standard Java VM. When debugging GWT application using Development Mode, assert statements are checked and throw assertion errors during runtime (they are displayed in Development Mode console). However, when compiling GWT application to JavaScript (Production Mode), assert statements are removed by GWT compiler, so they don't affect the application running in Production Mode. 
> 
> 
> 
> Cheers, 
> Vojtech 
> 
> 
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From vszocs at redhat.com  Tue Sep  4 12:45:47 2012
From: vszocs at redhat.com (Vojtech Szocs)
Date: Tue, 4 Sep 2012 08:45:47 -0400 (EDT)
Subject: [Engine-devel] UI Plugins configuration
In-Reply-To: <A24D665DC3ECC245A2EA0D9A958F80B62C491334@G9W0745.americas.hpqcorp.net>
Message-ID: <645576115.17154971.1346762747139.JavaMail.root@redhat.com>

Thank you Chris :)

I'd like to incorporate your ideas into upcoming PoC patch, which will be focused on server-side components of UI plugin infrastructure:
1) servlet that serves plugin-related static resources (plugin host page, actual plugin code, any 3rd party JavaScript libraries, CSS, etc.) from local filesystem
2) class responsible for parsing/validating/caching plugin descriptor information from local filesystem

Regarding the comment in WebadminDynamicHostingServlet that says "FIXME: do we load this everytime, or just once?": maybe we could use the same approach as in servlet containers, which use 'date last modified' attribute of JSP files to trigger recompilation of corresponding servlets. For example, each time WebAdmin host page gets requested through WebadminDynamicHostingServlet, we could iterate over plugin descriptor files (in /usr/share/ovirt-engine/ui-plugins), looking at 'date last modified' attribute, parsing/validating plugin meta-data, and caching it using pluginName + dateLastModified (compound cache key). We also need to synchronize the access to plugin meta-data. What do you think?

After the upcoming PoC patch (focused on server-side components) gets released, UI plugin infrastructure should be pretty much stable and we can focus on adding particular features, such as:
* adding custom sub-tabs
* adding custom context-menu items
* making REST API calls through plugin API
* investigating cross-origin plugin scenario (plugin gets loaded from page sitting on different origin than Engine JBoss instance)

Let me know what you think.

Cheers,
Vojtech


----- Original Message -----
From: "Chris Frantz" <Chris.Frantz at hp.com>
To: "Vojtech Szocs" <vszocs at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Thursday, August 30, 2012 10:03:02 PM
Subject: RE: UI Plugins configuration




Vojtech, 



Here is my patch against WIP-UI-Plugins-PoC-revision-4. I?ve also included 2 dummy plugins in sample.tar.gz. 



--Chris 







From: engine-devel-bounces at ovirt.org [mailto:engine-devel-bounces at ovirt.org] On Behalf Of Frantz, Chris 
Sent: Thursday, August 30, 2012 1:06 PM 
To: Vojtech Szocs 
Cc: engine-devel 
Subject: Re: [Engine-devel] UI Plugins configuration 



Vojtech, 



I agree with your formalized names: 



Plugin Descriptor is the JSON file containing plugin meta-data. The plugin descriptor may also contain the default configuration data. It is located in $DATADIR/ui-plugins. 



Plugin Configuration is the JSON file containing optional plugin configuration info. It is located in $CONFIGDIR/ui-plugins (unless the Plugin Descriptor contains an absolute path). 



Plugin Definition is the JavaScript object used by WebAdmin. In the current implementation, the Plugin Definition contains both the Plugin Descriptor and the Plugin Configuraion. 



Plugin Source Page is the HTML page used to invoke the plugin code and shall be referenced by the plugin descriptor?s ?url? attribute. 



I?ve implemented the config merging you?ve suggested: the structure in configFile gets merged with the structure of ?config?, with the data in configFile winning in the case of duplicate key names. 



BTW, the patch is against ovirt-engine + 0001-WIP-UI-Plugins-PoC-revision-2. 



Let me know what you think, 

--Chris 




From dfediuck at redhat.com  Tue Sep  4 14:10:19 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Tue, 4 Sep 2012 10:10:19 -0400 (EDT)
Subject: [Engine-devel] Update on UI Plugins: PoC patch revision 4
In-Reply-To: <627709453.17137271.1346760290602.JavaMail.root@redhat.com>
Message-ID: <29484463.5183046.1346767819084.JavaMail.root@redhat.com>

----- Original Message -----
> From: "Vojtech Szocs" <vszocs at redhat.com>
> To: "Juan Hernandez" <jhernand at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Tuesday, September 4, 2012 3:04:50 PM
> Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4
> 
> Hi Juan, thanks for your comments :)
> 
> Server-side components of UI plugin infrastructure (such as
> PluginSourcePageServlet) indeed need some more work, I agree with
> your points.
> 
> I was thinking that PluginSourcePageServlet and FileServlet are quite
> similar in their purpose, serving static resources from local
> filesystem. FileServlet is intended for general use, with 'file'
> parameter configured as servlet init-param. For example, FileServlet
> could be used to serve static resources from
> /usr/share/ovirt-engine/ui-plugins:
> 
> <servlet>
>   <servlet-name>pluginResourceServlet</servlet-name>
>   <servlet-class>org.ovirt.engine.core.FileServlet</servlet-class>
>   <init-param>
>     <param-name>file</param-name>
>     <param-value>/usr/share/ovirt-engine/ui-plugins</param-value>
>   </init-param>
> </servlet>
> <servlet-mapping>
>   <servlet-name>pluginResourceServlet</servlet-name>
>   <url-pattern>/plugins/*</url-pattern>
> </servlet-mapping>
> 
> Assuming following directory convention for UI plugin descriptors and
> actual plugin resources:
> /usr/share/ovirt-engine/ui-plugins/foo.json -> Plugin descriptor
> /usr/share/ovirt-engine/ui-plugins/foo/start.html -> Plugin host page
> /usr/share/ovirt-engine/ui-plugins/foo/foo.js -> Actual plugin code
> (referenced by plugin host page)
> 
> Such servlet could be used to map
> "http(s)://<EngineHost>:8700/plugins/foo/start.html" to
> /usr/share/ovirt-engine/ui-plugins/foo/start.html
> (note that FileServlet is in root WAR context)
> 
> The purpose of PluginSourcePageServlet is very similar, but in terms
> of FileServlet, the 'file' parameter is not static (defined in
> web.xml as init-param), but depends on plugin meta-data (defined in
> foo.json) for each plugin.
> 
> PluginSourcePageServlet was meant to map
> "http(s)://<EngineHost>:8700/webadmin/webadmin/plugin/foo/start.html"
> to /custom/plugin/base/directory/start.html
> (note that PluginSourcePageServlet is in WebAdmin WAR context)
> 
> Juan - do you think we could modify/reuse FileServlet for serving UI
> plugin static resources? As mentioned above, the only difference is
> that the 'file' parameter (base directory) would be potentially
> different for each plugin. Please let me know what you think about
> it.
> 
> Thanks,
> Vojtech
> 
> 
> ----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: "Vojtech Szocs" <vszocs at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Thursday, August 30, 2012 8:24:02 PM
> Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision
> 4
> 
> Nice work Vojtech, just some comments about the
> PluginSourcePageServlet:
> 
> * You can avoid the hardcoded plugin code location with something
> like this:
> 
>   import org.ovirt.engine.core.utils.LocalConfig;
> 
>   File dataDir = LocalConfig.getInstance().getUsrDir();
>   File pluginCodeLocation = new File(etcDir, "ui-plugins");
> 
> That will result in /usr/share/ovirt-engine/ui-plugins or whatever
> directory is configured in the ENGINE_USR parameter in the
> /etc/sysconfig/ovirt-engine file.
> 
> * It is very important to check the sanity of the value of the
> "plugin"
> parameter, otherwise an attacker could send you a name with
> backpaths,
> and that can result in accessing an unexpected file. In this
> particular
> case you are adding the ".js" extension, so it won't probably result
> in
> accessing dangerous files, but anyhow it is a good practice. I would
> recommend to do something like this:
> 
>   String pluginName = request.getParameter("plugin");
>   if (pluginName != null || !isSane(pluginName)) {
>       ...
>   }
> 
> The "isSane" method can do something similar to the "isSane" method
> in
> the "FileServlet" class (I think you already mentioned this at some
> point), maybe even forbid slashes as well.
> 
> * When copying the plugin file to the generated page you can avoid
> the
> extra Buffered reader/writer as you are already using your own buffer
> in
> the "copyChars" method (which is very good practice).
> 
> For the output you can directly use "response.getWriter()" instead of
> "response.getOutputStream()", that is already buffered by the
> container.
> 
> On 08/30/2012 05:39 PM, Vojtech Szocs wrote:
> > 
> > 
> > Hello everyone,
> > 
> > as a follow-up to my last email on improving plugin API, here comes
> > the latest revision of UI Plugins proof-of-concept patch (please
> > find it attached).
> > 
> > This patch is focused on improving JavaScript plugin API, along
> > with important changes and improvements made to plugin
> > infrastructure ( PluginManager ). Let's walk through the changes
> > step by step.
> > 
> > 
> > 
> > Improved plugin API, taking some inspiration from jQuery
> > 
> > Following is a sample plugin code that uses new plugin API:
> > 
> > var myPlugin = pluginApi('myPlugin'); // Obtain plugin API instance
> > for 'myPlugin'
> > var myPluginConfig = myPlugin.configObject(); // Obtain
> > plugin-specific configuration
> > 
> > // Register event handler functions to be invoked by WebAdmin
> > // Note: all functions are optional, the plugin only defines
> > functions for events it wants to handle
> > myPlugin.register({
> > UiInit: function() {
> > var testUrl = 'http://www.example.com/' + myPluginConfig.foo; //
> > Assume plugin configuration has 'foo' attribute
> > myPlugin.ui.addMainTab('Custom Tab', 'custom-tab', testUrl); //
> > Invoke some operation using plugin API
> > }
> > });
> > 
> > myPlugin.ready(); // Event handler functions are registered, we are
> > now ready to get initialized (UiInit)
> > 
> > 
> > 
> > UI plugin life-cycle, enforced by plugin infrastructure
> > 
> > The PluginState enumeration lists possible states of a plugin
> > during its runtime:
> > 
> >     * DEFINED : This is the initial state for all plugins. Plugin
> >     meta-data has been read by PluginManager and the corresponding
> >     iframe element has been created for the plugin. Note that at
> >     this point, the iframe element is not attached to DOM yet.
> >     * LOADING : The iframe element for the plugin has been attached
> >     to DOM, which causes plugin host page (previously known as
> >     plugin source page) to be fetched asynchronously in the
> >     background. We are now waiting for plugin to report in as
> >     ready. In practice, due to JavaScript runtime being
> >     single-threaded, WebAdmin startup logic will continue to
> >     execute until the JavaScript runtime is "idle" (browser event
> >     loop returns), and at this point JavaScript plugin code gets
> >     invoked through the plugin host page.
> >     * READY : The plugin has indicated that it is ready for use. We
> >     assume the plugin has already registered its event handler
> >     object (object containing various event handler functions to
> >     be called by WebAdmin) at this point. We can now proceed with
> >     plugin initialization.
> >     * INITIALIZED : The plugin has been initialized by calling
> >     UiInit function on its event handler object. We can now call
> >     other event handler functions, the plugin is now initialized
> >     and in use.
> > 
> > 
> > Note on plugin initialization: the UiInit function will be called
> > just once during the lifetime of the plugin, after the plugin
> > reports in as ready AND WebAdmin enters the state that allows
> > plugins to be invoked (entering main section for logged-in users),
> > and before other event handler functions are invoked by the plugin
> > infrastructure.
> > 
> > 
> > 
> > 
> > Plugin meta-data is now passed to client using different format
> > 
> > 
> > Previously, plugin meta-data was embedded into WebAdmin host page
> > as a simple JavaScript object, like so:
> > 
> > 
> > var pluginDefinitions = { myPlugin: "<URL>", anotherPlugin: "<URL>"
> > }
> > 
> > 
> > 
> > Now, plugin meta-data is embedded into WebAdmin host page as a
> > JavaScript array, like so:
> > 
> > 
> > 
> > var pluginDefinitions = [
> > { name: "myPlugin", url: "<URL>", config: { "foo": 1, "bar":
> > "whatever" } },
> > { name: "anotherPlugin", url: "<URL>" }
> > 
> > ];
> > 
> > 
> > As you can see, pluginDefinitions is now an array of JavaScript
> > objects, with each object representing plugin meta-data. The
> > "name" and "url" attributes are mandatory (we need to check them
> > when loading plugin descriptors). "config" is the plugin
> > configuration (JSON) object, obtained by merging default plugin
> > configuration (defined in plugin descriptor) with custom plugin
> > configuration (defined in external plugin configuration file).
> > Note that the "config" attribute is optional.
> > 
> > 
> > 
> > In terms of Java classes, pluginDefinitions is mapped to
> > PluginDefinitions overlay type, and each meta-data object within
> > the array is mapped to PluginMetaData overlay type.
> > 
> > 
> > 
> > 
> > 
> > Note on using assert statements in client code: you might notice
> > that I'm using a lot of assert statements in Plugin class. This is
> > to ensure consistency and guard against corrupted state during
> > development. In GWT, assert statements work in a different way
> > than in standard Java VM. When debugging GWT application using
> > Development Mode, assert statements are checked and throw
> > assertion errors during runtime (they are displayed in Development
> > Mode console). However, when compiling GWT application to
> > JavaScript (Production Mode), assert statements are removed by GWT
> > compiler, so they don't affect the application running in
> > Production Mode.
> > 
> > 
> > 
> > Cheers,
> > Vojtech
> > 

Hi Vojtech,
Looks very interesting.
In the context of the client code, I have a suggestion that I'd like to be considered;
Will it be possible to allow re-using some of the existing dialogs in the system? In
this way, I may be able to use the pop-up dialog of edit-policy for example.
This will keep the look and feel, and allow people to reduce issues when re-using
existing code.

Thanks!
Doron


From vszocs at redhat.com  Tue Sep  4 14:46:01 2012
From: vszocs at redhat.com (Vojtech Szocs)
Date: Tue, 4 Sep 2012 10:46:01 -0400 (EDT)
Subject: [Engine-devel] Update on UI Plugins: PoC patch revision 4
In-Reply-To: <29484463.5183046.1346767819084.JavaMail.root@redhat.com>
Message-ID: <220285395.17272000.1346769961137.JavaMail.root@redhat.com>

Hi Doron,

interesting use-case for UI plugins! Existing (standard) WebAdmin dialogs could be made extensible for plugin authors, so that additional UI elements (like text boxes) could be added to dialog UI.

On one side, extensible dialogs (like the Edit Cluster Policy dialog you mentioned) would provide a way to register new UI elements, controlled and driven through plugin API. On the other side, the code behind extensible dialogs would collect values from extra UI elements, and handle them as appropriate, e.g. pass them along the standard values when communicating with the backend.

I think we can add this to post-PoC-rev5 feature list.

Cheers,
Vojtech


----- Original Message -----
From: "Doron Fediuck" <dfediuck at redhat.com>
To: "Vojtech Szocs" <vszocs at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Tuesday, September 4, 2012 4:10:19 PM
Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4

----- Original Message -----
> From: "Vojtech Szocs" <vszocs at redhat.com>
> To: "Juan Hernandez" <jhernand at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Tuesday, September 4, 2012 3:04:50 PM
> Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4
> 
> Hi Juan, thanks for your comments :)
> 
> Server-side components of UI plugin infrastructure (such as
> PluginSourcePageServlet) indeed need some more work, I agree with
> your points.
> 
> I was thinking that PluginSourcePageServlet and FileServlet are quite
> similar in their purpose, serving static resources from local
> filesystem. FileServlet is intended for general use, with 'file'
> parameter configured as servlet init-param. For example, FileServlet
> could be used to serve static resources from
> /usr/share/ovirt-engine/ui-plugins:
> 
> <servlet>
>   <servlet-name>pluginResourceServlet</servlet-name>
>   <servlet-class>org.ovirt.engine.core.FileServlet</servlet-class>
>   <init-param>
>     <param-name>file</param-name>
>     <param-value>/usr/share/ovirt-engine/ui-plugins</param-value>
>   </init-param>
> </servlet>
> <servlet-mapping>
>   <servlet-name>pluginResourceServlet</servlet-name>
>   <url-pattern>/plugins/*</url-pattern>
> </servlet-mapping>
> 
> Assuming following directory convention for UI plugin descriptors and
> actual plugin resources:
> /usr/share/ovirt-engine/ui-plugins/foo.json -> Plugin descriptor
> /usr/share/ovirt-engine/ui-plugins/foo/start.html -> Plugin host page
> /usr/share/ovirt-engine/ui-plugins/foo/foo.js -> Actual plugin code
> (referenced by plugin host page)
> 
> Such servlet could be used to map
> "http(s)://<EngineHost>:8700/plugins/foo/start.html" to
> /usr/share/ovirt-engine/ui-plugins/foo/start.html
> (note that FileServlet is in root WAR context)
> 
> The purpose of PluginSourcePageServlet is very similar, but in terms
> of FileServlet, the 'file' parameter is not static (defined in
> web.xml as init-param), but depends on plugin meta-data (defined in
> foo.json) for each plugin.
> 
> PluginSourcePageServlet was meant to map
> "http(s)://<EngineHost>:8700/webadmin/webadmin/plugin/foo/start.html"
> to /custom/plugin/base/directory/start.html
> (note that PluginSourcePageServlet is in WebAdmin WAR context)
> 
> Juan - do you think we could modify/reuse FileServlet for serving UI
> plugin static resources? As mentioned above, the only difference is
> that the 'file' parameter (base directory) would be potentially
> different for each plugin. Please let me know what you think about
> it.
> 
> Thanks,
> Vojtech
> 
> 
> ----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: "Vojtech Szocs" <vszocs at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Thursday, August 30, 2012 8:24:02 PM
> Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision
> 4
> 
> Nice work Vojtech, just some comments about the
> PluginSourcePageServlet:
> 
> * You can avoid the hardcoded plugin code location with something
> like this:
> 
>   import org.ovirt.engine.core.utils.LocalConfig;
> 
>   File dataDir = LocalConfig.getInstance().getUsrDir();
>   File pluginCodeLocation = new File(etcDir, "ui-plugins");
> 
> That will result in /usr/share/ovirt-engine/ui-plugins or whatever
> directory is configured in the ENGINE_USR parameter in the
> /etc/sysconfig/ovirt-engine file.
> 
> * It is very important to check the sanity of the value of the
> "plugin"
> parameter, otherwise an attacker could send you a name with
> backpaths,
> and that can result in accessing an unexpected file. In this
> particular
> case you are adding the ".js" extension, so it won't probably result
> in
> accessing dangerous files, but anyhow it is a good practice. I would
> recommend to do something like this:
> 
>   String pluginName = request.getParameter("plugin");
>   if (pluginName != null || !isSane(pluginName)) {
>       ...
>   }
> 
> The "isSane" method can do something similar to the "isSane" method
> in
> the "FileServlet" class (I think you already mentioned this at some
> point), maybe even forbid slashes as well.
> 
> * When copying the plugin file to the generated page you can avoid
> the
> extra Buffered reader/writer as you are already using your own buffer
> in
> the "copyChars" method (which is very good practice).
> 
> For the output you can directly use "response.getWriter()" instead of
> "response.getOutputStream()", that is already buffered by the
> container.
> 
> On 08/30/2012 05:39 PM, Vojtech Szocs wrote:
> > 
> > 
> > Hello everyone,
> > 
> > as a follow-up to my last email on improving plugin API, here comes
> > the latest revision of UI Plugins proof-of-concept patch (please
> > find it attached).
> > 
> > This patch is focused on improving JavaScript plugin API, along
> > with important changes and improvements made to plugin
> > infrastructure ( PluginManager ). Let's walk through the changes
> > step by step.
> > 
> > 
> > 
> > Improved plugin API, taking some inspiration from jQuery
> > 
> > Following is a sample plugin code that uses new plugin API:
> > 
> > var myPlugin = pluginApi('myPlugin'); // Obtain plugin API instance
> > for 'myPlugin'
> > var myPluginConfig = myPlugin.configObject(); // Obtain
> > plugin-specific configuration
> > 
> > // Register event handler functions to be invoked by WebAdmin
> > // Note: all functions are optional, the plugin only defines
> > functions for events it wants to handle
> > myPlugin.register({
> > UiInit: function() {
> > var testUrl = 'http://www.example.com/' + myPluginConfig.foo; //
> > Assume plugin configuration has 'foo' attribute
> > myPlugin.ui.addMainTab('Custom Tab', 'custom-tab', testUrl); //
> > Invoke some operation using plugin API
> > }
> > });
> > 
> > myPlugin.ready(); // Event handler functions are registered, we are
> > now ready to get initialized (UiInit)
> > 
> > 
> > 
> > UI plugin life-cycle, enforced by plugin infrastructure
> > 
> > The PluginState enumeration lists possible states of a plugin
> > during its runtime:
> > 
> >     * DEFINED : This is the initial state for all plugins. Plugin
> >     meta-data has been read by PluginManager and the corresponding
> >     iframe element has been created for the plugin. Note that at
> >     this point, the iframe element is not attached to DOM yet.
> >     * LOADING : The iframe element for the plugin has been attached
> >     to DOM, which causes plugin host page (previously known as
> >     plugin source page) to be fetched asynchronously in the
> >     background. We are now waiting for plugin to report in as
> >     ready. In practice, due to JavaScript runtime being
> >     single-threaded, WebAdmin startup logic will continue to
> >     execute until the JavaScript runtime is "idle" (browser event
> >     loop returns), and at this point JavaScript plugin code gets
> >     invoked through the plugin host page.
> >     * READY : The plugin has indicated that it is ready for use. We
> >     assume the plugin has already registered its event handler
> >     object (object containing various event handler functions to
> >     be called by WebAdmin) at this point. We can now proceed with
> >     plugin initialization.
> >     * INITIALIZED : The plugin has been initialized by calling
> >     UiInit function on its event handler object. We can now call
> >     other event handler functions, the plugin is now initialized
> >     and in use.
> > 
> > 
> > Note on plugin initialization: the UiInit function will be called
> > just once during the lifetime of the plugin, after the plugin
> > reports in as ready AND WebAdmin enters the state that allows
> > plugins to be invoked (entering main section for logged-in users),
> > and before other event handler functions are invoked by the plugin
> > infrastructure.
> > 
> > 
> > 
> > 
> > Plugin meta-data is now passed to client using different format
> > 
> > 
> > Previously, plugin meta-data was embedded into WebAdmin host page
> > as a simple JavaScript object, like so:
> > 
> > 
> > var pluginDefinitions = { myPlugin: "<URL>", anotherPlugin: "<URL>"
> > }
> > 
> > 
> > 
> > Now, plugin meta-data is embedded into WebAdmin host page as a
> > JavaScript array, like so:
> > 
> > 
> > 
> > var pluginDefinitions = [
> > { name: "myPlugin", url: "<URL>", config: { "foo": 1, "bar":
> > "whatever" } },
> > { name: "anotherPlugin", url: "<URL>" }
> > 
> > ];
> > 
> > 
> > As you can see, pluginDefinitions is now an array of JavaScript
> > objects, with each object representing plugin meta-data. The
> > "name" and "url" attributes are mandatory (we need to check them
> > when loading plugin descriptors). "config" is the plugin
> > configuration (JSON) object, obtained by merging default plugin
> > configuration (defined in plugin descriptor) with custom plugin
> > configuration (defined in external plugin configuration file).
> > Note that the "config" attribute is optional.
> > 
> > 
> > 
> > In terms of Java classes, pluginDefinitions is mapped to
> > PluginDefinitions overlay type, and each meta-data object within
> > the array is mapped to PluginMetaData overlay type.
> > 
> > 
> > 
> > 
> > 
> > Note on using assert statements in client code: you might notice
> > that I'm using a lot of assert statements in Plugin class. This is
> > to ensure consistency and guard against corrupted state during
> > development. In GWT, assert statements work in a different way
> > than in standard Java VM. When debugging GWT application using
> > Development Mode, assert statements are checked and throw
> > assertion errors during runtime (they are displayed in Development
> > Mode console). However, when compiling GWT application to
> > JavaScript (Production Mode), assert statements are removed by GWT
> > compiler, so they don't affect the application running in
> > Production Mode.
> > 
> > 
> > 
> > Cheers,
> > Vojtech
> > 

Hi Vojtech,
Looks very interesting.
In the context of the client code, I have a suggestion that I'd like to be considered;
Will it be possible to allow re-using some of the existing dialogs in the system? In
this way, I may be able to use the pop-up dialog of edit-policy for example.
This will keep the look and feel, and allow people to reduce issues when re-using
existing code.

Thanks!
Doron


From dfediuck at redhat.com  Tue Sep  4 15:10:29 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Tue, 4 Sep 2012 11:10:29 -0400 (EDT)
Subject: [Engine-devel] Update on UI Plugins: PoC patch revision 4
In-Reply-To: <220285395.17272000.1346769961137.JavaMail.root@redhat.com>
Message-ID: <347095832.5228070.1346771429453.JavaMail.root@redhat.com>

----- Original Message -----
> From: "Vojtech Szocs" <vszocs at redhat.com>
> To: "Doron Fediuck" <dfediuck at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Tuesday, September 4, 2012 5:46:01 PM
> Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4
> 
> Hi Doron,
> 
> interesting use-case for UI plugins! Existing (standard) WebAdmin
> dialogs could be made extensible for plugin authors, so that
> additional UI elements (like text boxes) could be added to dialog
> UI.
> 
> On one side, extensible dialogs (like the Edit Cluster Policy dialog
> you mentioned) would provide a way to register new UI elements,
> controlled and driven through plugin API. On the other side, the
> code behind extensible dialogs would collect values from extra UI
> elements, and handle them as appropriate, e.g. pass them along the
> standard values when communicating with the backend.
> 
> I think we can add this to post-PoC-rev5 feature list.
> 
> Cheers,
> Vojtech
> 

Excellent, Vojtech.
I'll monitor for news in this thread.

Thanks!

> 
> ----- Original Message -----
> From: "Doron Fediuck" <dfediuck at redhat.com>
> To: "Vojtech Szocs" <vszocs at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Tuesday, September 4, 2012 4:10:19 PM
> Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision
> 4
> 
> ----- Original Message -----
> > From: "Vojtech Szocs" <vszocs at redhat.com>
> > To: "Juan Hernandez" <jhernand at redhat.com>
> > Cc: "engine-devel" <engine-devel at ovirt.org>
> > Sent: Tuesday, September 4, 2012 3:04:50 PM
> > Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch
> > revision 4
> > 
> > Hi Juan, thanks for your comments :)
> > 
> > Server-side components of UI plugin infrastructure (such as
> > PluginSourcePageServlet) indeed need some more work, I agree with
> > your points.
> > 
> > I was thinking that PluginSourcePageServlet and FileServlet are
> > quite
> > similar in their purpose, serving static resources from local
> > filesystem. FileServlet is intended for general use, with 'file'
> > parameter configured as servlet init-param. For example,
> > FileServlet
> > could be used to serve static resources from
> > /usr/share/ovirt-engine/ui-plugins:
> > 
> > <servlet>
> >   <servlet-name>pluginResourceServlet</servlet-name>
> >   <servlet-class>org.ovirt.engine.core.FileServlet</servlet-class>
> >   <init-param>
> >     <param-name>file</param-name>
> >     <param-value>/usr/share/ovirt-engine/ui-plugins</param-value>
> >   </init-param>
> > </servlet>
> > <servlet-mapping>
> >   <servlet-name>pluginResourceServlet</servlet-name>
> >   <url-pattern>/plugins/*</url-pattern>
> > </servlet-mapping>
> > 
> > Assuming following directory convention for UI plugin descriptors
> > and
> > actual plugin resources:
> > /usr/share/ovirt-engine/ui-plugins/foo.json -> Plugin descriptor
> > /usr/share/ovirt-engine/ui-plugins/foo/start.html -> Plugin host
> > page
> > /usr/share/ovirt-engine/ui-plugins/foo/foo.js -> Actual plugin code
> > (referenced by plugin host page)
> > 
> > Such servlet could be used to map
> > "http(s)://<EngineHost>:8700/plugins/foo/start.html" to
> > /usr/share/ovirt-engine/ui-plugins/foo/start.html
> > (note that FileServlet is in root WAR context)
> > 
> > The purpose of PluginSourcePageServlet is very similar, but in
> > terms
> > of FileServlet, the 'file' parameter is not static (defined in
> > web.xml as init-param), but depends on plugin meta-data (defined in
> > foo.json) for each plugin.
> > 
> > PluginSourcePageServlet was meant to map
> > "http(s)://<EngineHost>:8700/webadmin/webadmin/plugin/foo/start.html"
> > to /custom/plugin/base/directory/start.html
> > (note that PluginSourcePageServlet is in WebAdmin WAR context)
> > 
> > Juan - do you think we could modify/reuse FileServlet for serving
> > UI
> > plugin static resources? As mentioned above, the only difference is
> > that the 'file' parameter (base directory) would be potentially
> > different for each plugin. Please let me know what you think about
> > it.
> > 
> > Thanks,
> > Vojtech
> > 
> > 
> > ----- Original Message -----
> > From: "Juan Hernandez" <jhernand at redhat.com>
> > To: "Vojtech Szocs" <vszocs at redhat.com>
> > Cc: "engine-devel" <engine-devel at ovirt.org>
> > Sent: Thursday, August 30, 2012 8:24:02 PM
> > Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch
> > revision
> > 4
> > 
> > Nice work Vojtech, just some comments about the
> > PluginSourcePageServlet:
> > 
> > * You can avoid the hardcoded plugin code location with something
> > like this:
> > 
> >   import org.ovirt.engine.core.utils.LocalConfig;
> > 
> >   File dataDir = LocalConfig.getInstance().getUsrDir();
> >   File pluginCodeLocation = new File(etcDir, "ui-plugins");
> > 
> > That will result in /usr/share/ovirt-engine/ui-plugins or whatever
> > directory is configured in the ENGINE_USR parameter in the
> > /etc/sysconfig/ovirt-engine file.
> > 
> > * It is very important to check the sanity of the value of the
> > "plugin"
> > parameter, otherwise an attacker could send you a name with
> > backpaths,
> > and that can result in accessing an unexpected file. In this
> > particular
> > case you are adding the ".js" extension, so it won't probably
> > result
> > in
> > accessing dangerous files, but anyhow it is a good practice. I
> > would
> > recommend to do something like this:
> > 
> >   String pluginName = request.getParameter("plugin");
> >   if (pluginName != null || !isSane(pluginName)) {
> >       ...
> >   }
> > 
> > The "isSane" method can do something similar to the "isSane" method
> > in
> > the "FileServlet" class (I think you already mentioned this at some
> > point), maybe even forbid slashes as well.
> > 
> > * When copying the plugin file to the generated page you can avoid
> > the
> > extra Buffered reader/writer as you are already using your own
> > buffer
> > in
> > the "copyChars" method (which is very good practice).
> > 
> > For the output you can directly use "response.getWriter()" instead
> > of
> > "response.getOutputStream()", that is already buffered by the
> > container.
> > 
> > On 08/30/2012 05:39 PM, Vojtech Szocs wrote:
> > > 
> > > 
> > > Hello everyone,
> > > 
> > > as a follow-up to my last email on improving plugin API, here
> > > comes
> > > the latest revision of UI Plugins proof-of-concept patch (please
> > > find it attached).
> > > 
> > > This patch is focused on improving JavaScript plugin API, along
> > > with important changes and improvements made to plugin
> > > infrastructure ( PluginManager ). Let's walk through the changes
> > > step by step.
> > > 
> > > 
> > > 
> > > Improved plugin API, taking some inspiration from jQuery
> > > 
> > > Following is a sample plugin code that uses new plugin API:
> > > 
> > > var myPlugin = pluginApi('myPlugin'); // Obtain plugin API
> > > instance
> > > for 'myPlugin'
> > > var myPluginConfig = myPlugin.configObject(); // Obtain
> > > plugin-specific configuration
> > > 
> > > // Register event handler functions to be invoked by WebAdmin
> > > // Note: all functions are optional, the plugin only defines
> > > functions for events it wants to handle
> > > myPlugin.register({
> > > UiInit: function() {
> > > var testUrl = 'http://www.example.com/' + myPluginConfig.foo; //
> > > Assume plugin configuration has 'foo' attribute
> > > myPlugin.ui.addMainTab('Custom Tab', 'custom-tab', testUrl); //
> > > Invoke some operation using plugin API
> > > }
> > > });
> > > 
> > > myPlugin.ready(); // Event handler functions are registered, we
> > > are
> > > now ready to get initialized (UiInit)
> > > 
> > > 
> > > 
> > > UI plugin life-cycle, enforced by plugin infrastructure
> > > 
> > > The PluginState enumeration lists possible states of a plugin
> > > during its runtime:
> > > 
> > >     * DEFINED : This is the initial state for all plugins. Plugin
> > >     meta-data has been read by PluginManager and the
> > >     corresponding
> > >     iframe element has been created for the plugin. Note that at
> > >     this point, the iframe element is not attached to DOM yet.
> > >     * LOADING : The iframe element for the plugin has been
> > >     attached
> > >     to DOM, which causes plugin host page (previously known as
> > >     plugin source page) to be fetched asynchronously in the
> > >     background. We are now waiting for plugin to report in as
> > >     ready. In practice, due to JavaScript runtime being
> > >     single-threaded, WebAdmin startup logic will continue to
> > >     execute until the JavaScript runtime is "idle" (browser event
> > >     loop returns), and at this point JavaScript plugin code gets
> > >     invoked through the plugin host page.
> > >     * READY : The plugin has indicated that it is ready for use.
> > >     We
> > >     assume the plugin has already registered its event handler
> > >     object (object containing various event handler functions to
> > >     be called by WebAdmin) at this point. We can now proceed with
> > >     plugin initialization.
> > >     * INITIALIZED : The plugin has been initialized by calling
> > >     UiInit function on its event handler object. We can now call
> > >     other event handler functions, the plugin is now initialized
> > >     and in use.
> > > 
> > > 
> > > Note on plugin initialization: the UiInit function will be called
> > > just once during the lifetime of the plugin, after the plugin
> > > reports in as ready AND WebAdmin enters the state that allows
> > > plugins to be invoked (entering main section for logged-in
> > > users),
> > > and before other event handler functions are invoked by the
> > > plugin
> > > infrastructure.
> > > 
> > > 
> > > 
> > > 
> > > Plugin meta-data is now passed to client using different format
> > > 
> > > 
> > > Previously, plugin meta-data was embedded into WebAdmin host page
> > > as a simple JavaScript object, like so:
> > > 
> > > 
> > > var pluginDefinitions = { myPlugin: "<URL>", anotherPlugin:
> > > "<URL>"
> > > }
> > > 
> > > 
> > > 
> > > Now, plugin meta-data is embedded into WebAdmin host page as a
> > > JavaScript array, like so:
> > > 
> > > 
> > > 
> > > var pluginDefinitions = [
> > > { name: "myPlugin", url: "<URL>", config: { "foo": 1, "bar":
> > > "whatever" } },
> > > { name: "anotherPlugin", url: "<URL>" }
> > > 
> > > ];
> > > 
> > > 
> > > As you can see, pluginDefinitions is now an array of JavaScript
> > > objects, with each object representing plugin meta-data. The
> > > "name" and "url" attributes are mandatory (we need to check them
> > > when loading plugin descriptors). "config" is the plugin
> > > configuration (JSON) object, obtained by merging default plugin
> > > configuration (defined in plugin descriptor) with custom plugin
> > > configuration (defined in external plugin configuration file).
> > > Note that the "config" attribute is optional.
> > > 
> > > 
> > > 
> > > In terms of Java classes, pluginDefinitions is mapped to
> > > PluginDefinitions overlay type, and each meta-data object within
> > > the array is mapped to PluginMetaData overlay type.
> > > 
> > > 
> > > 
> > > 
> > > 
> > > Note on using assert statements in client code: you might notice
> > > that I'm using a lot of assert statements in Plugin class. This
> > > is
> > > to ensure consistency and guard against corrupted state during
> > > development. In GWT, assert statements work in a different way
> > > than in standard Java VM. When debugging GWT application using
> > > Development Mode, assert statements are checked and throw
> > > assertion errors during runtime (they are displayed in
> > > Development
> > > Mode console). However, when compiling GWT application to
> > > JavaScript (Production Mode), assert statements are removed by
> > > GWT
> > > compiler, so they don't affect the application running in
> > > Production Mode.
> > > 
> > > 
> > > 
> > > Cheers,
> > > Vojtech
> > > 
> 
> Hi Vojtech,
> Looks very interesting.
> In the context of the client code, I have a suggestion that I'd like
> to be considered;
> Will it be possible to allow re-using some of the existing dialogs in
> the system? In
> this way, I may be able to use the pop-up dialog of edit-policy for
> example.
> This will keep the look and feel, and allow people to reduce issues
> when re-using
> existing code.
> 
> Thanks!
> Doron
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From lpeer at redhat.com  Wed Sep  5 09:02:11 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 5 Sep 2012 05:02:11 -0400 (EDT)
Subject: [Engine-devel] Cancelled: ovirt network
Message-ID: <873751606.4104469.1346835731288.JavaMail.root@redhat.com>

A single instance of the following meeting has been cancelled:

Subject: ovirt network  
Organiser: "Livnat Peer" <lpeer at redhat.com> 

Time: Wednesday, 19 September, 2012, 4:00:00 PM - 5:00:00 PM GMT +02:00 Jerusalem
 
Invitees: engine-devel at ovirt.org; vdsm-devel at lists.fedorahosted.org; GARGYA at de.ibm.com; dyasny at redhat.com; simon at redhat.com; mkolesni at redhat.com; atal at redhat.com; ilvovsky at redhat.com; dfediuck at redhat.com; pradipta.banerjee at gmail.com; gkotton at redhat.com ... 


*~*~*~*~*~*~*~*~*~*

I'm on vacation Sep 19th, cancelling this meeting instance.
If you want to have the meeting please send a new invite to the dev lists.

Thanks, Livnat
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120905/35468157/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: meeting.ics
Type: text/calendar
Size: 3983 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120905/35468157/attachment.ics>

From thomas at parker.iz34.com  Wed Sep  5 09:25:05 2012
From: thomas at parker.iz34.com (thomas at parker.iz34.com)
Date: Wed, 5 Sep 2012 11:25:05 +0200
Subject: [Engine-devel] Change in ovirt-engine[master]: webadmin:
	DirectLUN - confirmation	dialog for SD LUN (#84806...
In-Reply-To: <201209050923.q859NWI9009399@gerrit.ovirt.org>
References: <201209050923.q859NWI9009399@gerrit.ovirt.org>
Message-ID: <201209050923.q859NWI9009399@gerrit.ovirt.org>

Daniel Erez has submitted this change and it was merged.

Change subject: webadmin: DirectLUN - confirmation dialog for SD LUN (#848067)
.....................................................................


webadmin: DirectLUN - confirmation dialog for SD LUN (#848067)

https://bugzilla.redhat.com/848067

DirectLUN disk - when choosing a LUN that is already part of a
storage domain, display a warning confirmation dialog with a latch.

Change-Id: I1f303381142d869d162ad6c181dc715a279d097a
Signed-off-by: Daniel Erez <derez at redhat.com>
---
M frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/disks/DiskListModel.java
M frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/storage/SanStorageModel.java
M frontend/webadmin/modules/uicommonweb/src/main/java/org/ovirt/engine/ui/uicommonweb/models/vms/VmDiskListModel.java
M frontend/webadmin/modules/uicompat/src/main/java/org/ovirt/engine/ui/uicompat/Constants.java
M frontend/webadmin/modules/uicompat/src/main/java/org/ovirt/engine/ui/uicompat/Messages.java
M frontend/webadmin/modules/webadmin/src/main/java/org/ovirt/engine/ui/webadmin/gin/uicommon/DiskModule.java
M frontend/webadmin/modules/webadmin/src/main/java/org/ovirt/engine/ui/webadmin/gin/uicommon/VirtualMachineModule.java
7 files changed, 176 insertions(+), 10 deletions(-)

Approvals:
  Daniel Erez: Verified; Looks good to me, approved


--
To view, visit http://gerrit.ovirt.org/7753
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: merged
Gerrit-Change-Id: I1f303381142d869d162ad6c181dc715a279d097a
Gerrit-PatchSet: 2
Gerrit-Project: ovirt-engine
Gerrit-Branch: master
Gerrit-Owner: Daniel Erez <derez at redhat.com>
Gerrit-Reviewer: Daniel Erez <derez at redhat.com>
Gerrit-Reviewer: Gilad Chaplik <gchaplik at redhat.com>
_______________________________________________
Engine-commits mailing list
Engine-commits at ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-commits
-------------- next part --------------
A non-text attachment was scrubbed...
Name: originalBody.txt
Type: application/octet-stream
Size: 2042 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120905/81d93d52/attachment.obj>

From lhornyak at redhat.com  Wed Sep  5 10:00:55 2012
From: lhornyak at redhat.com (Laszlo Hornyak)
Date: Wed, 5 Sep 2012 06:00:55 -0400 (EDT)
Subject: [Engine-devel] waiting for review
In-Reply-To: <1979910675.14187179.1346839103418.JavaMail.root@redhat.com>
Message-ID: <1147564236.14187524.1346839255821.JavaMail.root@redhat.com>

hi,

I have some patches waiting for review. Some of them are not quite new. But anyway, if you have some time, your reviews are welcome.
http://gerrit.ovirt.org/#/dashboard/1000027

Thx,
Laszlo


From Chris.Frantz at hp.com  Wed Sep  5 18:52:28 2012
From: Chris.Frantz at hp.com (Frantz, Chris)
Date: Wed, 5 Sep 2012 18:52:28 +0000
Subject: [Engine-devel] UI Plugins configuration
In-Reply-To: <645576115.17154971.1346762747139.JavaMail.root@redhat.com>
References: <A24D665DC3ECC245A2EA0D9A958F80B62C491334@G9W0745.americas.hpqcorp.net>
	<645576115.17154971.1346762747139.JavaMail.root@redhat.com>
Message-ID: <A24D665DC3ECC245A2EA0D9A958F80B62C4952E8@G4W3294.americas.hpqcorp.net>

Vojtech,

Please go ahead and include my ideas and/or code into your next patch.

With regards to the FIXME, I think using the 'date last modified' is a good idea.  However, the plugin descriptor file can also reference the plugin config file (via the configFile property).  I imagine that adding, removing or reconfiguring plugins will be a relatively rare event, so maybe there is a simpler method:

Remember the newest timestamp of ($DATADIR/ui-plugins/*, $CONFIGDIR/ui-plugins/*) and reload the plugin descriptors and configurations if the newest timestamp changes.

As far as synchronization goes, the current code's getPluginDefinitions() loads all of the plugin descriptors and configurations into a new HashMap and then assigns that hashmap to WebadminDynamicHostingServlet.pluginDefinitions.  As long as the JsonNode trees in pluginDefinitions aren't modified after being put into pluginDefinitions, I don't see a synchronization issue.  Successive calls to get data from the pluginDefinitions hashmap may see different data, but they'll never see inconsistent data (e.g. a partially constructed plugin descriptor).

However, if we want to add calls to modify the pluginDefinitions at runtime (such as a element in the GUI that disables certain plugins), then we will need synchronization.  We'll also have to concern ourselves with whether any such modifications need to be saved back to the files on disk.

--Chris



-----Original Message-----
From: Vojtech Szocs [mailto:vszocs at redhat.com] 
Sent: Tuesday, September 04, 2012 7:46 AM
To: Frantz, Chris
Cc: engine-devel
Subject: Re: UI Plugins configuration

Thank you Chris :)

I'd like to incorporate your ideas into upcoming PoC patch, which will be focused on server-side components of UI plugin infrastructure:
1) servlet that serves plugin-related static resources (plugin host page, actual plugin code, any 3rd party JavaScript libraries, CSS, etc.) from local filesystem
2) class responsible for parsing/validating/caching plugin descriptor information from local filesystem

Regarding the comment in WebadminDynamicHostingServlet that says "FIXME: do we load this everytime, or just once?": maybe we could use the same approach as in servlet containers, which use 'date last modified' attribute of JSP files to trigger recompilation of corresponding servlets. For example, each time WebAdmin host page gets requested through WebadminDynamicHostingServlet, we could iterate over plugin descriptor files (in /usr/share/ovirt-engine/ui-plugins), looking at 'date last modified' attribute, parsing/validating plugin meta-data, and caching it using pluginName + dateLastModified (compound cache key). We also need to synchronize the access to plugin meta-data. What do you think?

After the upcoming PoC patch (focused on server-side components) gets released, UI plugin infrastructure should be pretty much stable and we can focus on adding particular features, such as:
* adding custom sub-tabs
* adding custom context-menu items
* making REST API calls through plugin API
* investigating cross-origin plugin scenario (plugin gets loaded from page sitting on different origin than Engine JBoss instance)

Let me know what you think.

Cheers,
Vojtech


----- Original Message -----
From: "Chris Frantz" <Chris.Frantz at hp.com>
To: "Vojtech Szocs" <vszocs at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Thursday, August 30, 2012 10:03:02 PM
Subject: RE: UI Plugins configuration




Vojtech, 



Here is my patch against WIP-UI-Plugins-PoC-revision-4. I?ve also included 2 dummy plugins in sample.tar.gz. 



--Chris 







From: engine-devel-bounces at ovirt.org [mailto:engine-devel-bounces at ovirt.org] On Behalf Of Frantz, Chris 
Sent: Thursday, August 30, 2012 1:06 PM 
To: Vojtech Szocs 
Cc: engine-devel 
Subject: Re: [Engine-devel] UI Plugins configuration 



Vojtech, 



I agree with your formalized names: 



Plugin Descriptor is the JSON file containing plugin meta-data. The plugin descriptor may also contain the default configuration data. It is located in $DATADIR/ui-plugins. 



Plugin Configuration is the JSON file containing optional plugin configuration info. It is located in $CONFIGDIR/ui-plugins (unless the Plugin Descriptor contains an absolute path). 



Plugin Definition is the JavaScript object used by WebAdmin. In the current implementation, the Plugin Definition contains both the Plugin Descriptor and the Plugin Configuraion. 



Plugin Source Page is the HTML page used to invoke the plugin code and shall be referenced by the plugin descriptor?s ?url? attribute. 



I?ve implemented the config merging you?ve suggested: the structure in configFile gets merged with the structure of ?config?, with the data in configFile winning in the case of duplicate key names. 



BTW, the patch is against ovirt-engine + 0001-WIP-UI-Plugins-PoC-revision-2. 



Let me know what you think, 

--Chris 



From ecohen at redhat.com  Thu Sep  6 07:28:14 2012
From: ecohen at redhat.com (Einav Cohen)
Date: Thu, 6 Sep 2012 03:28:14 -0400 (EDT)
Subject: [Engine-devel] GUI patches: verify on 2 browsers
In-Reply-To: <1241011544.6308981.1346916359797.JavaMail.root@redhat.com>
Message-ID: <725301301.6309670.1346916494578.JavaMail.root@redhat.com>

oVirt-engine developers/reviewers: oVirt-engine patches that involve GUI changes (i.e. web admin and/or user portal) *must* be verified on at least two different browsers out of the following: firefox/chrome/ie

----
Thanks,
Einav


From ykaul at redhat.com  Thu Sep  6 07:32:59 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Thu, 06 Sep 2012 10:32:59 +0300
Subject: [Engine-devel] GUI patches: verify on 2 browsers
In-Reply-To: <725301301.6309670.1346916494578.JavaMail.root@redhat.com>
References: <725301301.6309670.1346916494578.JavaMail.root@redhat.com>
Message-ID: <504851AB.9050709@redhat.com>

On 09/06/2012 10:28 AM, Einav Cohen wrote:
> oVirt-engine developers/reviewers: oVirt-engine patches that involve GUI changes (i.e. web admin and/or user portal) *must* be verified on at least two different browsers out of the following: firefox/chrome/ie

What happens when it fails on Chrome? We know of several issues with it 
already.
Y.

>
> ----
> Thanks,
> Einav
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From ecohen at redhat.com  Thu Sep  6 07:37:08 2012
From: ecohen at redhat.com (Einav Cohen)
Date: Thu, 6 Sep 2012 03:37:08 -0400 (EDT)
Subject: [Engine-devel] GUI patches: verify on 2 browsers
In-Reply-To: <504851AB.9050709@redhat.com>
Message-ID: <873171430.6313183.1346917028181.JavaMail.root@redhat.com>

> ----- Original Message -----
> From: "Yaniv Kaul" <ykaul at redhat.com>
> Sent: Thursday, September 6, 2012 10:32:59 AM
> 
> On 09/06/2012 10:28 AM, Einav Cohen wrote:
> > oVirt-engine developers/reviewers: oVirt-engine patches that
> > involve GUI changes (i.e. web admin and/or user portal) *must* be
> > verified on at least two different browsers out of the following:
> > firefox/chrome/ie
> 
> What happens when it fails on Chrome? We know of several issues with
> it
> already.

we will handle the already-existing chrome issues eventually.
my e-mail is about not introducing any new GUI features/changes from now on that haven't been verified on at least two browsers.

> Y.
> 
> >
> > ----
> > Thanks,
> > Einav
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 
> 


From mpastern at redhat.com  Thu Sep  6 09:37:30 2012
From: mpastern at redhat.com (Michael Pasternak)
Date: Thu, 06 Sep 2012 12:37:30 +0300
Subject: [Engine-devel] updating disk /api/disks/xxx
Message-ID: <50486EDA.5000109@redhat.com>


i see in BackendDiskResource.update(), what is the status if this item?


    @Override
    public Disk update(Disk disk) {
          //TODO: implmenet when Backend is ready. Implementation stub:
//      return performUpdate(this, new QueryIdResolver(VdcQueryType.GetDiskByDiskId, GetDiskByDiskIdParameters.class), ?, ?);
        throw new NotImplementedException("Pending Backend support");
    }
-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D



From abaron at redhat.com  Thu Sep  6 09:41:37 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 6 Sep 2012 05:41:37 -0400 (EDT)
Subject: [Engine-devel] updating disk /api/disks/xxx
In-Reply-To: <50486EDA.5000109@redhat.com>
Message-ID: <535639676.65644361.1346924497641.JavaMail.root@redhat.com>



----- Original Message -----
> 
> i see in BackendDiskResource.update(), what is the status if this
> item?

Is there a bug tracking this?
I don't recall anyone is working on this.

> 
> 
>     @Override
>     public Disk update(Disk disk) {
>           //TODO: implmenet when Backend is ready. Implementation
>           stub:
> //      return performUpdate(this, new
> QueryIdResolver(VdcQueryType.GetDiskByDiskId,
> GetDiskByDiskIdParameters.class), ?, ?);
>         throw new NotImplementedException("Pending Backend support");
>     }
> --
> 
> Michael Pasternak
> RedHat, ENG-Virtualization R&D
> 
> 


From mpastern at redhat.com  Thu Sep  6 09:57:43 2012
From: mpastern at redhat.com (Michael Pasternak)
Date: Thu, 06 Sep 2012 12:57:43 +0300
Subject: [Engine-devel] updating disk /api/disks/xxx
In-Reply-To: <535639676.65644361.1346924497641.JavaMail.root@redhat.com>
References: <535639676.65644361.1346924497641.JavaMail.root@redhat.com>
Message-ID: <50487397.7080500@redhat.com>

On 09/06/2012 12:41 PM, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>>
>> i see in BackendDiskResource.update(), what is the status if this
>> item?
> 
> Is there a bug tracking this?

afaik no

> I don't recall anyone is working on this.

iirc this feature was done by mkublin

> 
>>
>>
>>     @Override
>>     public Disk update(Disk disk) {
>>           //TODO: implmenet when Backend is ready. Implementation
>>           stub:
>> //      return performUpdate(this, new
>> QueryIdResolver(VdcQueryType.GetDiskByDiskId,
>> GetDiskByDiskIdParameters.class), ?, ?);
>>         throw new NotImplementedException("Pending Backend support");
>>     }
>> --
>>
>> Michael Pasternak
>> RedHat, ENG-Virtualization R&D
>>
>>


-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D


From mkublin at redhat.com  Thu Sep  6 10:22:32 2012
From: mkublin at redhat.com (Michael Kublin)
Date: Thu, 6 Sep 2012 06:22:32 -0400 (EDT)
Subject: [Engine-devel] updating disk /api/disks/xxx
In-Reply-To: <50487397.7080500@redhat.com>
Message-ID: <583829101.4652970.1346926952353.JavaMail.root@redhat.com>

No, it is not my feature

----- Original Message -----
From: "Michael Pasternak" <mpastern at redhat.com>
To: "Ayal Baron" <abaron at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>, "Michael Kublin" <mkublin at redhat.com>
Sent: Thursday, September 6, 2012 12:57:43 PM
Subject: Re: updating disk /api/disks/xxx

On 09/06/2012 12:41 PM, Ayal Baron wrote:
> 
> 
> ----- Original Message -----
>>
>> i see in BackendDiskResource.update(), what is the status if this
>> item?
> 
> Is there a bug tracking this?

afaik no

> I don't recall anyone is working on this.

iirc this feature was done by mkublin

> 
>>
>>
>>     @Override
>>     public Disk update(Disk disk) {
>>           //TODO: implmenet when Backend is ready. Implementation
>>           stub:
>> //      return performUpdate(this, new
>> QueryIdResolver(VdcQueryType.GetDiskByDiskId,
>> GetDiskByDiskIdParameters.class), ?, ?);
>>         throw new NotImplementedException("Pending Backend support");
>>     }
>> --
>>
>> Michael Pasternak
>> RedHat, ENG-Virtualization R&D
>>
>>


-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D


From mpastern at redhat.com  Thu Sep  6 10:32:25 2012
From: mpastern at redhat.com (Michael Pasternak)
Date: Thu, 06 Sep 2012 13:32:25 +0300
Subject: [Engine-devel] updating disk /api/disks/xxx
In-Reply-To: <583829101.4652970.1346926952353.JavaMail.root@redhat.com>
References: <583829101.4652970.1346926952353.JavaMail.root@redhat.com>
Message-ID: <50487BB9.3060700@redhat.com>

On 09/06/2012 01:22 PM, Michael Kublin wrote:
> No, it is not my feature
> 
> ----- Original Message -----
> From: "Michael Pasternak" <mpastern at redhat.com>
> To: "Ayal Baron" <abaron at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>, "Michael Kublin" <mkublin at redhat.com>
> Sent: Thursday, September 6, 2012 12:57:43 PM
> Subject: Re: updating disk /api/disks/xxx
> 
> On 09/06/2012 12:41 PM, Ayal Baron wrote:
>>
>>
>> ----- Original Message -----
>>>
>>> i see in BackendDiskResource.update(), what is the status if this
>>> item?
>>
>> Is there a bug tracking this?

https://bugzilla.redhat.com/show_bug.cgi?id=854932

> 
> afaik no
> 
>> I don't recall anyone is working on this.
> 
> iirc this feature was done by mkublin
> 
>>
>>>
>>>
>>>     @Override
>>>     public Disk update(Disk disk) {
>>>           //TODO: implmenet when Backend is ready. Implementation
>>>           stub:
>>> //      return performUpdate(this, new
>>> QueryIdResolver(VdcQueryType.GetDiskByDiskId,
>>> GetDiskByDiskIdParameters.class), ?, ?);
>>>         throw new NotImplementedException("Pending Backend support");
>>>     }
>>> --
>>>
>>> Michael Pasternak
>>> RedHat, ENG-Virtualization R&D
>>>
>>>
> 
> 


-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D


From acathrow at redhat.com  Thu Sep  6 10:49:38 2012
From: acathrow at redhat.com (Andrew Cathrow)
Date: Thu, 6 Sep 2012 06:49:38 -0400 (EDT)
Subject: [Engine-devel] GUI patches: verify on 2 browsers
In-Reply-To: <873171430.6313183.1346917028181.JavaMail.root@redhat.com>
Message-ID: <1991041396.6373989.1346928578746.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Einav Cohen" <ecohen at redhat.com>
> To: "Yaniv Kaul" <ykaul at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Thursday, September 6, 2012 3:37:08 AM
> Subject: Re: [Engine-devel] GUI patches: verify on 2 browsers
> 
> > ----- Original Message -----
> > From: "Yaniv Kaul" <ykaul at redhat.com>
> > Sent: Thursday, September 6, 2012 10:32:59 AM
> > 
> > On 09/06/2012 10:28 AM, Einav Cohen wrote:
> > > oVirt-engine developers/reviewers: oVirt-engine patches that
> > > involve GUI changes (i.e. web admin and/or user portal) *must* be
> > > verified on at least two different browsers out of the following:
> > > firefox/chrome/ie
> > 
> > What happens when it fails on Chrome? We know of several issues
> > with
> > it
> > already.
> 
> we will handle the already-existing chrome issues eventually.
> my e-mail is about not introducing any new GUI features/changes from
> now on that haven't been verified on at least two browsers.

Why not 3? Obviously that's 50% more work but saying 2 our of firefox/chrome/ie doesn't make sense to me. Someone could pick to test on IE and Chrome and leave something that's broken in Firefox.


> 
> > Y.
> > 
> > >
> > > ----
> > > Thanks,
> > > Einav
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> > 
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From ecohen at redhat.com  Thu Sep  6 12:14:22 2012
From: ecohen at redhat.com (Einav Cohen)
Date: Thu, 6 Sep 2012 08:14:22 -0400 (EDT)
Subject: [Engine-devel] GUI patches: verify on 2 browsers
In-Reply-To: <1991041396.6373989.1346928578746.JavaMail.root@redhat.com>
Message-ID: <659952896.6400715.1346933662028.JavaMail.root@redhat.com>

> ----- Original Message -----
> From: "Andrew Cathrow" <acathrow at redhat.com>
> Sent: Thursday, September 6, 2012 1:49:38 PM
> 
> ----- Original Message -----
> > From: "Einav Cohen" <ecohen at redhat.com>
> > To: "Yaniv Kaul" <ykaul at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Thursday, September 6, 2012 3:37:08 AM
> > Subject: Re: [Engine-devel] GUI patches: verify on 2 browsers
> > 
> > > ----- Original Message -----
> > > From: "Yaniv Kaul" <ykaul at redhat.com>
> > > Sent: Thursday, September 6, 2012 10:32:59 AM
> > > 
> > > On 09/06/2012 10:28 AM, Einav Cohen wrote:
> > > > oVirt-engine developers/reviewers: oVirt-engine patches that
> > > > involve GUI changes (i.e. web admin and/or user portal) *must*
> > > > be
> > > > verified on at least two different browsers out of the
> > > > following:
> > > > firefox/chrome/ie
> > > 
> > > What happens when it fails on Chrome? We know of several issues
> > > with
> > > it
> > > already.
> > 
> > we will handle the already-existing chrome issues eventually.
> > my e-mail is about not introducing any new GUI features/changes
> > from
> > now on that haven't been verified on at least two browsers.
> 
> Why not 3? Obviously that's 50% more work but saying 2 our of
> firefox/chrome/ie doesn't make sense to me. Someone could pick to
> test on IE and Chrome and leave something that's broken in Firefox.
> 

Even testing on these 3 different browsers is not really enough - the GUI might look/behave differently even between different versions of FF, or between FF on Linux vs. FF on Windows.
Testing on two browsers (instead of one) should improve compatibility issue detection.

> 
> > 
> > > Y.
> > > 
> > > >
> > > > ----
> > > > Thanks,
> > > > Einav
> > > > _______________________________________________
> > > > Engine-devel mailing list
> > > > Engine-devel at ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > > 
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 
> 


From acathrow at redhat.com  Thu Sep  6 12:28:14 2012
From: acathrow at redhat.com (Andrew Cathrow)
Date: Thu, 6 Sep 2012 08:28:14 -0400 (EDT)
Subject: [Engine-devel] GUI patches: verify on 2 browsers
In-Reply-To: <659952896.6400715.1346933662028.JavaMail.root@redhat.com>
Message-ID: <1991382538.6403813.1346934494850.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Einav Cohen" <ecohen at redhat.com>
> To: "Andrew Cathrow" <acathrow at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Thursday, September 6, 2012 8:14:22 AM
> Subject: Re: [Engine-devel] GUI patches: verify on 2 browsers
> 
> > ----- Original Message -----
> > From: "Andrew Cathrow" <acathrow at redhat.com>
> > Sent: Thursday, September 6, 2012 1:49:38 PM
> > 
> > ----- Original Message -----
> > > From: "Einav Cohen" <ecohen at redhat.com>
> > > To: "Yaniv Kaul" <ykaul at redhat.com>
> > > Cc: engine-devel at ovirt.org
> > > Sent: Thursday, September 6, 2012 3:37:08 AM
> > > Subject: Re: [Engine-devel] GUI patches: verify on 2 browsers
> > > 
> > > > ----- Original Message -----
> > > > From: "Yaniv Kaul" <ykaul at redhat.com>
> > > > Sent: Thursday, September 6, 2012 10:32:59 AM
> > > > 
> > > > On 09/06/2012 10:28 AM, Einav Cohen wrote:
> > > > > oVirt-engine developers/reviewers: oVirt-engine patches that
> > > > > involve GUI changes (i.e. web admin and/or user portal)
> > > > > *must*
> > > > > be
> > > > > verified on at least two different browsers out of the
> > > > > following:
> > > > > firefox/chrome/ie
> > > > 
> > > > What happens when it fails on Chrome? We know of several issues
> > > > with
> > > > it
> > > > already.
> > > 
> > > we will handle the already-existing chrome issues eventually.
> > > my e-mail is about not introducing any new GUI features/changes
> > > from
> > > now on that haven't been verified on at least two browsers.
> > 
> > Why not 3? Obviously that's 50% more work but saying 2 our of
> > firefox/chrome/ie doesn't make sense to me. Someone could pick to
> > test on IE and Chrome and leave something that's broken in Firefox.
> > 
> 
> Even testing on these 3 different browsers is not really enough - the
> GUI might look/behave differently even between different versions of
> FF, or between FF on Linux vs. FF on Windows.
> Testing on two browsers (instead of one) should improve compatibility
> issue detection.
> 

I think the idea of saying two browsers Firefox + one other that was made elsewhere is the right way to go.



> > 
> > > 
> > > > Y.
> > > > 
> > > > >
> > > > > ----
> > > > > Thanks,
> > > > > Einav
> > > > > _______________________________________________
> > > > > Engine-devel mailing list
> > > > > Engine-devel at ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > 
> > > > _______________________________________________
> > > > Engine-devel mailing list
> > > > Engine-devel at ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > 
> > > > 
> > > > 
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> > 
> > 
> 


From sesubram at redhat.com  Fri Sep  7 15:03:53 2012
From: sesubram at redhat.com (Selvasundaram)
Date: Fri, 07 Sep 2012 20:33:53 +0530
Subject: [Engine-devel] No disk space in gerrit server?
Message-ID: <504A0CD9.2080009@redhat.com>

The following error seems to be no disk space in gerrit server.

Counting objects: 62, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (23/23), done.
Writing objects: 100% (38/38), 9.28 KiB, done.
Total 38 (delta 9), reused 0 (delta 0)
error: unpack failed: error No space left on device
fatal: Unpack error, check server log
To gerrit.ovirt.org:ovirt-engine
  ! [remote rejected] HEAD -> refs/for/master (n/a (unpacker error))
error: failed to push some refs to 'gerrit.ovirt.org:ovirt-engine'


--
Regards
Selvasundaram


From iheim at redhat.com  Fri Sep  7 16:29:47 2012
From: iheim at redhat.com (Itamar Heim)
Date: Fri, 07 Sep 2012 19:29:47 +0300
Subject: [Engine-devel] No disk space in gerrit server?
In-Reply-To: <504A0CD9.2080009@redhat.com>
References: <504A0CD9.2080009@redhat.com>
Message-ID: <504A20FB.3010903@redhat.com>

On 09/07/2012 06:03 PM, Selvasundaram wrote:
> The following error seems to be no disk space in gerrit server.
>
> Counting objects: 62, done.
> Delta compression using up to 4 threads.
> Compressing objects: 100% (23/23), done.
> Writing objects: 100% (38/38), 9.28 KiB, done.
> Total 38 (delta 9), reused 0 (delta 0)
> error: unpack failed: error No space left on device
> fatal: Unpack error, check server log
> To gerrit.ovirt.org:ovirt-engine
>   ! [remote rejected] HEAD -> refs/for/master (n/a (unpacker error))
> error: failed to push some refs to 'gerrit.ovirt.org:ovirt-engine'
>
>
> --
> Regards
> Selvasundaram
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

fixed



From eedri at redhat.com  Sun Sep  9 09:53:35 2012
From: eedri at redhat.com (Eyal Edri)
Date: Sun, 9 Sep 2012 05:53:35 -0400 (EDT)
Subject: [Engine-devel] commit breaks ovirt-engine due to compilation error
In-Reply-To: <2002945494.2730497.1347184335989.JavaMail.root@redhat.com>
Message-ID: <597188928.2730522.1347184415073.JavaMail.root@redhat.com>

this commit break ovirt-engine on jenkins.ovirt.org:

http://jenkins.ovirt.org/job/ovirt_engine_animal_sniffer_check/425/org.ovirt.engine.core$vdsbroker/changes#detail0

core: add vdsName to vdsBroker log info (#853883) (details)
Commit 35c2e7ceb5109c9349e3699c3f53414cbe0b4c4e by ybronhei
core: add vdsName to vdsBroker log info (#853883)

https://bugzilla.redhat.com/show_bug.cgi?id=853883

Logger gets VdsCommandBase.toString() method's output,
this patch adds call to getAdditionalInformation from
toString to add to log string VdsName.

Change-Id: Ieb0589bb5f5e157c04f85e46f6769d66ef98b2d1
Signed-off-by: Yaniv Bronhaim <ybronhei at redhat.com>


please check,

Eyal.










[INFO] 
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ vdsbroker ---
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR : 
[INFO] -------------------------------------------------------------
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/RemoveVdsVDSCommand.java:[5,7] error: RemoveVdsVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/UpdateVmDynamicDataVDSCommand.java:[5,7] error: UpdateVmDynamicDataVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/FailedToRunVmVDSCommand.java:[5,7] error: FailedToRunVmVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/ActivateVdsVDSCommand.java:[5,7] error: ActivateVdsVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/CreateVmVDSCommand.java:[28,7] error: CreateVmVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/UpdateVdsVMsClearedVDSCommand.java:[5,7] error: UpdateVdsVMsClearedVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/AddVdsVDSCommand.java:[12,7] error: AddVdsVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/SetVdsStatusVDSCommand.java:[13,7] error: SetVdsStatusVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/ResumeVDSCommand.java:[10,7] error: ResumeVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/DestroyVmVDSCommand.java:[18,7] error: DestroyVmVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/MigrateVDSCommand.java:[21,7] error: MigrateVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/UpdateVdsDynamicDataVDSCommand.java:[5,7] error: UpdateVdsDynamicDataVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
[INFO] 12 errors 
[INFO] -------------------------------------------------------------
mojoFailed org.apache.maven.plugins:maven-compiler-plugin:2.3.2(default-compile)
projectFailed org.ovirt.engine.core:vdsbroker:3.1.0


From masayag at redhat.com  Sun Sep  9 10:18:39 2012
From: masayag at redhat.com (Moti Asayag)
Date: Sun, 09 Sep 2012 13:18:39 +0300
Subject: [Engine-devel] commit breaks ovirt-engine due to compilation
 error
In-Reply-To: <597188928.2730522.1347184415073.JavaMail.root@redhat.com>
References: <597188928.2730522.1347184415073.JavaMail.root@redhat.com>
Message-ID: <504C6CFF.9010409@redhat.com>

I've submitted a patch to bypass the failure created by that commit.

http://gerrit.ovirt.org/#/c/7865/1

Yaniv, please submit a proper solution for it if meant differently.


On 09/09/2012 12:53 PM, Eyal Edri wrote:
> this commit break ovirt-engine on jenkins.ovirt.org:
> 
> http://jenkins.ovirt.org/job/ovirt_engine_animal_sniffer_check/425/org.ovirt.engine.core$vdsbroker/changes#detail0
> 
> core: add vdsName to vdsBroker log info (#853883) (details)
> Commit 35c2e7ceb5109c9349e3699c3f53414cbe0b4c4e by ybronhei
> core: add vdsName to vdsBroker log info (#853883)
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=853883
> 
> Logger gets VdsCommandBase.toString() method's output,
> this patch adds call to getAdditionalInformation from
> toString to add to log string VdsName.
> 
> Change-Id: Ieb0589bb5f5e157c04f85e46f6769d66ef98b2d1
> Signed-off-by: Yaniv Bronhaim <ybronhei at redhat.com>
> 
> 
> please check,
> 
> Eyal.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> [INFO] 
> [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ vdsbroker ---
> [INFO] -------------------------------------------------------------
> [ERROR] COMPILATION ERROR : 
> [INFO] -------------------------------------------------------------
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/RemoveVdsVDSCommand.java:[5,7] error: RemoveVdsVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/UpdateVmDynamicDataVDSCommand.java:[5,7] error: UpdateVmDynamicDataVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/FailedToRunVmVDSCommand.java:[5,7] error: FailedToRunVmVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/ActivateVdsVDSCommand.java:[5,7] error: ActivateVdsVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/CreateVmVDSCommand.java:[28,7] error: CreateVmVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/UpdateVdsVMsClearedVDSCommand.java:[5,7] error: UpdateVdsVMsClearedVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/AddVdsVDSCommand.java:[12,7] error: AddVdsVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/SetVdsStatusVDSCommand.java:[13,7] error: SetVdsStatusVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/ResumeVDSCommand.java:[10,7] error: ResumeVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/DestroyVmVDSCommand.java:[18,7] error: DestroyVmVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/MigrateVDSCommand.java:[21,7] error: MigrateVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [ERROR] /home/jenkins/workspace/ovirt_engine_animal_sniffer_check/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/UpdateVdsDynamicDataVDSCommand.java:[5,7] error: UpdateVdsDynamicDataVDSCommand is not abstract and does not override abstract method getAdditionalInformation() in VDSCommandBase
> [INFO] 12 errors 
> [INFO] -------------------------------------------------------------
> mojoFailed org.apache.maven.plugins:maven-compiler-plugin:2.3.2(default-compile)
> projectFailed org.ovirt.engine.core:vdsbroker:3.1.0
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 



From alonbl at redhat.com  Sun Sep  9 21:58:00 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Sun, 9 Sep 2012 17:58:00 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <337981190.5561906.1347224272137.JavaMail.root@redhat.com>
Message-ID: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>

Hello All,

I would like to discuss the method we use to manage default options. I believe it can be significantly simplified.

Please read though and comment.

Thank you,
Alon Bar-Lev

---

CURRENT STATE

Most options are located in three different locations.

Let's explain by example... The FenceQuietTimeBetweenOperationsInSec option.

---
ConfigValues.java
---
public enum ConfigValues {
<snip>
    @Reloadable
    @TypeConverterAttribute(Integer.class)
    @DefaultValueAttribute("180")
    FenceQuietTimeBetweenOperationsInSec(30),
<snip>
}
---

---
0000_config.sql
---
<snip>
select fn_db_add_config_value('FenceQuietTimeBetweenOperationsInSec','180','general');
<snip>
---

---
engine-config.properties
---
<snip>
FenceQuietTimeBetweenOperationsInSec.type=Integer
FenceQuietTimeBetweenOperationsInSec.validValues=60..600
FenceQuietTimeBetweenOperationsInSec.description="Fence quiet time between operations (in seconds)"
FenceQuietTimeBetweenOperationsInSec.alternateKey=Fence_Quiet_Time
<snip>
---

ConfigValues.java is the base, it defines all options and appropriate metadata as annotations. It defines the option name, type, default value, and if reloadable.

0000_config.sql adds all parameters into the database using their default value, we actually duplicate the parameter name and default value into the sql script. Please note that even if we do not add the parameter to the database the engine infrastructure will use the default value specify at ConfigValues.

engine-config.properties is used as an interface to engine-config utility, a command-line utility that can manipulate engine options. engine-config manages only options that are specified in this property files. Property files specifies user friendly description, valid values, alias, but duplicate the option name and type that already specified in ConfigValues. engine-config will only manage options that exists in database.

QUESTIONS

1. Why do we store default values in database?

>From what I managed to gather, we store default values in database under the assumption that we need to keep defaults when we upgrade from one version to another.

However, in most cases when upgrading an application the new defaults should apply as long as they were not overridden by user. Keeping old default as a new value is an exception that can be taken care of during the upgrade process for selected options.

2. Why do we keep properties file?

In practice we could specify the metadata within the property files as annotations in ConfigValues.java. So why do we actually need the properties file?

>From what I managed to gather, we use the property file as an interface to support personal, to allow adding/removing options exposed to the engine-config utility. An addition of option to the property file exposes it.

SUGGESTED STATE

Establish a single place to manage options.

Do not store default values in database.

Provide alternate method of exposing internal options to engine-config utility.

---
ConfigValues.java
---
public enum ConfigValues {
<snip>
    @Reloadable
    @Public
    @TypeConverterAttribute(Integer.class)
    @Restriction.IntegerRange(60, 600)
    @DefaultValueAttribute("180")
    @Description("Fence quiet time between operations (in seconds)")
    FenceQuietTimeBetweenOperationsInSec(30),
<snip>
}
---

BENEFITS

No duplication of option name, type, default value, etc... between multiple files. One place to role them all.

Simpler upgrade sequence, in most cases as we do want the new defaults to apply.

METHOD

1. Add @Public annotation for ConfigValues, to expose options to engine-config instead of the properties file.

2. Add @Restrict annotation for ConfigValues, instead of validValues of properties file.

3. Add @Description annotation for ConfigValues, instead of description of properties file.

4. Add engine-config --internal parameter to allow get/set/list of none @Public option, instead of the need to update the properties file.

5. Modify enigne-config [set] to add option if does not exist in database and does not match default value.

6. Modify engine-config [set] to delete option if value matches default value.

7. Modify engine-config [get] to retrieve default from ConfigValues if value is missing from database.

8. Create upgrade script which deletes all options with value that matches the default from the database.

IMPLICATIONS

The option alias is removed. Can be added at later time if required using own @Alias annotation. Not sure it is actually required.



From yzaslavs at redhat.com  Mon Sep 10 05:22:11 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Mon, 10 Sep 2012 08:22:11 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
References: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
Message-ID: <504D7903.2000100@redhat.com>



On 09/10/2012 12:58 AM, Alon Bar-Lev wrote:
> Hello All,
>
> I would like to discuss the method we use to manage default options. I believe it can be significantly simplified.
>
> Please read though and comment.
>
> Thank you,
> Alon Bar-Lev
>
> ---
>
> CURRENT STATE
>
> Most options are located in three different locations.
>
> Let's explain by example... The FenceQuietTimeBetweenOperationsInSec option.
>
> ---
> ConfigValues.java
> ---
> public enum ConfigValues {
> <snip>
>      @Reloadable
>      @TypeConverterAttribute(Integer.class)
>      @DefaultValueAttribute("180")
>      FenceQuietTimeBetweenOperationsInSec(30),
> <snip>
> }
> ---
>
> ---
> 0000_config.sql
> ---
> <snip>
> select fn_db_add_config_value('FenceQuietTimeBetweenOperationsInSec','180','general');
> <snip>
> ---
>
> ---
> engine-config.properties
> ---
> <snip>
> FenceQuietTimeBetweenOperationsInSec.type=Integer
> FenceQuietTimeBetweenOperationsInSec.validValues=60..600
> FenceQuietTimeBetweenOperationsInSec.description="Fence quiet time between operations (in seconds)"
> FenceQuietTimeBetweenOperationsInSec.alternateKey=Fence_Quiet_Time
> <snip>
> ---
>
> ConfigValues.java is the base, it defines all options and appropriate metadata as annotations. It defines the option name, type, default value, and if reloadable.
>
> 0000_config.sql adds all parameters into the database using their default value, we actually duplicate the parameter name and default value into the sql script. Please note that even if we do not add the parameter to the database the engine infrastructure will use the default value specify at ConfigValues.
>
> engine-config.properties is used as an interface to engine-config utility, a command-line utility that can manipulate engine options. engine-config manages only options that are specified in this property files. Property files specifies user friendly description, valid values, alias, but duplicate the option name and type that already specified in ConfigValues. engine-config will only manage options that exists in database.
>
> QUESTIONS
>
> 1. Why do we store default values in database?
>
>  From what I managed to gather, we store default values in database under the assumption that we need to keep defaults when we upgrade from one version to another.
>
> However, in most cases when upgrading an application the new defaults should apply as long as they were not overridden by user. Keeping old default as a new value is an exception that can be taken care of during the upgrade process for selected options.
>
> 2. Why do we keep properties file?
>
> In practice we could specify the metadata within the property files as annotations in ConfigValues.java. So why do we actually need the properties file?
>
>  From what I managed to gather, we use the property file as an interface to support personal, to allow adding/removing options exposed to the engine-config utility. An addition of option to the property file exposes it.
>
> SUGGESTED STATE
>
> Establish a single place to manage options.
>
> Do not store default values in database.
>
> Provide alternate method of exposing internal options to engine-config utility.
>
> ---
> ConfigValues.java
> ---
> public enum ConfigValues {
> <snip>
>      @Reloadable
>      @Public
>      @TypeConverterAttribute(Integer.class)
>      @Restriction.IntegerRange(60, 600)
This definition will probably not work, should be
@IntegerRestrictionRange(60,600)
>      @DefaultValueAttribute("180")
>      @Description("Fence quiet time between operations (in seconds)")
>      FenceQuietTimeBetweenOperationsInSec(30),
> <snip>
> }
> ---
>
> BENEFITS
>
> No duplication of option name, type, default value, etc... between multiple files. One place to role them all.
>
> Simpler upgrade sequence, in most cases as we do want the new defaults to apply.
>
> METHOD
>
> 1. Add @Public annotation for ConfigValues, to expose options to engine-config instead of the properties file.

I would still recommend in this case to have an optional properties file 
that will allow you to override the behaviour of the @Public annotation 
(i.e - cancel it).
In evolution of java frameworks from the stage where metadata was 
provided via external configuration file (XML, properties file ,etc...) 
to a stage where metadata was provided via annotations, in order to 
support changes at metadata without needing to recompile and to support 
backward compatibility, many frameworks simply made the configuration 
file to be optional for purpose of overriding the annotation deceleration

>
> 2. Add @Restrict annotation for ConfigValues, instead of validValues of properties file.

Bare in mind that annotations are limited in the values they can accept. 
Most likely it will work, but we need to check it.

>
> 3. Add @Description annotation for ConfigValues, instead of description of properties file.
>
> 4. Add engine-config --internal parameter to allow get/set/list of none @Public option, instead of the need to update the properties file.
>
> 5. Modify enigne-config [set] to add option if does not exist in database and does not match default value.

>
> 6. Modify engine-config [set] to delete option if value matches default value.
>
> 7. Modify engine-config [get] to retrieve default from ConfigValues if value is missing from database.

I would recommend as a future feature to consider modifying [set] to 
provide an option to reset the value to the default value, as presented 
at ConfigValue
>
> 8. Create upgrade script which deletes all options with value that matches the default from the database.
>
> IMPLICATIONS
>
> The option alias is removed. Can be added at later time if required using own @Alias annotation. Not sure it is actually required.
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>


From lpeer at redhat.com  Mon Sep 10 05:55:32 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Mon, 10 Sep 2012 08:55:32 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
References: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
Message-ID: <504D80D4.8010706@redhat.com>

On 10/09/12 00:58, Alon Bar-Lev wrote:
> Hello All,
> 
> I would like to discuss the method we use to manage default options. I believe it can be significantly simplified.
> 
> Please read though and comment.
> 
> Thank you,
> Alon Bar-Lev
> 
> ---
> 
> CURRENT STATE
> 
> Most options are located in three different locations.
> 
> Let's explain by example... The FenceQuietTimeBetweenOperationsInSec option.
> 
> ---
> ConfigValues.java
> ---
> public enum ConfigValues {
> <snip>
>     @Reloadable
>     @TypeConverterAttribute(Integer.class)
>     @DefaultValueAttribute("180")
>     FenceQuietTimeBetweenOperationsInSec(30),
> <snip>
> }
> ---
> 
> ---
> 0000_config.sql
> ---
> <snip>
> select fn_db_add_config_value('FenceQuietTimeBetweenOperationsInSec','180','general');
> <snip>
> ---
> 
> ---
> engine-config.properties
> ---
> <snip>
> FenceQuietTimeBetweenOperationsInSec.type=Integer
> FenceQuietTimeBetweenOperationsInSec.validValues=60..600
> FenceQuietTimeBetweenOperationsInSec.description="Fence quiet time between operations (in seconds)"
> FenceQuietTimeBetweenOperationsInSec.alternateKey=Fence_Quiet_Time
> <snip>
> ---
> 
> ConfigValues.java is the base, it defines all options and appropriate metadata as annotations. It defines the option name, type, default value, and if reloadable.
> 
> 0000_config.sql adds all parameters into the database using their default value, we actually duplicate the parameter name and default value into the sql script. Please note that even if we do not add the parameter to the database the engine infrastructure will use the default value specify at ConfigValues.
> 
> engine-config.properties is used as an interface to engine-config utility, a command-line utility that can manipulate engine options. engine-config manages only options that are specified in this property files. Property files specifies user friendly description, valid values, alias, but duplicate the option name and type that already specified in ConfigValues. engine-config will only manage options that exists in database.
> 
> QUESTIONS
> 
> 1. Why do we store default values in database?
> 

Our intention so far was to remove default values from the code.

The values in the data base are not default values but the values of the
properties and we keep them per version were we need different values
per version.

The obvious advantage of keeping them in the DB vs. in code is they can
be changed with no compilation required and today you don't need to
restart the application for changing some of them (WIP).

> From what I managed to gather, we store default values in database under the assumption that we need to keep defaults when we upgrade from one version to another.
> 
> However, in most cases when upgrading an application the new defaults should apply as long as they were not overridden by user. Keeping old default as a new value is an exception that can be taken care of during the upgrade process for selected options.
> 
> 2. Why do we keep properties file?


The properties file primary use was to add translation/description for
each property. The reason we store it in the file was that at some point
we'll support multilingual description for each property.

> 
> In practice we could specify the metadata within the property files as annotations in ConfigValues.java. So why do we actually need the properties file?
> 
> From what I managed to gather, we use the property file as an interface to support personal, to allow adding/removing options exposed to the engine-config utility. An addition of option to the property file exposes it.
> 
> SUGGESTED STATE
> 
> Establish a single place to manage options.
> Do not store default values in database.
> 
> Provide alternate method of exposing internal options to engine-config utility.
> 

Maintaining that in a single place would be great.

1. Were do you store the description of each property (and optionally
support multilingual)?

2. Where do you store value per version?

3. I personally find holding values in code cumbersome, usually you end
up with adding option to override the values by entries in the data
base.....


> ---
> ConfigValues.java
> ---
> public enum ConfigValues {
> <snip>
>     @Reloadable
>     @Public
>     @TypeConverterAttribute(Integer.class)
>     @Restriction.IntegerRange(60, 600)
>     @DefaultValueAttribute("180")
>     @Description("Fence quiet time between operations (in seconds)")
>     FenceQuietTimeBetweenOperationsInSec(30),
> <snip>
> }
> ---
> 
> BENEFITS
> 
> No duplication of option name, type, default value, etc... between multiple files. One place to role them all.
> 
> Simpler upgrade sequence, in most cases as we do want the new defaults to apply.
> 
> METHOD
> 
> 1. Add @Public annotation for ConfigValues, to expose options to engine-config instead of the properties file.
> 
> 2. Add @Restrict annotation for ConfigValues, instead of validValues of properties file.
> 
> 3. Add @Description annotation for ConfigValues, instead of description of properties file.
> 
> 4. Add engine-config --internal parameter to allow get/set/list of none @Public option, instead of the need to update the properties file.
> 
> 5. Modify enigne-config [set] to add option if does not exist in database and does not match default value.
> 
> 6. Modify engine-config [set] to delete option if value matches default value.
> 
> 7. Modify engine-config [get] to retrieve default from ConfigValues if value is missing from database.
> 
> 8. Create upgrade script which deletes all options with value that matches the default from the database.
> 
> IMPLICATIONS
> 
> The option alias is removed. Can be added at later time if required using own @Alias annotation. Not sure it is actually required.
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 



From iheim at redhat.com  Mon Sep 10 06:04:00 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 10 Sep 2012 09:04:00 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504D80D4.8010706@redhat.com>
References: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
	<504D80D4.8010706@redhat.com>
Message-ID: <504D82D0.4050208@redhat.com>

On 09/10/2012 08:55 AM, Livnat Peer wrote:
> The obvious advantage of keeping them in the DB vs. in code is they can
> be changed with no compilation required and today you don't need to
> restart the application for changing some of them (WIP).

you can still change them in the db without code by overriding them in 
the db - code is just the default if there is no value in the db, right?



From yzaslavs at redhat.com  Mon Sep 10 06:05:33 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Mon, 10 Sep 2012 09:05:33 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504D82D0.4050208@redhat.com>
References: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
	<504D80D4.8010706@redhat.com> <504D82D0.4050208@redhat.com>
Message-ID: <504D832D.5030202@redhat.com>



On 09/10/2012 09:04 AM, Itamar Heim wrote:
> On 09/10/2012 08:55 AM, Livnat Peer wrote:
>> The obvious advantage of keeping them in the DB vs. in code is they can
>> be changed with no compilation required and today you don't need to
>> restart the application for changing some of them (WIP).
>
> you can still change them in the db without code by overriding them in
> the db - code is just the default if there is no value in the db, right?

Yes, you also get a warning message indicating they cannot be found, so 
the default value is used.

>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel


From iheim at redhat.com  Mon Sep 10 06:09:55 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 10 Sep 2012 09:09:55 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504D832D.5030202@redhat.com>
References: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
	<504D80D4.8010706@redhat.com> <504D82D0.4050208@redhat.com>
	<504D832D.5030202@redhat.com>
Message-ID: <504D8433.2020700@redhat.com>

On 09/10/2012 09:05 AM, Yair Zaslavsky wrote:
>
>
> On 09/10/2012 09:04 AM, Itamar Heim wrote:
>> On 09/10/2012 08:55 AM, Livnat Peer wrote:
>>> The obvious advantage of keeping them in the DB vs. in code is they can
>>> be changed with no compilation required and today you don't need to
>>> restart the application for changing some of them (WIP).
>>
>> you can still change them in the db without code by overriding them in
>> the db - code is just the default if there is no value in the db, right?
>
> Yes, you also get a warning message indicating they cannot be found, so
> the default value is used.

if we remove the defaults from the db for those that don't have per 
version values, i don't think we'll want that warning in the log  not in 
debug mode...


From lpeer at redhat.com  Mon Sep 10 06:19:00 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Mon, 10 Sep 2012 09:19:00 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504D82D0.4050208@redhat.com>
References: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
	<504D80D4.8010706@redhat.com> <504D82D0.4050208@redhat.com>
Message-ID: <504D8654.9040306@redhat.com>

On 10/09/12 09:04, Itamar Heim wrote:
> On 09/10/2012 08:55 AM, Livnat Peer wrote:
>> The obvious advantage of keeping them in the DB vs. in code is they can
>> be changed with no compilation required and today you don't need to
>> restart the application for changing some of them (WIP).
> 
> you can still change them in the db without code by overriding them in
> the db - code is just the default if there is no value in the db, right?
> 

copy paste from my previous mail -

"
3. I personally find holding values in code cumbersome, usually you end
up with adding option to override the values by entries in the data
base.....
"

if you are going to maintain a table in the data base why start with
code from the first place....



From alonbl at redhat.com  Mon Sep 10 06:30:51 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 10 Sep 2012 02:30:51 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504D8654.9040306@redhat.com>
Message-ID: <169905926.5586081.1347258651077.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Itamar Heim" <iheim at redhat.com>
> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, engine-devel at ovirt.org
> Sent: Monday, September 10, 2012 9:19:00 AM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> On 10/09/12 09:04, Itamar Heim wrote:
> > On 09/10/2012 08:55 AM, Livnat Peer wrote:
> >> The obvious advantage of keeping them in the DB vs. in code is
> >> they can
> >> be changed with no compilation required and today you don't need
> >> to
> >> restart the application for changing some of them (WIP).
> > 
> > you can still change them in the db without code by overriding them
> > in
> > the db - code is just the default if there is no value in the db,
> > right?
> > 
> 
> copy paste from my previous mail -
> 
> "
> 3. I personally find holding values in code cumbersome, usually you
> end
> up with adding option to override the values by entries in the data
> base.....
> "
> 
> if you are going to maintain a table in the data base why start with
> code from the first place....
> 
> 

Because upgrading an application becomes more complex, as you need not only install application binaries (rpm), but also touch the database.

Touching the database is a failure potential.

I will get back to your previous response later, just wanted to quick comment this one.

However, if you believe that any option needs to be in database, then the metadata (description, restriction, internal flag, type, default) should also be moved to the database, into its own table, so that vdc_options will not contain any default, so that application can enjoy modifying defaults upon upgrade, and there will be no duplicate between code and database.

Microsoft "Registry" is a good example of application options handling. The "Registry" holds only non-default values, allowing the application to evolve (both in term of more variables and in term of different defaults), without actually touching the "Registry".

Alon.


From yzaslavs at redhat.com  Mon Sep 10 06:34:07 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Mon, 10 Sep 2012 09:34:07 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504D8433.2020700@redhat.com>
References: <2140443832.5562593.1347227880953.JavaMail.root@redhat.com>
	<504D80D4.8010706@redhat.com> <504D82D0.4050208@redhat.com>
	<504D832D.5030202@redhat.com> <504D8433.2020700@redhat.com>
Message-ID: <504D89DF.5020603@redhat.com>



On 09/10/2012 09:09 AM, Itamar Heim wrote:
> On 09/10/2012 09:05 AM, Yair Zaslavsky wrote:
>>
>>
>> On 09/10/2012 09:04 AM, Itamar Heim wrote:
>>> On 09/10/2012 08:55 AM, Livnat Peer wrote:
>>>> The obvious advantage of keeping them in the DB vs. in code is they can
>>>> be changed with no compilation required and today you don't need to
>>>> restart the application for changing some of them (WIP).
>>>
>>> you can still change them in the db without code by overriding them in
>>> the db - code is just the default if there is no value in the db, right?
>>
>> Yes, you also get a warning message indicating they cannot be found, so
>> the default value is used.
>
> if we remove the defaults from the db for those that don't have per
> version values, i don't think we'll want that warning in the log  not in
> debug mode...

I agree.


From lpeer at redhat.com  Mon Sep 10 07:01:37 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Mon, 10 Sep 2012 10:01:37 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <169905926.5586081.1347258651077.JavaMail.root@redhat.com>
References: <169905926.5586081.1347258651077.JavaMail.root@redhat.com>
Message-ID: <504D9051.1030706@redhat.com>

On 10/09/12 09:30, Alon Bar-Lev wrote:
> 
> 
> ----- Original Message -----
>> From: "Livnat Peer" <lpeer at redhat.com>
>> To: "Itamar Heim" <iheim at redhat.com>
>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, engine-devel at ovirt.org
>> Sent: Monday, September 10, 2012 9:19:00 AM
>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
>>
>> On 10/09/12 09:04, Itamar Heim wrote:
>>> On 09/10/2012 08:55 AM, Livnat Peer wrote:
>>>> The obvious advantage of keeping them in the DB vs. in code is
>>>> they can
>>>> be changed with no compilation required and today you don't need
>>>> to
>>>> restart the application for changing some of them (WIP).
>>>
>>> you can still change them in the db without code by overriding them
>>> in
>>> the db - code is just the default if there is no value in the db,
>>> right?
>>>
>>
>> copy paste from my previous mail -
>>
>> "
>> 3. I personally find holding values in code cumbersome, usually you
>> end
>> up with adding option to override the values by entries in the data
>> base.....
>> "
>>
>> if you are going to maintain a table in the data base why start with
>> code from the first place....
>>
>>
> 
> Because upgrading an application becomes more complex, as you need not only install application binaries (rpm), but also touch the database.
> 

I don't think you can avoid upgrading the data base, regardless of the
the configuration table.


> Touching the database is a failure potential.
> 
> I will get back to your previous response later, just wanted to quick comment this one.
> 
> However, if you believe that any option needs to be in database, then the metadata (description, restriction, internal flag, type, default) should also be moved to the database, into its own table, so that vdc_options will not contain any default, so that application can enjoy modifying defaults upon upgrade, and there will be no duplicate between code and database.
> 

I would like to have everything stored in the data base.
We just need to do it (which we did not have time to do so far) and we
need to think of solution for the description field,(multilingual
support is typically done via properties file).

> Microsoft "Registry" is a good example of application options handling. The "Registry" holds only non-default values, allowing the application to evolve (both in term of more variables and in term of different defaults), without actually touching the "Registry".
> 
> Alon.
> 



From alonbl at redhat.com  Mon Sep 10 08:04:16 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 10 Sep 2012 04:04:16 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504D7903.2000100@redhat.com>
Message-ID: <1575953118.5602830.1347264256741.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Yair Zaslavsky" <yzaslavs at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Monday, September 10, 2012 8:22:11 AM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> 
> 
> On 09/10/2012 12:58 AM, Alon Bar-Lev wrote:
> > Hello All,
> >
> > I would like to discuss the method we use to manage default
> > options. I believe it can be significantly simplified.
> >
> > Please read though and comment.
> >
> > Thank you,
> > Alon Bar-Lev
> >
> > ---
> >
> > CURRENT STATE
> >
> > Most options are located in three different locations.
> >
> > Let's explain by example... The
> > FenceQuietTimeBetweenOperationsInSec option.
> >
> > ---
> > ConfigValues.java
> > ---
> > public enum ConfigValues {
> > <snip>
> >      @Reloadable
> >      @TypeConverterAttribute(Integer.class)
> >      @DefaultValueAttribute("180")
> >      FenceQuietTimeBetweenOperationsInSec(30),
> > <snip>
> > }
> > ---
> >
> > ---
> > 0000_config.sql
> > ---
> > <snip>
> > select
> > fn_db_add_config_value('FenceQuietTimeBetweenOperationsInSec','180','general');
> > <snip>
> > ---
> >
> > ---
> > engine-config.properties
> > ---
> > <snip>
> > FenceQuietTimeBetweenOperationsInSec.type=Integer
> > FenceQuietTimeBetweenOperationsInSec.validValues=60..600
> > FenceQuietTimeBetweenOperationsInSec.description="Fence quiet time
> > between operations (in seconds)"
> > FenceQuietTimeBetweenOperationsInSec.alternateKey=Fence_Quiet_Time
> > <snip>
> > ---
> >
> > ConfigValues.java is the base, it defines all options and
> > appropriate metadata as annotations. It defines the option name,
> > type, default value, and if reloadable.
> >
> > 0000_config.sql adds all parameters into the database using their
> > default value, we actually duplicate the parameter name and
> > default value into the sql script. Please note that even if we do
> > not add the parameter to the database the engine infrastructure
> > will use the default value specify at ConfigValues.
> >
> > engine-config.properties is used as an interface to engine-config
> > utility, a command-line utility that can manipulate engine
> > options. engine-config manages only options that are specified in
> > this property files. Property files specifies user friendly
> > description, valid values, alias, but duplicate the option name
> > and type that already specified in ConfigValues. engine-config
> > will only manage options that exists in database.
> >
> > QUESTIONS
> >
> > 1. Why do we store default values in database?
> >
> >  From what I managed to gather, we store default values in database
> >  under the assumption that we need to keep defaults when we
> >  upgrade from one version to another.
> >
> > However, in most cases when upgrading an application the new
> > defaults should apply as long as they were not overridden by user.
> > Keeping old default as a new value is an exception that can be
> > taken care of during the upgrade process for selected options.
> >
> > 2. Why do we keep properties file?
> >
> > In practice we could specify the metadata within the property files
> > as annotations in ConfigValues.java. So why do we actually need
> > the properties file?
> >
> >  From what I managed to gather, we use the property file as an
> >  interface to support personal, to allow adding/removing options
> >  exposed to the engine-config utility. An addition of option to
> >  the property file exposes it.
> >
> > SUGGESTED STATE
> >
> > Establish a single place to manage options.
> >
> > Do not store default values in database.
> >
> > Provide alternate method of exposing internal options to
> > engine-config utility.
> >
> > ---
> > ConfigValues.java
> > ---
> > public enum ConfigValues {
> > <snip>
> >      @Reloadable
> >      @Public
> >      @TypeConverterAttribute(Integer.class)
> >      @Restriction.IntegerRange(60, 600)
> This definition will probably not work, should be
> @IntegerRestrictionRange(60,600)
> >      @DefaultValueAttribute("180")
> >      @Description("Fence quiet time between operations (in
> >      seconds)")
> >      FenceQuietTimeBetweenOperationsInSec(30),
> > <snip>
> > }
> > ---
> >
> > BENEFITS
> >
> > No duplication of option name, type, default value, etc... between
> > multiple files. One place to role them all.
> >
> > Simpler upgrade sequence, in most cases as we do want the new
> > defaults to apply.
> >
> > METHOD
> >
> > 1. Add @Public annotation for ConfigValues, to expose options to
> > engine-config instead of the properties file.
> 
> I would still recommend in this case to have an optional properties
> file
> that will allow you to override the behaviour of the @Public
> annotation
> (i.e - cancel it).

Why would you want to cancel it?
I understand why you want to expose more...
But cancel?
I don't mind adding a /etc/<something> to allow exposing more, but why adding --internal parameter to engine-config is not enough?

> In evolution of java frameworks from the stage where metadata was
> provided via external configuration file (XML, properties file
> ,etc...)
> to a stage where metadata was provided via annotations, in order to
> support changes at metadata without needing to recompile and to
> support
> backward compatibility, many frameworks simply made the configuration
> file to be optional for purpose of overriding the annotation
> deceleration

I don't mind to use XML for holding metadata, or any format, as long as we have single location to declare new option.

We can even use a table within database...

*BUT* the metadata should be in one place, and there should be no default values within the options tables so an update to the metadata will be in effect as long as the user had not overridden the default.

> 
> >
> > 2. Add @Restrict annotation for ConfigValues, instead of
> > validValues of properties file.
> 
> Bare in mind that annotations are limited in the values they can
> accept.
> Most likely it will work, but we need to check it.

Sure...

> 
> >
> > 3. Add @Description annotation for ConfigValues, instead of
> > description of properties file.
> >
> > 4. Add engine-config --internal parameter to allow get/set/list of
> > none @Public option, instead of the need to update the properties
> > file.
> >
> > 5. Modify enigne-config [set] to add option if does not exist in
> > database and does not match default value.
> 
> >
> > 6. Modify engine-config [set] to delete option if value matches
> > default value.
> >
> > 7. Modify engine-config [get] to retrieve default from ConfigValues
> > if value is missing from database.
> 
> I would recommend as a future feature to consider modifying [set] to
> provide an option to reset the value to the default value, as
> presented
> at ConfigValue

Good idea! so we can have --set "[default]" or similar to revert to default value.

> >
> > 8. Create upgrade script which deletes all options with value that
> > matches the default from the database.
> >
> > IMPLICATIONS
> >
> > The option alias is removed. Can be added at later time if required
> > using own @Alias annotation. Not sure it is actually required.
> >
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> >
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From alonbl at redhat.com  Mon Sep 10 08:30:40 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 10 Sep 2012 04:30:40 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504D80D4.8010706@redhat.com>
Message-ID: <1489794069.5607482.1347265840945.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Monday, September 10, 2012 8:55:32 AM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> On 10/09/12 00:58, Alon Bar-Lev wrote:
> > Hello All,
> > 
> > I would like to discuss the method we use to manage default
> > options. I believe it can be significantly simplified.
> > 
> > Please read though and comment.
> > 
> > Thank you,
> > Alon Bar-Lev
> > 
> > ---
> > 
> > CURRENT STATE
> > 
> > Most options are located in three different locations.
> > 
> > Let's explain by example... The
> > FenceQuietTimeBetweenOperationsInSec option.
> > 
> > ---
> > ConfigValues.java
> > ---
> > public enum ConfigValues {
> > <snip>
> >     @Reloadable
> >     @TypeConverterAttribute(Integer.class)
> >     @DefaultValueAttribute("180")
> >     FenceQuietTimeBetweenOperationsInSec(30),
> > <snip>
> > }
> > ---
> > 
> > ---
> > 0000_config.sql
> > ---
> > <snip>
> > select
> > fn_db_add_config_value('FenceQuietTimeBetweenOperationsInSec','180','general');
> > <snip>
> > ---
> > 
> > ---
> > engine-config.properties
> > ---
> > <snip>
> > FenceQuietTimeBetweenOperationsInSec.type=Integer
> > FenceQuietTimeBetweenOperationsInSec.validValues=60..600
> > FenceQuietTimeBetweenOperationsInSec.description="Fence quiet time
> > between operations (in seconds)"
> > FenceQuietTimeBetweenOperationsInSec.alternateKey=Fence_Quiet_Time
> > <snip>
> > ---
> > 
> > ConfigValues.java is the base, it defines all options and
> > appropriate metadata as annotations. It defines the option name,
> > type, default value, and if reloadable.
> > 
> > 0000_config.sql adds all parameters into the database using their
> > default value, we actually duplicate the parameter name and
> > default value into the sql script. Please note that even if we do
> > not add the parameter to the database the engine infrastructure
> > will use the default value specify at ConfigValues.
> > 
> > engine-config.properties is used as an interface to engine-config
> > utility, a command-line utility that can manipulate engine
> > options. engine-config manages only options that are specified in
> > this property files. Property files specifies user friendly
> > description, valid values, alias, but duplicate the option name
> > and type that already specified in ConfigValues. engine-config
> > will only manage options that exists in database.
> > 
> > QUESTIONS
> > 
> > 1. Why do we store default values in database?
> > 
> 
> Our intention so far was to remove default values from the code.
> 
> The values in the data base are not default values but the values of
> the
> properties and we keep them per version were we need different values
> per version.
> 
> The obvious advantage of keeping them in the DB vs. in code is they
> can
> be changed with no compilation required and today you don't need to
> restart the application for changing some of them (WIP).

The above statement is true if you use database or any other format, such as XML file for the metadata.

As long as there is no duplication between code and metadata storage, and as long as the default values are not stored within the options tables, so that after upgrade the new defaults are in effect unless explicitly overridden by user.

Let's assume we would like to avoid compilation. What is the benefit in storing the metadata within the database and not in plain text file / XML file / properties file?

> > From what I managed to gather, we store default values in database
> > under the assumption that we need to keep defaults when we upgrade
> > from one version to another.
> > 
> > However, in most cases when upgrading an application the new
> > defaults should apply as long as they were not overridden by user.
> > Keeping old default as a new value is an exception that can be
> > taken care of during the upgrade process for selected options.
> > 
> > 2. Why do we keep properties file?
> 
> 
> The properties file primary use was to add translation/description
> for
> each property. The reason we store it in the file was that at some
> point
> we'll support multilingual description for each property.

Usually, multilingual has the "C" (latin) within code, while overriding the "other" lingual within language support pack. So that application without any language support pack will speak latin.

The properties file also contains non multilingual items: restrictions and alias, so if the propose is to be lingual, it should contain only description.

> > 
> > In practice we could specify the metadata within the property files
> > as annotations in ConfigValues.java. So why do we actually need
> > the properties file?
> > 
> > From what I managed to gather, we use the property file as an
> > interface to support personal, to allow adding/removing options
> > exposed to the engine-config utility. An addition of option to the
> > property file exposes it.
> > 
> > SUGGESTED STATE
> > 
> > Establish a single place to manage options.
> > Do not store default values in database.
> > 
> > Provide alternate method of exposing internal options to
> > engine-config utility.
> > 
> 
> Maintaining that in a single place would be great.
> 
> 1. Were do you store the description of each property (and optionally
> support multilingual)?

Description (non C multi-lingual) should be separate from other (operative) metadata.

The C (latin) description should be placed within core product metadata. Each other supported linual should be within its own separate file.

This way you install ovirt-engine-lang-pack-es and application speaks es without change in database or any of the installed files.

The search path for description is (1) use application locale, and if not found (2) use the C locale. So when no description available at language pack, the core C is used.

> 2. Where do you store value per version?

When user set value it will be in database no change in that.

The question is where we keep the metadata.

We can keep the metadata within ConfigValues as annotations.
We can keep the metadata within XML or whatever format.
We can keep the metadata in *SEPARATE* table in database.

The benefit of using XML is that you can have one common metadata and one per product variant. So that metadata search path can be (1) product variant, and if not found (2) use the common metadata.

Same logic can be used within database only that it will be somewhat more complex to implement.
 
> 3. I personally find holding values in code cumbersome, usually you
> end
> up with adding option to override the values by entries in the data
> base.....

Right,
This is what default is all about.
However, when introducing a new variable, the database is left as-is.
When changing default value that was not overridden by user, the database is left as-is.
And... as time goes by, developers collect the common use case, to apply into next version, reducing the need to override defaults.

> I would like to have everything stored in the data base.
> We just need to do it (which we did not have time to do so far) and
> we
> need to think of solution for the description field,(multilingual
> support is typically done via properties file).

I am curios to read why database is better mechanism for static metadata than plain XML.

> 
> > ---
> > ConfigValues.java
> > ---
> > public enum ConfigValues {
> > <snip>
> >     @Reloadable
> >     @Public
> >     @TypeConverterAttribute(Integer.class)
> >     @Restriction.IntegerRange(60, 600)
> >     @DefaultValueAttribute("180")
> >     @Description("Fence quiet time between operations (in
> >     seconds)")
> >     FenceQuietTimeBetweenOperationsInSec(30),
> > <snip>
> > }
> > ---
> > 
> > BENEFITS
> > 
> > No duplication of option name, type, default value, etc... between
> > multiple files. One place to role them all.
> > 
> > Simpler upgrade sequence, in most cases as we do want the new
> > defaults to apply.
> > 
> > METHOD
> > 
> > 1. Add @Public annotation for ConfigValues, to expose options to
> > engine-config instead of the properties file.
> > 
> > 2. Add @Restrict annotation for ConfigValues, instead of
> > validValues of properties file.
> > 
> > 3. Add @Description annotation for ConfigValues, instead of
> > description of properties file.
> > 
> > 4. Add engine-config --internal parameter to allow get/set/list of
> > none @Public option, instead of the need to update the properties
> > file.
> > 
> > 5. Modify enigne-config [set] to add option if does not exist in
> > database and does not match default value.
> > 
> > 6. Modify engine-config [set] to delete option if value matches
> > default value.
> > 
> > 7. Modify engine-config [get] to retrieve default from ConfigValues
> > if value is missing from database.
> > 
> > 8. Create upgrade script which deletes all options with value that
> > matches the default from the database.
> > 
> > IMPLICATIONS
> > 
> > The option alias is removed. Can be added at later time if required
> > using own @Alias annotation. Not sure it is actually required.
> > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 
> 


From lhornyak at redhat.com  Mon Sep 10 12:51:59 2012
From: lhornyak at redhat.com (Laszlo Hornyak)
Date: Mon, 10 Sep 2012 08:51:59 -0400 (EDT)
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <1219751295.16326231.1347281142395.JavaMail.root@redhat.com>
Message-ID: <200947044.16330358.1347281519376.JavaMail.root@redhat.com>

hi,

I am trying to change a behavior in vdsm. When you pass 100% load on a VM, it will stop reporting further load and will keep telling 100% until the load drops under 100% again in it's cpuIdle information. This is totally correct if you have only single-cpu VM's, but it is false when you have multiple vcpu's, I think the cpuIdle information should not be on a 0-100 scale, but on a 0-100*vcpus scale.

So I submitted this patch to vdsm: http://gerrit.ovirt.org/#/c/7892/2 and Dan pointed out that some functionality may depend on the value in the 0-100 interval. For me it seems it is ignored and the load is calculated only from sysCpu + userCpu. Does anyone build on the cpuIdle value?

Thanks,
Laszlo


From dfediuck at redhat.com  Mon Sep 10 16:03:20 2012
From: dfediuck at redhat.com (Doron Fediuck)
Date: Mon, 10 Sep 2012 12:03:20 -0400 (EDT)
Subject: [Engine-devel] Java EE 7 roadmap changes
Message-ID: <1781404708.8356635.1347293000140.JavaMail.root@redhat.com>

https://blogs.oracle.com/theaquarium/entry/java_ee_7_roadmap


From djasa at redhat.com  Mon Sep 10 16:14:03 2012
From: djasa at redhat.com (David =?UTF-8?Q?Ja=C5=A1a?=)
Date: Mon, 10 Sep 2012 18:14:03 +0200
Subject: [Engine-devel] i18n status / availability of pot files for
	translators
Message-ID: <1347293643.11758.22.camel@dhcp-29-7.brq.redhat.com>

Hi,

I was wondering about $SUBJ. When will translators be able to lay their
hands on the initial .pot files, sometimes around 3.2 freeze?

The only mention of translation I've seen is in release notes of 3.1
promising translations to be included in 3.2, so it would be quite nice
to give translators some time to work on them...

David

-- 

David Ja?a, RHCE

SPICE QE based in Brno
GPG Key:     22C33E24 
Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24





From iheim at redhat.com  Mon Sep 10 20:21:57 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 10 Sep 2012 23:21:57 +0300
Subject: [Engine-devel] i18n status / availability of pot files for
	translators
In-Reply-To: <1347293643.11758.22.camel@dhcp-29-7.brq.redhat.com>
References: <1347293643.11758.22.camel@dhcp-29-7.brq.redhat.com>
Message-ID: <504E4BE5.9070602@redhat.com>

On 09/10/2012 07:14 PM, David Ja?a wrote:
> Hi,
>
> I was wondering about $SUBJ. When will translators be able to lay their
> hands on the initial .pot files, sometimes around 3.2 freeze?
>
> The only mention of translation I've seen is in release notes of 3.1
> promising translations to be included in 3.2, so it would be quite nice
> to give translators some time to work on them...
>
> David
>

translators already started to work:
https://translate.zanata.org/zanata/project/view/ovirt



From ofrenkel at redhat.com  Tue Sep 11 05:55:01 2012
From: ofrenkel at redhat.com (Omer Frenkel)
Date: Tue, 11 Sep 2012 01:55:01 -0400 (EDT)
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <200947044.16330358.1347281519376.JavaMail.root@redhat.com>
Message-ID: <538784509.8618112.1347342901035.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Laszlo Hornyak" <lhornyak at redhat.com>
> To: "engine-devel" <engine-devel at ovirt.org>
> Sent: Monday, September 10, 2012 3:51:59 PM
> Subject: [Engine-devel] what does engine with cpuIdle?
> 
> hi,
> 
> I am trying to change a behavior in vdsm. When you pass 100% load on
> a VM, it will stop reporting further load and will keep telling 100%
> until the load drops under 100% again in it's cpuIdle information.
> This is totally correct if you have only single-cpu VM's, but it is
> false when you have multiple vcpu's, I think the cpuIdle information
> should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> 
> So I submitted this patch to vdsm: http://gerrit.ovirt.org/#/c/7892/2
> and Dan pointed out that some functionality may depend on the value
> in the 0-100 interval. For me it seems it is ignored and the load is
> calculated only from sysCpu + userCpu. Does anyone build on the
> cpuIdle value?
> 
> Thanks,
> Laszlo
>

you are right, engine doesn't save cpuIdle for vm,
so it's not in use in the engine.
 _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From lpeer at redhat.com  Tue Sep 11 06:21:31 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 11 Sep 2012 09:21:31 +0300
Subject: [Engine-devel] network subnet
In-Reply-To: <571855213.4031461.1346357490996.JavaMail.root@redhat.com>
References: <571855213.4031461.1346357490996.JavaMail.root@redhat.com>
Message-ID: <504ED86B.7050005@redhat.com>

On 30/08/12 23:11, Alon Bar-Lev wrote:
> 
> 
> ----- Original Message -----
>> From: "Livnat Peer" <lpeer at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Thursday, August 30, 2012 10:16:05 PM
>> Subject: Re: [Engine-devel] network subnet
>>
>> On 30/08/12 21:39, Alon Bar-Lev wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Sent: Thursday, August 30, 2012 3:22:29 PM
>>>> Subject: [Engine-devel] network subnet
>>>>
>>>> Hi All,
>>>>
>>>> Today when a user wants to define a network subnet mask, he does
>>>> it
>>>> when
>>>> he attaches the network to a host NIC.
>>>>
>>>> I was wondering if there is a reason not to define the network
>>>> subnet
>>>> on
>>>> the logical network entity (Data center level).
>>>>
>>>> Thanks, Livnat
>>>
>>> Hello,
>>>
>>> I am sorry, maybe I do not understand... the IP scheme enforces the
>>> use of address mask in order to properly route packets.
>>
>> of course. My proposal is related to our user usage of the system.
>> Today
>> ovirt user, who wants to define a network subnet, has to type the
>> subnet
>> per host (per network), I think the user should only define it once
>> on
>> the logical network entity in the Data Center.
>> Propagating the value to all hosts is needed but it should be our
>> internal implementation detail.
>>
>>>
>>> Network mask is used in any case, I guess it can be dropped from
>>> configuration in favour of using the address class as mask, is
>>> that what you suggest?
>>>
>>
>> No, hope the above paragraph made it more clear.
>>
> 
> Hello,
> 
> Then you assume that a logical network, which is actually layer 2 network in our implementation, has layer 3 characteristics, right?
> 
> In our current implementation "data center logical network" is pure layer 2 segment aka layer 2 broadcast domain.
> 
> One can use the same logical network for multiple layer 3 segments, which is totally valid and consistent with standard physical layer 2 setup.
> 
> Unless I am missing something crucial, I would suggest to keep the consistent physical->virtual mapping, unless we emulate layer 3 switching. Layer 2 does not have layer 3 characteristics.
> 
> Regards,
> Alon.
> 

Generally I agree with what you wrote.

I would like to open for discussion the definition mentioned above that
a logical network is a layer 2 broadcast domain.

We have 2 types of logical networks, VM networks and 'other' (host
functionality?) networks.

When talking about VM networks, I think the definition above is
accurate, the guests using the network should get a layer 2 broadcast
domain.
It can be implemented over a single (physical) layer 2 domain or it can
be implemented over multiple (physical) layer 2 domains using
technologies like tunneling or vxlan.

When talking about other networks like storage, display, migration,
etc. we are actually discussing a network which represent a common
functionality the hosts are using but I am not sure guaranteeing a layer
2 broadcast domain is interesting for such network.

Going forward I expect we'll associate layer 3 services with logical
networks. For example DHCP, DNS, LB etc.

Any thoughts/comments on the above are welcomed.


Thanks, Livnat


From danken at redhat.com  Tue Sep 11 07:22:15 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Tue, 11 Sep 2012 10:22:15 +0300
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <538784509.8618112.1347342901035.JavaMail.root@redhat.com>
References: <200947044.16330358.1347281519376.JavaMail.root@redhat.com>
	<538784509.8618112.1347342901035.JavaMail.root@redhat.com>
Message-ID: <20120911072215.GL4630@redhat.com>

On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> 
> 
> ----- Original Message -----
> > From: "Laszlo Hornyak" <lhornyak at redhat.com>
> > To: "engine-devel" <engine-devel at ovirt.org>
> > Sent: Monday, September 10, 2012 3:51:59 PM
> > Subject: [Engine-devel] what does engine with cpuIdle?
> > 
> > hi,
> > 
> > I am trying to change a behavior in vdsm. When you pass 100% load on
> > a VM, it will stop reporting further load and will keep telling 100%
> > until the load drops under 100% again in it's cpuIdle information.
> > This is totally correct if you have only single-cpu VM's, but it is
> > false when you have multiple vcpu's, I think the cpuIdle information
> > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > 
> > So I submitted this patch to vdsm: http://gerrit.ovirt.org/#/c/7892/2
> > and Dan pointed out that some functionality may depend on the value
> > in the 0-100 interval. For me it seems it is ignored and the load is
> > calculated only from sysCpu + userCpu. Does anyone build on the
> > cpuIdle value?
> > 
> > Thanks,
> > Laszlo
> >
> 
> you are right, engine doesn't save cpuIdle for vm,
> so it's not in use in the engine.

Laszlo, in this case, I think it would be best to drop this bogus piece
of information.

Dan.


From lpeer at redhat.com  Tue Sep 11 10:28:46 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Tue, 11 Sep 2012 13:28:46 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <1489794069.5607482.1347265840945.JavaMail.root@redhat.com>
References: <1489794069.5607482.1347265840945.JavaMail.root@redhat.com>
Message-ID: <504F125E.4050606@redhat.com>

On 10/09/12 11:30, Alon Bar-Lev wrote:
> 
> 
> ----- Original Message -----
>> From: "Livnat Peer" <lpeer at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Monday, September 10, 2012 8:55:32 AM
>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
>>
>> On 10/09/12 00:58, Alon Bar-Lev wrote:
>>> Hello All,
>>>
>>> I would like to discuss the method we use to manage default
>>> options. I believe it can be significantly simplified.
>>>
>>> Please read though and comment.
>>>
>>> Thank you,
>>> Alon Bar-Lev
>>>
>>> ---
>>>
>>> CURRENT STATE
>>>
>>> Most options are located in three different locations.
>>>
>>> Let's explain by example... The
>>> FenceQuietTimeBetweenOperationsInSec option.
>>>
>>> ---
>>> ConfigValues.java
>>> ---
>>> public enum ConfigValues {
>>> <snip>
>>>     @Reloadable
>>>     @TypeConverterAttribute(Integer.class)
>>>     @DefaultValueAttribute("180")
>>>     FenceQuietTimeBetweenOperationsInSec(30),
>>> <snip>
>>> }
>>> ---
>>>
>>> ---
>>> 0000_config.sql
>>> ---
>>> <snip>
>>> select
>>> fn_db_add_config_value('FenceQuietTimeBetweenOperationsInSec','180','general');
>>> <snip>
>>> ---
>>>
>>> ---
>>> engine-config.properties
>>> ---
>>> <snip>
>>> FenceQuietTimeBetweenOperationsInSec.type=Integer
>>> FenceQuietTimeBetweenOperationsInSec.validValues=60..600
>>> FenceQuietTimeBetweenOperationsInSec.description="Fence quiet time
>>> between operations (in seconds)"
>>> FenceQuietTimeBetweenOperationsInSec.alternateKey=Fence_Quiet_Time
>>> <snip>
>>> ---
>>>
>>> ConfigValues.java is the base, it defines all options and
>>> appropriate metadata as annotations. It defines the option name,
>>> type, default value, and if reloadable.
>>>
>>> 0000_config.sql adds all parameters into the database using their
>>> default value, we actually duplicate the parameter name and
>>> default value into the sql script. Please note that even if we do
>>> not add the parameter to the database the engine infrastructure
>>> will use the default value specify at ConfigValues.
>>>
>>> engine-config.properties is used as an interface to engine-config
>>> utility, a command-line utility that can manipulate engine
>>> options. engine-config manages only options that are specified in
>>> this property files. Property files specifies user friendly
>>> description, valid values, alias, but duplicate the option name
>>> and type that already specified in ConfigValues. engine-config
>>> will only manage options that exists in database.
>>>
>>> QUESTIONS
>>>
>>> 1. Why do we store default values in database?
>>>
>>
>> Our intention so far was to remove default values from the code.
>>
>> The values in the data base are not default values but the values of
>> the
>> properties and we keep them per version were we need different values
>> per version.
>>
>> The obvious advantage of keeping them in the DB vs. in code is they
>> can
>> be changed with no compilation required and today you don't need to
>> restart the application for changing some of them (WIP).
> 
> The above statement is true if you use database or any other format, such as XML file for the metadata.
> 
> As long as there is no duplication between code and metadata storage, and as long as the default values are not stored within the options tables, so that after upgrade the new defaults are in effect unless explicitly overridden by user.
> 
> Let's assume we would like to avoid compilation. What is the benefit in storing the metadata within the database and not in plain text file / XML file / properties file?

I think that the main advantage of using data base over XML is that we
already maintain data base and work with it. I have nothing against XML
but I think the question we should ask ourselves is why add another
mechanism like XML file for managing static configuration when we can
use data base?

Also I think there was/is a thought to expose the configuration via the
UI (both for view and edit) I think that if this is still the intention
then working with DB would be easier than XML.

> 
>>> From what I managed to gather, we store default values in database
>>> under the assumption that we need to keep defaults when we upgrade
>>> from one version to another.
>>>
>>> However, in most cases when upgrading an application the new
>>> defaults should apply as long as they were not overridden by user.
>>> Keeping old default as a new value is an exception that can be
>>> taken care of during the upgrade process for selected options.
>>>
>>> 2. Why do we keep properties file?
>>
>>
>> The properties file primary use was to add translation/description
>> for
>> each property. The reason we store it in the file was that at some
>> point
>> we'll support multilingual description for each property.
> 
> Usually, multilingual has the "C" (latin) within code, while overriding the "other" lingual within language support pack. So that application without any language support pack will speak latin.
>
That's not the way I am familiar with, Usually when you work in java you
use ResourceBundle which uses keys in the code and the translation is
provided in external properties file, one per language, distinguished by
their extensions: sample.property.en sample.property.jp etc.

> The properties file also contains non multilingual items: restrictions and alias, so if the propose is to be lingual, it should contain only description.
> 
 I agree.

>>>
>>> In practice we could specify the metadata within the property files
>>> as annotations in ConfigValues.java. So why do we actually need
>>> the properties file?
>>>
>>> From what I managed to gather, we use the property file as an
>>> interface to support personal, to allow adding/removing options
>>> exposed to the engine-config utility. An addition of option to the
>>> property file exposes it.
>>>
>>> SUGGESTED STATE
>>>
>>> Establish a single place to manage options.
>>> Do not store default values in database.
>>>
>>> Provide alternate method of exposing internal options to
>>> engine-config utility.
>>>
>>
>> Maintaining that in a single place would be great.
>>
>> 1. Were do you store the description of each property (and optionally
>> support multilingual)?
> 
> Description (non C multi-lingual) should be separate from other (operative) metadata.
> 
> The C (latin) description should be placed within core product metadata. Each other supported linual should be within its own separate file.
> 
> This way you install ovirt-engine-lang-pack-es and application speaks es without change in database or any of the installed files.
> 
> The search path for description is (1) use application locale, and if not found (2) use the C locale. So when no description available at language pack, the core C is used.

>> 2. Where do you store value per version?
> 
> When user set value it will be in database no change in that.
> 
> The question is where we keep the metadata.
> 
> We can keep the metadata within ConfigValues as annotations.
> We can keep the metadata within XML or whatever format.
> We can keep the metadata in *SEPARATE* table in database.
> 
> The benefit of using XML is that you can have one common metadata and one per product variant. So that metadata search path can be (1) product variant, and if not found (2) use the common metadata.
> 
> Same logic can be used within database only that it will be somewhat more complex to implement.
>  
>> 3. I personally find holding values in code cumbersome, usually you
>> end
>> up with adding option to override the values by entries in the data
>> base.....
> 
> Right,
> This is what default is all about.
> However, when introducing a new variable, the database is left as-is.
> When changing default value that was not overridden by user, the database is left as-is.
> And... as time goes by, developers collect the common use case, to apply into next version, reducing the need to override defaults.
> 
>> I would like to have everything stored in the data base.
>> We just need to do it (which we did not have time to do so far) and
>> we
>> need to think of solution for the description field,(multilingual
>> support is typically done via properties file).
> 
> I am curios to read why database is better mechanism for static metadata than plain XML.
> 
>>
>>> ---
>>> ConfigValues.java
>>> ---
>>> public enum ConfigValues {
>>> <snip>
>>>     @Reloadable
>>>     @Public
>>>     @TypeConverterAttribute(Integer.class)
>>>     @Restriction.IntegerRange(60, 600)
>>>     @DefaultValueAttribute("180")
>>>     @Description("Fence quiet time between operations (in
>>>     seconds)")
>>>     FenceQuietTimeBetweenOperationsInSec(30),
>>> <snip>
>>> }
>>> ---
>>>
>>> BENEFITS
>>>
>>> No duplication of option name, type, default value, etc... between
>>> multiple files. One place to role them all.
>>>
>>> Simpler upgrade sequence, in most cases as we do want the new
>>> defaults to apply.
>>>
>>> METHOD
>>>
>>> 1. Add @Public annotation for ConfigValues, to expose options to
>>> engine-config instead of the properties file.
>>>
>>> 2. Add @Restrict annotation for ConfigValues, instead of
>>> validValues of properties file.
>>>
>>> 3. Add @Description annotation for ConfigValues, instead of
>>> description of properties file.
>>>
>>> 4. Add engine-config --internal parameter to allow get/set/list of
>>> none @Public option, instead of the need to update the properties
>>> file.
>>>
>>> 5. Modify enigne-config [set] to add option if does not exist in
>>> database and does not match default value.
>>>
>>> 6. Modify engine-config [set] to delete option if value matches
>>> default value.
>>>
>>> 7. Modify engine-config [get] to retrieve default from ConfigValues
>>> if value is missing from database.
>>>
>>> 8. Create upgrade script which deletes all options with value that
>>> matches the default from the database.
>>>
>>> IMPLICATIONS
>>>
>>> The option alias is removed. Can be added at later time if required
>>> using own @Alias annotation. Not sure it is actually required.
>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>
>>



From djasa at redhat.com  Tue Sep 11 10:54:32 2012
From: djasa at redhat.com (David =?UTF-8?Q?Ja=C5=A1a?=)
Date: Tue, 11 Sep 2012 12:54:32 +0200
Subject: [Engine-devel] i18n status / availability of pot files for
 translators
In-Reply-To: <504E4BE5.9070602@redhat.com>
References: <1347293643.11758.22.camel@dhcp-29-7.brq.redhat.com>
	<504E4BE5.9070602@redhat.com>
Message-ID: <1347360872.11758.195.camel@dhcp-29-7.brq.redhat.com>

Itamar Heim p??e v Po 10. 09. 2012 v 23:21 +0300:
> On 09/10/2012 07:14 PM, David Ja?a wrote:
> > Hi,
> >
> > I was wondering about $SUBJ. When will translators be able to lay their
> > hands on the initial .pot files, sometimes around 3.2 freeze?
> >
> > The only mention of translation I've seen is in release notes of 3.1
> > promising translations to be included in 3.2, so it would be quite nice
> > to give translators some time to work on them...
> >
> > David
> >
> 
> translators already started to work:
> https://translate.zanata.org/zanata/project/view/ovirt

This is very well hidden, there's no mention of Zanata neither on the
web nor on the wiki nor in the mailing lists...

David

> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

-- 

David Ja?a, RHCE

SPICE QE based in Brno
GPG Key:     22C33E24 
Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24





From lhornyak at redhat.com  Tue Sep 11 11:38:11 2012
From: lhornyak at redhat.com (Laszlo Hornyak)
Date: Tue, 11 Sep 2012 07:38:11 -0400 (EDT)
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <20120911072215.GL4630@redhat.com>
Message-ID: <750873512.16928083.1347363491068.JavaMail.root@redhat.com>


----- Original Message -----
> From: "Dan Kenigsberg" <danken at redhat.com>
> To: "Omer Frenkel" <ofrenkel at redhat.com>
> Cc: "Laszlo Hornyak" <lhornyak at redhat.com>, "engine-devel" <engine-devel at ovirt.org>
> Sent: Tuesday, September 11, 2012 9:22:15 AM
> Subject: Re: [Engine-devel] what does engine with cpuIdle?
> 
> On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> > 
> > 
> > ----- Original Message -----
> > > From: "Laszlo Hornyak" <lhornyak at redhat.com>
> > > To: "engine-devel" <engine-devel at ovirt.org>
> > > Sent: Monday, September 10, 2012 3:51:59 PM
> > > Subject: [Engine-devel] what does engine with cpuIdle?
> > > 
> > > hi,
> > > 
> > > I am trying to change a behavior in vdsm. When you pass 100% load
> > > on
> > > a VM, it will stop reporting further load and will keep telling
> > > 100%
> > > until the load drops under 100% again in it's cpuIdle
> > > information.
> > > This is totally correct if you have only single-cpu VM's, but it
> > > is
> > > false when you have multiple vcpu's, I think the cpuIdle
> > > information
> > > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > > 
> > > So I submitted this patch to vdsm:
> > > http://gerrit.ovirt.org/#/c/7892/2
> > > and Dan pointed out that some functionality may depend on the
> > > value
> > > in the 0-100 interval. For me it seems it is ignored and the load
> > > is
> > > calculated only from sysCpu + userCpu. Does anyone build on the
> > > cpuIdle value?
> > > 
> > > Thanks,
> > > Laszlo
> > >
> > 
> > you are right, engine doesn't save cpuIdle for vm,
> > so it's not in use in the engine.
> 
> Laszlo, in this case, I think it would be best to drop this bogus
> piece
> of information.

Ok.

However, before I abandon this patch: we have a requirement to report cpuSys and cpuUser separately. Afaik in libvirt cpuUser and cpuSys does not include the actual guest time (at least not with KVM), and in this way if we only report cpuSys and cpuUser, the sum does not give the actual load, only a relatively little percentage of it. If we have the cpuIdle information in engine, we can calculate the guest time. Therefore, should I
- include the guest time in cpuSys or cpuUser?
- add another exported field?

And in both case, we will still have to calculate from cpuIdle because libvirt does not tell the guest cpu time :-(

> 
> Dan.
> 


From djasa at redhat.com  Tue Sep 11 11:47:56 2012
From: djasa at redhat.com (David =?UTF-8?Q?Ja=C5=A1a?=)
Date: Tue, 11 Sep 2012 13:47:56 +0200
Subject: [Engine-devel] network subnet
In-Reply-To: <504ED86B.7050005@redhat.com>
References: <571855213.4031461.1346357490996.JavaMail.root@redhat.com>
	<504ED86B.7050005@redhat.com>
Message-ID: <1347364076.11758.250.camel@dhcp-29-7.brq.redhat.com>

Livnat Peer p??e v ?t 11. 09. 2012 v 09:21 +0300:
> On 30/08/12 23:11, Alon Bar-Lev wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Livnat Peer" <lpeer at redhat.com>
> >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >> Cc: engine-devel at ovirt.org
> >> Sent: Thursday, August 30, 2012 10:16:05 PM
> >> Subject: Re: [Engine-devel] network subnet
> >>
> >> On 30/08/12 21:39, Alon Bar-Lev wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Livnat Peer" <lpeer at redhat.com>
> >>>> To: engine-devel at ovirt.org
> >>>> Sent: Thursday, August 30, 2012 3:22:29 PM
> >>>> Subject: [Engine-devel] network subnet
> >>>>
> >>>> Hi All,
> >>>>
> >>>> Today when a user wants to define a network subnet mask, he does
> >>>> it
> >>>> when
> >>>> he attaches the network to a host NIC.
> >>>>
> >>>> I was wondering if there is a reason not to define the network
> >>>> subnet
> >>>> on
> >>>> the logical network entity (Data center level).
> >>>>
> >>>> Thanks, Livnat
> >>>
> >>> Hello,
> >>>
> >>> I am sorry, maybe I do not understand... the IP scheme enforces the
> >>> use of address mask in order to properly route packets.
> >>
> >> of course. My proposal is related to our user usage of the system.
> >> Today
> >> ovirt user, who wants to define a network subnet, has to type the
> >> subnet
> >> per host (per network), I think the user should only define it once
> >> on
> >> the logical network entity in the Data Center.
> >> Propagating the value to all hosts is needed but it should be our
> >> internal implementation detail.
> >>
> >>>
> >>> Network mask is used in any case, I guess it can be dropped from
> >>> configuration in favour of using the address class as mask, is
> >>> that what you suggest?
> >>>
> >>
> >> No, hope the above paragraph made it more clear.
> >>
> > 
> > Hello,
> > 
> > Then you assume that a logical network, which is actually layer 2 network in our implementation, has layer 3 characteristics, right?
> > 
> > In our current implementation "data center logical network" is pure layer 2 segment aka layer 2 broadcast domain.
> > 
> > One can use the same logical network for multiple layer 3 segments, which is totally valid and consistent with standard physical layer 2 setup.
> > 
> > Unless I am missing something crucial, I would suggest to keep the consistent physical->virtual mapping, unless we emulate layer 3 switching. Layer 2 does not have layer 3 characteristics.
> > 
> > Regards,
> > Alon.
> > 
> 
> Generally I agree with what you wrote.
> 
> I would like to open for discussion the definition mentioned above that
> a logical network is a layer 2 broadcast domain.
> 
> We have 2 types of logical networks, VM networks and 'other' (host
> functionality?) networks.
> 
> When talking about VM networks, I think the definition above is
> accurate, the guests using the network should get a layer 2 broadcast
> domain.
> It can be implemented over a single (physical) layer 2 domain or it can
> be implemented over multiple (physical) layer 2 domains using
> technologies like tunneling or vxlan.
> 
> When talking about other networks like storage, display, migration,
> etc. we are actually discussing a network which represent a common
> functionality the hosts are using but I am not sure guaranteeing a layer
> 2 broadcast domain is interesting for such network.
> 

IMHO the requirements for the "logical networks" could be differentiated
depending on their function:
      * management: engine must reach each host on L3
      * storage: each host can reach the storage on L3 (FCP or IP)
      * VM network: all VNICs in one  are in L2 broadcast domain by
        default
      * migration: mutual reachability of hosts on L3
      * display: reachability on from clients
that in turn could make 

Note that engine or hosts can not really test if display network is
working in conditions like split-horizon DNS.

David

> Going forward I expect we'll associate layer 3 services with logical
> networks. For example DHCP, DNS, LB etc.
> 
> Any thoughts/comments on the above are welcomed.
> 
> 
> Thanks, Livnat
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel

-- 

David Ja?a, RHCE

SPICE QE based in Brno
GPG Key:     22C33E24 
Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24





From danken at redhat.com  Tue Sep 11 12:34:13 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Tue, 11 Sep 2012 15:34:13 +0300
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <750873512.16928083.1347363491068.JavaMail.root@redhat.com>
References: <20120911072215.GL4630@redhat.com>
	<750873512.16928083.1347363491068.JavaMail.root@redhat.com>
Message-ID: <20120911123413.GB11836@redhat.com>

On Tue, Sep 11, 2012 at 07:38:11AM -0400, Laszlo Hornyak wrote:
> 
> ----- Original Message -----
> > From: "Dan Kenigsberg" <danken at redhat.com>
> > To: "Omer Frenkel" <ofrenkel at redhat.com>
> > Cc: "Laszlo Hornyak" <lhornyak at redhat.com>, "engine-devel" <engine-devel at ovirt.org>
> > Sent: Tuesday, September 11, 2012 9:22:15 AM
> > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > 
> > On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Laszlo Hornyak" <lhornyak at redhat.com>
> > > > To: "engine-devel" <engine-devel at ovirt.org>
> > > > Sent: Monday, September 10, 2012 3:51:59 PM
> > > > Subject: [Engine-devel] what does engine with cpuIdle?
> > > > 
> > > > hi,
> > > > 
> > > > I am trying to change a behavior in vdsm. When you pass 100% load
> > > > on
> > > > a VM, it will stop reporting further load and will keep telling
> > > > 100%
> > > > until the load drops under 100% again in it's cpuIdle
> > > > information.
> > > > This is totally correct if you have only single-cpu VM's, but it
> > > > is
> > > > false when you have multiple vcpu's, I think the cpuIdle
> > > > information
> > > > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > > > 
> > > > So I submitted this patch to vdsm:
> > > > http://gerrit.ovirt.org/#/c/7892/2
> > > > and Dan pointed out that some functionality may depend on the
> > > > value
> > > > in the 0-100 interval. For me it seems it is ignored and the load
> > > > is
> > > > calculated only from sysCpu + userCpu. Does anyone build on the
> > > > cpuIdle value?
> > > > 
> > > > Thanks,
> > > > Laszlo
> > > >
> > > 
> > > you are right, engine doesn't save cpuIdle for vm,
> > > so it's not in use in the engine.
> > 
> > Laszlo, in this case, I think it would be best to drop this bogus
> > piece
> > of information.
> 
> Ok.
> 
> However, before I abandon this patch:

Why abandon? I've suggested you to keep it, just make it even simpler.

> we have a requirement to report cpuSys and cpuUser separately. Afaik
> in libvirt cpuUser and cpuSys does not include the actual guest time
> (at least not with KVM), and in this way if we only report cpuSys and
> cpuUser, the sum does not give the actual load, only a relatively
> little percentage of it.

I am not sure I understand what you are saying, but afaik, libvirt's
relatively-new
http://libvirt.org/html/libvirt-libvirt.html#virDomainGetCPUStats
reports the cpu time spent by the entire qemu process - in guest and
host modes.

> If we have the cpuIdle information in engine,
> we can calculate the guest time. Therefore, should I - include the
> guest time in cpuSys or cpuUser?
> - add another exported field?
> 
> And in both case, we will still have to calculate from cpuIdle because
> libvirt does not tell the guest cpu time :-(

Now I'm completely at loss. Why should we calculate cpuIdle per VM?
Haven't we agreed that it is useless?

Dan.


From vszocs at redhat.com  Tue Sep 11 12:38:47 2012
From: vszocs at redhat.com (Vojtech Szocs)
Date: Tue, 11 Sep 2012 08:38:47 -0400 (EDT)
Subject: [Engine-devel] UI Plugins configuration
In-Reply-To: <A24D665DC3ECC245A2EA0D9A958F80B62C4952E8@G4W3294.americas.hpqcorp.net>
Message-ID: <1702837334.19927565.1347367127378.JavaMail.root@redhat.com>

Hi Chris,

sorry for my late response, I'm working on PoC patch rev.5 this week, and was thinking about your ideas, and came up with some modifications. Please let me know what you think.

Regarding 'configFile' property in plugin descriptor, maybe we could remove it, in favor of introducing configuration file naming convention '<descriptorFileName>-config.json':
(+) there would be a standard plugin configuration file naming convention [making it simpler for plugin end users]
(+) Engine (WebAdmin servlet) wouldn't have to parse plugin descriptor just to determine configuration file name

This way, reloading plugin descriptors/configuration could work like this:
a, iterate through plugin descriptor [*.json] files in $DATADIR/ui-plugins/ directory, remember 'dateLastModified' value
b, iterate through plugin configuration [<descriptorFileName>-config.json] files in $CONFIGDIR/ui-plugins/ directory, remember 'dateLastModified' value
c, for each plugin descriptor:
   c1, if (descriptorFileNameNotInCache || currentDescriptorDateLastModified > cachedDescriptorDateLastModified) { goto c3; }
   c2, else if (currentConfigDateLastModified > cachedConfigDateLastModified) { goto c4; }
   c3, reload (parse/validate/cache) plugin descriptor
   c4, reload plugin configuration

This is based on following assumptions:
1. plugin descriptor [$DATADIR/ui-plugins/<descriptorFileName>.json] is NOT intended to be modified by end users [packaging system handles this]
   -> we don't expect plugin descriptor data to change too often
2. plugin configuration [$CONFIGDIR/ui-plugins/<descriptorFileName>-config.json] is intended to be modified by end users
   -> we expect plugin configuration data to change more frequently

This is why I proposed the above mentioned reloading process, in favor of reloading everything when any timestamp (descriptor/configuration) changes to a newer value. Let me know what you think.

> As far as synchronization goes, the current code's getPluginDefinitions() loads all of the plugin descriptors and configurations into a new HashMap and then assigns that hashmap to WebadminDynamicHostingServlet.pluginDefinitions.  As long as the JsonNode trees in pluginDefinitions aren't modified after being put into pluginDefinitions, I don't see a synchronization issue.  Successive calls to get data from the pluginDefinitions hashmap may see different data, but they'll never see inconsistent data (e.g. a partially constructed plugin descriptor).

This is correct. 'pluginDefinitions' HashMap state is not expected to be modified, at least for now (e.g. using Collections.unmodifiableMap). But consider following use-case when invoking WebadminDynamicHostingServlet:

[thread #1] nothing needs to be reloaded, writes plugin meta-data into WebAdmin HTML page, writes data for 'plugin1' (1/2 plugins)
[thread #2] both 'plugin1' and 'plugin2' descriptor/configuration has changed, reloads them, writes plugin meta-data into WebAdmin HTML page (both plugins), request finished
[thread #1] writes data for 'plugin2', request finished

As you can see, thread #1 will produce plugin meta-data with 'old' data for 'plugin1', and 'new' data for 'plugin2'. I guess a more appropriate behavior would be that thread #1 will produce 'old' data for both plugins, since nothing new was detected at time when thread #1 checked for changes ("nothing needs to be reloaded"). What do you think?

> However, if we want to add calls to modify the pluginDefinitions at runtime (such as a element in the GUI that disables certain plugins), then we will need synchronization.  We'll also have to concern ourselves with whether any such modifications need to be saved back to the files on disk.

I'd like to avoid such synchronization. Plugin descriptor/configuration data, after being loaded, should be treated as immutable for the given timestamp.

For example, if the user wants to disable pluginX across multiple WebAdmin runs, he can do so via WebAdmin GUI, with "disable pluginX" information persisted in HTML5 local storage (through WebAdmin's ClientStorage abstraction, using HTML5 if possible, falling back to cookies when not available). What do you think?

Regarding GUI -> plugin file changes, I'd also like to avoid it, we could use above mentioned HTML5 local storage for any changes on top of descriptor/configuration data.

Cheers,
Vojtech


----- Original Message -----
From: "Chris Frantz" <Chris.Frantz at hp.com>
To: "Vojtech Szocs" <vszocs at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Wednesday, September 5, 2012 8:52:28 PM
Subject: RE: UI Plugins configuration

Vojtech,

Please go ahead and include my ideas and/or code into your next patch.

With regards to the FIXME, I think using the 'date last modified' is a good idea.  However, the plugin descriptor file can also reference the plugin config file (via the configFile property).  I imagine that adding, removing or reconfiguring plugins will be a relatively rare event, so maybe there is a simpler method:

Remember the newest timestamp of ($DATADIR/ui-plugins/*, $CONFIGDIR/ui-plugins/*) and reload the plugin descriptors and configurations if the newest timestamp changes.

As far as synchronization goes, the current code's getPluginDefinitions() loads all of the plugin descriptors and configurations into a new HashMap and then assigns that hashmap to WebadminDynamicHostingServlet.pluginDefinitions.  As long as the JsonNode trees in pluginDefinitions aren't modified after being put into pluginDefinitions, I don't see a synchronization issue.  Successive calls to get data from the pluginDefinitions hashmap may see different data, but they'll never see inconsistent data (e.g. a partially constructed plugin descriptor).

However, if we want to add calls to modify the pluginDefinitions at runtime (such as a element in the GUI that disables certain plugins), then we will need synchronization.  We'll also have to concern ourselves with whether any such modifications need to be saved back to the files on disk.

--Chris



-----Original Message-----
From: Vojtech Szocs [mailto:vszocs at redhat.com] 
Sent: Tuesday, September 04, 2012 7:46 AM
To: Frantz, Chris
Cc: engine-devel
Subject: Re: UI Plugins configuration

Thank you Chris :)

I'd like to incorporate your ideas into upcoming PoC patch, which will be focused on server-side components of UI plugin infrastructure:
1) servlet that serves plugin-related static resources (plugin host page, actual plugin code, any 3rd party JavaScript libraries, CSS, etc.) from local filesystem
2) class responsible for parsing/validating/caching plugin descriptor information from local filesystem

Regarding the comment in WebadminDynamicHostingServlet that says "FIXME: do we load this everytime, or just once?": maybe we could use the same approach as in servlet containers, which use 'date last modified' attribute of JSP files to trigger recompilation of corresponding servlets. For example, each time WebAdmin host page gets requested through WebadminDynamicHostingServlet, we could iterate over plugin descriptor files (in /usr/share/ovirt-engine/ui-plugins), looking at 'date last modified' attribute, parsing/validating plugin meta-data, and caching it using pluginName + dateLastModified (compound cache key). We also need to synchronize the access to plugin meta-data. What do you think?

After the upcoming PoC patch (focused on server-side components) gets released, UI plugin infrastructure should be pretty much stable and we can focus on adding particular features, such as:
* adding custom sub-tabs
* adding custom context-menu items
* making REST API calls through plugin API
* investigating cross-origin plugin scenario (plugin gets loaded from page sitting on different origin than Engine JBoss instance)

Let me know what you think.

Cheers,
Vojtech


----- Original Message -----
From: "Chris Frantz" <Chris.Frantz at hp.com>
To: "Vojtech Szocs" <vszocs at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Thursday, August 30, 2012 10:03:02 PM
Subject: RE: UI Plugins configuration




Vojtech, 



Here is my patch against WIP-UI-Plugins-PoC-revision-4. I?ve also included 2 dummy plugins in sample.tar.gz. 



--Chris 







From: engine-devel-bounces at ovirt.org [mailto:engine-devel-bounces at ovirt.org] On Behalf Of Frantz, Chris 
Sent: Thursday, August 30, 2012 1:06 PM 
To: Vojtech Szocs 
Cc: engine-devel 
Subject: Re: [Engine-devel] UI Plugins configuration 



Vojtech, 



I agree with your formalized names: 



Plugin Descriptor is the JSON file containing plugin meta-data. The plugin descriptor may also contain the default configuration data. It is located in $DATADIR/ui-plugins. 



Plugin Configuration is the JSON file containing optional plugin configuration info. It is located in $CONFIGDIR/ui-plugins (unless the Plugin Descriptor contains an absolute path). 



Plugin Definition is the JavaScript object used by WebAdmin. In the current implementation, the Plugin Definition contains both the Plugin Descriptor and the Plugin Configuraion. 



Plugin Source Page is the HTML page used to invoke the plugin code and shall be referenced by the plugin descriptor?s ?url? attribute. 



I?ve implemented the config merging you?ve suggested: the structure in configFile gets merged with the structure of ?config?, with the data in configFile winning in the case of duplicate key names. 



BTW, the patch is against ovirt-engine + 0001-WIP-UI-Plugins-PoC-revision-2. 



Let me know what you think, 

--Chris 




From vszocs at redhat.com  Tue Sep 11 12:44:40 2012
From: vszocs at redhat.com (Vojtech Szocs)
Date: Tue, 11 Sep 2012 08:44:40 -0400 (EDT)
Subject: [Engine-devel] UI Plugins configuration
In-Reply-To: <1702837334.19927565.1347367127378.JavaMail.root@redhat.com>
Message-ID: <1888013341.19929070.1347367480472.JavaMail.root@redhat.com>

Hi,

> This is correct. 'pluginDefinitions' HashMap state is not expected to be modified, at least for now (e.g. using Collections.unmodifiableMap).

forgot about changing JsonNode values themselves, within the HashMap :) we should treat them as immutable for now.

Vojtech


----- Original Message -----
From: "Vojtech Szocs" <vszocs at redhat.com>
To: "Chris Frantz" <Chris.Frantz at hp.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Tuesday, September 11, 2012 2:38:47 PM
Subject: Re: [Engine-devel] UI Plugins configuration

Hi Chris,

sorry for my late response, I'm working on PoC patch rev.5 this week, and was thinking about your ideas, and came up with some modifications. Please let me know what you think.

Regarding 'configFile' property in plugin descriptor, maybe we could remove it, in favor of introducing configuration file naming convention '<descriptorFileName>-config.json':
(+) there would be a standard plugin configuration file naming convention [making it simpler for plugin end users]
(+) Engine (WebAdmin servlet) wouldn't have to parse plugin descriptor just to determine configuration file name

This way, reloading plugin descriptors/configuration could work like this:
a, iterate through plugin descriptor [*.json] files in $DATADIR/ui-plugins/ directory, remember 'dateLastModified' value
b, iterate through plugin configuration [<descriptorFileName>-config.json] files in $CONFIGDIR/ui-plugins/ directory, remember 'dateLastModified' value
c, for each plugin descriptor:
   c1, if (descriptorFileNameNotInCache || currentDescriptorDateLastModified > cachedDescriptorDateLastModified) { goto c3; }
   c2, else if (currentConfigDateLastModified > cachedConfigDateLastModified) { goto c4; }
   c3, reload (parse/validate/cache) plugin descriptor
   c4, reload plugin configuration

This is based on following assumptions:
1. plugin descriptor [$DATADIR/ui-plugins/<descriptorFileName>.json] is NOT intended to be modified by end users [packaging system handles this]
   -> we don't expect plugin descriptor data to change too often
2. plugin configuration [$CONFIGDIR/ui-plugins/<descriptorFileName>-config.json] is intended to be modified by end users
   -> we expect plugin configuration data to change more frequently

This is why I proposed the above mentioned reloading process, in favor of reloading everything when any timestamp (descriptor/configuration) changes to a newer value. Let me know what you think.

> As far as synchronization goes, the current code's getPluginDefinitions() loads all of the plugin descriptors and configurations into a new HashMap and then assigns that hashmap to WebadminDynamicHostingServlet.pluginDefinitions.  As long as the JsonNode trees in pluginDefinitions aren't modified after being put into pluginDefinitions, I don't see a synchronization issue.  Successive calls to get data from the pluginDefinitions hashmap may see different data, but they'll never see inconsistent data (e.g. a partially constructed plugin descriptor).

This is correct. 'pluginDefinitions' HashMap state is not expected to be modified, at least for now (e.g. using Collections.unmodifiableMap). But consider following use-case when invoking WebadminDynamicHostingServlet:

[thread #1] nothing needs to be reloaded, writes plugin meta-data into WebAdmin HTML page, writes data for 'plugin1' (1/2 plugins)
[thread #2] both 'plugin1' and 'plugin2' descriptor/configuration has changed, reloads them, writes plugin meta-data into WebAdmin HTML page (both plugins), request finished
[thread #1] writes data for 'plugin2', request finished

As you can see, thread #1 will produce plugin meta-data with 'old' data for 'plugin1', and 'new' data for 'plugin2'. I guess a more appropriate behavior would be that thread #1 will produce 'old' data for both plugins, since nothing new was detected at time when thread #1 checked for changes ("nothing needs to be reloaded"). What do you think?

> However, if we want to add calls to modify the pluginDefinitions at runtime (such as a element in the GUI that disables certain plugins), then we will need synchronization.  We'll also have to concern ourselves with whether any such modifications need to be saved back to the files on disk.

I'd like to avoid such synchronization. Plugin descriptor/configuration data, after being loaded, should be treated as immutable for the given timestamp.

For example, if the user wants to disable pluginX across multiple WebAdmin runs, he can do so via WebAdmin GUI, with "disable pluginX" information persisted in HTML5 local storage (through WebAdmin's ClientStorage abstraction, using HTML5 if possible, falling back to cookies when not available). What do you think?

Regarding GUI -> plugin file changes, I'd also like to avoid it, we could use above mentioned HTML5 local storage for any changes on top of descriptor/configuration data.

Cheers,
Vojtech


----- Original Message -----
From: "Chris Frantz" <Chris.Frantz at hp.com>
To: "Vojtech Szocs" <vszocs at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Wednesday, September 5, 2012 8:52:28 PM
Subject: RE: UI Plugins configuration

Vojtech,

Please go ahead and include my ideas and/or code into your next patch.

With regards to the FIXME, I think using the 'date last modified' is a good idea.  However, the plugin descriptor file can also reference the plugin config file (via the configFile property).  I imagine that adding, removing or reconfiguring plugins will be a relatively rare event, so maybe there is a simpler method:

Remember the newest timestamp of ($DATADIR/ui-plugins/*, $CONFIGDIR/ui-plugins/*) and reload the plugin descriptors and configurations if the newest timestamp changes.

As far as synchronization goes, the current code's getPluginDefinitions() loads all of the plugin descriptors and configurations into a new HashMap and then assigns that hashmap to WebadminDynamicHostingServlet.pluginDefinitions.  As long as the JsonNode trees in pluginDefinitions aren't modified after being put into pluginDefinitions, I don't see a synchronization issue.  Successive calls to get data from the pluginDefinitions hashmap may see different data, but they'll never see inconsistent data (e.g. a partially constructed plugin descriptor).

However, if we want to add calls to modify the pluginDefinitions at runtime (such as a element in the GUI that disables certain plugins), then we will need synchronization.  We'll also have to concern ourselves with whether any such modifications need to be saved back to the files on disk.

--Chris



-----Original Message-----
From: Vojtech Szocs [mailto:vszocs at redhat.com] 
Sent: Tuesday, September 04, 2012 7:46 AM
To: Frantz, Chris
Cc: engine-devel
Subject: Re: UI Plugins configuration

Thank you Chris :)

I'd like to incorporate your ideas into upcoming PoC patch, which will be focused on server-side components of UI plugin infrastructure:
1) servlet that serves plugin-related static resources (plugin host page, actual plugin code, any 3rd party JavaScript libraries, CSS, etc.) from local filesystem
2) class responsible for parsing/validating/caching plugin descriptor information from local filesystem

Regarding the comment in WebadminDynamicHostingServlet that says "FIXME: do we load this everytime, or just once?": maybe we could use the same approach as in servlet containers, which use 'date last modified' attribute of JSP files to trigger recompilation of corresponding servlets. For example, each time WebAdmin host page gets requested through WebadminDynamicHostingServlet, we could iterate over plugin descriptor files (in /usr/share/ovirt-engine/ui-plugins), looking at 'date last modified' attribute, parsing/validating plugin meta-data, and caching it using pluginName + dateLastModified (compound cache key). We also need to synchronize the access to plugin meta-data. What do you think?

After the upcoming PoC patch (focused on server-side components) gets released, UI plugin infrastructure should be pretty much stable and we can focus on adding particular features, such as:
* adding custom sub-tabs
* adding custom context-menu items
* making REST API calls through plugin API
* investigating cross-origin plugin scenario (plugin gets loaded from page sitting on different origin than Engine JBoss instance)

Let me know what you think.

Cheers,
Vojtech


----- Original Message -----
From: "Chris Frantz" <Chris.Frantz at hp.com>
To: "Vojtech Szocs" <vszocs at redhat.com>
Cc: "engine-devel" <engine-devel at ovirt.org>
Sent: Thursday, August 30, 2012 10:03:02 PM
Subject: RE: UI Plugins configuration




Vojtech, 



Here is my patch against WIP-UI-Plugins-PoC-revision-4. I?ve also included 2 dummy plugins in sample.tar.gz. 



--Chris 







From: engine-devel-bounces at ovirt.org [mailto:engine-devel-bounces at ovirt.org] On Behalf Of Frantz, Chris 
Sent: Thursday, August 30, 2012 1:06 PM 
To: Vojtech Szocs 
Cc: engine-devel 
Subject: Re: [Engine-devel] UI Plugins configuration 



Vojtech, 



I agree with your formalized names: 



Plugin Descriptor is the JSON file containing plugin meta-data. The plugin descriptor may also contain the default configuration data. It is located in $DATADIR/ui-plugins. 



Plugin Configuration is the JSON file containing optional plugin configuration info. It is located in $CONFIGDIR/ui-plugins (unless the Plugin Descriptor contains an absolute path). 



Plugin Definition is the JavaScript object used by WebAdmin. In the current implementation, the Plugin Definition contains both the Plugin Descriptor and the Plugin Configuraion. 



Plugin Source Page is the HTML page used to invoke the plugin code and shall be referenced by the plugin descriptor?s ?url? attribute. 



I?ve implemented the config merging you?ve suggested: the structure in configFile gets merged with the structure of ?config?, with the data in configFile winning in the case of duplicate key names. 



BTW, the patch is against ovirt-engine + 0001-WIP-UI-Plugins-PoC-revision-2. 



Let me know what you think, 

--Chris 


_______________________________________________
Engine-devel mailing list
Engine-devel at ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


From lhornyak at redhat.com  Tue Sep 11 12:52:51 2012
From: lhornyak at redhat.com (Laszlo Hornyak)
Date: Tue, 11 Sep 2012 08:52:51 -0400 (EDT)
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <20120911123413.GB11836@redhat.com>
Message-ID: <937059412.16951072.1347367971075.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Dan Kenigsberg" <danken at redhat.com>
> To: "Laszlo Hornyak" <lhornyak at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>, "Omer Frenkel" <ofrenkel at redhat.com>
> Sent: Tuesday, September 11, 2012 2:34:13 PM
> Subject: Re: [Engine-devel] what does engine with cpuIdle?
> 
> On Tue, Sep 11, 2012 at 07:38:11AM -0400, Laszlo Hornyak wrote:
> > 
> > ----- Original Message -----
> > > From: "Dan Kenigsberg" <danken at redhat.com>
> > > To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > Cc: "Laszlo Hornyak" <lhornyak at redhat.com>, "engine-devel"
> > > <engine-devel at ovirt.org>
> > > Sent: Tuesday, September 11, 2012 9:22:15 AM
> > > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > > 
> > > On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > From: "Laszlo Hornyak" <lhornyak at redhat.com>
> > > > > To: "engine-devel" <engine-devel at ovirt.org>
> > > > > Sent: Monday, September 10, 2012 3:51:59 PM
> > > > > Subject: [Engine-devel] what does engine with cpuIdle?
> > > > > 
> > > > > hi,
> > > > > 
> > > > > I am trying to change a behavior in vdsm. When you pass 100%
> > > > > load
> > > > > on
> > > > > a VM, it will stop reporting further load and will keep
> > > > > telling
> > > > > 100%
> > > > > until the load drops under 100% again in it's cpuIdle
> > > > > information.
> > > > > This is totally correct if you have only single-cpu VM's, but
> > > > > it
> > > > > is
> > > > > false when you have multiple vcpu's, I think the cpuIdle
> > > > > information
> > > > > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > > > > 
> > > > > So I submitted this patch to vdsm:
> > > > > http://gerrit.ovirt.org/#/c/7892/2
> > > > > and Dan pointed out that some functionality may depend on the
> > > > > value
> > > > > in the 0-100 interval. For me it seems it is ignored and the
> > > > > load
> > > > > is
> > > > > calculated only from sysCpu + userCpu. Does anyone build on
> > > > > the
> > > > > cpuIdle value?
> > > > > 
> > > > > Thanks,
> > > > > Laszlo
> > > > >
> > > > 
> > > > you are right, engine doesn't save cpuIdle for vm,
> > > > so it's not in use in the engine.
> > > 
> > > Laszlo, in this case, I think it would be best to drop this bogus
> > > piece
> > > of information.
> > 
> > Ok.
> > 
> > However, before I abandon this patch:
> 
> Why abandon? I've suggested you to keep it, just make it even
> simpler.

Ok, it is only burocracy, but the new patch will do something completely different than the original, so it does not seem to make sense to continue this patch. It is more simple to make another one.

> 
> > we have a requirement to report cpuSys and cpuUser separately.
> > Afaik
> > in libvirt cpuUser and cpuSys does not include the actual guest
> > time
> > (at least not with KVM), and in this way if we only report cpuSys
> > and
> > cpuUser, the sum does not give the actual load, only a relatively
> > little percentage of it.
> 
> I am not sure I understand what you are saying, but afaik, libvirt's
> relatively-new
> http://libvirt.org/html/libvirt-libvirt.html#virDomainGetCPUStats
> reports the cpu time spent by the entire qemu process - in guest and
> host modes.

It seems like sysCpu + userCpu < cpuTime, therefore something is missing. I will give it another try, maybe something wrong with my hosts.

> 
> > If we have the cpuIdle information in engine,
> > we can calculate the guest time. Therefore, should I - include the
> > guest time in cpuSys or cpuUser?
> > - add another exported field?
> > 
> > And in both case, we will still have to calculate from cpuIdle
> > because
> > libvirt does not tell the guest cpu time :-(
> 
> Now I'm completely at loss. Why should we calculate cpuIdle per VM?
> Haven't we agreed that it is useless?

Well, if libvirt exports the guest time in sysCpu, then we do not have to. But it seems it does not.

> 
> Dan.
> 


From danken at redhat.com  Tue Sep 11 13:11:37 2012
From: danken at redhat.com (Dan Kenigsberg)
Date: Tue, 11 Sep 2012 16:11:37 +0300
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <937059412.16951072.1347367971075.JavaMail.root@redhat.com>
References: <20120911123413.GB11836@redhat.com>
	<937059412.16951072.1347367971075.JavaMail.root@redhat.com>
Message-ID: <20120911131137.GE11836@redhat.com>

On Tue, Sep 11, 2012 at 08:52:51AM -0400, Laszlo Hornyak wrote:
> 
> 
> ----- Original Message -----
> > From: "Dan Kenigsberg" <danken at redhat.com>
> > To: "Laszlo Hornyak" <lhornyak at redhat.com>
> > Cc: "engine-devel" <engine-devel at ovirt.org>, "Omer Frenkel" <ofrenkel at redhat.com>
> > Sent: Tuesday, September 11, 2012 2:34:13 PM
> > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > 
> > On Tue, Sep 11, 2012 at 07:38:11AM -0400, Laszlo Hornyak wrote:
> > > 
> > > ----- Original Message -----
> > > > From: "Dan Kenigsberg" <danken at redhat.com>
> > > > To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > Cc: "Laszlo Hornyak" <lhornyak at redhat.com>, "engine-devel"
> > > > <engine-devel at ovirt.org>
> > > > Sent: Tuesday, September 11, 2012 9:22:15 AM
> > > > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > > > 
> > > > On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > > > From: "Laszlo Hornyak" <lhornyak at redhat.com>
> > > > > > To: "engine-devel" <engine-devel at ovirt.org>
> > > > > > Sent: Monday, September 10, 2012 3:51:59 PM
> > > > > > Subject: [Engine-devel] what does engine with cpuIdle?
> > > > > > 
> > > > > > hi,
> > > > > > 
> > > > > > I am trying to change a behavior in vdsm. When you pass 100%
> > > > > > load
> > > > > > on
> > > > > > a VM, it will stop reporting further load and will keep
> > > > > > telling
> > > > > > 100%
> > > > > > until the load drops under 100% again in it's cpuIdle
> > > > > > information.
> > > > > > This is totally correct if you have only single-cpu VM's, but
> > > > > > it
> > > > > > is
> > > > > > false when you have multiple vcpu's, I think the cpuIdle
> > > > > > information
> > > > > > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > > > > > 
> > > > > > So I submitted this patch to vdsm:
> > > > > > http://gerrit.ovirt.org/#/c/7892/2
> > > > > > and Dan pointed out that some functionality may depend on the
> > > > > > value
> > > > > > in the 0-100 interval. For me it seems it is ignored and the
> > > > > > load
> > > > > > is
> > > > > > calculated only from sysCpu + userCpu. Does anyone build on
> > > > > > the
> > > > > > cpuIdle value?
> > > > > > 
> > > > > > Thanks,
> > > > > > Laszlo
> > > > > >
> > > > > 
> > > > > you are right, engine doesn't save cpuIdle for vm,
> > > > > so it's not in use in the engine.
> > > > 
> > > > Laszlo, in this case, I think it would be best to drop this bogus
> > > > piece
> > > > of information.
> > > 
> > > Ok.
> > > 
> > > However, before I abandon this patch:
> > 
> > Why abandon? I've suggested you to keep it, just make it even
> > simpler.
> 
> Ok, it is only burocracy, but the new patch will do something
> completely different than the original, so it does not seem to make
> sense to continue this patch. It is more simple to make another one.

Actually, if the patch has the same intentions, I prefer to keep it as
another version - this way it is easier to see if former comments have
been addressed. But it's your code and your call.

> 
> > 
> > > we have a requirement to report cpuSys and cpuUser separately.
> > > Afaik
> > > in libvirt cpuUser and cpuSys does not include the actual guest
> > > time
> > > (at least not with KVM), and in this way if we only report cpuSys
> > > and
> > > cpuUser, the sum does not give the actual load, only a relatively
> > > little percentage of it.
> > 
> > I am not sure I understand what you are saying, but afaik, libvirt's
> > relatively-new
> > http://libvirt.org/html/libvirt-libvirt.html#virDomainGetCPUStats
> > reports the cpu time spent by the entire qemu process - in guest and
> > host modes.
> 
> It seems like sysCpu + userCpu < cpuTime, therefore something is
> missing. I will give it another try, maybe something wrong with my
> hosts.

I may well be wrong.
http://libvirt.org/html/libvirt-libvirt.html#VIR_DOMAIN_CPU_STATS_CPUTIME

someone would have to compare your report requirement with what libvirt
provides.

> 
> > 
> > > If we have the cpuIdle information in engine,
> > > we can calculate the guest time. Therefore, should I - include the
> > > guest time in cpuSys or cpuUser?
> > > - add another exported field?
> > > 
> > > And in both case, we will still have to calculate from cpuIdle
> > > because
> > > libvirt does not tell the guest cpu time :-(
> > 
> > Now I'm completely at loss. Why should we calculate cpuIdle per VM?
> > Haven't we agreed that it is useless?
> 
> Well, if libvirt exports the guest time in sysCpu, then we do not have
> to. But it seems it does not.

Even we should jump throguh hoops to get the data you require, I do not
see how cpuIdle is related (or actually, what it means).

Dan.



From mpastern at redhat.com  Tue Sep 11 13:50:03 2012
From: mpastern at redhat.com (Michael Pasternak)
Date: Tue, 11 Sep 2012 16:50:03 +0300
Subject: [Engine-devel] Fwd: [oss-security] CVE Request (minor) -- JVM: heap
 memory disclosure (possibly various JDKs)
Message-ID: <504F418B.3080704@redhat.com>


-------- Original Message --------
Subject: [oss-security] CVE Request (minor) -- JVM: heap memory disclosure (possibly various JDKs)

Hello Kurt, Steve, vendors,

  an information disclosure flaw was found in the way certain
Java Virtual Machines (JVM) used to initialize integer arrays
(they have had nonzero elements right after the allocation in
certain circumstances). An attacker could use this flaw to
obtain potentially sensitive information.

References (including the reproducer, workaround and further details):
[1] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7196857
[2] https://bugzilla.redhat.com/show_bug.cgi?id=856124

Could you allocate a CVE id for this?

Thank you && Regards, Jan.
--
Jan iankko Lieskovsky / Red Hat Security Response Team

P.S.:  Issue brought to us by Florian Weimer, Red Hat Product Security Team
       (for case someone is tracking the initial reporter)

P.S#2: Oracle Security Team Cc-ed on this request too (to clarify
       if CVE id has been assigned to this already or not).

-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D


From sgordon at redhat.com  Tue Sep 11 14:47:36 2012
From: sgordon at redhat.com (Steve Gordon)
Date: Tue, 11 Sep 2012 10:47:36 -0400 (EDT)
Subject: [Engine-devel] i18n status / availability of pot files for
 translators
In-Reply-To: <1347360872.11758.195.camel@dhcp-29-7.brq.redhat.com>
Message-ID: <2111859999.33162705.1347374856214.JavaMail.root@redhat.com>



----- Original Message -----
> From: "David Ja?a" <djasa at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Tuesday, September 11, 2012 6:54:32 AM
> Subject: Re: [Engine-devel] i18n status / availability of pot files for translators
> 
> Itamar Heim p??e v Po 10. 09. 2012 v 23:21 +0300:
> > On 09/10/2012 07:14 PM, David Ja?a wrote:
> > > Hi,
> > >
> > > I was wondering about $SUBJ. When will translators be able to lay
> > > their
> > > hands on the initial .pot files, sometimes around 3.2 freeze?
> > >
> > > The only mention of translation I've seen is in release notes of
> > > 3.1
> > > promising translations to be included in 3.2, so it would be
> > > quite nice
> > > to give translators some time to work on them...
> > >
> > > David
> > >
> > 
> > translators already started to work:
> > https://translate.zanata.org/zanata/project/view/ovirt
> 
> This is very well hidden, there's no mention of Zanata neither on the
> web nor on the wiki nor in the mailing lists...
> 
> David

Maybe we need a link to the zanata instance from the 'community' page on the website?

Steve


From lhornyak at redhat.com  Tue Sep 11 16:27:34 2012
From: lhornyak at redhat.com (Laszlo Hornyak)
Date: Tue, 11 Sep 2012 12:27:34 -0400 (EDT)
Subject: [Engine-devel] AsyncTaskManager and the irsbroker
In-Reply-To: <1520419731.17068939.1347380006536.JavaMail.root@redhat.com>
Message-ID: <1163133771.17075037.1347380854215.JavaMail.root@redhat.com>

hi,

I have just noticed that AsyncTaskManager is
- almost all of the public methods are synchronized
- and it is singleton
- calling to VDS operations (line 291)

Therefore this may be a performance bottleneck. Even when you want to create an asynchronous task or query a task's state, you may have to wait till a remote vdsm responds. Would it be ok to remove the synchronization from e.g. the query-like methods? Or the one initiating remote calls?

Laszlo


From alonbl at redhat.com  Wed Sep 12 09:40:03 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 12 Sep 2012 05:40:03 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <504F125E.4050606@redhat.com>
Message-ID: <1002781018.6106654.1347442803758.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Livnat Peer" <lpeer at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Tuesday, September 11, 2012 1:28:46 PM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 

<snip>

> >>> QUESTIONS
> >>>
> >>> 1. Why do we store default values in database?
> >>>
> >>
> >> Our intention so far was to remove default values from the code.
> >>
> >> The values in the data base are not default values but the values
> >> of
> >> the
> >> properties and we keep them per version were we need different
> >> values
> >> per version.
> >>
> >> The obvious advantage of keeping them in the DB vs. in code is
> >> they
> >> can
> >> be changed with no compilation required and today you don't need
> >> to
> >> restart the application for changing some of them (WIP).
> > 
> > The above statement is true if you use database or any other
> > format, such as XML file for the metadata.
> > 
> > As long as there is no duplication between code and metadata
> > storage, and as long as the default values are not stored within
> > the options tables, so that after upgrade the new defaults are in
> > effect unless explicitly overridden by user.
> > 
> > Let's assume we would like to avoid compilation. What is the
> > benefit in storing the metadata within the database and not in
> > plain text file / XML file / properties file?
> 
> I think that the main advantage of using data base over XML is that
> we
> already maintain data base and work with it. I have nothing against
> XML
> but I think the question we should ask ourselves is why add another
> mechanism like XML file for managing static configuration when we can
> use data base?

Because:

1. During upgrade database is not touched, less chance of failure.

2. A patch that adds a new option is touching a single file.

It is obvious why if we use Java annotations....

And I will answer the obvious question of why single file if it is XML... still need ConfigValues like enums... The answer is that we generate ConfigValues.java (just the enum name) during build.

3. When using SQL database, the schema should be modified when each new feature / restriction is introduced, while XML or code annotations are more flexible in this regards.

4. The metadata of options (or any release dependent metadata) is unchanged between one release to another, RDBMS role is to manage the data that is to be modified.

> Also I think there was/is a thought to expose the configuration via
> the
> UI (both for view and edit) I think that if this is still the
> intention
> then working with DB would be easier than XML.

As all options are to be loaded to memory at initialization time, I don't think there is any difference.

Please note that there are two entities:
- options' metadata
- options' values

The options' values is stored in database, I have no intention to change this.
The options' metadata is the one we are discussing.

Options' metadata can be read to memory to appropriate data format from XML or from database, the end result is the same, there should be no difference for the consumer were you got the metadata from.

> 
> > 
> >>> From what I managed to gather, we store default values in
> >>> database
> >>> under the assumption that we need to keep defaults when we
> >>> upgrade
> >>> from one version to another.
> >>>
> >>> However, in most cases when upgrading an application the new
> >>> defaults should apply as long as they were not overridden by
> >>> user.
> >>> Keeping old default as a new value is an exception that can be
> >>> taken care of during the upgrade process for selected options.
> >>>
> >>> 2. Why do we keep properties file?
> >>
> >>
> >> The properties file primary use was to add translation/description
> >> for
> >> each property. The reason we store it in the file was that at some
> >> point
> >> we'll support multilingual description for each property.
> > 
> > Usually, multilingual has the "C" (latin) within code, while
> > overriding the "other" lingual within language support pack. So
> > that application without any language support pack will speak
> > latin.
> >
> That's not the way I am familiar with, Usually when you work in java
> you
> use ResourceBundle which uses keys in the code and the translation is
> provided in external properties file, one per language, distinguished
> by
> their extensions: sample.property.en sample.property.jp etc.

I prefer to use multiple properties files so that a language pack can be maintained and installed separately.

ResourceBundle.getBundle() supports this as far as I understand.

Having said that, I still like to encourage the modification of single file when adding one entity, much easier in review process, conflicts, code management, blame management, learning curve etc...

So ether generate the C linguas property file during build time from java annotations or XML metadata, and allow external linguas properties file to be added at later time.

Or maintain the translation within the metadata XML (this is the first true advantage of storing metadata within XML over storing metadata using java annotations), and either generate the linguas property files at built time, or just read it and create ResourceBundle at runtime when reading the metadata (I think this is the simplest solution).

<snip>


Thanks,
Alon


From alonbl at redhat.com  Wed Sep 12 09:51:50 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 12 Sep 2012 05:51:50 -0400 (EDT)
Subject: [Engine-devel] network subnet
In-Reply-To: <1347364076.11758.250.camel@dhcp-29-7.brq.redhat.com>
Message-ID: <1273999591.6107354.1347443510920.JavaMail.root@redhat.com>



----- Original Message -----
> From: "David Ja?a" <djasa at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Tuesday, September 11, 2012 2:47:56 PM
> Subject: Re: [Engine-devel] network subnet
> 
> Livnat Peer p??e v ?t 11. 09. 2012 v 09:21 +0300:
> > On 30/08/12 23:11, Alon Bar-Lev wrote:
> > > 
> > > 
> > > ----- Original Message -----
> > >> From: "Livnat Peer" <lpeer at redhat.com>
> > >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> > >> Cc: engine-devel at ovirt.org
> > >> Sent: Thursday, August 30, 2012 10:16:05 PM
> > >> Subject: Re: [Engine-devel] network subnet
> > >>
> > >> On 30/08/12 21:39, Alon Bar-Lev wrote:
> > >>>
> > >>>
> > >>> ----- Original Message -----
> > >>>> From: "Livnat Peer" <lpeer at redhat.com>
> > >>>> To: engine-devel at ovirt.org
> > >>>> Sent: Thursday, August 30, 2012 3:22:29 PM
> > >>>> Subject: [Engine-devel] network subnet
> > >>>>
> > >>>> Hi All,
> > >>>>
> > >>>> Today when a user wants to define a network subnet mask, he
> > >>>> does
> > >>>> it
> > >>>> when
> > >>>> he attaches the network to a host NIC.
> > >>>>
> > >>>> I was wondering if there is a reason not to define the network
> > >>>> subnet
> > >>>> on
> > >>>> the logical network entity (Data center level).
> > >>>>
> > >>>> Thanks, Livnat
> > >>>
> > >>> Hello,
> > >>>
> > >>> I am sorry, maybe I do not understand... the IP scheme enforces
> > >>> the
> > >>> use of address mask in order to properly route packets.
> > >>
> > >> of course. My proposal is related to our user usage of the
> > >> system.
> > >> Today
> > >> ovirt user, who wants to define a network subnet, has to type
> > >> the
> > >> subnet
> > >> per host (per network), I think the user should only define it
> > >> once
> > >> on
> > >> the logical network entity in the Data Center.
> > >> Propagating the value to all hosts is needed but it should be
> > >> our
> > >> internal implementation detail.
> > >>
> > >>>
> > >>> Network mask is used in any case, I guess it can be dropped
> > >>> from
> > >>> configuration in favour of using the address class as mask, is
> > >>> that what you suggest?
> > >>>
> > >>
> > >> No, hope the above paragraph made it more clear.
> > >>
> > > 
> > > Hello,
> > > 
> > > Then you assume that a logical network, which is actually layer 2
> > > network in our implementation, has layer 3 characteristics,
> > > right?
> > > 
> > > In our current implementation "data center logical network" is
> > > pure layer 2 segment aka layer 2 broadcast domain.
> > > 
> > > One can use the same logical network for multiple layer 3
> > > segments, which is totally valid and consistent with standard
> > > physical layer 2 setup.
> > > 
> > > Unless I am missing something crucial, I would suggest to keep
> > > the consistent physical->virtual mapping, unless we emulate
> > > layer 3 switching. Layer 2 does not have layer 3
> > > characteristics.
> > > 
> > > Regards,
> > > Alon.
> > > 
> > 
> > Generally I agree with what you wrote.
> > 
> > I would like to open for discussion the definition mentioned above
> > that
> > a logical network is a layer 2 broadcast domain.
> > 
> > We have 2 types of logical networks, VM networks and 'other' (host
> > functionality?) networks.
> > 
> > When talking about VM networks, I think the definition above is
> > accurate, the guests using the network should get a layer 2
> > broadcast
> > domain.
> > It can be implemented over a single (physical) layer 2 domain or it
> > can
> > be implemented over multiple (physical) layer 2 domains using
> > technologies like tunneling or vxlan.
> > 
> > When talking about other networks like storage, display, migration,
> > etc. we are actually discussing a network which represent a common
> > functionality the hosts are using but I am not sure guaranteeing a
> > layer
> > 2 broadcast domain is interesting for such network.
> > 
> 
> IMHO the requirements for the "logical networks" could be
> differentiated
> depending on their function:
>       * management: engine must reach each host on L3
>       * storage: each host can reach the storage on L3 (FCP or IP)
>       * VM network: all VNICs in one  are in L2 broadcast domain by
>         default
>       * migration: mutual reachability of hosts on L3
>       * display: reachability on from clients
> that in turn could make
> 
> Note that engine or hosts can not really test if display network is
> working in conditions like split-horizon DNS.
> 
> David
> 
> > Going forward I expect we'll associate layer 3 services with
> > logical
> > networks. For example DHCP, DNS, LB etc.
> > 
> > Any thoughts/comments on the above are welcomed.
> > 

I agree that we should distinguish between terms.

"Layer 2 Segment" which is what you connect "Network Element" to.

Now I understand the we have "XXX Group", I guess this is what you refer as "Storage Network" for example?

Most of these groups only need network reachability so the term "Network" in this sense is somewhat confusing.

But I may got this wrong.

Alon.


From alonbl at redhat.com  Wed Sep 12 09:58:06 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 12 Sep 2012 05:58:06 -0400 (EDT)
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <937059412.16951072.1347367971075.JavaMail.root@redhat.com>
Message-ID: <1011784487.6107796.1347443886150.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Laszlo Hornyak" <lhornyak at redhat.com>
> To: "Dan Kenigsberg" <danken at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Tuesday, September 11, 2012 3:52:51 PM
> Subject: Re: [Engine-devel] what does engine with cpuIdle?
> 
> 
> 
> ----- Original Message -----
> > From: "Dan Kenigsberg" <danken at redhat.com>
> > To: "Laszlo Hornyak" <lhornyak at redhat.com>
> > Cc: "engine-devel" <engine-devel at ovirt.org>, "Omer Frenkel"
> > <ofrenkel at redhat.com>
> > Sent: Tuesday, September 11, 2012 2:34:13 PM
> > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > 
> > On Tue, Sep 11, 2012 at 07:38:11AM -0400, Laszlo Hornyak wrote:
> > > 
> > > ----- Original Message -----
> > > > From: "Dan Kenigsberg" <danken at redhat.com>
> > > > To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > Cc: "Laszlo Hornyak" <lhornyak at redhat.com>, "engine-devel"
> > > > <engine-devel at ovirt.org>
> > > > Sent: Tuesday, September 11, 2012 9:22:15 AM
> > > > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > > > 
> > > > On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > > > From: "Laszlo Hornyak" <lhornyak at redhat.com>
> > > > > > To: "engine-devel" <engine-devel at ovirt.org>
> > > > > > Sent: Monday, September 10, 2012 3:51:59 PM
> > > > > > Subject: [Engine-devel] what does engine with cpuIdle?
> > > > > > 
> > > > > > hi,
> > > > > > 
> > > > > > I am trying to change a behavior in vdsm. When you pass
> > > > > > 100%
> > > > > > load
> > > > > > on
> > > > > > a VM, it will stop reporting further load and will keep
> > > > > > telling
> > > > > > 100%
> > > > > > until the load drops under 100% again in it's cpuIdle
> > > > > > information.
> > > > > > This is totally correct if you have only single-cpu VM's,
> > > > > > but
> > > > > > it
> > > > > > is
> > > > > > false when you have multiple vcpu's, I think the cpuIdle
> > > > > > information
> > > > > > should not be on a 0-100 scale, but on a 0-100*vcpus scale.
> > > > > > 
> > > > > > So I submitted this patch to vdsm:
> > > > > > http://gerrit.ovirt.org/#/c/7892/2
> > > > > > and Dan pointed out that some functionality may depend on
> > > > > > the
> > > > > > value
> > > > > > in the 0-100 interval. For me it seems it is ignored and
> > > > > > the
> > > > > > load
> > > > > > is
> > > > > > calculated only from sysCpu + userCpu. Does anyone build on
> > > > > > the
> > > > > > cpuIdle value?
> > > > > > 
> > > > > > Thanks,
> > > > > > Laszlo
> > > > > >
> > > > > 
> > > > > you are right, engine doesn't save cpuIdle for vm,
> > > > > so it's not in use in the engine.
> > > > 
> > > > Laszlo, in this case, I think it would be best to drop this
> > > > bogus
> > > > piece
> > > > of information.
> > > 
> > > Ok.
> > > 
> > > However, before I abandon this patch:
> > 
> > Why abandon? I've suggested you to keep it, just make it even
> > simpler.
> 
> Ok, it is only burocracy, but the new patch will do something
> completely different than the original, so it does not seem to make
> sense to continue this patch. It is more simple to make another one.

No... what you do is split the patch into two, leaving the change id for the 2nd PPC patch.
Then push the two patches.
You will get the PPC patch depend on the platform patch.
It works pretty well... :)

Alon.


From alonbl at redhat.com  Wed Sep 12 09:59:01 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 12 Sep 2012 05:59:01 -0400 (EDT)
Subject: [Engine-devel] what does engine with cpuIdle?
In-Reply-To: <1011784487.6107796.1347443886150.JavaMail.root@redhat.com>
Message-ID: <1886036412.6107847.1347443941825.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Alon Bar-Lev" <alonbl at redhat.com>
> To: "Laszlo Hornyak" <lhornyak at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Wednesday, September 12, 2012 12:58:06 PM
> Subject: Re: [Engine-devel] what does engine with cpuIdle?
> 
> 
> 
> ----- Original Message -----
> > From: "Laszlo Hornyak" <lhornyak at redhat.com>
> > To: "Dan Kenigsberg" <danken at redhat.com>
> > Cc: "engine-devel" <engine-devel at ovirt.org>
> > Sent: Tuesday, September 11, 2012 3:52:51 PM
> > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Dan Kenigsberg" <danken at redhat.com>
> > > To: "Laszlo Hornyak" <lhornyak at redhat.com>
> > > Cc: "engine-devel" <engine-devel at ovirt.org>, "Omer Frenkel"
> > > <ofrenkel at redhat.com>
> > > Sent: Tuesday, September 11, 2012 2:34:13 PM
> > > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > > 
> > > On Tue, Sep 11, 2012 at 07:38:11AM -0400, Laszlo Hornyak wrote:
> > > > 
> > > > ----- Original Message -----
> > > > > From: "Dan Kenigsberg" <danken at redhat.com>
> > > > > To: "Omer Frenkel" <ofrenkel at redhat.com>
> > > > > Cc: "Laszlo Hornyak" <lhornyak at redhat.com>, "engine-devel"
> > > > > <engine-devel at ovirt.org>
> > > > > Sent: Tuesday, September 11, 2012 9:22:15 AM
> > > > > Subject: Re: [Engine-devel] what does engine with cpuIdle?
> > > > > 
> > > > > On Tue, Sep 11, 2012 at 01:55:01AM -0400, Omer Frenkel wrote:
> > > > > > 
> > > > > > 
> > > > > > ----- Original Message -----
> > > > > > > From: "Laszlo Hornyak" <lhornyak at redhat.com>
> > > > > > > To: "engine-devel" <engine-devel at ovirt.org>
> > > > > > > Sent: Monday, September 10, 2012 3:51:59 PM
> > > > > > > Subject: [Engine-devel] what does engine with cpuIdle?
> > > > > > > 
> > > > > > > hi,
> > > > > > > 
> > > > > > > I am trying to change a behavior in vdsm. When you pass
> > > > > > > 100%
> > > > > > > load
> > > > > > > on
> > > > > > > a VM, it will stop reporting further load and will keep
> > > > > > > telling
> > > > > > > 100%
> > > > > > > until the load drops under 100% again in it's cpuIdle
> > > > > > > information.
> > > > > > > This is totally correct if you have only single-cpu VM's,
> > > > > > > but
> > > > > > > it
> > > > > > > is
> > > > > > > false when you have multiple vcpu's, I think the cpuIdle
> > > > > > > information
> > > > > > > should not be on a 0-100 scale, but on a 0-100*vcpus
> > > > > > > scale.
> > > > > > > 
> > > > > > > So I submitted this patch to vdsm:
> > > > > > > http://gerrit.ovirt.org/#/c/7892/2
> > > > > > > and Dan pointed out that some functionality may depend on
> > > > > > > the
> > > > > > > value
> > > > > > > in the 0-100 interval. For me it seems it is ignored and
> > > > > > > the
> > > > > > > load
> > > > > > > is
> > > > > > > calculated only from sysCpu + userCpu. Does anyone build
> > > > > > > on
> > > > > > > the
> > > > > > > cpuIdle value?
> > > > > > > 
> > > > > > > Thanks,
> > > > > > > Laszlo
> > > > > > >
> > > > > > 
> > > > > > you are right, engine doesn't save cpuIdle for vm,
> > > > > > so it's not in use in the engine.
> > > > > 
> > > > > Laszlo, in this case, I think it would be best to drop this
> > > > > bogus
> > > > > piece
> > > > > of information.
> > > > 
> > > > Ok.
> > > > 
> > > > However, before I abandon this patch:
> > > 
> > > Why abandon? I've suggested you to keep it, just make it even
> > > simpler.
> > 
> > Ok, it is only burocracy, but the new patch will do something
> > completely different than the original, so it does not seem to make
> > sense to continue this patch. It is more simple to make another
> > one.
> 
> No... what you do is split the patch into two, leaving the change id
> for the 2nd PPC patch.
> Then push the two patches.
> You will get the PPC patch depend on the platform patch.
> It works pretty well... :)
> 
> Alon.

Oh... wrong thread.

Alon.


From lpeer at redhat.com  Wed Sep 12 10:11:15 2012
From: lpeer at redhat.com (Livnat Peer)
Date: Wed, 12 Sep 2012 13:11:15 +0300
Subject: [Engine-devel] network subnet
In-Reply-To: <1273999591.6107354.1347443510920.JavaMail.root@redhat.com>
References: <1273999591.6107354.1347443510920.JavaMail.root@redhat.com>
Message-ID: <50505FC3.6080500@redhat.com>

On 12/09/12 12:51, Alon Bar-Lev wrote:
> 
> 
> ----- Original Message -----
>> From: "David Ja?a" <djasa at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Tuesday, September 11, 2012 2:47:56 PM
>> Subject: Re: [Engine-devel] network subnet
>>
>> Livnat Peer p??e v ?t 11. 09. 2012 v 09:21 +0300:
>>> On 30/08/12 23:11, Alon Bar-Lev wrote:
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>>> Cc: engine-devel at ovirt.org
>>>>> Sent: Thursday, August 30, 2012 10:16:05 PM
>>>>> Subject: Re: [Engine-devel] network subnet
>>>>>
>>>>> On 30/08/12 21:39, Alon Bar-Lev wrote:
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Livnat Peer" <lpeer at redhat.com>
>>>>>>> To: engine-devel at ovirt.org
>>>>>>> Sent: Thursday, August 30, 2012 3:22:29 PM
>>>>>>> Subject: [Engine-devel] network subnet
>>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> Today when a user wants to define a network subnet mask, he
>>>>>>> does
>>>>>>> it
>>>>>>> when
>>>>>>> he attaches the network to a host NIC.
>>>>>>>
>>>>>>> I was wondering if there is a reason not to define the network
>>>>>>> subnet
>>>>>>> on
>>>>>>> the logical network entity (Data center level).
>>>>>>>
>>>>>>> Thanks, Livnat
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I am sorry, maybe I do not understand... the IP scheme enforces
>>>>>> the
>>>>>> use of address mask in order to properly route packets.
>>>>>
>>>>> of course. My proposal is related to our user usage of the
>>>>> system.
>>>>> Today
>>>>> ovirt user, who wants to define a network subnet, has to type
>>>>> the
>>>>> subnet
>>>>> per host (per network), I think the user should only define it
>>>>> once
>>>>> on
>>>>> the logical network entity in the Data Center.
>>>>> Propagating the value to all hosts is needed but it should be
>>>>> our
>>>>> internal implementation detail.
>>>>>
>>>>>>
>>>>>> Network mask is used in any case, I guess it can be dropped
>>>>>> from
>>>>>> configuration in favour of using the address class as mask, is
>>>>>> that what you suggest?
>>>>>>
>>>>>
>>>>> No, hope the above paragraph made it more clear.
>>>>>
>>>>
>>>> Hello,
>>>>
>>>> Then you assume that a logical network, which is actually layer 2
>>>> network in our implementation, has layer 3 characteristics,
>>>> right?
>>>>
>>>> In our current implementation "data center logical network" is
>>>> pure layer 2 segment aka layer 2 broadcast domain.
>>>>
>>>> One can use the same logical network for multiple layer 3
>>>> segments, which is totally valid and consistent with standard
>>>> physical layer 2 setup.
>>>>
>>>> Unless I am missing something crucial, I would suggest to keep
>>>> the consistent physical->virtual mapping, unless we emulate
>>>> layer 3 switching. Layer 2 does not have layer 3
>>>> characteristics.
>>>>
>>>> Regards,
>>>> Alon.
>>>>
>>>
>>> Generally I agree with what you wrote.
>>>
>>> I would like to open for discussion the definition mentioned above
>>> that
>>> a logical network is a layer 2 broadcast domain.
>>>
>>> We have 2 types of logical networks, VM networks and 'other' (host
>>> functionality?) networks.
>>>
>>> When talking about VM networks, I think the definition above is
>>> accurate, the guests using the network should get a layer 2
>>> broadcast
>>> domain.
>>> It can be implemented over a single (physical) layer 2 domain or it
>>> can
>>> be implemented over multiple (physical) layer 2 domains using
>>> technologies like tunneling or vxlan.
>>>
>>> When talking about other networks like storage, display, migration,
>>> etc. we are actually discussing a network which represent a common
>>> functionality the hosts are using but I am not sure guaranteeing a
>>> layer
>>> 2 broadcast domain is interesting for such network.
>>>
>>
>> IMHO the requirements for the "logical networks" could be
>> differentiated
>> depending on their function:
>>       * management: engine must reach each host on L3
>>       * storage: each host can reach the storage on L3 (FCP or IP)
>>       * VM network: all VNICs in one  are in L2 broadcast domain by
>>         default
>>       * migration: mutual reachability of hosts on L3
>>       * display: reachability on from clients
>> that in turn could make
>>
>> Note that engine or hosts can not really test if display network is
>> working in conditions like split-horizon DNS.
>>
>> David
>>
>>> Going forward I expect we'll associate layer 3 services with
>>> logical
>>> networks. For example DHCP, DNS, LB etc.
>>>
>>> Any thoughts/comments on the above are welcomed.
>>>
> 
> I agree that we should distinguish between terms.
> 
> "Layer 2 Segment" which is what you connect "Network Element" to.
> 
> Now I understand the we have "XXX Group", I guess this is what you refer as "Storage Network" for example?
> 
> Most of these groups only need network reachability so the term "Network" in this sense is somewhat confusing.
> 
> But I may got this wrong.

You got it exactly right, I'm not sure we should create a network for
such groups.

Anyway it is just a set of initial thoughts, I need to build a more
formal proposal what to do with this and I'll send it to the list.

Thanks for your inputs.

Livnat

> 
> Alon.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 



From jhernand at redhat.com  Wed Sep 12 12:32:28 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Wed, 12 Sep 2012 14:32:28 +0200
Subject: [Engine-devel] Fixed -Xmx could kill your JVM
Message-ID: <505080DC.4030609@redhat.com>

Hello,

I recently discovered that I made a mistake in the engine service
script. The problem is that were running the JVM with the "-Xms option
twice, like this:

  java -Xms1g -Xms1g ...

That doesn't correctly limit the max amount of heap space. It has been
fixed in a recent commit [1], so with the updated code the engine will
run by default like this:

  java -Xms1g -Xmx1g ...

With this change you could start to see OutOfMemory errors if your
engine was using a lot of memory, if you are running stress tests, for
example. In that case remember that you can adjust this changing the
ENGINE_HEAP_MIN and ENGINE_HEAP_MAX parameters in
/etc/sysconfig/ovirt-engine. For example, to use 2g instead of the
default of 1g:

ENGINE_HEAP_MIN=2g
ENGINE_HEAP_MAX=2g

The restart the engine.

Let me know if you have issues.

Regards,
Juan Hernandez

[1] http://gerrit.ovirt.org/7952

-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From lhornyak at redhat.com  Wed Sep 12 12:44:25 2012
From: lhornyak at redhat.com (Laszlo Hornyak)
Date: Wed, 12 Sep 2012 08:44:25 -0400 (EDT)
Subject: [Engine-devel] Fixed -Xmx could kill your JVM
In-Reply-To: <505080DC.4030609@redhat.com>
Message-ID: <76249789.17627678.1347453865264.JavaMail.root@redhat.com>

Hi Juan,

Just wondering why the initial memory size for the JVM. Isn't it better if we just start from the default and let it grow to a maximum size?

Thx,
Laszlo

----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Wednesday, September 12, 2012 2:32:28 PM
> Subject: [Engine-devel] Fixed -Xmx could kill your JVM
> 
> Hello,
> 
> I recently discovered that I made a mistake in the engine service
> script. The problem is that were running the JVM with the "-Xms
> option
> twice, like this:
> 
>   java -Xms1g -Xms1g ...
> 
> That doesn't correctly limit the max amount of heap space. It has
> been
> fixed in a recent commit [1], so with the updated code the engine
> will
> run by default like this:
> 
>   java -Xms1g -Xmx1g ...
> 
> With this change you could start to see OutOfMemory errors if your
> engine was using a lot of memory, if you are running stress tests,
> for
> example. In that case remember that you can adjust this changing the
> ENGINE_HEAP_MIN and ENGINE_HEAP_MAX parameters in
> /etc/sysconfig/ovirt-engine. For example, to use 2g instead of the
> default of 1g:
> 
> ENGINE_HEAP_MIN=2g
> ENGINE_HEAP_MAX=2g
> 
> The restart the engine.
> 
> Let me know if you have issues.
> 
> Regards,
> Juan Hernandez
> 
> [1] http://gerrit.ovirt.org/7952
> 
> --
> Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
> 3?D, 28016 Madrid, Spain
> Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat
> S.L.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From emesika at redhat.com  Wed Sep 12 12:48:00 2012
From: emesika at redhat.com (Eli Mesika)
Date: Wed, 12 Sep 2012 08:48:00 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <1002781018.6106654.1347442803758.JavaMail.root@redhat.com>
Message-ID: <1509358691.68328052.1347454080804.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Alon Bar-Lev" <alonbl at redhat.com>
> To: "Livnat Peer" <lpeer at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Wednesday, September 12, 2012 12:40:03 PM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> 
> 
> ----- Original Message -----
> > From: "Livnat Peer" <lpeer at redhat.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Tuesday, September 11, 2012 1:28:46 PM
> > Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default
> > options
> > 
> 
> <snip>
> 
> > >>> QUESTIONS
> > >>>
> > >>> 1. Why do we store default values in database?

I wanted to add few important points 
a) We have default values in DB in order to enable overriding values in the 0000_config.sql file only if a customer did not change the default value, if a customer changed the default in some entries, we want to honour the customer settings
b)There may be other requirements on configuration that are easier to manage in the database, for example, I have heard that one of the suggested features regarding configuration is to keep a kind of configuration history and know exactly which configuration values was changed, when and by whom.
c) The version mechanism in the config is working like this : there is a 'general' version , means that this value is not version dependant, or the version can be a real version like 3.1 3.2 etc , in such case the value is version dependant and an entry is required for each version.

I agree however, that the default value in the code is redundant and error prone and should be removed.


> > >>>
> > >>
> > >> Our intention so far was to remove default values from the code.
> > >>
> > >> The values in the data base are not default values but the
> > >> values
> > >> of
> > >> the
> > >> properties and we keep them per version were we need different
> > >> values
> > >> per version.
> > >>
> > >> The obvious advantage of keeping them in the DB vs. in code is
> > >> they
> > >> can
> > >> be changed with no compilation required and today you don't need
> > >> to
> > >> restart the application for changing some of them (WIP).
> > > 
> > > The above statement is true if you use database or any other
> > > format, such as XML file for the metadata.
> > > 
> > > As long as there is no duplication between code and metadata
> > > storage, and as long as the default values are not stored within
> > > the options tables, so that after upgrade the new defaults are in
> > > effect unless explicitly overridden by user.
> > > 
> > > Let's assume we would like to avoid compilation. What is the
> > > benefit in storing the metadata within the database and not in
> > > plain text file / XML file / properties file?
> > 
> > I think that the main advantage of using data base over XML is that
> > we
> > already maintain data base and work with it. I have nothing against
> > XML
> > but I think the question we should ask ourselves is why add another
> > mechanism like XML file for managing static configuration when we
> > can
> > use data base?
> 
> Because:
> 
> 1. During upgrade database is not touched, less chance of failure.
> 
> 2. A patch that adds a new option is touching a single file.
> 
> It is obvious why if we use Java annotations....
> 
> And I will answer the obvious question of why single file if it is
> XML... still need ConfigValues like enums... The answer is that we
> generate ConfigValues.java (just the enum name) during build.
> 
> 3. When using SQL database, the schema should be modified when each
> new feature / restriction is introduced, while XML or code
> annotations are more flexible in this regards.
> 
> 4. The metadata of options (or any release dependent metadata) is
> unchanged between one release to another, RDBMS role is to manage
> the data that is to be modified.
> 
> > Also I think there was/is a thought to expose the configuration via
> > the
> > UI (both for view and edit) I think that if this is still the
> > intention
> > then working with DB would be easier than XML.
> 
> As all options are to be loaded to memory at initialization time, I
> don't think there is any difference.
> 
> Please note that there are two entities:
> - options' metadata
> - options' values
> 
> The options' values is stored in database, I have no intention to
> change this.
> The options' metadata is the one we are discussing.
> 
> Options' metadata can be read to memory to appropriate data format
> from XML or from database, the end result is the same, there should
> be no difference for the consumer were you got the metadata from.
> 
> > 
> > > 
> > >>> From what I managed to gather, we store default values in
> > >>> database
> > >>> under the assumption that we need to keep defaults when we
> > >>> upgrade
> > >>> from one version to another.
> > >>>
> > >>> However, in most cases when upgrading an application the new
> > >>> defaults should apply as long as they were not overridden by
> > >>> user.
> > >>> Keeping old default as a new value is an exception that can be
> > >>> taken care of during the upgrade process for selected options.
> > >>>
> > >>> 2. Why do we keep properties file?
> > >>
> > >>
> > >> The properties file primary use was to add
> > >> translation/description
> > >> for
> > >> each property. The reason we store it in the file was that at
> > >> some
> > >> point
> > >> we'll support multilingual description for each property.
> > > 
> > > Usually, multilingual has the "C" (latin) within code, while
> > > overriding the "other" lingual within language support pack. So
> > > that application without any language support pack will speak
> > > latin.
> > >
> > That's not the way I am familiar with, Usually when you work in
> > java
> > you
> > use ResourceBundle which uses keys in the code and the translation
> > is
> > provided in external properties file, one per language,
> > distinguished
> > by
> > their extensions: sample.property.en sample.property.jp etc.
> 
> I prefer to use multiple properties files so that a language pack can
> be maintained and installed separately.
> 
> ResourceBundle.getBundle() supports this as far as I understand.
> 
> Having said that, I still like to encourage the modification of
> single file when adding one entity, much easier in review process,
> conflicts, code management, blame management, learning curve etc...
> 
> So ether generate the C linguas property file during build time from
> java annotations or XML metadata, and allow external linguas
> properties file to be added at later time.
> 
> Or maintain the translation within the metadata XML (this is the
> first true advantage of storing metadata within XML over storing
> metadata using java annotations), and either generate the linguas
> property files at built time, or just read it and create
> ResourceBundle at runtime when reading the metadata (I think this is
> the simplest solution).
> 
> <snip>
> 
> 
> Thanks,
> Alon
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From alonbl at redhat.com  Wed Sep 12 12:57:13 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 12 Sep 2012 08:57:13 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <1509358691.68328052.1347454080804.JavaMail.root@redhat.com>
Message-ID: <1202206087.6136904.1347454633345.JavaMail.root@redhat.com>


Hello Eli,

----- Original Message -----
> From: "Eli Mesika" <emesika at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> Sent: Wednesday, September 12, 2012 3:48:00 PM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> > > >>> QUESTIONS
> > > >>>
> > > >>> 1. Why do we store default values in database?
> 
> I wanted to add few important points
> a) We have default values in DB in order to enable overriding values
> in the 0000_config.sql file only if a customer did not change the
> default value, if a customer changed the default in some entries, we
> want to honour the customer settings

Default != value.
Default is what should be used if not specified explicitly 

Current implementation puts the default value as a value, so you cannot distinguish between what user enforced.

When a default is changed, because of field feedback, we should push this into the database instead of keeping in database only options that were modified by the user and fetch the default from the option metadata.

In another words.... there should be no 000_config.sql, the option table should be empty as long as the user does not modify any of the options.

> b)There may be other requirements on configuration that are easier to
> manage in the database, for example, I have heard that one of the
> suggested features regarding configuration is to keep a kind of
> configuration history and know exactly which configuration values
> was changed, when and by whom.

There is no conflict. You can have audit table when modifying options.
Keep in mind that I discuss the option metadata.
I believe you are discussing the option data.

> c) The version mechanism in the config is working like this : there
> is a 'general' version , means that this value is not version
> dependant, or the version can be a real version like 3.1 3.2 etc ,
> in such case the value is version dependant and an entry is required
> for each version.

Again, there is no conflict, as the default value within the metadata can be version specific too.

Having said that, I don't understand how two different versions can work with the different data models and share one database and options.

> I agree however, that the default value in the code is redundant and
> error prone and should be removed.

>From what you wrote above, I don't understand how you reached to this conclusion. Can you please explain?

Thanks,
Alon.


From jhernand at redhat.com  Wed Sep 12 12:57:33 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Wed, 12 Sep 2012 14:57:33 +0200
Subject: [Engine-devel] Fixed -Xmx could kill your JVM
In-Reply-To: <76249789.17627678.1347453865264.JavaMail.root@redhat.com>
References: <76249789.17627678.1347453865264.JavaMail.root@redhat.com>
Message-ID: <505086BD.7030303@redhat.com>

On 09/12/2012 02:44 PM, Laszlo Hornyak wrote:
> Just wondering why the initial memory size for the JVM. Isn't it better if we just start from the default and let it grow to a maximum size?

The reason for setting the min ad max values equal by default is that in
that way the JVM doesn't have to resize the heap during execution, which
takes some additional time (barely noticeable) and can fail if some
other program takes the virtual address space after starting the JVM.

In my experience you get slightly better performance and less problems
if you determine the right size for your environment and then set the
min and max to those values, but I don't really have hard data to
confirm it is good for the engine.

Worst case we can change the defaults for ENGINE_HEAP_MIN and
ENGINE_HEAP_MAX to get that behavior, for example:

ENGINE_HEAP_MIN=128m
ENGINE_HEAP_MAX=1g

I am open to do that change, specially if you show me hard data proving
that it is better for performance ;-) .

> ----- Original Message -----
>> From: "Juan Hernandez" <jhernand at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Wednesday, September 12, 2012 2:32:28 PM
>> Subject: [Engine-devel] Fixed -Xmx could kill your JVM
>>
>> Hello,
>>
>> I recently discovered that I made a mistake in the engine service
>> script. The problem is that were running the JVM with the "-Xms
>> option
>> twice, like this:
>>
>>   java -Xms1g -Xms1g ...
>>
>> That doesn't correctly limit the max amount of heap space. It has
>> been
>> fixed in a recent commit [1], so with the updated code the engine
>> will
>> run by default like this:
>>
>>   java -Xms1g -Xmx1g ...
>>
>> With this change you could start to see OutOfMemory errors if your
>> engine was using a lot of memory, if you are running stress tests,
>> for
>> example. In that case remember that you can adjust this changing the
>> ENGINE_HEAP_MIN and ENGINE_HEAP_MAX parameters in
>> /etc/sysconfig/ovirt-engine. For example, to use 2g instead of the
>> default of 1g:
>>
>> ENGINE_HEAP_MIN=2g
>> ENGINE_HEAP_MAX=2g
>>
>> The restart the engine.
>>
>> Let me know if you have issues.
>>
>> Regards,
>> Juan Hernandez
>>
>> [1] http://gerrit.ovirt.org/7952
>>
>> --
>> Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
>> 3?D, 28016 Madrid, Spain
>> Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat
>> S.L.
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From alonbl at redhat.com  Wed Sep 12 13:06:43 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 12 Sep 2012 09:06:43 -0400 (EDT)
Subject: [Engine-devel] Fixed -Xmx could kill your JVM
In-Reply-To: <505086BD.7030303@redhat.com>
Message-ID: <1112951670.6139472.1347455203353.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: "Laszlo Hornyak" <lhornyak at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Wednesday, September 12, 2012 3:57:33 PM
> Subject: Re: [Engine-devel] Fixed -Xmx could kill your JVM

<snip>

> 
> 
> Worst case we can change the defaults for ENGINE_HEAP_MIN and
> ENGINE_HEAP_MAX to get that behavior, for example:
> 
> ENGINE_HEAP_MIN=128m
> ENGINE_HEAP_MAX=1g
> 
> I am open to do that change, specially if you show me hard data
> proving
> that it is better for performance ;-) .

We are talking about virtual memory, allocation of the heap is done as reserved, so there is no real impact on the system if memory is unused. When a reserved page is actually used it is committed.

Unless there is a system with small page file, there is no real reason to use lower values in HEAP_MIN.

Alon.


From sanjal at redhat.com  Wed Sep 12 14:38:33 2012
From: sanjal at redhat.com (Shireesh Anjal)
Date: Wed, 12 Sep 2012 20:08:33 +0530
Subject: [Engine-devel] Fixed -Xmx could kill your JVM
In-Reply-To: <1112951670.6139472.1347455203353.JavaMail.root@redhat.com>
References: <1112951670.6139472.1347455203353.JavaMail.root@redhat.com>
Message-ID: <50509E69.3000102@redhat.com>

On Wednesday 12 September 2012 06:36 PM, Alon Bar-Lev wrote:
>
> ----- Original Message -----
>> From: "Juan Hernandez" <jhernand at redhat.com>
>> To: "Laszlo Hornyak" <lhornyak at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Wednesday, September 12, 2012 3:57:33 PM
>> Subject: Re: [Engine-devel] Fixed -Xmx could kill your JVM
> <snip>
>
>>
>> Worst case we can change the defaults for ENGINE_HEAP_MIN and
>> ENGINE_HEAP_MAX to get that behavior, for example:
>>
>> ENGINE_HEAP_MIN=128m
>> ENGINE_HEAP_MAX=1g
>>
>> I am open to do that change, specially if you show me hard data
>> proving
>> that it is better for performance ;-) .
> We are talking about virtual memory, allocation of the heap is done as reserved, so there is no real impact on the system if memory is unused. When a reserved page is actually used it is committed.
>
> Unless there is a system with small page file, there is no real reason to use lower values in HEAP_MIN.
>
> Alon.

I've seen that Java experts from Sun (now Oracle) recommend to identify 
a good (high enough) value for the heap size and set the same value for 
both min and max, during the sessions on Java performance in the Java 
One conference.

> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From sanjal at redhat.com  Wed Sep 12 14:43:35 2012
From: sanjal at redhat.com (Shireesh Anjal)
Date: Wed, 12 Sep 2012 20:13:35 +0530
Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
Message-ID: <50509F97.1070801@redhat.com>




From alonbl at redhat.com  Wed Sep 12 14:45:18 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 12 Sep 2012 10:45:18 -0400 (EDT)
Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
In-Reply-To: <50509F97.1070801@redhat.com>
Message-ID: <267812570.6166390.1347461118470.JavaMail.root@redhat.com>

yes.

----- Original Message -----
> From: "Shireesh Anjal" <sanjal at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Wednesday, September 12, 2012 5:43:35 PM
> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
> 
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From shuming at linux.vnet.ibm.com  Wed Sep 12 14:50:14 2012
From: shuming at linux.vnet.ibm.com (Shu Ming)
Date: Wed, 12 Sep 2012 22:50:14 +0800
Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
In-Reply-To: <267812570.6166390.1347461118470.JavaMail.root@redhat.com>
References: <267812570.6166390.1347461118470.JavaMail.root@redhat.com>
Message-ID: <5050A126.1090100@linux.vnet.ibm.com>

It seems gerrit has downed for several times recently. Is there any 
special reason?
? 2012-9-12 22:45, Alon Bar-Lev:
> yes.
>
> ----- Original Message -----
>> From: "Shireesh Anjal" <sanjal at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Wednesday, September 12, 2012 5:43:35 PM
>> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
>>
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>


-- 
---
?? Shu Ming
Open Virtualization Engineerning; CSTL, IBM Corp.
Tel: 86-10-82451626  Tieline: 9051626 E-mail: shuming at cn.ibm.com or shuming at linux.vnet.ibm.com
Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian District, Beijing 100193, PRC




From alonbl at redhat.com  Wed Sep 12 14:52:39 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 12 Sep 2012 10:52:39 -0400 (EDT)
Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
In-Reply-To: <5050A126.1090100@linux.vnet.ibm.com>
Message-ID: <933533053.6168376.1347461559383.JavaMail.root@redhat.com>


Yes, I am experiencing this too...

Itamar?

----- Original Message -----
> From: "Shu Ming" <shuming at linux.vnet.ibm.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org, "VDSM Project Development"
> <vdsm-devel at lists.fedorahosted.org>
> Sent: Wednesday, September 12, 2012 5:50:14 PM
> Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
> 
> It seems gerrit has downed for several times recently. Is there any
> special reason?
> ? 2012-9-12 22:45, Alon Bar-Lev:
> > yes.
> >
> > ----- Original Message -----
> >> From: "Shireesh Anjal" <sanjal at redhat.com>
> >> To: engine-devel at ovirt.org
> >> Sent: Wednesday, September 12, 2012 5:43:35 PM
> >> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
> >>
> >>
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> >
> 
> 
> --
> ---
> ?? Shu Ming
> Open Virtualization Engineerning; CSTL, IBM Corp.
> Tel: 86-10-82451626  Tieline: 9051626 E-mail: shuming at cn.ibm.com or
> shuming at linux.vnet.ibm.com
> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
> District, Beijing 100193, PRC
> 
> 
> 


From ewoud+ovirt at kohlvanwijngaarden.nl  Wed Sep 12 15:13:25 2012
From: ewoud+ovirt at kohlvanwijngaarden.nl (Ewoud Kohl van Wijngaarden)
Date: Wed, 12 Sep 2012 17:13:25 +0200
Subject: [Engine-devel] [vdsm]  is gerrit.ovirt.org down? <eom>
In-Reply-To: <933533053.6168376.1347461559383.JavaMail.root@redhat.com>
References: <5050A126.1090100@linux.vnet.ibm.com>
	<933533053.6168376.1347461559383.JavaMail.root@redhat.com>
Message-ID: <20120912151324.GK9201@bogey.xentower.nl>

In cases like these it's best to mail infra as well, or poke on #ovirt.

On Wed, Sep 12, 2012 at 10:52:39AM -0400, Alon Bar-Lev wrote:
> 
> Yes, I am experiencing this too...
> 
> Itamar?
> 
> ----- Original Message -----
> > From: "Shu Ming" <shuming at linux.vnet.ibm.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org, "VDSM Project Development"
> > <vdsm-devel at lists.fedorahosted.org>
> > Sent: Wednesday, September 12, 2012 5:50:14 PM
> > Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
> > 
> > It seems gerrit has downed for several times recently. Is there any
> > special reason?
> > ? 2012-9-12 22:45, Alon Bar-Lev:
> > > yes.
> > >
> > > ----- Original Message -----
> > >> From: "Shireesh Anjal" <sanjal at redhat.com>
> > >> To: engine-devel at ovirt.org
> > >> Sent: Wednesday, September 12, 2012 5:43:35 PM
> > >> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>


From ashakarc at redhat.com  Wed Sep 12 15:23:04 2012
From: ashakarc at redhat.com (Asaf Shakarchi)
Date: Wed, 12 Sep 2012 11:23:04 -0400 (EDT)
Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
In-Reply-To: <933533053.6168376.1347461559383.JavaMail.root@redhat.com>
Message-ID: <864749424.9590586.1347463384753.JavaMail.root@redhat.com>

It happens from time to time, restart is required, Itamar only.

----- Original Message -----
> 
> Yes, I am experiencing this too...
> 
> Itamar?
> 
> ----- Original Message -----
> > From: "Shu Ming" <shuming at linux.vnet.ibm.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org,
> > "VDSM Project Development"
> > <vdsm-devel at lists.fedorahosted.org>
> > Sent: Wednesday, September 12, 2012 5:50:14 PM
> > Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
> > 
> > It seems gerrit has downed for several times recently. Is there any
> > special reason?
> > ? 2012-9-12 22:45, Alon Bar-Lev:
> > > yes.
> > >
> > > ----- Original Message -----
> > >> From: "Shireesh Anjal" <sanjal at redhat.com>
> > >> To: engine-devel at ovirt.org
> > >> Sent: Wednesday, September 12, 2012 5:43:35 PM
> > >> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
> > >>
> > >>
> > >> _______________________________________________
> > >> Engine-devel mailing list
> > >> Engine-devel at ovirt.org
> > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >
> > 
> > 
> > --
> > ---
> > ?? Shu Ming
> > Open Virtualization Engineerning; CSTL, IBM Corp.
> > Tel: 86-10-82451626  Tieline: 9051626 E-mail: shuming at cn.ibm.com or
> > shuming at linux.vnet.ibm.com
> > Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
> > District, Beijing 100193, PRC
> > 
> > 
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From iheim at redhat.com  Wed Sep 12 15:34:56 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 12 Sep 2012 18:34:56 +0300
Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
In-Reply-To: <864749424.9590586.1347463384753.JavaMail.root@redhat.com>
References: <864749424.9590586.1347463384753.JavaMail.root@redhat.com>
Message-ID: <5050ABA0.9090101@redhat.com>

On 09/12/2012 06:23 PM, Asaf Shakarchi wrote:
> It happens from time to time, restart is required, Itamar only.

restarted.
eyal - can we make progress on the jenkins job with permission to  more 
people to restart gerrit?
others - please email infra on gerrit issues (well, me personally always 
help as well)

>
> ----- Original Message -----
>>
>> Yes, I am experiencing this too...
>>
>> Itamar?
>>
>> ----- Original Message -----
>>> From: "Shu Ming" <shuming at linux.vnet.ibm.com>
>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org,
>>> "VDSM Project Development"
>>> <vdsm-devel at lists.fedorahosted.org>
>>> Sent: Wednesday, September 12, 2012 5:50:14 PM
>>> Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
>>>
>>> It seems gerrit has downed for several times recently. Is there any
>>> special reason?
>>> ? 2012-9-12 22:45, Alon Bar-Lev:
>>>> yes.
>>>>
>>>> ----- Original Message -----
>>>>> From: "Shireesh Anjal" <sanjal at redhat.com>
>>>>> To: engine-devel at ovirt.org
>>>>> Sent: Wednesday, September 12, 2012 5:43:35 PM
>>>>> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
>>>
>>>
>>> --
>>> ---
>>> ?? Shu Ming
>>> Open Virtualization Engineerning; CSTL, IBM Corp.
>>> Tel: 86-10-82451626  Tieline: 9051626 E-mail: shuming at cn.ibm.com or
>>> shuming at linux.vnet.ibm.com
>>> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
>>> District, Beijing 100193, PRC
>>>
>>>
>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>




From eedri at redhat.com  Wed Sep 12 15:56:44 2012
From: eedri at redhat.com (Eyal Edri)
Date: Wed, 12 Sep 2012 11:56:44 -0400 (EDT)
Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
In-Reply-To: <5050ABA0.9090101@redhat.com>
Message-ID: <1406165301.4334192.1347465404631.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Asaf Shakarchi" <ashakarc at redhat.com>
> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org, "VDSM Project
> Development" <vdsm-devel at lists.fedorahosted.org>, "Shu Ming" <shuming at linux.vnet.ibm.com>, "Eyal Edri"
> <eedri at redhat.com>
> Sent: Wednesday, September 12, 2012 6:34:56 PM
> Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
> 
> On 09/12/2012 06:23 PM, Asaf Shakarchi wrote:
> > It happens from time to time, restart is required, Itamar only.
> 
> restarted.
> eyal - can we make progress on the jenkins job with permission to
>  more
> people to restart gerrit?

the job is ready http://jenkins.ovirt.org/view/system-monitoring/job/restart_gerrit_service
but i need to have jenkins user access to gerrit server + sudo access to run 'service' restart... 

it has access to www.ovirt.org but not to gerrit.ovirt.org. 

> others - please email infra on gerrit issues (well, me personally
> always
> help as well)
> 
> >
> > ----- Original Message -----
> >>
> >> Yes, I am experiencing this too...
> >>
> >> Itamar?
> >>
> >> ----- Original Message -----
> >>> From: "Shu Ming" <shuming at linux.vnet.ibm.com>
> >>> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >>> Cc: "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org,
> >>> "VDSM Project Development"
> >>> <vdsm-devel at lists.fedorahosted.org>
> >>> Sent: Wednesday, September 12, 2012 5:50:14 PM
> >>> Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
> >>>
> >>> It seems gerrit has downed for several times recently. Is there
> >>> any
> >>> special reason?
> >>> ? 2012-9-12 22:45, Alon Bar-Lev:
> >>>> yes.
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Shireesh Anjal" <sanjal at redhat.com>
> >>>>> To: engine-devel at ovirt.org
> >>>>> Sent: Wednesday, September 12, 2012 5:43:35 PM
> >>>>> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Engine-devel mailing list
> >>>>> Engine-devel at ovirt.org
> >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>>
> >>>> _______________________________________________
> >>>> Engine-devel mailing list
> >>>> Engine-devel at ovirt.org
> >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>
> >>>
> >>>
> >>> --
> >>> ---
> >>> ?? Shu Ming
> >>> Open Virtualization Engineerning; CSTL, IBM Corp.
> >>> Tel: 86-10-82451626  Tieline: 9051626 E-mail: shuming at cn.ibm.com
> >>> or
> >>> shuming at linux.vnet.ibm.com
> >>> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
> >>> District, Beijing 100193, PRC
> >>>
> >>>
> >>>
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>
> 
> 
>


From agl at us.ibm.com  Wed Sep 12 17:15:06 2012
From: agl at us.ibm.com (Adam Litke)
Date: Wed, 12 Sep 2012 12:15:06 -0500
Subject: [Engine-devel] [vdsm]  is gerrit.ovirt.org down? <eom>
In-Reply-To: <1406165301.4334192.1347465404631.JavaMail.root@redhat.com>
References: <5050ABA0.9090101@redhat.com>
	<1406165301.4334192.1347465404631.JavaMail.root@redhat.com>
Message-ID: <20120912171506.GB5415@aglitke>

So the fix is to just regularly restart gerrit?  Do we have any idea about the
real, underlying problem?

On Wed, Sep 12, 2012 at 11:56:44AM -0400, Eyal Edri wrote:
> 
> 
> ----- Original Message -----
> > From: "Itamar Heim" <iheim at redhat.com>
> > To: "Asaf Shakarchi" <ashakarc at redhat.com>
> > Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org, "VDSM Project
> > Development" <vdsm-devel at lists.fedorahosted.org>, "Shu Ming" <shuming at linux.vnet.ibm.com>, "Eyal Edri"
> > <eedri at redhat.com>
> > Sent: Wednesday, September 12, 2012 6:34:56 PM
> > Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
> > 
> > On 09/12/2012 06:23 PM, Asaf Shakarchi wrote:
> > > It happens from time to time, restart is required, Itamar only.
> > 
> > restarted.
> > eyal - can we make progress on the jenkins job with permission to
> >  more
> > people to restart gerrit?
> 
> the job is ready http://jenkins.ovirt.org/view/system-monitoring/job/restart_gerrit_service
> but i need to have jenkins user access to gerrit server + sudo access to run 'service' restart... 
> 
> it has access to www.ovirt.org but not to gerrit.ovirt.org. 
> 
> > others - please email infra on gerrit issues (well, me personally
> > always
> > help as well)
> > 
> > >
> > > ----- Original Message -----
> > >>
> > >> Yes, I am experiencing this too...
> > >>
> > >> Itamar?
> > >>
> > >> ----- Original Message -----
> > >>> From: "Shu Ming" <shuming at linux.vnet.ibm.com>
> > >>> To: "Alon Bar-Lev" <alonbl at redhat.com>
> > >>> Cc: "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org,
> > >>> "VDSM Project Development"
> > >>> <vdsm-devel at lists.fedorahosted.org>
> > >>> Sent: Wednesday, September 12, 2012 5:50:14 PM
> > >>> Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
> > >>>
> > >>> It seems gerrit has downed for several times recently. Is there
> > >>> any
> > >>> special reason?
> > >>> ? 2012-9-12 22:45, Alon Bar-Lev:
> > >>>> yes.
> > >>>>
> > >>>> ----- Original Message -----
> > >>>>> From: "Shireesh Anjal" <sanjal at redhat.com>
> > >>>>> To: engine-devel at ovirt.org
> > >>>>> Sent: Wednesday, September 12, 2012 5:43:35 PM
> > >>>>> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
> > >>>>>
> > >>>>>
> > >>>>> _______________________________________________
> > >>>>> Engine-devel mailing list
> > >>>>> Engine-devel at ovirt.org
> > >>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>>>>
> > >>>> _______________________________________________
> > >>>> Engine-devel mailing list
> > >>>> Engine-devel at ovirt.org
> > >>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>> ---
> > >>> ?? Shu Ming
> > >>> Open Virtualization Engineerning; CSTL, IBM Corp.
> > >>> Tel: 86-10-82451626  Tieline: 9051626 E-mail: shuming at cn.ibm.com
> > >>> or
> > >>> shuming at linux.vnet.ibm.com
> > >>> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
> > >>> District, Beijing 100193, PRC
> > >>>
> > >>>
> > >>>
> > >> _______________________________________________
> > >> Engine-devel mailing list
> > >> Engine-devel at ovirt.org
> > >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> > >>
> > 
> > 
> >
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

-- 
Adam Litke <agl at us.ibm.com>
IBM Linux Technology Center



From iheim at redhat.com  Wed Sep 12 20:45:21 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 12 Sep 2012 23:45:21 +0300
Subject: [Engine-devel] [vdsm]  is gerrit.ovirt.org down? <eom>
In-Reply-To: <5050A126.1090100@linux.vnet.ibm.com>
References: <267812570.6166390.1347461118470.JavaMail.root@redhat.com>
	<5050A126.1090100@linux.vnet.ibm.com>
Message-ID: <5050F461.3050009@redhat.com>

On 09/12/2012 05:50 PM, Shu Ming wrote:
> It seems gerrit has downed for several times recently. Is there any
> special reason?

I assume the jenkins/gerrit integration is overloading it a bit.


> ? 2012-9-12 22:45, Alon Bar-Lev:
>> yes.
>>
>> ----- Original Message -----
>>> From: "Shireesh Anjal" <sanjal at redhat.com>
>>> To: engine-devel at ovirt.org
>>> Sent: Wednesday, September 12, 2012 5:43:35 PM
>>> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
>>>
>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>
>




From iheim at redhat.com  Wed Sep 12 22:48:26 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 13 Sep 2012 01:48:26 +0300
Subject: [Engine-devel] [vdsm]  is gerrit.ovirt.org down? <eom>
In-Reply-To: <20120912171506.GB5415@aglitke>
References: <5050ABA0.9090101@redhat.com>
	<1406165301.4334192.1347465404631.JavaMail.root@redhat.com>
	<20120912171506.GB5415@aglitke>
Message-ID: <5051113A.3040406@redhat.com>

On 09/12/2012 08:15 PM, Adam Litke wrote:
> So the fix is to just regularly restart gerrit?  Do we have any idea about the
> real, underlying problem?

the error is in jetty, i didn't look at fixing the core issue in jetty.
moving to another container is possible, but makes managing it for 
upgrades much less trivial.
so far most threads around this on google mention a cron to detect this 
and restart.
I actually expect moving to a hosted physical server will make this less 
of an issue based on the previous experience of extending the EC2 instance.

if it won't help, we may need to tackle the core jetty issue ourselves.

>
> On Wed, Sep 12, 2012 at 11:56:44AM -0400, Eyal Edri wrote:
>>
>>
>> ----- Original Message -----
>>> From: "Itamar Heim" <iheim at redhat.com>
>>> To: "Asaf Shakarchi" <ashakarc at redhat.com>
>>> Cc: "Alon Bar-Lev" <alonbl at redhat.com>, "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org, "VDSM Project
>>> Development" <vdsm-devel at lists.fedorahosted.org>, "Shu Ming" <shuming at linux.vnet.ibm.com>, "Eyal Edri"
>>> <eedri at redhat.com>
>>> Sent: Wednesday, September 12, 2012 6:34:56 PM
>>> Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
>>>
>>> On 09/12/2012 06:23 PM, Asaf Shakarchi wrote:
>>>> It happens from time to time, restart is required, Itamar only.
>>>
>>> restarted.
>>> eyal - can we make progress on the jenkins job with permission to
>>>   more
>>> people to restart gerrit?
>>
>> the job is ready http://jenkins.ovirt.org/view/system-monitoring/job/restart_gerrit_service
>> but i need to have jenkins user access to gerrit server + sudo access to run 'service' restart...
>>
>> it has access to www.ovirt.org but not to gerrit.ovirt.org.
>>
>>> others - please email infra on gerrit issues (well, me personally
>>> always
>>> help as well)
>>>
>>>>
>>>> ----- Original Message -----
>>>>>
>>>>> Yes, I am experiencing this too...
>>>>>
>>>>> Itamar?
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Shu Ming" <shuming at linux.vnet.ibm.com>
>>>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>>>> Cc: "Shireesh Anjal" <sanjal at redhat.com>, engine-devel at ovirt.org,
>>>>>> "VDSM Project Development"
>>>>>> <vdsm-devel at lists.fedorahosted.org>
>>>>>> Sent: Wednesday, September 12, 2012 5:50:14 PM
>>>>>> Subject: Re: [Engine-devel] is gerrit.ovirt.org down? <eom>
>>>>>>
>>>>>> It seems gerrit has downed for several times recently. Is there
>>>>>> any
>>>>>> special reason?
>>>>>> ? 2012-9-12 22:45, Alon Bar-Lev:
>>>>>>> yes.
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Shireesh Anjal" <sanjal at redhat.com>
>>>>>>>> To: engine-devel at ovirt.org
>>>>>>>> Sent: Wednesday, September 12, 2012 5:43:35 PM
>>>>>>>> Subject: [Engine-devel] is gerrit.ovirt.org down? <eom>
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Engine-devel mailing list
>>>>>>>> Engine-devel at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Engine-devel mailing list
>>>>>>> Engine-devel at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> ---
>>>>>> ?? Shu Ming
>>>>>> Open Virtualization Engineerning; CSTL, IBM Corp.
>>>>>> Tel: 86-10-82451626  Tieline: 9051626 E-mail: shuming at cn.ibm.com
>>>>>> or
>>>>>> shuming at linux.vnet.ibm.com
>>>>>> Address: 3/F Ring Building, ZhongGuanCun Software Park, Haidian
>>>>>> District, Beijing 100193, PRC
>>>>>>
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>>>
>>>
>>>
>> _______________________________________________
>> vdsm-devel mailing list
>> vdsm-devel at lists.fedorahosted.org
>> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>




From emesika at redhat.com  Thu Sep 13 07:46:41 2012
From: emesika at redhat.com (Eli Mesika)
Date: Thu, 13 Sep 2012 03:46:41 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <1202206087.6136904.1347454633345.JavaMail.root@redhat.com>
Message-ID: <1605287998.68861376.1347522401524.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Alon Bar-Lev" <alonbl at redhat.com>
> To: "Eli Mesika" <emesika at redhat.com>
> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> Sent: Wednesday, September 12, 2012 3:57:13 PM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> 
> Hello Eli,
> 
> ----- Original Message -----
> > From: "Eli Mesika" <emesika at redhat.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> > Sent: Wednesday, September 12, 2012 3:48:00 PM
> > Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default
> > options
> > 
> > > > >>> QUESTIONS
> > > > >>>
> > > > >>> 1. Why do we store default values in database?
> > 
> > I wanted to add few important points
> > a) We have default values in DB in order to enable overriding
> > values
> > in the 0000_config.sql file only if a customer did not change the
> > default value, if a customer changed the default in some entries,
> > we
> > want to honour the customer settings
> 
> Default != value.
> Default is what should be used if not specified explicitly
> 
> Current implementation puts the default value as a value, so you
> cannot distinguish between what user enforced.
> 
> When a default is changed, because of field feedback, we should push
> this into the database instead of keeping in database only options
> that were modified by the user and fetch the default from the option
> metadata.
> 
> In another words.... there should be no 000_config.sql, the option
> table should be empty as long as the user does not modify any of the
> options.
> 

Lets assume that we are going for it, how would you upgrade from the current state to your suggested solution keeping all user current settings ?

> > b)There may be other requirements on configuration that are easier
> > to
> > manage in the database, for example, I have heard that one of the
> > suggested features regarding configuration is to keep a kind of
> > configuration history and know exactly which configuration values
> > was changed, when and by whom.
> 
> There is no conflict. You can have audit table when modifying
> options.
> Keep in mind that I discuss the option metadata.
> I believe you are discussing the option data.
You are right 

> 
> > c) The version mechanism in the config is working like this : there
> > is a 'general' version , means that this value is not version
> > dependant, or the version can be a real version like 3.1 3.2 etc ,
> > in such case the value is version dependant and an entry is
> > required
> > for each version.
> 
> Again, there is no conflict, as the default value within the metadata
> can be version specific too.
I agree , however we should consider all changes in current code + tools + upgrade since this seems a major change
> 
> Having said that, I don't understand how two different versions can
> work with the different data models and share one database and
> options.
> 
> > I agree however, that the default value in the code is redundant
> > and
> > error prone and should be removed.
> 
> From what you wrote above, I don't understand how you reached to this
> conclusion. Can you please explain?
I mean that we have now defaults in code (ConfigValues.java) and in the database they may not match.
I think as you that only one place defined the defaults, so , it should be removed from code which is less flexible if we want to change something without recompiling ...
> 
> Thanks,
> Alon.
> 


From alonbl at redhat.com  Thu Sep 13 08:36:34 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Thu, 13 Sep 2012 04:36:34 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <1605287998.68861376.1347522401524.JavaMail.root@redhat.com>
Message-ID: <1425474616.6319236.1347525394582.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Eli Mesika" <emesika at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> Sent: Thursday, September 13, 2012 10:46:41 AM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> 
> 
> ----- Original Message -----
> > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > To: "Eli Mesika" <emesika at redhat.com>
> > Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> > Sent: Wednesday, September 12, 2012 3:57:13 PM
> > Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default
> > options
> > 
> > 
> > Hello Eli,
> > 
> > ----- Original Message -----
> > > From: "Eli Mesika" <emesika at redhat.com>
> > > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > > Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> > > Sent: Wednesday, September 12, 2012 3:48:00 PM
> > > Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
> > > default
> > > options
> > > 
> > > > > >>> QUESTIONS
> > > > > >>>
> > > > > >>> 1. Why do we store default values in database?
> > > 
> > > I wanted to add few important points
> > > a) We have default values in DB in order to enable overriding
> > > values
> > > in the 0000_config.sql file only if a customer did not change the
> > > default value, if a customer changed the default in some entries,
> > > we
> > > want to honour the customer settings
> > 
> > Default != value.
> > Default is what should be used if not specified explicitly
> > 
> > Current implementation puts the default value as a value, so you
> > cannot distinguish between what user enforced.
> > 
> > When a default is changed, because of field feedback, we should
> > push
> > this into the database instead of keeping in database only options
> > that were modified by the user and fetch the default from the
> > option
> > metadata.
> > 
> > In another words.... there should be no 000_config.sql, the option
> > table should be empty as long as the user does not modify any of
> > the
> > options.
> > 
> 
> Lets assume that we are going for it, how would you upgrade from the
> current state to your suggested solution keeping all user current
> settings ?

Either by:

1. leaving all existing data within database... so current users will not enjoy updating defaults in future. I really don't like this solution, but it will work.

2. during upgrade, go over the values, and remove all that matches the metadata default. This way, future change in metadata will apply.

> 
> > > b)There may be other requirements on configuration that are
> > > easier
> > > to
> > > manage in the database, for example, I have heard that one of the
> > > suggested features regarding configuration is to keep a kind of
> > > configuration history and know exactly which configuration values
> > > was changed, when and by whom.
> > 
> > There is no conflict. You can have audit table when modifying
> > options.
> > Keep in mind that I discuss the option metadata.
> > I believe you are discussing the option data.
> You are right
> 
> > 
> > > c) The version mechanism in the config is working like this :
> > > there
> > > is a 'general' version , means that this value is not version
> > > dependant, or the version can be a real version like 3.1 3.2 etc
> > > ,
> > > in such case the value is version dependant and an entry is
> > > required
> > > for each version.
> > 
> > Again, there is no conflict, as the default value within the
> > metadata
> > can be version specific too.
> I agree , however we should consider all changes in current code +
> tools + upgrade since this seems a major change

Right. I outlined all changes I collected in the initial message.

> > 
> > Having said that, I don't understand how two different versions can
> > work with the different data models and share one database and
> > options.
> > 
> > > I agree however, that the default value in the code is redundant
> > > and
> > > error prone and should be removed.
> > 
> > From what you wrote above, I don't understand how you reached to
> > this
> > conclusion. Can you please explain?
> I mean that we have now defaults in code (ConfigValues.java) and in
> the database they may not match.
> I think as you that only one place defined the defaults, so , it
> should be removed from code which is less flexible if we want to
> change something without recompiling ...

If we decide to use java annotations for metadata, the default will be in annotation.
If we decide to use XML for metadata, the default will be in XML.
At any case, I would like to reach to a state where the metadata is at one place, and not spread between code, property/xml, database.

Thanks,
Alon


From jhernand at redhat.com  Thu Sep 13 12:45:34 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Thu, 13 Sep 2012 14:45:34 +0200
Subject: [Engine-devel] Update on UI Plugins: PoC patch revision 4
In-Reply-To: <627709453.17137271.1346760290602.JavaMail.root@redhat.com>
References: <627709453.17137271.1346760290602.JavaMail.root@redhat.com>
Message-ID: <5051D56E.2010906@redhat.com>

On 09/04/2012 02:04 PM, Vojtech Szocs wrote:
> Hi Juan, thanks for your comments :)
> 
> Server-side components of UI plugin infrastructure (such as PluginSourcePageServlet) indeed need some more work, I agree with your points.
> 
> I was thinking that PluginSourcePageServlet and FileServlet are quite similar in their purpose, serving static resources from local filesystem. FileServlet is intended for general use, with 'file' parameter configured as servlet init-param. For example, FileServlet could be used to serve static resources from /usr/share/ovirt-engine/ui-plugins:
> 
> <servlet>
>   <servlet-name>pluginResourceServlet</servlet-name>
>   <servlet-class>org.ovirt.engine.core.FileServlet</servlet-class>
>   <init-param>
>     <param-name>file</param-name>
>     <param-value>/usr/share/ovirt-engine/ui-plugins</param-value>
>   </init-param>
> </servlet>
> <servlet-mapping>
>   <servlet-name>pluginResourceServlet</servlet-name>
>   <url-pattern>/plugins/*</url-pattern>
> </servlet-mapping>
> 
> Assuming following directory convention for UI plugin descriptors and actual plugin resources:
> /usr/share/ovirt-engine/ui-plugins/foo.json -> Plugin descriptor
> /usr/share/ovirt-engine/ui-plugins/foo/start.html -> Plugin host page
> /usr/share/ovirt-engine/ui-plugins/foo/foo.js -> Actual plugin code (referenced by plugin host page)
> 
> Such servlet could be used to map "http(s)://<EngineHost>:8700/plugins/foo/start.html" to /usr/share/ovirt-engine/ui-plugins/foo/start.html
> (note that FileServlet is in root WAR context)

Using the FileServlet in this way looks perfectly ok from my point of
view. Only thing to tkae into account is that then we should not put any
content that is not to be public in the
/usr/share/ovirt-engine/ui-plugins directory as the FileServlet has no
mechanism to filter files: it serves anything inside that directory.

> 
> The purpose of PluginSourcePageServlet is very similar, but in terms of FileServlet, the 'file' parameter is not static (defined in web.xml as init-param), but depends on plugin meta-data (defined in foo.json) for each plugin.
> 
> PluginSourcePageServlet was meant to map "http(s)://<EngineHost>:8700/webadmin/webadmin/plugin/foo/start.html" to /custom/plugin/base/directory/start.html
> (note that PluginSourcePageServlet is in WebAdmin WAR context)
> 
> Juan - do you think we could modify/reuse FileServlet for serving UI plugin static resources? As mentioned above, the only difference is that the 'file' parameter (base directory) would be potentially different for each plugin. Please let me know what you think about it.

I would rather move the useful methods from FileServlet to the
ServletUtils class (isSane, loading of MIME types, probably more) and
then we can use them from FileServlet and PluginSourcePageServket, thus
simplifying both.


> ----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: "Vojtech Szocs" <vszocs at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>
> Sent: Thursday, August 30, 2012 8:24:02 PM
> Subject: Re: [Engine-devel] Update on UI Plugins: PoC patch revision 4
> 
> Nice work Vojtech, just some comments about the PluginSourcePageServlet:
> 
> * You can avoid the hardcoded plugin code location with something like this:
> 
>   import org.ovirt.engine.core.utils.LocalConfig;
> 
>   File dataDir = LocalConfig.getInstance().getUsrDir();
>   File pluginCodeLocation = new File(etcDir, "ui-plugins");
> 
> That will result in /usr/share/ovirt-engine/ui-plugins or whatever
> directory is configured in the ENGINE_USR parameter in the
> /etc/sysconfig/ovirt-engine file.
> 
> * It is very important to check the sanity of the value of the "plugin"
> parameter, otherwise an attacker could send you a name with backpaths,
> and that can result in accessing an unexpected file. In this particular
> case you are adding the ".js" extension, so it won't probably result in
> accessing dangerous files, but anyhow it is a good practice. I would
> recommend to do something like this:
> 
>   String pluginName = request.getParameter("plugin");
>   if (pluginName != null || !isSane(pluginName)) {
>       ...
>   }
> 
> The "isSane" method can do something similar to the "isSane" method in
> the "FileServlet" class (I think you already mentioned this at some
> point), maybe even forbid slashes as well.
> 
> * When copying the plugin file to the generated page you can avoid the
> extra Buffered reader/writer as you are already using your own buffer in
> the "copyChars" method (which is very good practice).
> 
> For the output you can directly use "response.getWriter()" instead of
> "response.getOutputStream()", that is already buffered by the container.
> 
> On 08/30/2012 05:39 PM, Vojtech Szocs wrote:
>>
>>
>> Hello everyone, 
>>
>> as a follow-up to my last email on improving plugin API, here comes the latest revision of UI Plugins proof-of-concept patch (please find it attached). 
>>
>> This patch is focused on improving JavaScript plugin API, along with important changes and improvements made to plugin infrastructure ( PluginManager ). Let's walk through the changes step by step. 
>>
>>
>>
>> Improved plugin API, taking some inspiration from jQuery 
>>
>> Following is a sample plugin code that uses new plugin API: 
>>
>> var myPlugin = pluginApi('myPlugin'); // Obtain plugin API instance for 'myPlugin' 
>> var myPluginConfig = myPlugin.configObject(); // Obtain plugin-specific configuration 
>>
>> // Register event handler functions to be invoked by WebAdmin 
>> // Note: all functions are optional, the plugin only defines functions for events it wants to handle 
>> myPlugin.register({ 
>> UiInit: function() { 
>> var testUrl = 'http://www.example.com/' + myPluginConfig.foo; // Assume plugin configuration has 'foo' attribute 
>> myPlugin.ui.addMainTab('Custom Tab', 'custom-tab', testUrl); // Invoke some operation using plugin API 
>> } 
>> }); 
>>
>> myPlugin.ready(); // Event handler functions are registered, we are now ready to get initialized (UiInit) 
>>
>>
>>
>> UI plugin life-cycle, enforced by plugin infrastructure 
>>
>> The PluginState enumeration lists possible states of a plugin during its runtime: 
>>
>>     * DEFINED : This is the initial state for all plugins. Plugin meta-data has been read by PluginManager and the corresponding iframe element has been created for the plugin. Note that at this point, the iframe element is not attached to DOM yet. 
>>     * LOADING : The iframe element for the plugin has been attached to DOM, which causes plugin host page (previously known as plugin source page) to be fetched asynchronously in the background. We are now waiting for plugin to report in as ready. In practice, due to JavaScript runtime being single-threaded, WebAdmin startup logic will continue to execute until the JavaScript runtime is "idle" (browser event loop returns), and at this point JavaScript plugin code gets invoked through the plugin host page. 
>>     * READY : The plugin has indicated that it is ready for use. We assume the plugin has already registered its event handler object (object containing various event handler functions to be called by WebAdmin) at this point. We can now proceed with plugin initialization. 
>>     * INITIALIZED : The plugin has been initialized by calling UiInit function on its event handler object. We can now call other event handler functions, the plugin is now initialized and in use. 
>>
>>
>> Note on plugin initialization: the UiInit function will be called just once during the lifetime of the plugin, after the plugin reports in as ready AND WebAdmin enters the state that allows plugins to be invoked (entering main section for logged-in users), and before other event handler functions are invoked by the plugin infrastructure. 
>>
>>
>>
>>
>> Plugin meta-data is now passed to client using different format 
>>
>>
>> Previously, plugin meta-data was embedded into WebAdmin host page as a simple JavaScript object, like so: 
>>
>>
>> var pluginDefinitions = { myPlugin: "<URL>", anotherPlugin: "<URL>" } 
>>
>>
>>
>> Now, plugin meta-data is embedded into WebAdmin host page as a JavaScript array, like so: 
>>
>>
>>
>> var pluginDefinitions = [ 
>> { name: "myPlugin", url: "<URL>", config: { "foo": 1, "bar": "whatever" } }, 
>> { name: "anotherPlugin", url: "<URL>" } 
>>
>> ]; 
>>
>>
>> As you can see, pluginDefinitions is now an array of JavaScript objects, with each object representing plugin meta-data. The "name" and "url" attributes are mandatory (we need to check them when loading plugin descriptors). "config" is the plugin configuration (JSON) object, obtained by merging default plugin configuration (defined in plugin descriptor) with custom plugin configuration (defined in external plugin configuration file). Note that the "config" attribute is optional. 
>>
>>
>>
>> In terms of Java classes, pluginDefinitions is mapped to PluginDefinitions overlay type, and each meta-data object within the array is mapped to PluginMetaData overlay type. 
>>
>>
>>
>>
>>
>> Note on using assert statements in client code: you might notice that I'm using a lot of assert statements in Plugin class. This is to ensure consistency and guard against corrupted state during development. In GWT, assert statements work in a different way than in standard Java VM. When debugging GWT application using Development Mode, assert statements are checked and throw assertion errors during runtime (they are displayed in Development Mode console). However, when compiling GWT application to JavaScript (Production Mode), assert statements are removed by GWT compiler, so they don't affect the application running in Production Mode. 
>>
>>
>>
>> Cheers, 
>> Vojtech 
>>
>>
>>
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> 
> 


-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From iheim at redhat.com  Thu Sep 13 14:50:06 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 13 Sep 2012 17:50:06 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <1425474616.6319236.1347525394582.JavaMail.root@redhat.com>
References: <1425474616.6319236.1347525394582.JavaMail.root@redhat.com>
Message-ID: <5051F29E.2010000@redhat.com>

On 09/13/2012 11:36 AM, Alon Bar-Lev wrote:
>
>
> ----- Original Message -----
>> From: "Eli Mesika" <emesika at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
>> Sent: Thursday, September 13, 2012 10:46:41 AM
>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
>>
>>
>>
>> ----- Original Message -----
>>> From: "Alon Bar-Lev" <alonbl at redhat.com>
>>> To: "Eli Mesika" <emesika at redhat.com>
>>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
>>> Sent: Wednesday, September 12, 2012 3:57:13 PM
>>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default
>>> options
>>>
>>>
>>> Hello Eli,
>>>
>>> ----- Original Message -----
>>>> From: "Eli Mesika" <emesika at redhat.com>
>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
>>>> Sent: Wednesday, September 12, 2012 3:48:00 PM
>>>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
>>>> default
>>>> options
>>>>
>>>>>>>>> QUESTIONS
>>>>>>>>>
>>>>>>>>> 1. Why do we store default values in database?
>>>>
>>>> I wanted to add few important points
>>>> a) We have default values in DB in order to enable overriding
>>>> values
>>>> in the 0000_config.sql file only if a customer did not change the
>>>> default value, if a customer changed the default in some entries,
>>>> we
>>>> want to honour the customer settings
>>>
>>> Default != value.
>>> Default is what should be used if not specified explicitly
>>>
>>> Current implementation puts the default value as a value, so you
>>> cannot distinguish between what user enforced.
>>>
>>> When a default is changed, because of field feedback, we should
>>> push
>>> this into the database instead of keeping in database only options
>>> that were modified by the user and fetch the default from the
>>> option
>>> metadata.
>>>
>>> In another words.... there should be no 000_config.sql, the option
>>> table should be empty as long as the user does not modify any of
>>> the
>>> options.
>>>
>>
>> Lets assume that we are going for it, how would you upgrade from the
>> current state to your suggested solution keeping all user current
>> settings ?
>
> Either by:
>
> 1. leaving all existing data within database... so current users will not enjoy updating defaults in future. I really don't like this solution, but it will work.
>
> 2. during upgrade, go over the values, and remove all that matches the metadata default. This way, future change in metadata will apply.
>
>>
>>>> b)There may be other requirements on configuration that are
>>>> easier
>>>> to
>>>> manage in the database, for example, I have heard that one of the
>>>> suggested features regarding configuration is to keep a kind of
>>>> configuration history and know exactly which configuration values
>>>> was changed, when and by whom.
>>>
>>> There is no conflict. You can have audit table when modifying
>>> options.
>>> Keep in mind that I discuss the option metadata.
>>> I believe you are discussing the option data.
>> You are right
>>
>>>
>>>> c) The version mechanism in the config is working like this :
>>>> there
>>>> is a 'general' version , means that this value is not version
>>>> dependant, or the version can be a real version like 3.1 3.2 etc
>>>> ,
>>>> in such case the value is version dependant and an entry is
>>>> required
>>>> for each version.
>>>
>>> Again, there is no conflict, as the default value within the
>>> metadata
>>> can be version specific too.
>> I agree , however we should consider all changes in current code +
>> tools + upgrade since this seems a major change
>
> Right. I outlined all changes I collected in the initial message.
>
>>>
>>> Having said that, I don't understand how two different versions can
>>> work with the different data models and share one database and
>>> options.
>>>
>>>> I agree however, that the default value in the code is redundant
>>>> and
>>>> error prone and should be removed.
>>>
>>>  From what you wrote above, I don't understand how you reached to
>>> this
>>> conclusion. Can you please explain?
>> I mean that we have now defaults in code (ConfigValues.java) and in
>> the database they may not match.
>> I think as you that only one place defined the defaults, so , it
>> should be removed from code which is less flexible if we want to
>> change something without recompiling ...
>
> If we decide to use java annotations for metadata, the default will be in annotation.
> If we decide to use XML for metadata, the default will be in XML.
> At any case, I would like to reach to a state where the metadata is at one place, and not spread between code, property/xml, database.

I may be missing something.
are you suggesting moving all the vdc_options table to an xml file?
I assume it will be placed under /etc?

then if we update the rpm, the edited config will remain unmodified, and 
we'll get the new one as rpmsave (iirc).
then during upgrade process we'll need to merge these files?


From yzaslavs at redhat.com  Thu Sep 13 14:57:48 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Thu, 13 Sep 2012 17:57:48 +0300
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <5051F29E.2010000@redhat.com>
References: <1425474616.6319236.1347525394582.JavaMail.root@redhat.com>
	<5051F29E.2010000@redhat.com>
Message-ID: <5051F46C.5070402@redhat.com>



On 09/13/2012 05:50 PM, Itamar Heim wrote:
> On 09/13/2012 11:36 AM, Alon Bar-Lev wrote:
>>
>>
>> ----- Original Message -----
>>> From: "Eli Mesika" <emesika at redhat.com>
>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
>>> Sent: Thursday, September 13, 2012 10:46:41 AM
>>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default
>>> options
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Alon Bar-Lev" <alonbl at redhat.com>
>>>> To: "Eli Mesika" <emesika at redhat.com>
>>>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
>>>> Sent: Wednesday, September 12, 2012 3:57:13 PM
>>>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default
>>>> options
>>>>
>>>>
>>>> Hello Eli,
>>>>
>>>> ----- Original Message -----
>>>>> From: "Eli Mesika" <emesika at redhat.com>
>>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
>>>>> Sent: Wednesday, September 12, 2012 3:48:00 PM
>>>>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
>>>>> default
>>>>> options
>>>>>
>>>>>>>>>> QUESTIONS
>>>>>>>>>>
>>>>>>>>>> 1. Why do we store default values in database?
>>>>>
>>>>> I wanted to add few important points
>>>>> a) We have default values in DB in order to enable overriding
>>>>> values
>>>>> in the 0000_config.sql file only if a customer did not change the
>>>>> default value, if a customer changed the default in some entries,
>>>>> we
>>>>> want to honour the customer settings
>>>>
>>>> Default != value.
>>>> Default is what should be used if not specified explicitly
I agree here, maybe we should consider a future possibility of "reset to 
default value"
>>>>
>>>> Current implementation puts the default value as a value, so you
>>>> cannot distinguish between what user enforced.
>>>>
>>>> When a default is changed, because of field feedback, we should
>>>> push
>>>> this into the database instead of keeping in database only options
>>>> that were modified by the user and fetch the default from the
>>>> option
>>>> metadata.
>>>>
>>>> In another words.... there should be no 000_config.sql, the option
>>>> table should be empty as long as the user does not modify any of
>>>> the
>>>> options.

Not sure I fully understand how upgrade of configuration will look?
I'm talking about update of existing data (+meta data - for example, 
change of existing description) vs introducing of new data (+ metadata)
>>>>
>>>
>>> Lets assume that we are going for it, how would you upgrade from the
>>> current state to your suggested solution keeping all user current
>>> settings ?
>>
>> Either by:
>>
>> 1. leaving all existing data within database... so current users will
>> not enjoy updating defaults in future. I really don't like this
>> solution, but it will work.
>>
>> 2. during upgrade, go over the values, and remove all that matches the
>> metadata default. This way, future change in metadata will apply.
>>
>>>
>>>>> b)There may be other requirements on configuration that are
>>>>> easier
>>>>> to
>>>>> manage in the database, for example, I have heard that one of the
>>>>> suggested features regarding configuration is to keep a kind of
>>>>> configuration history and know exactly which configuration values
>>>>> was changed, when and by whom.
>>>>
>>>> There is no conflict. You can have audit table when modifying
>>>> options.
>>>> Keep in mind that I discuss the option metadata.
>>>> I believe you are discussing the option data.
>>> You are right
>>>
>>>>
>>>>> c) The version mechanism in the config is working like this :
>>>>> there
>>>>> is a 'general' version , means that this value is not version
>>>>> dependant, or the version can be a real version like 3.1 3.2 etc
>>>>> ,
>>>>> in such case the value is version dependant and an entry is
>>>>> required
>>>>> for each version.
>>>>
>>>> Again, there is no conflict, as the default value within the
>>>> metadata
>>>> can be version specific too.
>>> I agree , however we should consider all changes in current code +
>>> tools + upgrade since this seems a major change
>>
>> Right. I outlined all changes I collected in the initial message.
>>
>>>>
>>>> Having said that, I don't understand how two different versions can
>>>> work with the different data models and share one database and
>>>> options.
>>>>
>>>>> I agree however, that the default value in the code is redundant
>>>>> and
>>>>> error prone and should be removed.
>>>>
>>>>  From what you wrote above, I don't understand how you reached to
>>>> this
>>>> conclusion. Can you please explain?
>>> I mean that we have now defaults in code (ConfigValues.java) and in
>>> the database they may not match.
>>> I think as you that only one place defined the defaults, so , it
>>> should be removed from code which is less flexible if we want to
>>> change something without recompiling ...
>>
>> If we decide to use java annotations for metadata, the default will be
>> in annotation.
>> If we decide to use XML for metadata, the default will be in XML.
>> At any case, I would like to reach to a state where the metadata is at
>> one place, and not spread between code, property/xml, database.
>
> I may be missing something.
> are you suggesting moving all the vdc_options table to an xml file?
> I assume it will be placed under /etc?
>
> then if we update the rpm, the edited config will remain unmodified, and
> we'll get the new one as rpmsave (iirc).
> then during upgrade process we'll need to merge these files?
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel


From alonbl at redhat.com  Thu Sep 13 18:23:41 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Thu, 13 Sep 2012 14:23:41 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <5051F29E.2010000@redhat.com>
Message-ID: <1817767230.6439667.1347560621703.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "Eli Mesika" <emesika at redhat.com>, engine-devel at ovirt.org
> Sent: Thursday, September 13, 2012 5:50:06 PM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> On 09/13/2012 11:36 AM, Alon Bar-Lev wrote:
> >
> >
> > ----- Original Message -----
> >> From: "Eli Mesika" <emesika at redhat.com>
> >> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> >> Sent: Thursday, September 13, 2012 10:46:41 AM
> >> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
> >> default options
> >>
> >>
> >>
> >> ----- Original Message -----
> >>> From: "Alon Bar-Lev" <alonbl at redhat.com>
> >>> To: "Eli Mesika" <emesika at redhat.com>
> >>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> >>> Sent: Wednesday, September 12, 2012 3:57:13 PM
> >>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
> >>> default
> >>> options
> >>>
> >>>
> >>> Hello Eli,
> >>>
> >>> ----- Original Message -----
> >>>> From: "Eli Mesika" <emesika at redhat.com>
> >>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >>>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> >>>> Sent: Wednesday, September 12, 2012 3:48:00 PM
> >>>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
> >>>> default
> >>>> options
> >>>>
> >>>>>>>>> QUESTIONS
> >>>>>>>>>
> >>>>>>>>> 1. Why do we store default values in database?
> >>>>
> >>>> I wanted to add few important points
> >>>> a) We have default values in DB in order to enable overriding
> >>>> values
> >>>> in the 0000_config.sql file only if a customer did not change
> >>>> the
> >>>> default value, if a customer changed the default in some
> >>>> entries,
> >>>> we
> >>>> want to honour the customer settings
> >>>
> >>> Default != value.
> >>> Default is what should be used if not specified explicitly
> >>>
> >>> Current implementation puts the default value as a value, so you
> >>> cannot distinguish between what user enforced.
> >>>
> >>> When a default is changed, because of field feedback, we should
> >>> push
> >>> this into the database instead of keeping in database only
> >>> options
> >>> that were modified by the user and fetch the default from the
> >>> option
> >>> metadata.
> >>>
> >>> In another words.... there should be no 000_config.sql, the
> >>> option
> >>> table should be empty as long as the user does not modify any of
> >>> the
> >>> options.
> >>>
> >>
> >> Lets assume that we are going for it, how would you upgrade from
> >> the
> >> current state to your suggested solution keeping all user current
> >> settings ?
> >
> > Either by:
> >
> > 1. leaving all existing data within database... so current users
> > will not enjoy updating defaults in future. I really don't like
> > this solution, but it will work.
> >
> > 2. during upgrade, go over the values, and remove all that matches
> > the metadata default. This way, future change in metadata will
> > apply.
> >
> >>
> >>>> b)There may be other requirements on configuration that are
> >>>> easier
> >>>> to
> >>>> manage in the database, for example, I have heard that one of
> >>>> the
> >>>> suggested features regarding configuration is to keep a kind of
> >>>> configuration history and know exactly which configuration
> >>>> values
> >>>> was changed, when and by whom.
> >>>
> >>> There is no conflict. You can have audit table when modifying
> >>> options.
> >>> Keep in mind that I discuss the option metadata.
> >>> I believe you are discussing the option data.
> >> You are right
> >>
> >>>
> >>>> c) The version mechanism in the config is working like this :
> >>>> there
> >>>> is a 'general' version , means that this value is not version
> >>>> dependant, or the version can be a real version like 3.1 3.2 etc
> >>>> ,
> >>>> in such case the value is version dependant and an entry is
> >>>> required
> >>>> for each version.
> >>>
> >>> Again, there is no conflict, as the default value within the
> >>> metadata
> >>> can be version specific too.
> >> I agree , however we should consider all changes in current code +
> >> tools + upgrade since this seems a major change
> >
> > Right. I outlined all changes I collected in the initial message.
> >
> >>>
> >>> Having said that, I don't understand how two different versions
> >>> can
> >>> work with the different data models and share one database and
> >>> options.
> >>>
> >>>> I agree however, that the default value in the code is redundant
> >>>> and
> >>>> error prone and should be removed.
> >>>
> >>>  From what you wrote above, I don't understand how you reached to
> >>> this
> >>> conclusion. Can you please explain?
> >> I mean that we have now defaults in code (ConfigValues.java) and
> >> in
> >> the database they may not match.
> >> I think as you that only one place defined the defaults, so , it
> >> should be removed from code which is less flexible if we want to
> >> change something without recompiling ...
> >
> > If we decide to use java annotations for metadata, the default will
> > be in annotation.
> > If we decide to use XML for metadata, the default will be in XML.
> > At any case, I would like to reach to a state where the metadata is
> > at one place, and not spread between code, property/xml, database.
> 
> I may be missing something.
> are you suggesting moving all the vdc_options table to an xml file?
> I assume it will be placed under /etc?
> 
> then if we update the rpm, the edited config will remain unmodified,
> and
> we'll get the new one as rpmsave (iirc).
> then during upgrade process we'll need to merge these files?
> 

No... the vdc_options stays exactly as-is.
I am suggesting to move the metadata to one place.
metadata = option name, description, reloadable, password, default value, restriction.

metadata is part of release, and not changed by user.

Alon.


From alonbl at redhat.com  Thu Sep 13 18:27:42 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Thu, 13 Sep 2012 14:27:42 -0400 (EDT)
Subject: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
In-Reply-To: <5051F46C.5070402@redhat.com>
Message-ID: <440622197.6440600.1347560862106.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Yair Zaslavsky" <yzaslavs at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Thursday, September 13, 2012 5:57:48 PM
> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config default options
> 
> 
> 
> On 09/13/2012 05:50 PM, Itamar Heim wrote:
> > On 09/13/2012 11:36 AM, Alon Bar-Lev wrote:
> >>
> >>
> >> ----- Original Message -----
> >>> From: "Eli Mesika" <emesika at redhat.com>
> >>> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> >>> Sent: Thursday, September 13, 2012 10:46:41 AM
> >>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
> >>> default
> >>> options
> >>>
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Alon Bar-Lev" <alonbl at redhat.com>
> >>>> To: "Eli Mesika" <emesika at redhat.com>
> >>>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> >>>> Sent: Wednesday, September 12, 2012 3:57:13 PM
> >>>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
> >>>> default
> >>>> options
> >>>>
> >>>>
> >>>> Hello Eli,
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Eli Mesika" <emesika at redhat.com>
> >>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
> >>>>> Cc: engine-devel at ovirt.org, "Livnat Peer" <lpeer at redhat.com>
> >>>>> Sent: Wednesday, September 12, 2012 3:48:00 PM
> >>>>> Subject: Re: [Engine-devel] [RFC] ovirt-engine - vdc_config
> >>>>> default
> >>>>> options
> >>>>>
> >>>>>>>>>> QUESTIONS
> >>>>>>>>>>
> >>>>>>>>>> 1. Why do we store default values in database?
> >>>>>
> >>>>> I wanted to add few important points
> >>>>> a) We have default values in DB in order to enable overriding
> >>>>> values
> >>>>> in the 0000_config.sql file only if a customer did not change
> >>>>> the
> >>>>> default value, if a customer changed the default in some
> >>>>> entries,
> >>>>> we
> >>>>> want to honour the customer settings
> >>>>
> >>>> Default != value.
> >>>> Default is what should be used if not specified explicitly
> I agree here, maybe we should consider a future possibility of "reset
> to
> default value"

Right, will be implemented.

> >>>>
> >>>> Current implementation puts the default value as a value, so you
> >>>> cannot distinguish between what user enforced.
> >>>>
> >>>> When a default is changed, because of field feedback, we should
> >>>> push
> >>>> this into the database instead of keeping in database only
> >>>> options
> >>>> that were modified by the user and fetch the default from the
> >>>> option
> >>>> metadata.
> >>>>
> >>>> In another words.... there should be no 000_config.sql, the
> >>>> option
> >>>> table should be empty as long as the user does not modify any of
> >>>> the
> >>>> options.
> 
> Not sure I fully understand how upgrade of configuration will look?
> I'm talking about update of existing data (+meta data - for example,
> change of existing description) vs introducing of new data (+
> metadata)

Metadata is not updated. it is delivered with release.
It comes as resource as any other resource we have within application.

Adding a new option to metadata implies that this option was not available at previous release, so after upgrade if application tries to access this option it will not find it in database and then retrieve the default from the metadata.

If a description is updated within metadata, it will be available immediately after upgrade, as description is taken directly from metadata.

I may miss something from your question...

> >>>>
> >>>
> >>> Lets assume that we are going for it, how would you upgrade from
> >>> the
> >>> current state to your suggested solution keeping all user current
> >>> settings ?
> >>
> >> Either by:
> >>
> >> 1. leaving all existing data within database... so current users
> >> will
> >> not enjoy updating defaults in future. I really don't like this
> >> solution, but it will work.
> >>
> >> 2. during upgrade, go over the values, and remove all that matches
> >> the
> >> metadata default. This way, future change in metadata will apply.
> >>
> >>>
> >>>>> b)There may be other requirements on configuration that are
> >>>>> easier
> >>>>> to
> >>>>> manage in the database, for example, I have heard that one of
> >>>>> the
> >>>>> suggested features regarding configuration is to keep a kind of
> >>>>> configuration history and know exactly which configuration
> >>>>> values
> >>>>> was changed, when and by whom.
> >>>>
> >>>> There is no conflict. You can have audit table when modifying
> >>>> options.
> >>>> Keep in mind that I discuss the option metadata.
> >>>> I believe you are discussing the option data.
> >>> You are right
> >>>
> >>>>
> >>>>> c) The version mechanism in the config is working like this :
> >>>>> there
> >>>>> is a 'general' version , means that this value is not version
> >>>>> dependant, or the version can be a real version like 3.1 3.2
> >>>>> etc
> >>>>> ,
> >>>>> in such case the value is version dependant and an entry is
> >>>>> required
> >>>>> for each version.
> >>>>
> >>>> Again, there is no conflict, as the default value within the
> >>>> metadata
> >>>> can be version specific too.
> >>> I agree , however we should consider all changes in current code
> >>> +
> >>> tools + upgrade since this seems a major change
> >>
> >> Right. I outlined all changes I collected in the initial message.
> >>
> >>>>
> >>>> Having said that, I don't understand how two different versions
> >>>> can
> >>>> work with the different data models and share one database and
> >>>> options.
> >>>>
> >>>>> I agree however, that the default value in the code is
> >>>>> redundant
> >>>>> and
> >>>>> error prone and should be removed.
> >>>>
> >>>>  From what you wrote above, I don't understand how you reached
> >>>>  to
> >>>> this
> >>>> conclusion. Can you please explain?
> >>> I mean that we have now defaults in code (ConfigValues.java) and
> >>> in
> >>> the database they may not match.
> >>> I think as you that only one place defined the defaults, so , it
> >>> should be removed from code which is less flexible if we want to
> >>> change something without recompiling ...
> >>
> >> If we decide to use java annotations for metadata, the default
> >> will be
> >> in annotation.
> >> If we decide to use XML for metadata, the default will be in XML.
> >> At any case, I would like to reach to a state where the metadata
> >> is at
> >> one place, and not spread between code, property/xml, database.
> >
> > I may be missing something.
> > are you suggesting moving all the vdc_options table to an xml file?
> > I assume it will be placed under /etc?
> >
> > then if we update the rpm, the edited config will remain
> > unmodified, and
> > we'll get the new one as rpmsave (iirc).
> > then during upgrade process we'll need to merge these files?
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From workshop-pc at ovirt.org  Thu Sep 13 22:30:10 2012
From: workshop-pc at ovirt.org (workshop-pc at ovirt.org)
Date: Thu, 13 Sep 2012 15:30:10 -0700
Subject: [Engine-devel] oVirt Workshop Europe 2012: Call For
	Participation
In-Reply-To: <20120814214405.GN20407@x200.localdomain>
References: <20120814214405.GN20407@x200.localdomain>
Message-ID: <20120913223010.GE12386@x200.localdomain>

* workshop-pc at ovirt.org (workshop-pc at ovirt.org) wrote:
> =================================================================
> oVirt Workshop Europe 2012: Call For Participation
> November 7-9, 2012 - Hotel Fira Palace - Barcelona, Spain
> 
> (All submissions must be received before midnight Sep 14th, 2012)
> =================================================================

The submission deadline is being pushed out one week to Sep 21st, 2012.

There was a glitch in the CFP system causing the oVirt Workshop option
during CFP submission to disappear.  It has been fixed and we're going
to extend the submission deadline by one week to accommodate.  If you
were having trouble submitting a proposal, please try again.  Sorry for
any inconvenience.

> The oVirt Project is an open virtualization project for anyone who cares
> about Linux-based KVM virtualization. Providing a feature-rich server
> virtualization management system with advanced capabilities for hosts
> and guests, including high availability, live migration, storage
> management, system scheduler, and more. By open we mean open source &
> open governance, done right.
> 
> During this workshop you?ll learn about the technical background and
> direction of the oVirt project. You?ll meet the developers, and have an
> opportunity to see and dive into the code right away. The workshop is
> open to all who want to use, get involved with, or  learn about the
> comprehensive open virtualization management platform, oVirt. The
> sessions cover the technical projects details, governance, getting
> involved, usage, and much more. If you have any interest in an Open
> Virtualization Management platform, this workshop is for you!
> 
> We are excited to announce that this oVirt Workshop will be held in
> conjunction with the KVM Forum.
> 
> http://events.linuxfoundation.org/events/kvm-forum/
> 
> The KVM Forum and oVirt Workshop are co-located with the Linux
> Foundation's 2012 LinuxCon Europe in Barcelona, Spain.
> 
> oVirt Workshop attendees will be able to attend KVM Forum sessions and
> are eligible to attend LinuxCon Europe for a discounted rate.
> 
> http://events.linuxfoundation.org/events/kvm-forum/register
> 
> We invite you to lead part of the discussion by submitting a speaking
> proposal for oVirt Workshop 2012.
> 
> http://events.linuxfoundation.org/cfp
> 
> Suggested topics:
> 
>  - community use case/stories
>  - roadmaps
>  - deep dives into features/areas
>  - deep dives into code/debugging/tuning
>  - integration and extensions
>  - components: engine, vdsm, node, sdk/cli, reports, mom, guest agent, etc.
>  - subjects: network, storage, vm life cycle, scheduling & sla, gluster, etc.
>  - packaging, installation and distributions
>  - community infrastructure and services
> 
> SUBMISSION REQUIREMENTS
> 
> Abstracts due: Sep 14th, 2012
> Notification: Sep 28th, 2012
> 
> Please submit a short abstract (~150 words) describing your presentation
> proposal.  In your submission please note how long your talk will take.
> Slots vary in length up to 45 minutes.  Also include in your proposal
> the proposal type -- one of:
> 
> - technical talk
> - end-user talk
> - birds of a feather (BOF) session
> 
> Submit your proposal here:
> 
> http://events.linuxfoundation.org/cfp
> 
> You will receive a notification whether or not your presentation proposal
> was accepted by Sep 14th.
> 
> END-USER COLLABORATION
> 
> One of the big challenges as developers is to know what, where and how
> people actually use our software.  We will reserve a few slots for end
> users talking about their deployment challenges and achievements.
> 
> If you are using oVirt in production you are encouraged submit a speaking
> proposal.  Simply mark it as an end-user collaboration proposal.  As an
> end user, this is a unique opportunity to get your input to developers.
> 
> BOF SESSION
> 
> We will reserve some slots in the evening after the main conference
> tracks, for birds of a feather (BOF) sessions. These sessions will be
> less formal than presentation tracks and targetted for people who would
> like to discuss specific issues with other developers and/or users.
> If you are interested in getting developers and/or uses together to
> discuss a specific problem, please submit a BOF proposal.
> 
> LIGHTNING TALKS
> 
> In addition to submitted talks we will also have some room for lightning
> talks. These are short (5 minute) discussions to highlight new work or
> ideas that aren't complete enough to warrant a full presentation slot.
> Lightning talk submissions and scheduling will be handled on-site at
> oVirt Workshop.
> 
> HOTEL / TRAVEL
> 
> The oVirt Workshop Europe 2012 will be held in Barcelona, Spain at the
> Hotel Fira Palace.
> 
> http://events.linuxfoundation.org/events/kvm-forum/hotel
> 
> Thank you for your interest in oVirt.  We're looking forward to your
> submissions and seeing you at the oVirt Workshop Europe 2012 in November!
> 
> Thanks,
> your oVirt Workshop Europe 2012 Program Commitee
> 
> Please contact us with any questions or comments.
> workshop-pc at ovirt.org


From yzaslavs at redhat.com  Sun Sep 16 08:18:44 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Sun, 16 Sep 2012 11:18:44 +0300
Subject: [Engine-devel] upgrade scripts -  Please keep keys ordered
Message-ID: <50558B64.3030109@redhat.com>

At config_0000.sql (backend/manager/dbscripts/upgrade/pre_upgrade 
directory) at Alphabetic order.

Kind regards,
Yair




From dgopal at redhat.com  Mon Sep 17 06:12:30 2012
From: dgopal at redhat.com (Dhandapani)
Date: Mon, 17 Sep 2012 11:42:30 +0530
Subject: [Engine-devel] Upstream git repo hung
Message-ID: <5056BF4E.7090507@redhat.com>

I got,
/fatal: The remote end hung up unexpectedly/

--
Regards,
Dhandapani
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120917/d45c7ad6/attachment.html>

From kmayilsa at redhat.com  Mon Sep 17 06:17:38 2012
From: kmayilsa at redhat.com (Kanagaraj Mayilsamy)
Date: Mon, 17 Sep 2012 02:17:38 -0400 (EDT)
Subject: [Engine-devel] Upstream git repo hung
In-Reply-To: <5056BF4E.7090507@redhat.com>
Message-ID: <937342494.109178.1347862658158.JavaMail.root@redhat.com>

I am facing the same issue, this happens when we use Anonymous Git. After that i tried with ssh, its working fine. Gerrit doesn't show the SSH/HTTPS url's for me even though I logged in. So you can manually modify the url's before doing a cherry-pick/fetch/checkout and try. 
Instead of git://gerrit... you can use ssh://username at gerrit...

Regards
Kanagaraj M

----- Original Message -----
From: "Dhandapani" <dgopal at redhat.com>
To: engine-devel at ovirt.org
Sent: Monday, September 17, 2012 11:42:30 AM
Subject: [Engine-devel] Upstream git repo hung


I got, 
fatal: The remote end hung up unexpectedly 

-- 
Regards, 
Dhandapani 

_______________________________________________
Engine-devel mailing list
Engine-devel at ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


From deadhorseconsulting at gmail.com  Wed Sep 19 02:47:15 2012
From: deadhorseconsulting at gmail.com (Dead Horse)
Date: Tue, 18 Sep 2012 21:47:15 -0500
Subject: [Engine-devel] gwt error with nightly engine builds and IE8/9
Message-ID: <CAEWPe=qKBXXken7pJ1bcCZh9Mu9cV69fxdQKpahUoyCjcx9u=g@mail.gmail.com>

I have been seeing this issue with the latest nightly engine builds for
quite a few weeks now with Internet Explorer Version 8 and 9.

ERROR: Possible problem with your *.gwt.xml module file.
The compile time user.agent value (gecko1_8) does not match the runtime
user.agent value (ie9). Expect more errors.

Is this a known issue?

- DHC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120918/702abc90/attachment.html>

From iheim at redhat.com  Wed Sep 19 07:05:24 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 19 Sep 2012 10:05:24 +0300
Subject: [Engine-devel] gwt error with nightly engine builds and IE8/9
In-Reply-To: <CAEWPe=qKBXXken7pJ1bcCZh9Mu9cV69fxdQKpahUoyCjcx9u=g@mail.gmail.com>
References: <CAEWPe=qKBXXken7pJ1bcCZh9Mu9cV69fxdQKpahUoyCjcx9u=g@mail.gmail.com>
Message-ID: <50596EB4.3070709@redhat.com>

On 09/19/2012 05:47 AM, Dead Horse wrote:
> I have been seeing this issue with the latest nightly engine builds for
> quite a few weeks now with Internet Explorer Version 8 and 9.
>
> ERROR: Possible problem with your *.gwt.xml module file.
> The compile time user.agent value (gecko1_8) does not match the runtime
> user.agent value (ie9). Expect more errors.

since compiling for different browsers is very expensive on cpu/time 
with gwt, the devel build is done with only gecko1_8 by default.
it could be that the nightly build isn't enabling the other browsers, 
which it should.


From jhernand at redhat.com  Wed Sep 19 08:08:26 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Wed, 19 Sep 2012 10:08:26 +0200
Subject: [Engine-devel] [Users] base url of ovirt
In-Reply-To: <1348001438.2458.40.camel@localhost>
References: <1348001438.2458.40.camel@localhost>
Message-ID: <50597D7A.1070403@redhat.com>

Copying engine-devel, as I think this is something we should discuss and
maybe do.

On 09/18/2012 10:50 PM, Jon Thomas wrote:
> Is there some config in the engine to set up the web interface base url
> so that instead of https://localhost.localdomain/ it is
> https://localhost.localdomain/ovirt ?

No, there is no such config.

I think this should be the default, I mean, we should have this /ovirt
prefix in all our URLs, to make coexistence with other users of the web
server easy.

-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From alonbl at redhat.com  Wed Sep 19 08:19:42 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Wed, 19 Sep 2012 04:19:42 -0400 (EDT)
Subject: [Engine-devel] [Users] base url of ovirt
In-Reply-To: <50597D7A.1070403@redhat.com>
Message-ID: <1379032552.493336.1348042782497.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: "Jon Thomas" <jthomas at redhat.com>
> Cc: engine-devel at ovirt.org, users at ovirt.org
> Sent: Wednesday, September 19, 2012 11:08:26 AM
> Subject: Re: [Users] base url of ovirt
> 
> Copying engine-devel, as I think this is something we should discuss
> and
> maybe do.
> 
> On 09/18/2012 10:50 PM, Jon Thomas wrote:
> > Is there some config in the engine to set up the web interface base
> > url
> > so that instead of https://localhost.localdomain/ it is
> > https://localhost.localdomain/ovirt ?
> 
> No, there is no such config.
> 
> I think this should be the default, I mean, we should have this
> /ovirt
> prefix in all our URLs, to make coexistence with other users of the
> web
> server easy.

Totally agree.

We discussed that, Itamar agreed to go ahead URL change for ovirt-engine-4.0...

Moving namespace out of root provides many advantages including including simpler apache configuration, easier to use proxies, ability to host multiple applications.

Alon.


From jhernand at redhat.com  Wed Sep 19 08:53:00 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Wed, 19 Sep 2012 10:53:00 +0200
Subject: [Engine-devel] [Users] base url of ovirt
In-Reply-To: <1379032552.493336.1348042782497.JavaMail.root@redhat.com>
References: <1379032552.493336.1348042782497.JavaMail.root@redhat.com>
Message-ID: <505987EC.5070501@redhat.com>

On 09/19/2012 10:19 AM, Alon Bar-Lev wrote:
> 
> 
> ----- Original Message -----
>> From: "Juan Hernandez" <jhernand at redhat.com>
>> To: "Jon Thomas" <jthomas at redhat.com>
>> Cc: engine-devel at ovirt.org, users at ovirt.org
>> Sent: Wednesday, September 19, 2012 11:08:26 AM
>> Subject: Re: [Users] base url of ovirt
>>
>> Copying engine-devel, as I think this is something we should discuss
>> and
>> maybe do.
>>
>> On 09/18/2012 10:50 PM, Jon Thomas wrote:
>>> Is there some config in the engine to set up the web interface base
>>> url
>>> so that instead of https://localhost.localdomain/ it is
>>> https://localhost.localdomain/ovirt ?
>>
>> No, there is no such config.
>>
>> I think this should be the default, I mean, we should have this
>> /ovirt
>> prefix in all our URLs, to make coexistence with other users of the
>> web
>> server easy.
> 
> Totally agree.
> 
> We discussed that, Itamar agreed to go ahead URL change for ovirt-engine-4.0...
> 
> Moving namespace out of root provides many advantages including including simpler apache configuration, easier to use proxies, ability to host multiple applications.
> 
> Alon.

Jon, as you see this will probably go in release 4.0, which is the
future. Meanwhile if what you need is to use the web server with other
applications you could try to replace the directives in
/etc/httpd/conf.d/ovirt-engine.conf with the following:

ProxyPassMatch ^/(ca.crt|engine.ssh.key.txt)$ ajp://localhost:8009/$1
ProxyPassMatch ^/(api|webadmin|UserPortal|OvirtEngineWeb)(/.*)?$
ajp://localhost:8009/$1$2

That will change your configuration so that only the URLs that are
really required for the engine will be redirected to it. A notable
exception will be the welcome page, but you probably can live without
it, just use the following to get to the UI:

https://whatever.example.com/webadmin
https://whatever.example.com/UserPortal

Take into account that ProxyPassMatch directives are processed in the
order they appear, so if you have another application with conflicting
ProxyPassMatch directives the result can be unexpected.

-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From amureini at redhat.com  Wed Sep 19 12:19:20 2012
From: amureini at redhat.com (Allon Mureinik)
Date: Wed, 19 Sep 2012 08:19:20 -0400 (EDT)
Subject: [Engine-devel] Upstream git repo hung
In-Reply-To: <937342494.109178.1347862658158.JavaMail.root@redhat.com>
Message-ID: <1434073603.2443335.1348057160892.JavaMail.root@redhat.com>

FYI: http (/not/ https) also seems to work

----- Original Message -----
> From: "Kanagaraj Mayilsamy" <kmayilsa at redhat.com>
> To: "Dhandapani" <dgopal at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Monday, September 17, 2012 9:17:38 AM
> Subject: Re: [Engine-devel] Upstream git repo hung
> 
> I am facing the same issue, this happens when we use Anonymous Git.
> After that i tried with ssh, its working fine. Gerrit doesn't show
> the SSH/HTTPS url's for me even though I logged in. So you can
> manually modify the url's before doing a cherry-pick/fetch/checkout
> and try.
> Instead of git://gerrit... you can use ssh://username at gerrit...
> 
> Regards
> Kanagaraj M
> 
> ----- Original Message -----
> From: "Dhandapani" <dgopal at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Monday, September 17, 2012 11:42:30 AM
> Subject: [Engine-devel] Upstream git repo hung
> 
> 
> I got,
> fatal: The remote end hung up unexpectedly
> 
> --
> Regards,
> Dhandapani
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From iheim at redhat.com  Wed Sep 19 13:11:56 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 19 Sep 2012 16:11:56 +0300
Subject: [Engine-devel] Upstream git repo hung
In-Reply-To: <937342494.109178.1347862658158.JavaMail.root@redhat.com>
References: <937342494.109178.1347862658158.JavaMail.root@redhat.com>
Message-ID: <5059C49C.80808@redhat.com>

On 09/17/2012 09:17 AM, Kanagaraj Mayilsamy wrote:
> I am facing the same issue, this happens when we use Anonymous Git. After that i tried with ssh, its working fine. Gerrit doesn't show the SSH/HTTPS url's for me even though I logged in. So you can manually modify the url's before doing a cherry-pick/fetch/checkout and try.
> Instead of git://gerrit... you can use ssh://username at gerrit...

last time this was around file descriptors, but it is currently set to 
'unlimited'.
looking at other potential issues.
as allon mentioned, using http based urls works without an issue.

>
> Regards
> Kanagaraj M
>
> ----- Original Message -----
> From: "Dhandapani" <dgopal at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Monday, September 17, 2012 11:42:30 AM
> Subject: [Engine-devel] Upstream git repo hung
>
>
> I got,
> fatal: The remote end hung up unexpectedly
>




From kmayilsa at redhat.com  Wed Sep 19 13:27:17 2012
From: kmayilsa at redhat.com (Kanagaraj Mayilsamy)
Date: Wed, 19 Sep 2012 09:27:17 -0400 (EDT)
Subject: [Engine-devel] Upstream git repo hung
In-Reply-To: <5059C49C.80808@redhat.com>
Message-ID: <81714546.825271.1348061237767.JavaMail.root@redhat.com>

Thanks for your information, http based urls are working fine.

But i am wondering, why gerrit doesn't show the SSH/HTTP based urls even though i logged in. It shows only the Anonymous Git url, the ones starts with git:// . But previously the gerrit used to show all 3 type of urls (GIT/SSH/HTTPS). It would be easier to copy & paste, otherwise we have to manually edit the url before proceeding. 

----- Original Message -----
From: "Itamar Heim" <iheim at redhat.com>
To: "Kanagaraj Mayilsamy" <kmayilsa at redhat.com>
Cc: "Dhandapani" <dgopal at redhat.com>, engine-devel at ovirt.org
Sent: Wednesday, September 19, 2012 6:41:56 PM
Subject: Re: [Engine-devel] Upstream git repo hung

On 09/17/2012 09:17 AM, Kanagaraj Mayilsamy wrote:
> I am facing the same issue, this happens when we use Anonymous Git. After that i tried with ssh, its working fine. Gerrit doesn't show the SSH/HTTPS url's for me even though I logged in. So you can manually modify the url's before doing a cherry-pick/fetch/checkout and try.
> Instead of git://gerrit... you can use ssh://username at gerrit...

last time this was around file descriptors, but it is currently set to 
'unlimited'.
looking at other potential issues.
as allon mentioned, using http based urls works without an issue.

>
> Regards
> Kanagaraj M
>
> ----- Original Message -----
> From: "Dhandapani" <dgopal at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Monday, September 17, 2012 11:42:30 AM
> Subject: [Engine-devel] Upstream git repo hung
>
>
> I got,
> fatal: The remote end hung up unexpectedly
>



From snmishra at linux.vnet.ibm.com  Wed Sep 19 15:51:57 2012
From: snmishra at linux.vnet.ibm.com (snmishra at linux.vnet.ibm.com)
Date: Wed, 19 Sep 2012 08:51:57 -0700
Subject: [Engine-devel] Frontend questions.
Message-ID: <20120919085157.Horde.LPvybJir309QWeoda9Fl8pA@imap.linux.ibm.com>


Hi,


     I am looking at adding support for novnc in ovirt. I have not  
worked on the frontend and thought of asking some questions to the  
list to save time.

     Currently, when a user selects to open console on a VM with VNC.  
They get a popup window with IP, port and password for VNC session. I  
plan to take this info to host and run the command -

   novnc_server --vnc IP:(port-5900)

This command returns a URL, use this url to open a new browser window  
which will be a vnc session to the VM.

    My questions are -

1. How to run the novnc-server command on the host?

2. How to open a new URL for VNC session?

Thanks
Sharad Mishra



From iheim at redhat.com  Wed Sep 19 16:50:43 2012
From: iheim at redhat.com (Itamar Heim)
Date: Wed, 19 Sep 2012 19:50:43 +0300
Subject: [Engine-devel] Upstream git repo hung
In-Reply-To: <81714546.825271.1348061237767.JavaMail.root@redhat.com>
References: <81714546.825271.1348061237767.JavaMail.root@redhat.com>
Message-ID: <5059F7E3.8050305@redhat.com>

On 09/19/2012 04:27 PM, Kanagaraj Mayilsamy wrote:
> Thanks for your information, http based urls are working fine.
>
> But i am wondering, why gerrit doesn't show the SSH/HTTP based urls even though i logged in. It shows only the Anonymous Git url, the ones starts with git:// . But previously the gerrit used to show all 3 type of urls (GIT/SSH/HTTPS). It would be easier to copy & paste, otherwise we have to manually edit the url before proceeding.

are you sure you are looking in the same place?
gitweb is the one showing all urls.
i.e., after i login to gerrit, i see for this url:
http://gerrit.ovirt.org/gitweb?p=vdsm.git

these 3 repo locations (last one is the personalized one)
http://gerrit.ovirt.org/p/vdsm.git
git://gerrit.ovirt.org/vdsm.git
ssh://iheim at gerrit.ovirt.org:29418/vdsm.git

>
> ----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Kanagaraj Mayilsamy" <kmayilsa at redhat.com>
> Cc: "Dhandapani" <dgopal at redhat.com>, engine-devel at ovirt.org
> Sent: Wednesday, September 19, 2012 6:41:56 PM
> Subject: Re: [Engine-devel] Upstream git repo hung
>
> On 09/17/2012 09:17 AM, Kanagaraj Mayilsamy wrote:
>> I am facing the same issue, this happens when we use Anonymous Git. After that i tried with ssh, its working fine. Gerrit doesn't show the SSH/HTTPS url's for me even though I logged in. So you can manually modify the url's before doing a cherry-pick/fetch/checkout and try.
>> Instead of git://gerrit... you can use ssh://username at gerrit...
>
> last time this was around file descriptors, but it is currently set to
> 'unlimited'.
> looking at other potential issues.
> as allon mentioned, using http based urls works without an issue.
>
>>
>> Regards
>> Kanagaraj M
>>
>> ----- Original Message -----
>> From: "Dhandapani" <dgopal at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Monday, September 17, 2012 11:42:30 AM
>> Subject: [Engine-devel] Upstream git repo hung
>>
>>
>> I got,
>> fatal: The remote end hung up unexpectedly
>>
>




From kmayilsa at redhat.com  Wed Sep 19 17:24:33 2012
From: kmayilsa at redhat.com (Kanagaraj Mayilsamy)
Date: Wed, 19 Sep 2012 13:24:33 -0400 (EDT)
Subject: [Engine-devel] Upstream git repo hung
In-Reply-To: <5059F7E3.8050305@redhat.com>
Message-ID: <2091400833.884971.1348075473071.JavaMail.root@redhat.com>

Sorry i am not looking at http://gerrit.ovirt.org/gitweb?p=vdsm.git , but at http://gerrit.ovirt.org.
Just open any one of the patch, and expand a patch set, there will be download section. we will have 4 options there 
checkout,pull,cherry-pick and patch. In parallel to that we used have Anonymous GIT, SSH and HTTPS. Previously i used to all see these 3 options, but i could see only Anonymous GIT.


----- Original Message -----
From: "Itamar Heim" <iheim at redhat.com>
To: "Kanagaraj Mayilsamy" <kmayilsa at redhat.com>
Cc: "Allon Mureinik" <amureini at redhat.com>, "Dhandapani" <dgopal at redhat.com>, engine-devel at ovirt.org
Sent: Wednesday, September 19, 2012 10:20:43 PM
Subject: Re: [Engine-devel] Upstream git repo hung

On 09/19/2012 04:27 PM, Kanagaraj Mayilsamy wrote:
> Thanks for your information, http based urls are working fine.
>
> But i am wondering, why gerrit doesn't show the SSH/HTTP based urls even though i logged in. It shows only the Anonymous Git url, the ones starts with git:// . But previously the gerrit used to show all 3 type of urls (GIT/SSH/HTTPS). It would be easier to copy & paste, otherwise we have to manually edit the url before proceeding.

are you sure you are looking in the same place?
gitweb is the one showing all urls.
i.e., after i login to gerrit, i see for this url:
http://gerrit.ovirt.org/gitweb?p=vdsm.git

these 3 repo locations (last one is the personalized one)
http://gerrit.ovirt.org/p/vdsm.git
git://gerrit.ovirt.org/vdsm.git
ssh://iheim at gerrit.ovirt.org:29418/vdsm.git

>
> ----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Kanagaraj Mayilsamy" <kmayilsa at redhat.com>
> Cc: "Dhandapani" <dgopal at redhat.com>, engine-devel at ovirt.org
> Sent: Wednesday, September 19, 2012 6:41:56 PM
> Subject: Re: [Engine-devel] Upstream git repo hung
>
> On 09/17/2012 09:17 AM, Kanagaraj Mayilsamy wrote:
>> I am facing the same issue, this happens when we use Anonymous Git. After that i tried with ssh, its working fine. Gerrit doesn't show the SSH/HTTPS url's for me even though I logged in. So you can manually modify the url's before doing a cherry-pick/fetch/checkout and try.
>> Instead of git://gerrit... you can use ssh://username at gerrit...
>
> last time this was around file descriptors, but it is currently set to
> 'unlimited'.
> looking at other potential issues.
> as allon mentioned, using http based urls works without an issue.
>
>>
>> Regards
>> Kanagaraj M
>>
>> ----- Original Message -----
>> From: "Dhandapani" <dgopal at redhat.com>
>> To: engine-devel at ovirt.org
>> Sent: Monday, September 17, 2012 11:42:30 AM
>> Subject: [Engine-devel] Upstream git repo hung
>>
>>
>> I got,
>> fatal: The remote end hung up unexpectedly
>>
>



From iheim at redhat.com  Thu Sep 20 10:04:53 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 20 Sep 2012 13:04:53 +0300
Subject: [Engine-devel] Upstream git repo hung
In-Reply-To: <2091400833.884971.1348075473071.JavaMail.root@redhat.com>
References: <2091400833.884971.1348075473071.JavaMail.root@redhat.com>
Message-ID: <505AEA45.9020402@redhat.com>

On 09/19/2012 08:24 PM, Kanagaraj Mayilsamy wrote:
> Sorry i am not looking at http://gerrit.ovirt.org/gitweb?p=vdsm.git , but at http://gerrit.ovirt.org.
> Just open any one of the patch, and expand a patch set, there will be download section. we will have 4 options there
> checkout,pull,cherry-pick and patch. In parallel to that we used have Anonymous GIT, SSH and HTTPS. Previously i used to all see these 3 options, but i could see only Anonymous GIT.

well, anonymous git should be ok now.
i ran git gc based on this helper gerrit script to pack the repo.
https://gerrit.googlesource.com/gerrit/+/43a4e17fbb54a9a3f3bca7ea0fd6eb675d1c71bb

>
>
> ----- Original Message -----
> From: "Itamar Heim" <iheim at redhat.com>
> To: "Kanagaraj Mayilsamy" <kmayilsa at redhat.com>
> Cc: "Allon Mureinik" <amureini at redhat.com>, "Dhandapani" <dgopal at redhat.com>, engine-devel at ovirt.org
> Sent: Wednesday, September 19, 2012 10:20:43 PM
> Subject: Re: [Engine-devel] Upstream git repo hung
>
> On 09/19/2012 04:27 PM, Kanagaraj Mayilsamy wrote:
>> Thanks for your information, http based urls are working fine.
>>
>> But i am wondering, why gerrit doesn't show the SSH/HTTP based urls even though i logged in. It shows only the Anonymous Git url, the ones starts with git:// . But previously the gerrit used to show all 3 type of urls (GIT/SSH/HTTPS). It would be easier to copy & paste, otherwise we have to manually edit the url before proceeding.
>
> are you sure you are looking in the same place?
> gitweb is the one showing all urls.
> i.e., after i login to gerrit, i see for this url:
> http://gerrit.ovirt.org/gitweb?p=vdsm.git
>
> these 3 repo locations (last one is the personalized one)
> http://gerrit.ovirt.org/p/vdsm.git
> git://gerrit.ovirt.org/vdsm.git
> ssh://iheim at gerrit.ovirt.org:29418/vdsm.git
>
>>
>> ----- Original Message -----
>> From: "Itamar Heim" <iheim at redhat.com>
>> To: "Kanagaraj Mayilsamy" <kmayilsa at redhat.com>
>> Cc: "Dhandapani" <dgopal at redhat.com>, engine-devel at ovirt.org
>> Sent: Wednesday, September 19, 2012 6:41:56 PM
>> Subject: Re: [Engine-devel] Upstream git repo hung
>>
>> On 09/17/2012 09:17 AM, Kanagaraj Mayilsamy wrote:
>>> I am facing the same issue, this happens when we use Anonymous Git. After that i tried with ssh, its working fine. Gerrit doesn't show the SSH/HTTPS url's for me even though I logged in. So you can manually modify the url's before doing a cherry-pick/fetch/checkout and try.
>>> Instead of git://gerrit... you can use ssh://username at gerrit...
>>
>> last time this was around file descriptors, but it is currently set to
>> 'unlimited'.
>> looking at other potential issues.
>> as allon mentioned, using http based urls works without an issue.
>>
>>>
>>> Regards
>>> Kanagaraj M
>>>
>>> ----- Original Message -----
>>> From: "Dhandapani" <dgopal at redhat.com>
>>> To: engine-devel at ovirt.org
>>> Sent: Monday, September 17, 2012 11:42:30 AM
>>> Subject: [Engine-devel] Upstream git repo hung
>>>
>>>
>>> I got,
>>> fatal: The remote end hung up unexpectedly
>>>
>>
>




From alkaplan at redhat.com  Thu Sep 20 10:50:15 2012
From: alkaplan at redhat.com (Alona Kaplan)
Date: Thu, 20 Sep 2012 06:50:15 -0400 (EDT)
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <739472843.2591870.1348124394898.JavaMail.root@redhat.com>
Message-ID: <1850100433.2675811.1348138215181.JavaMail.root@redhat.com>

Hi all,

Please review the wiki and add your comments.

http://wiki.ovirt.org/wiki/Feature/NetworkMainTab

RFE:
https://bugzilla.redhat.com/858742

Thanks,
Alona.


From mpastern at redhat.com  Thu Sep 20 11:24:30 2012
From: mpastern at redhat.com (Michael Pasternak)
Date: Thu, 20 Sep 2012 14:24:30 +0300
Subject: [Engine-devel] ovirt-cli 3.2.0.3 - important changes
Message-ID: <505AFCEE.7050304@redhat.com>


Two commands renamed:
--------------------

- "create" command renamed with "add" #855773.
- "delete" command renamed with "remove" #855769.


Changed authentication procedure:
--------------------------------

- added username/password prompt/conf-file functionality (see [1] for more details).

[1] http://wiki.ovirt.org/wiki/CLI#Connect


* For complete list of changes, see change log.

-- 

Michael Pasternak
RedHat, ENG-Virtualization R&D


From jhernand at redhat.com  Thu Sep 20 14:40:06 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Thu, 20 Sep 2012 16:40:06 +0200
Subject: [Engine-devel] [Users]   base url of ovirt
In-Reply-To: <1348151765.2566.3.camel@localhost>
References: <1379032552.493336.1348042782497.JavaMail.root@redhat.com>
	<505987EC.5070501@redhat.com> <1348151765.2566.3.camel@localhost>
Message-ID: <505B2AC6.4060601@redhat.com>

On 09/20/2012 04:36 PM, Jon Thomas wrote:
> On Wed, 2012-09-19 at 10:53 +0200, Juan Hernandez wrote:
>> On 09/19/2012 10:19 AM, Alon Bar-Lev wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Juan Hernandez" <jhernand at redhat.com>
>>>> To: "Jon Thomas" <jthomas at redhat.com>
>>>> Cc: engine-devel at ovirt.org, users at ovirt.org
>>>> Sent: Wednesday, September 19, 2012 11:08:26 AM
>>>> Subject: Re: [Users] base url of ovirt
>>>>
>>>> Copying engine-devel, as I think this is something we should discuss
>>>> and
>>>> maybe do.
>>>>
>>>> On 09/18/2012 10:50 PM, Jon Thomas wrote:
>>>>> Is there some config in the engine to set up the web interface base
>>>>> url
>>>>> so that instead of https://localhost.localdomain/ it is
>>>>> https://localhost.localdomain/ovirt ?
>>>>
>>>> No, there is no such config.
>>>>
>>>> I think this should be the default, I mean, we should have this
>>>> /ovirt
>>>> prefix in all our URLs, to make coexistence with other users of the
>>>> web
>>>> server easy.
>>>
>>> Totally agree.
>>>
>>> We discussed that, Itamar agreed to go ahead URL change for ovirt-engine-4.0...
>>>
>>> Moving namespace out of root provides many advantages including including simpler apache configuration, easier to use proxies, ability to host multiple applications.
>>>
>>> Alon.
>>
>> Jon, as you see this will probably go in release 4.0, which is the
>> future. Meanwhile if what you need is to use the web server with other
>> applications you could try to replace the directives in
>> /etc/httpd/conf.d/ovirt-engine.conf with the following:
>>
>> ProxyPassMatch ^/(ca.crt|engine.ssh.key.txt)$ ajp://localhost:8009/$1
>> ProxyPassMatch ^/(api|webadmin|UserPortal|OvirtEngineWeb)(/.*)?$
>> ajp://localhost:8009/$1$2
>>
>> That will change your configuration so that only the URLs that are
>> really required for the engine will be redirected to it. A notable
>> exception will be the welcome page, but you probably can live without
>> it, just use the following to get to the UI:
>>
>> https://whatever.example.com/webadmin
>> https://whatever.example.com/UserPortal
>>
>> Take into account that ProxyPassMatch directives are processed in the
>> order they appear, so if you have another application with conflicting
>> ProxyPassMatch directives the result can be unexpected.
> 
> Thx, that worked better than what I was trying. BTW, I still get the
> welcome page. I also had to add
> 
> WSGIScriptAlias /auth/login /usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi
> 
> to openstack-dashboard.conf for dashboard to work.

It is strange that you still get the welcome page, you shouldn't. Maybe
it is in your browser's cache. Try to reload it.

Take into account that I may have missed some of the required URLs. I
would really appreciate if you report back here if you have any further
problem.
-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From jthomas at redhat.com  Thu Sep 20 14:36:05 2012
From: jthomas at redhat.com (Jon Thomas)
Date: Thu, 20 Sep 2012 10:36:05 -0400
Subject: [Engine-devel] [Users] base url of ovirt
In-Reply-To: <505987EC.5070501@redhat.com>
References: <1379032552.493336.1348042782497.JavaMail.root@redhat.com>
	<505987EC.5070501@redhat.com>
Message-ID: <1348151765.2566.3.camel@localhost>

On Wed, 2012-09-19 at 10:53 +0200, Juan Hernandez wrote:
> On 09/19/2012 10:19 AM, Alon Bar-Lev wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Juan Hernandez" <jhernand at redhat.com>
> >> To: "Jon Thomas" <jthomas at redhat.com>
> >> Cc: engine-devel at ovirt.org, users at ovirt.org
> >> Sent: Wednesday, September 19, 2012 11:08:26 AM
> >> Subject: Re: [Users] base url of ovirt
> >>
> >> Copying engine-devel, as I think this is something we should discuss
> >> and
> >> maybe do.
> >>
> >> On 09/18/2012 10:50 PM, Jon Thomas wrote:
> >>> Is there some config in the engine to set up the web interface base
> >>> url
> >>> so that instead of https://localhost.localdomain/ it is
> >>> https://localhost.localdomain/ovirt ?
> >>
> >> No, there is no such config.
> >>
> >> I think this should be the default, I mean, we should have this
> >> /ovirt
> >> prefix in all our URLs, to make coexistence with other users of the
> >> web
> >> server easy.
> > 
> > Totally agree.
> > 
> > We discussed that, Itamar agreed to go ahead URL change for ovirt-engine-4.0...
> > 
> > Moving namespace out of root provides many advantages including including simpler apache configuration, easier to use proxies, ability to host multiple applications.
> > 
> > Alon.
> 
> Jon, as you see this will probably go in release 4.0, which is the
> future. Meanwhile if what you need is to use the web server with other
> applications you could try to replace the directives in
> /etc/httpd/conf.d/ovirt-engine.conf with the following:
> 
> ProxyPassMatch ^/(ca.crt|engine.ssh.key.txt)$ ajp://localhost:8009/$1
> ProxyPassMatch ^/(api|webadmin|UserPortal|OvirtEngineWeb)(/.*)?$
> ajp://localhost:8009/$1$2
> 
> That will change your configuration so that only the URLs that are
> really required for the engine will be redirected to it. A notable
> exception will be the welcome page, but you probably can live without
> it, just use the following to get to the UI:
> 
> https://whatever.example.com/webadmin
> https://whatever.example.com/UserPortal
> 
> Take into account that ProxyPassMatch directives are processed in the
> order they appear, so if you have another application with conflicting
> ProxyPassMatch directives the result can be unexpected.

Thx, that worked better than what I was trying. BTW, I still get the
welcome page. I also had to add

WSGIScriptAlias /auth/login /usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi

to openstack-dashboard.conf for dashboard to work.



> 




From jthomas at redhat.com  Thu Sep 20 15:07:46 2012
From: jthomas at redhat.com (Jon Thomas)
Date: Thu, 20 Sep 2012 11:07:46 -0400
Subject: [Engine-devel] [Users]   base url of ovirt
In-Reply-To: <505B2AC6.4060601@redhat.com>
References: <1379032552.493336.1348042782497.JavaMail.root@redhat.com>
	<505987EC.5070501@redhat.com> <1348151765.2566.3.camel@localhost>
	<505B2AC6.4060601@redhat.com>
Message-ID: <1348153666.2566.6.camel@localhost>

On Thu, 2012-09-20 at 16:40 +0200, Juan Hernandez wrote:
> On 09/20/2012 04:36 PM, Jon Thomas wrote:
> > On Wed, 2012-09-19 at 10:53 +0200, Juan Hernandez wrote:
> >> On 09/19/2012 10:19 AM, Alon Bar-Lev wrote:
> >>>
> >>>
> >>> ----- Original Message -----
> >>>> From: "Juan Hernandez" <jhernand at redhat.com>
> >>>> To: "Jon Thomas" <jthomas at redhat.com>
> >>>> Cc: engine-devel at ovirt.org, users at ovirt.org
> >>>> Sent: Wednesday, September 19, 2012 11:08:26 AM
> >>>> Subject: Re: [Users] base url of ovirt
> >>>>
> >>>> Copying engine-devel, as I think this is something we should discuss
> >>>> and
> >>>> maybe do.
> >>>>
> >>>> On 09/18/2012 10:50 PM, Jon Thomas wrote:
> >>>>> Is there some config in the engine to set up the web interface base
> >>>>> url
> >>>>> so that instead of https://localhost.localdomain/ it is
> >>>>> https://localhost.localdomain/ovirt ?
> >>>>
> >>>> No, there is no such config.
> >>>>
> >>>> I think this should be the default, I mean, we should have this
> >>>> /ovirt
> >>>> prefix in all our URLs, to make coexistence with other users of the
> >>>> web
> >>>> server easy.
> >>>
> >>> Totally agree.
> >>>
> >>> We discussed that, Itamar agreed to go ahead URL change for ovirt-engine-4.0...
> >>>
> >>> Moving namespace out of root provides many advantages including including simpler apache configuration, easier to use proxies, ability to host multiple applications.
> >>>
> >>> Alon.
> >>
> >> Jon, as you see this will probably go in release 4.0, which is the
> >> future. Meanwhile if what you need is to use the web server with other
> >> applications you could try to replace the directives in
> >> /etc/httpd/conf.d/ovirt-engine.conf with the following:
> >>
> >> ProxyPassMatch ^/(ca.crt|engine.ssh.key.txt)$ ajp://localhost:8009/$1
> >> ProxyPassMatch ^/(api|webadmin|UserPortal|OvirtEngineWeb)(/.*)?$
> >> ajp://localhost:8009/$1$2
> >>
> >> That will change your configuration so that only the URLs that are
> >> really required for the engine will be redirected to it. A notable
> >> exception will be the welcome page, but you probably can live without
> >> it, just use the following to get to the UI:
> >>
> >> https://whatever.example.com/webadmin
> >> https://whatever.example.com/UserPortal
> >>
> >> Take into account that ProxyPassMatch directives are processed in the
> >> order they appear, so if you have another application with conflicting
> >> ProxyPassMatch directives the result can be unexpected.
> > 
> > Thx, that worked better than what I was trying. BTW, I still get the
> > welcome page. I also had to add
> > 
> > WSGIScriptAlias /auth/login /usr/share/openstack-dashboard/openstack_dashboard/wsgi/django.wsgi
> > 
> > to openstack-dashboard.conf for dashboard to work.
> 
> It is strange that you still get the welcome page, you shouldn't. Maybe
> it is in your browser's cache. Try to reload it.

So it's at http://localhost:8080/ and the default fedora test page is at
http://localhost

> 
> Take into account that I may have missed some of the required URLs. I
> would really appreciate if you report back here if you have any further
> problem.




From djasa at redhat.com  Thu Sep 20 15:35:09 2012
From: djasa at redhat.com (David =?UTF-8?Q?Ja=C5=A1a?=)
Date: Thu, 20 Sep 2012 17:35:09 +0200
Subject: [Engine-devel] Duplicities in tranlations
Message-ID: <1348155309.3380.90.camel@dhcp-29-7.brq.redhat.com>

Hi,

given that my language is not yet included in language set in Zanata
(why?), I've had a look at existing translation - and I noticed
non-negligible number of duplicities in the translation: for example,
just trio of "Create", "Delete" and "Edit Properties" was repeated six
times on the first page of LocalizedEnums!

This is a recipe for translater frutration at best and translation
errors at worst. Can there be some guideline and effort to limit
proliferation of new strings by reusing existing ones?

David

-- 

David Ja?a, RHCE

SPICE QE based in Brno
GPG Key:     22C33E24 
Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24





From atal at redhat.com  Thu Sep 20 15:58:05 2012
From: atal at redhat.com (Avi Tal)
Date: Thu, 20 Sep 2012 11:58:05 -0400 (EDT)
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <1850100433.2675811.1348138215181.JavaMail.root@redhat.com>
Message-ID: <24490165.15.1348156667359.JavaMail.atal@atal-fedora.tlv.redhat.com>

1. Why not having Actions for Hosts, Vms and Templates ?
I believe we should have those actions instead of making the user browse between 'network -> vms -> network (sub tab)' just in order to update VM's network.

2. Why having such a big window which most of the time will be empty?
   How many logical network would be existed in the system? 
   My suggestion is to go for more graphical window which will present:
   A. logical networks
   B. Multiple networks with the host configurations at the same window: very important because most of the System admins would like to have a bigger picture of the entire network configuration.





----- Original Message -----
From: "Alona Kaplan" <alkaplan at redhat.com>
To: engine-devel at ovirt.org
Sent: Thursday, September 20, 2012 1:50:15 PM
Subject: [Engine-devel] RFE: Netwrok Main Tab

Hi all,

Please review the wiki and add your comments.

http://wiki.ovirt.org/wiki/Feature/NetworkMainTab



RFE:
https://bugzilla.redhat.com/858742

Thanks,
Alona.
_______________________________________________
Engine-devel mailing list
Engine-devel at ovirt.org
http://lists.ovirt.org/mailman/listinfo/engine-devel


From iheim at redhat.com  Thu Sep 20 18:44:05 2012
From: iheim at redhat.com (Itamar Heim)
Date: Thu, 20 Sep 2012 21:44:05 +0300
Subject: [Engine-devel] Frontend questions.
In-Reply-To: <20120919085157.Horde.LPvybJir309QWeoda9Fl8pA@imap.linux.ibm.com>
References: <20120919085157.Horde.LPvybJir309QWeoda9Fl8pA@imap.linux.ibm.com>
Message-ID: <505B63F5.1080300@redhat.com>

On 09/19/2012 06:51 PM, snmishra at linux.vnet.ibm.com wrote:
>
> Hi,
>
>
>      I am looking at adding support for novnc in ovirt. I have not
> worked on the frontend and thought of asking some questions to the list
> to save time.
>
>      Currently, when a user selects to open console on a VM with VNC.
> They get a popup window with IP, port and password for VNC session. I
> plan to take this info to host and run the command -
>
>    novnc_server --vnc IP:(port-5900)
>
> This command returns a URL, use this url to open a new browser window
> which will be a vnc session to the VM.
>
>     My questions are -
>
> 1. How to run the novnc-server command on the host?

i can't say i like the idea of launching these on the engine, but we can 
start with making this work first with these limitations. hopefully 
these aren't too resource intensive.
ohad - how do you deal with this in foreman?
adam - i saw you pushed these rpms to fedora - can you share how this is 
launched in openstack?

on the technical side, you can launch an external process with the 
relevant parameters.
though we probably need to consider how we could do this not via shell 
locally, rather via a remote call to allow setting this up on another 
server than the engine.


>
> 2. How to open a new URL for VNC session?

this should be easy enough to launch another window with the requested url.
vojtech?

>
> Thanks
> Sharad Mishra
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel




From iheim at redhat.com  Thu Sep 20 21:03:46 2012
From: iheim at redhat.com (Itamar Heim)
Date: Fri, 21 Sep 2012 00:03:46 +0300
Subject: [Engine-devel] Duplicities in tranlations
In-Reply-To: <1348155309.3380.90.camel@dhcp-29-7.brq.redhat.com>
References: <1348155309.3380.90.camel@dhcp-29-7.brq.redhat.com>
Message-ID: <505B84B2.1040102@redhat.com>

On 09/20/2012 06:35 PM, David Ja?a wrote:
> Hi,
>
> given that my language is not yet included in language set in Zanata
> (why?), I've had a look at existing translation - and I noticed
> non-negligible number of duplicities in the translation: for example,
> just trio of "Create", "Delete" and "Edit Properties" was repeated six
> times on the first page of LocalizedEnums!
>
> This is a recipe for translater frutration at best and translation
> errors at worst. Can there be some guideline and effort to limit
> proliferation of new strings by reusing existing ones?

yes, its a cleanup task we need to get to.
we don't enjoy adding the strings to all these places to begin with.
hopefully for 3.2.


From ayoung at redhat.com  Thu Sep 20 21:11:54 2012
From: ayoung at redhat.com (Adam Young)
Date: Thu, 20 Sep 2012 17:11:54 -0400
Subject: [Engine-devel] Frontend questions.
In-Reply-To: <505B63F5.1080300@redhat.com>
References: <20120919085157.Horde.LPvybJir309QWeoda9Fl8pA@imap.linux.ibm.com>
	<505B63F5.1080300@redhat.com>
Message-ID: <505B869A.9030507@redhat.com>

On 09/20/2012 02:44 PM, Itamar Heim wrote:
> On 09/19/2012 06:51 PM, snmishra at linux.vnet.ibm.com wrote:
>>
>> Hi,
>>
>>
>>      I am looking at adding support for novnc in ovirt. I have not
>> worked on the frontend and thought of asking some questions to the list
>> to save time.
>>
>>      Currently, when a user selects to open console on a VM with VNC.
>> They get a popup window with IP, port and password for VNC session. I
>> plan to take this info to host and run the command -
>>
>>    novnc_server --vnc IP:(port-5900)
>>
>> This command returns a URL, use this url to open a new browser window
>> which will be a vnc session to the VM.
>>
>>     My questions are -
>>
>> 1. How to run the novnc-server command on the host?
>
> i can't say i like the idea of launching these on the engine, but we 
> can start with making this work first with these limitations. 
> hopefully these aren't too resource intensive.
> ohad - how do you deal with this in foreman?
> adam - i saw you pushed these rpms to fedora - can you share how this 
> is launched in openstack? 
  I have to admit I have not looked at it in months.  As I recall, there 
was an URL that included all of the necessary setup information.  I'm on 
PTO this week.  If you still don't have it settled by next week, as k me 
again, and I will check on a live server.


>
> on the technical side, you can launch an external process with the 
> relevant parameters.
> though we probably need to consider how we could do this not via shell 
> locally, rather via a remote call to allow setting this up on another 
> server than the engine.
>
>
>>
>> 2. How to open a new URL for VNC session?
>
> this should be easy enough to launch another window with the requested 
> url.
> vojtech?
>
>>
>> Thanks
>> Sharad Mishra
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
>



From vszocs at redhat.com  Fri Sep 21 20:37:31 2012
From: vszocs at redhat.com (Vojtech Szocs)
Date: Fri, 21 Sep 2012 16:37:31 -0400 (EDT)
Subject: [Engine-devel] UI Plugins: PoC patch revision 5 is here
In-Reply-To: <694594112.2447860.1348243049095.JavaMail.root@redhat.com>
Message-ID: <994646689.2529178.1348259851832.JavaMail.root@redhat.com>

Hi guys, 

it's been a while but here comes the latest revision of UI Plugins proof-of-concept patch (please find it attached). 

This revision was originally meant to focus solely on server-side components of the plugin infrastructure. However, I ended up implementing all the major concepts and ideas as discussed on engine-devel mailing list, impacting both client-side and server-side parts of the plugin infrastructure. As a result, UI plugin infrastructure should be pretty much complete now, so we can focus on specific plugin API features in upcoming PoC revisions. 

There's a whole bunch of changes and improvements in this revision, so I'll try to cover all the relevant parts step by step. If you have any comments, questions or ideas, please let me know! 

So here we go... (or if you just want to get the patch, find the link at the end of this message) 



0. Added new Engine configuration values 

UI plugin data path is represented by ConfigValues.UIPluginDataPath enum option ("UIPluginDataPath" in vdc_options table), and resolved relative to ConfigValues.DataDir if possible. Following (default) values: 

    * UIPluginDataPath = ui-plugins 
    * DataDir = /usr/share/ovirt-engine 

result in UI plugin data path: /usr/share/ovirt-engine/ui-plugins 

UI plugin config path is represented by ConfigValues.UIPluginConfigPath enum option ("UIPluginConfigPath" in vdc_options table), and resolved relative to ConfigValues.ConfigDir if possible. Following (default) values: 

    * UIPluginConfigPath = ui-plugins 
    * ConfigDir = /etc/ovirt-engine 


result in UI plugin config path: /etc/ovirt-engine/ ui-plugins 
1. Processing UI plugin data on the server 

PluginDataManager is the class responsible for reading, validating and caching UI plugin descriptor/configuration data on the server (Engine). It has two main responsibilities: 

    * return a snapshot of currently valid plugin data ( getCurrentData method ) 
    * reload plugin data from local file system if necessary ( reloadData method) 


The reloadData method doesn't modify "live" plugin data directly. Instead, it creates a local working copy of current plugin data, updates this copy as it reads/validates plugin descriptor and configuration files, and attempts to update "live" plugin data through conditional reference re-assignment (using java.util.concurrent.atomic.AtomicReference.compareAndSet method). 


In other words, reloadData method makes no attempts with regard to Java lock-based synchronization, in favor of dealing with "live" data through AtomicReference (reference that involves atomic volatile reads and writes): 

    * In the best case, a thread will succeed in updating "live" data ( AtomicReference.compareAndSet == true), which means that "live" data remained unchanged since this thread acquired a reference of current plugin data. 
    * In the worst case, a thread will NOT succeed in updating "live" data ( AtomicReference.compareAndSet == false), which means that "live" data was already changed by another thread since this thread acquired a reference of current plugin data. 


In my opinion, when dealing with external resources like the local file system, this is a good compromise between performance and up-to-date data. While we might not get "completely-up-to-date" data at the given point in time ( reloadData + getCurrentData ), we are guaranteed to get "recently-up-to-date" and consistent data. In other words, the requirement of "completely-up-to-date" data would involve synchronized statements that would hurt performance. In my (very humble) opinion, the benefit of having "completely-up-to-date" data , at the cost of reduced performance, is not really worth it, especially in our case when the user can just hit refresh (F5) to reload WebAdmin and its plugin data. 



Plugin descriptor files are expected to be placed in UI plugin data path , for example: /usr/share/ovirt-engine/ui-plugins/foo.json 


Following descriptor file attributes are implemented and recognized by the plugin infrastructure: 

    * name : A name that uniquely identifies the plugin (required attribute). 
    * url : URL of plugin host page that invokes the plugin code (required attribute). 
    * config : Default configuration object associated with the plugin (optional attribute). 
    * resourcePath : Path to plugin static resources, relative to UI plugin data path (optional attribute). This is used when serving plugin files through Engine PluginResourceServlet (more on this below). 


Plugin configuration files are expected to be placed in UI plugin config path, for example: /etc/engine/ui-plugins/foo-config.json 



Note that plugin configuration files follow the "<descriptorFileName>-config.json" convention. 


Following configuration file attributes are implemented and recognized by the plugin infrastructure: 

    * config : Custom configuration object associated with the plugin (optional attribute). This overrides the default plugin descriptor configuration, if any. 
    * enabled : Indicates whether the plugin should be loaded on WebAdmin startup (optional attribute). Default value is 'true'. 
    * order : Defines the relative order in which the plugin will be loaded on WebAdmin startup (optional attribute). Default value is Integer.MAX_VALUE (lowest order). 


The concept of merging custom configuration ( config attribute in foo-config.json ), if any, on top of default configuration ( config attribute in foo.json ), if any, remains unchanged. This makes the plugin configuration quite flexible - in my opinion, the added complexity of handling/merging such configuration is definitely worth the effort. 



The enabled attribute is straight-forward, allowing users to turn the given plugin off, if necessary. In future, users should still be able to load such plugins through WebAdmin GUI. 



The order attribute controls the order in which plugins are loaded on WebAdmin startup. Since plugin resources are fetched asynchronously by the browser, this is basically a way of imposing some degree of determinism in the "generally-non-deterministic" plugin environment, which is helpful when troubleshooting problems with multiple plugins. This attribute is also helpful due to file listing methods in java.io.File giving no guarantees that files would be listed in any particular order (otherwise we could just go for the "NN-<descriptorFileName>.json" convention, with NN being the order number). 

2. Modified behavior of WebadminDynamicHostingServlet 

WebadminDynamicHostingServlet is the servlet used to serve WebAdmin application host page (HTML page that bootstraps WebAdmin JavaScript code). 

In addition to its former behavior, as part of handling the given request, WebadminDynamicHostingServlet : 

    * reloads descriptor/configuration data from local file system if necessary, and obtains a snapshot of currently valid plugin data ( PluginDataManager.reloadAndGetCurrentData ) 
    * embeds all plugin meta-data, suitable for use in client-side plugin infrastructure , into WebAdmin host page as "pluginDefinitions" JavaScript a rray ( PluginDefinitions ) 

As a result, reloading UI plugin descriptor/configuration data is as simple as refreshing (F5) WebAdmin application in the browser (no need to restart Engine). 

3. Added servlet for serving plugin static resources 

PluginResourceServlet is the servlet used to serve UI plugin static files (plugin host page, 3rd party JavaScript, etc.) from the local file system. 

For example, requesting URL: 

    * http://<EngineManagerHost>:8700/webadmin/webadmin/plugin/foo/content/start.html 


will send the content of: 

    * /usr/share/ovirt-engine/ui-plugins/<resourcePath>/ content/start.html 


to the client. 


As shown in the above example: 

    * /webadmin/webadmin/plugin/ is the servlet root path for PluginResourceServlet 
    * in the extra path beyond the servlet root path ( /foo/content/start.html ): 


        * /foo represents the name of the plugin 
        * /content/start.html represents the path to requested resource, relative to " UIPluginDataPath / < resourcePath >" 

Note that each plugin using PluginResourceServlet to serve its static files must declare non-empty resourcePath attribute in within the plugin descriptor. 

Also note that PluginResourceServlet , unlike WebadminDynamicHostingServlet , does NOT reload descriptor/configuration data from local file system as part of handling the given request. In other words, it's assumed that plugin data has already been (re)loaded when serving WebAdmin application host page, with subsequent requests to PluginResourceServlet reading the current plugin information. 

Until we solve the cross-origin issue in a clean way, PluginResourceServlet should be used to serve all plugin resources from local file system. 

4. Plugin lifecycle improved to deal with misbehaving plugins 

PluginState enum has been modified to deal with plugins that allow uncaught exceptions to escape from plugin event handler functions (e.g. "UiInit"): 

    * removed state INITIALIZED 
    * added state INITIALIZING : The plugin is (currently) being initialized by calling UiInit event handler function. 
    * added state IN_USE : Plugin's UiInit event handler function has completed successfully, we can now call other event handler functions as necessary. The plugin is in use now. 
    * added state FAILED : An uncaught exception escaped while calling an event handler function, which indicates internal error within the plugin code. The plugin is removed from service. 

I've attached a simple state diagram that illustrates different states and transitions between them (green color is initial state, red color is end state). 

Uncaught exceptions in plugin event handler functions will be caught and handled by the plugin infrastructure. This prevents a misbehaving plugin from breaking WebAdmin application, since WebAdmin is the caller (initiator) of the function call. In such case, the plugin will be removed from service. 

Update on cross-origin issue (consequence of same-origin policy) 

In order for the plugin to access WebAdmin plugin API, plugin host page (e.g. start.html ) must be served from URL on same origin as Engine origin. Otherwise, plugin code running in the context of an iframe'd host page will fail to evaluate "parent.pluginApi" expression, with "parent" being top-level (WebAdmin) window, and "pluginApi" being the global plugin API object exposed by WebAdmin. 

This is why PluginResourceServlet , available on Engine origin, should be used to serve all plugin resources from local file system. 

There's only one issue that remains to be solved: cross-origin "plugin vs. remote service" communication, with "remote service" being anything other than Engine (REST API). In future, we'll address this with Apache reverse proxy configuration, so that users can configure Apache server (placed in front of Engine JBoss AS) to put arbitrary (local or remote non-Engine) services on same origin. However, this requires a change in current Apache configuration. Until then, users can manually edit the Engine Apache configuration file ( /etc/httpd/conf.d/ovirt-engine.conf ). 



I've attached some sample plugin files for you to experiment with. Instead of attaching actual patch file (92 kB) to this email, I've submitted the patch to oVirt Gerrit: http://gerrit.ovirt.org/8120 

Let me know what you think! 

Cheers, 
Vojtech 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120921/579207ff/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ui-plugin-sample-files.tar.gz
Type: application/x-compressed-tar
Size: 2100 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120921/579207ff/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ui-plugin-lifecycle.png
Type: image/png
Size: 14616 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120921/579207ff/attachment.png>

From mkublin at redhat.com  Sun Sep 23 10:41:05 2012
From: mkublin at redhat.com (Michael Kublin)
Date: Sun, 23 Sep 2012 06:41:05 -0400 (EDT)
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <50363D1B.8030809@redhat.com>
Message-ID: <180214377.123398.1348396865566.JavaMail.root@redhat.com>


>>>>>>>>>> Hi guys,
>>>>>>>>>>
>>>>>>>>>> As you may know the engine currently has the ability to fire
>>>>>>>>>> an
>>>>>>>>>> SPM
>>>>>>>>>> task, and be asynchronously be "woken-up" when it ends.
>>>>>>>>>> This is great, but we found the for the Live Storage
>>>>>>>>>> Migration
>>>>>>>>>> feature we need something a bit complex - the ability to
>>>>>>>>>> have a
>>>>>>>>>> series of async tasks in a single control flow.
>>>>>>>>>>
>>>>>>>>>> Here's my initial design for this, your comments and
>>>>>>>>>> criticism
>>>>>>>>>> would
>>>>>>>>>> be welcome:
>>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
>>>>>>>> -successful execution -
>>>>>>>> * "CommandBase iterates over its SPMAsyncTaskHandlers" - when?
>>>>>>> This is the new suggested format of executeCommand(). I'll
>>>>>>> clarify
>>>>>>> this too.
>>>>>>>
>>>>>>>> * If the second task is an HSM command (vs. SPM command), I
>>>>>>>> think you
>>>>>>>> should explain in the design how to handle such flows as well.
>>>>>>> HSM commands do not create AsyncTasks, as they do today - I
>>>>>>> will
>>>>>>> clarify this.
>>>>>>>
>>>>>>>> * Why do we need before task? can you give a concrete example
>>>>>>>> of what
>>>>>>>> would you do in such a method.
>>>>>>> Basically, /today/, command look like this:
>>>>>>> executeCommand() {
>>>>>>>     doStuffInTheDB();
>>>>>>>     runVdsCommand(someCommand);
>>>>>>> }
>>>>>>>
>>>>>>> endSuccessfully() {
>>>>>>>     doMoreStuffInTheDB();
>>>>>>> }
>>>>>>>
>>>>>>> endWithFailure() {
>>>>>>>     doMoreStuffForFailureInTheDB();
>>>>>>> }
>>>>>>>
>>>>>>> In the new design, the entire doStuffInTheDB() should be moved
>>>>>>> to a
>>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
>>>>>>>
>>>>>>>>
>>>>>>>> - I see you added SPMAsyncTaskHandler, any reason not to use
>>>>>>>> SPMAsyncTasK to manage it own life-cycle?
>>>>>>> Conserving today's design - The SPMAsyncTaskHandler is the
>>>>>>> place to
>>>>>>> add additional, non-SPM, logic around the SPM task execution,
>>>>>>> like
>>>>>>> CommandBase allows today.
>>>>>>>
>>>>>>>>
>>>>>>>> - In the life-cycle managed by the SPMAsyncTaskHandler there
>>>>>>>> is a
>>>>>>>> step
>>>>>>>> 'createTask - how to create the async task' can you please
>>>>>>>> elaborate
>>>>>>>> what are the options.
>>>>>>> new [any type of async task]

(I cleaned thread a little.)
The following design and it is implementation http://gerrit.ovirt.org/#/c/7956/
is bad.
I don't see any reason for creating a new SPMAsyncTaskHandler and especially in the 
way as it's done in patch.
The reason are following:
1. Performance , increased memory footprint, created CYCLIC REFERENCE.
2. Readability and robust of code: the code which is written as cyclic references is unreadable
   and difficult for debug.
3. Why I need a generic implementation and changes all over whole project because of 
   series of async commands, for me it is a private case?



From amureini at redhat.com  Sun Sep 23 10:51:44 2012
From: amureini at redhat.com (Allon Mureinik)
Date: Sun, 23 Sep 2012 06:51:44 -0400 (EDT)
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <180214377.123398.1348396865566.JavaMail.root@redhat.com>
Message-ID: <354178440.179031.1348397504157.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Michael Kublin" <mkublin at redhat.com>
> To: "Allon Mureinik" <amureini at redhat.com>
> Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Liron Aravot" <laravot at redhat.com>, "Maor Lipchuk"
> <mlipchuk at redhat.com>, "engine-devel" <engine-devel at ovirt.org>
> Sent: Sunday, September 23, 2012 12:41:05 PM
> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> 
> 
> >>>>>>>>>> Hi guys,
> >>>>>>>>>>
> >>>>>>>>>> As you may know the engine currently has the ability to
> >>>>>>>>>> fire
> >>>>>>>>>> an
> >>>>>>>>>> SPM
> >>>>>>>>>> task, and be asynchronously be "woken-up" when it ends.
> >>>>>>>>>> This is great, but we found the for the Live Storage
> >>>>>>>>>> Migration
> >>>>>>>>>> feature we need something a bit complex - the ability to
> >>>>>>>>>> have a
> >>>>>>>>>> series of async tasks in a single control flow.
> >>>>>>>>>>
> >>>>>>>>>> Here's my initial design for this, your comments and
> >>>>>>>>>> criticism
> >>>>>>>>>> would
> >>>>>>>>>> be welcome:
> >>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
> >>>>>>>> -successful execution -
> >>>>>>>> * "CommandBase iterates over its SPMAsyncTaskHandlers" -
> >>>>>>>> when?
> >>>>>>> This is the new suggested format of executeCommand(). I'll
> >>>>>>> clarify
> >>>>>>> this too.
> >>>>>>>
> >>>>>>>> * If the second task is an HSM command (vs. SPM command), I
> >>>>>>>> think you
> >>>>>>>> should explain in the design how to handle such flows as
> >>>>>>>> well.
> >>>>>>> HSM commands do not create AsyncTasks, as they do today - I
> >>>>>>> will
> >>>>>>> clarify this.
> >>>>>>>
> >>>>>>>> * Why do we need before task? can you give a concrete
> >>>>>>>> example
> >>>>>>>> of what
> >>>>>>>> would you do in such a method.
> >>>>>>> Basically, /today/, command look like this:
> >>>>>>> executeCommand() {
> >>>>>>>     doStuffInTheDB();
> >>>>>>>     runVdsCommand(someCommand);
> >>>>>>> }
> >>>>>>>
> >>>>>>> endSuccessfully() {
> >>>>>>>     doMoreStuffInTheDB();
> >>>>>>> }
> >>>>>>>
> >>>>>>> endWithFailure() {
> >>>>>>>     doMoreStuffForFailureInTheDB();
> >>>>>>> }
> >>>>>>>
> >>>>>>> In the new design, the entire doStuffInTheDB() should be
> >>>>>>> moved
> >>>>>>> to a
> >>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> - I see you added SPMAsyncTaskHandler, any reason not to use
> >>>>>>>> SPMAsyncTasK to manage it own life-cycle?
> >>>>>>> Conserving today's design - The SPMAsyncTaskHandler is the
> >>>>>>> place to
> >>>>>>> add additional, non-SPM, logic around the SPM task execution,
> >>>>>>> like
> >>>>>>> CommandBase allows today.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> - In the life-cycle managed by the SPMAsyncTaskHandler there
> >>>>>>>> is a
> >>>>>>>> step
> >>>>>>>> 'createTask - how to create the async task' can you please
> >>>>>>>> elaborate
> >>>>>>>> what are the options.
> >>>>>>> new [any type of async task]
> 
> (I cleaned thread a little.)
> The following design and it is implementation
> http://gerrit.ovirt.org/#/c/7956/
> is bad.
> I don't see any reason for creating a new SPMAsyncTaskHandler and
> especially in the
> way as it's done in patch.
> The reason are following:
> 1. Performance , increased memory footprint, created CYCLIC
> REFERENCE.
> 2. Readability and robust of code: the code which is written as
> cyclic references is unreadable
>    and difficult for debug.
> 3. Why I need a generic implementation and changes all over whole
> project because of
>    series of async commands, for me it is a private case?
This will occur all over the storage commands (which are the only usages of tasks nowadys).
Moreover, async task handling is done at the Commandbase level (see the end* methods) - instead of hacking it in X different places whenever we need it, I'd prefer doing it once, properly.
> 
> 


From abaron at redhat.com  Sun Sep 23 11:10:07 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 23 Sep 2012 07:10:07 -0400 (EDT)
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <354178440.179031.1348397504157.JavaMail.root@redhat.com>
Message-ID: <1239991594.75727.1348398607629.JavaMail.root@redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Michael Kublin" <mkublin at redhat.com>
> > To: "Allon Mureinik" <amureini at redhat.com>
> > Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Liron Aravot"
> > <laravot at redhat.com>, "Maor Lipchuk"
> > <mlipchuk at redhat.com>, "engine-devel" <engine-devel at ovirt.org>
> > Sent: Sunday, September 23, 2012 12:41:05 PM
> > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > 
> > 
> > >>>>>>>>>> Hi guys,
> > >>>>>>>>>>
> > >>>>>>>>>> As you may know the engine currently has the ability to
> > >>>>>>>>>> fire
> > >>>>>>>>>> an
> > >>>>>>>>>> SPM
> > >>>>>>>>>> task, and be asynchronously be "woken-up" when it ends.
> > >>>>>>>>>> This is great, but we found the for the Live Storage
> > >>>>>>>>>> Migration
> > >>>>>>>>>> feature we need something a bit complex - the ability to
> > >>>>>>>>>> have a
> > >>>>>>>>>> series of async tasks in a single control flow.
> > >>>>>>>>>>
> > >>>>>>>>>> Here's my initial design for this, your comments and
> > >>>>>>>>>> criticism
> > >>>>>>>>>> would
> > >>>>>>>>>> be welcome:
> > >>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
> > >>>>>>>> -successful execution -
> > >>>>>>>> * "CommandBase iterates over its SPMAsyncTaskHandlers" -
> > >>>>>>>> when?
> > >>>>>>> This is the new suggested format of executeCommand(). I'll
> > >>>>>>> clarify
> > >>>>>>> this too.
> > >>>>>>>
> > >>>>>>>> * If the second task is an HSM command (vs. SPM command),
> > >>>>>>>> I
> > >>>>>>>> think you
> > >>>>>>>> should explain in the design how to handle such flows as
> > >>>>>>>> well.
> > >>>>>>> HSM commands do not create AsyncTasks, as they do today - I
> > >>>>>>> will
> > >>>>>>> clarify this.
> > >>>>>>>
> > >>>>>>>> * Why do we need before task? can you give a concrete
> > >>>>>>>> example
> > >>>>>>>> of what
> > >>>>>>>> would you do in such a method.
> > >>>>>>> Basically, /today/, command look like this:
> > >>>>>>> executeCommand() {
> > >>>>>>>     doStuffInTheDB();
> > >>>>>>>     runVdsCommand(someCommand);
> > >>>>>>> }
> > >>>>>>>
> > >>>>>>> endSuccessfully() {
> > >>>>>>>     doMoreStuffInTheDB();
> > >>>>>>> }
> > >>>>>>>
> > >>>>>>> endWithFailure() {
> > >>>>>>>     doMoreStuffForFailureInTheDB();
> > >>>>>>> }
> > >>>>>>>
> > >>>>>>> In the new design, the entire doStuffInTheDB() should be
> > >>>>>>> moved
> > >>>>>>> to a
> > >>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> - I see you added SPMAsyncTaskHandler, any reason not to
> > >>>>>>>> use
> > >>>>>>>> SPMAsyncTasK to manage it own life-cycle?
> > >>>>>>> Conserving today's design - The SPMAsyncTaskHandler is the
> > >>>>>>> place to
> > >>>>>>> add additional, non-SPM, logic around the SPM task
> > >>>>>>> execution,
> > >>>>>>> like
> > >>>>>>> CommandBase allows today.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> - In the life-cycle managed by the SPMAsyncTaskHandler
> > >>>>>>>> there
> > >>>>>>>> is a
> > >>>>>>>> step
> > >>>>>>>> 'createTask - how to create the async task' can you please
> > >>>>>>>> elaborate
> > >>>>>>>> what are the options.
> > >>>>>>> new [any type of async task]
> > 
> > (I cleaned thread a little.)
> > The following design and it is implementation
> > http://gerrit.ovirt.org/#/c/7956/
> > is bad.
> > I don't see any reason for creating a new SPMAsyncTaskHandler and
> > especially in the
> > way as it's done in patch.
> > The reason are following:
> > 1. Performance , increased memory footprint, created CYCLIC
> > REFERENCE.
> > 2. Readability and robust of code: the code which is written as
> > cyclic references is unreadable
> >    and difficult for debug.
> > 3. Why I need a generic implementation and changes all over whole
> > project because of
> >    series of async commands, for me it is a private case?

What is the private case here exactly?
Every task can have multiple jobs.  We've identified several such places (e.g. live storage migration, move disk, move vm) and I have no doubt more will popup.
As Allon notes below, task handling is done at CommandBase, if you think task management should be for storage only, you're welcome to push it down to StorageHandlingCommandBase (or get rid of inheritance here altogether).

> This will occur all over the storage commands (which are the only
> usages of tasks nowadys).
> Moreover, async task handling is done at the Commandbase level (see
> the end* methods) - instead of hacking it in X different places
> whenever we need it, I'd prefer doing it once, properly.
> > 
> > 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From alkaplan at redhat.com  Sun Sep 23 11:31:32 2012
From: alkaplan at redhat.com (Alona Kaplan)
Date: Sun, 23 Sep 2012 07:31:32 -0400 (EDT)
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <24490165.15.1348156667359.JavaMail.atal@atal-fedora.tlv.redhat.com>
Message-ID: <284609224.292171.1348399892415.JavaMail.root@redhat.com>

1. What actions do you suggest to add?
2. I don't understand your suggestion. You mean add SetupNetworks to the Networks main tab?

----- Original Message -----
> From: "Avi Tal" <atal at redhat.com>
> To: "Alona Kaplan" <alkaplan at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Thursday, September 20, 2012 6:58:05 PM
> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> 1. Why not having Actions for Hosts, Vms and Templates ?
> I believe we should have those actions instead of making the user
> browse between 'network -> vms -> network (sub tab)' just in order
> to update VM's network.
> 
> 2. Why having such a big window which most of the time will be empty?
>    How many logical network would be existed in the system?
>    My suggestion is to go for more graphical window which will
>    present:
>    A. logical networks
>    B. Multiple networks with the host configurations at the same
>    window: very important because most of the System admins would
>    like to have a bigger picture of the entire network
>    configuration
> 
> 
> 
> 
> 
> ----- Original Message -----
> From: "Alona Kaplan" <alkaplan at redhat.com>
> To: engine-devel at ovirt.org
> Sent: Thursday, September 20, 2012 1:50:15 PM
> Subject: [Engine-devel] RFE: Netwrok Main Tab
> 
> Hi all,
> 
> Please review the wiki and add your comments.
> 
> http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> 
> 
> 
> RFE:
> https://bugzilla.redhat.com/858742
> 
> Thanks,
> Alona.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From mkublin at redhat.com  Sun Sep 23 12:03:38 2012
From: mkublin at redhat.com (Michael Kublin)
Date: Sun, 23 Sep 2012 08:03:38 -0400 (EDT)
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <1239991594.75727.1348398607629.JavaMail.root@redhat.com>
Message-ID: <1732039544.128498.1348401818357.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Allon Mureinik" <amureini at redhat.com>, "Michael Kublin" <mkublin at redhat.com>
> Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel" <engine-devel at ovirt.org>, "Eduardo Warszawski"
> <ewarszaw at redhat.com>
> Sent: Sunday, September 23, 2012 1:10:07 PM
> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> 
> 
> 
> ----- Original Message -----
> > 
> > 
> > ----- Original Message -----
> > > From: "Michael Kublin" <mkublin at redhat.com>
> > > To: "Allon Mureinik" <amureini at redhat.com>
> > > Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Liron Aravot"
> > > <laravot at redhat.com>, "Maor Lipchuk"
> > > <mlipchuk at redhat.com>, "engine-devel" <engine-devel at ovirt.org>
> > > Sent: Sunday, September 23, 2012 12:41:05 PM
> > > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > > 
> > > 
> > > >>>>>>>>>> Hi guys,
> > > >>>>>>>>>>
> > > >>>>>>>>>> As you may know the engine currently has the ability
> > > >>>>>>>>>> to
> > > >>>>>>>>>> fire
> > > >>>>>>>>>> an
> > > >>>>>>>>>> SPM
> > > >>>>>>>>>> task, and be asynchronously be "woken-up" when it
> > > >>>>>>>>>> ends.
> > > >>>>>>>>>> This is great, but we found the for the Live Storage
> > > >>>>>>>>>> Migration
> > > >>>>>>>>>> feature we need something a bit complex - the ability
> > > >>>>>>>>>> to
> > > >>>>>>>>>> have a
> > > >>>>>>>>>> series of async tasks in a single control flow.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Here's my initial design for this, your comments and
> > > >>>>>>>>>> criticism
> > > >>>>>>>>>> would
> > > >>>>>>>>>> be welcome:
> > > >>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
> > > >>>>>>>> -successful execution -
> > > >>>>>>>> * "CommandBase iterates over its SPMAsyncTaskHandlers" -
> > > >>>>>>>> when?
> > > >>>>>>> This is the new suggested format of executeCommand().
> > > >>>>>>> I'll
> > > >>>>>>> clarify
> > > >>>>>>> this too.
> > > >>>>>>>
> > > >>>>>>>> * If the second task is an HSM command (vs. SPM
> > > >>>>>>>> command),
> > > >>>>>>>> I
> > > >>>>>>>> think you
> > > >>>>>>>> should explain in the design how to handle such flows as
> > > >>>>>>>> well.
> > > >>>>>>> HSM commands do not create AsyncTasks, as they do today -
> > > >>>>>>> I
> > > >>>>>>> will
> > > >>>>>>> clarify this.
> > > >>>>>>>
> > > >>>>>>>> * Why do we need before task? can you give a concrete
> > > >>>>>>>> example
> > > >>>>>>>> of what
> > > >>>>>>>> would you do in such a method.
> > > >>>>>>> Basically, /today/, command look like this:
> > > >>>>>>> executeCommand() {
> > > >>>>>>>     doStuffInTheDB();
> > > >>>>>>>     runVdsCommand(someCommand);
> > > >>>>>>> }
> > > >>>>>>>
> > > >>>>>>> endSuccessfully() {
> > > >>>>>>>     doMoreStuffInTheDB();
> > > >>>>>>> }
> > > >>>>>>>
> > > >>>>>>> endWithFailure() {
> > > >>>>>>>     doMoreStuffForFailureInTheDB();
> > > >>>>>>> }
> > > >>>>>>>
> > > >>>>>>> In the new design, the entire doStuffInTheDB() should be
> > > >>>>>>> moved
> > > >>>>>>> to a
> > > >>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> - I see you added SPMAsyncTaskHandler, any reason not to
> > > >>>>>>>> use
> > > >>>>>>>> SPMAsyncTasK to manage it own life-cycle?
> > > >>>>>>> Conserving today's design - The SPMAsyncTaskHandler is
> > > >>>>>>> the
> > > >>>>>>> place to
> > > >>>>>>> add additional, non-SPM, logic around the SPM task
> > > >>>>>>> execution,
> > > >>>>>>> like
> > > >>>>>>> CommandBase allows today.
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> - In the life-cycle managed by the SPMAsyncTaskHandler
> > > >>>>>>>> there
> > > >>>>>>>> is a
> > > >>>>>>>> step
> > > >>>>>>>> 'createTask - how to create the async task' can you
> > > >>>>>>>> please
> > > >>>>>>>> elaborate
> > > >>>>>>>> what are the options.
> > > >>>>>>> new [any type of async task]
> > > 
> > > (I cleaned thread a little.)
> > > The following design and it is implementation
> > > http://gerrit.ovirt.org/#/c/7956/
> > > is bad.
> > > I don't see any reason for creating a new SPMAsyncTaskHandler and
> > > especially in the
> > > way as it's done in patch.
> > > The reason are following:
> > > 1. Performance , increased memory footprint, created CYCLIC
> > > REFERENCE.
> > > 2. Readability and robust of code: the code which is written as
> > > cyclic references is unreadable
> > >    and difficult for debug.
> > > 3. Why I need a generic implementation and changes all over whole
> > > project because of
> > >    series of async commands, for me it is a private case?
> 
> What is the private case here exactly?
> Every task can have multiple jobs.  We've identified several such
> places (e.g. live storage migration, move disk, move vm) and I have
> no doubt more will popup.
> As Allon notes below, task handling is done at CommandBase, if you
> think task management should be for storage only, you're welcome to
> push it down to StorageHandlingCommandBase (or get rid of
> inheritance here altogether).
Interesting , regards cyclic reference no response, but who cares,
it is difficult to answer , that's why better not to response?
Regards private case:
1. We have command that not creating any task
2. We have command that will create a one task.
3. And we have 3 commands meanwhile which will create more than one task.
I think that 3 is private case and not common? (In the future, I removed too many
lines of code that were preparation for future that never come)
The handling done at CommandBase it means that it is influence all system.
Now regards architecture why I need some handler which will be inside a command 
and will call for command methods? Please explain.

  
> > This will occur all over the storage commands (which are the only
> > usages of tasks nowadys).
> > Moreover, async task handling is done at the Commandbase level (see
> > the end* methods) - instead of hacking it in X different places
> > whenever we need it, I'd prefer doing it once, properly.
> > > 
> > > 
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 


From atal at redhat.com  Sun Sep 23 14:17:26 2012
From: atal at redhat.com (Avi Tal)
Date: Sun, 23 Sep 2012 10:17:26 -0400 (EDT)
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <284609224.292171.1348399892415.JavaMail.root@redhat.com>
Message-ID: <7388614.48.1348409840002.JavaMail.atal@atal-fedora.tlv.redhat.com>



----- Original Message -----
> From: "Alona Kaplan" <alkaplan at redhat.com>
> To: "Avi Tal" <atal at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Sunday, September 23, 2012 1:31:32 PM
> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> 1. What actions do you suggest to add?

Add, Edit and remove actions of each component.

> 2. I don't understand your suggestion. You mean add SetupNetworks to
> the Networks main tab?

No.
Just Thinking of a nice display which will present logical network and his connected host/vm configuration instead of passing through dub-tabs
I should give it a deeper thought.


> 
> ----- Original Message -----
> > From: "Avi Tal" <atal at redhat.com>
> > To: "Alona Kaplan" <alkaplan at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Thursday, September 20, 2012 6:58:05 PM
> > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > 
> > 1. Why not having Actions for Hosts, Vms and Templates ?
> > I believe we should have those actions instead of making the user
> > browse between 'network -> vms -> network (sub tab)' just in order
> > to update VM's network.
> > 
> > 2. Why having such a big window which most of the time will be
> > empty?
> >    How many logical network would be existed in the system?
> >    My suggestion is to go for more graphical window which will
> >    present:
> >    A. logical networks
> >    B. Multiple networks with the host configurations at the same
> >    window: very important because most of the System admins would
> >    like to have a bigger picture of the entire network
> >    configuration
> > 
> > 
> > 
> > 
> > 
> > ----- Original Message -----
> > From: "Alona Kaplan" <alkaplan at redhat.com>
> > To: engine-devel at ovirt.org
> > Sent: Thursday, September 20, 2012 1:50:15 PM
> > Subject: [Engine-devel] RFE: Netwrok Main Tab
> > 
> > Hi all,
> > 
> > Please review the wiki and add your comments.
> > 
> > http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> > 
> > 
> > 
> > RFE:
> > https://bugzilla.redhat.com/858742
> > 
> > Thanks,
> > Alona.
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
> 


From alkaplan at redhat.com  Sun Sep 23 14:55:54 2012
From: alkaplan at redhat.com (Alona Kaplan)
Date: Sun, 23 Sep 2012 10:55:54 -0400 (EDT)
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <7388614.48.1348409840002.JavaMail.atal@atal-fedora.tlv.redhat.com>
Message-ID: <157698272.342581.1348412154540.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Avi Tal" <atal at redhat.com>
> To: "Alona Kaplan" <alkaplan at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Sunday, September 23, 2012 4:17:26 PM
> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> 
> 
> ----- Original Message -----
> > From: "Alona Kaplan" <alkaplan at redhat.com>
> > To: "Avi Tal" <atal at redhat.com>
> > Cc: engine-devel at ovirt.org
> > Sent: Sunday, September 23, 2012 1:31:32 PM
> > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > 
> > 1. What actions do you suggest to add?
> 
> Add, Edit and remove actions of each component.
> 
Host:
Add- What will add action on Networks->Hosts sub tab do? Choose an existing host and attach the network to one of its nics?
How will it work? after choosing the host you will open setupNetworks window and just enable dragging of the selected network (in the main tab) to a nic? I think it is too much.

Edit- same as add.

Remove- What is the meaning of Remove host in network's context? The network will be detached from the host? I think it is confusing.

Vm/Template:
Add: What will it do? You will choose a vm and then add a vnic to the vm? Where will you see the vnic details?
Edit: Same as add.
Remove: You will remove all the vnics that use the selected network from the vm? Or do you mean to add a remove per vnic?
> > 2. I don't understand your suggestion. You mean add SetupNetworks
> > to
> > the Networks main tab?
> 
> No.
> Just Thinking of a nice display which will present logical network
> and his connected host/vm configuration instead of passing through
> dub-tabs
> I should give it a deeper thought.
> 
> 
> > 
> > ----- Original Message -----
> > > From: "Avi Tal" <atal at redhat.com>
> > > To: "Alona Kaplan" <alkaplan at redhat.com>
> > > Cc: engine-devel at ovirt.org
> > > Sent: Thursday, September 20, 2012 6:58:05 PM
> > > Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> > > 
> > > 1. Why not having Actions for Hosts, Vms and Templates ?
> > > I believe we should have those actions instead of making the user
> > > browse between 'network -> vms -> network (sub tab)' just in
> > > order
> > > to update VM's network.
> > > 
> > > 2. Why having such a big window which most of the time will be
> > > empty?
> > >    How many logical network would be existed in the system?
> > >    My suggestion is to go for more graphical window which will
> > >    present:
> > >    A. logical networks
> > >    B. Multiple networks with the host configurations at the same
> > >    window: very important because most of the System admins would
> > >    like to have a bigger picture of the entire network
> > >    configuration
> > > 
> > > 
> > > 
> > > 
> > > 
> > > ----- Original Message -----
> > > From: "Alona Kaplan" <alkaplan at redhat.com>
> > > To: engine-devel at ovirt.org
> > > Sent: Thursday, September 20, 2012 1:50:15 PM
> > > Subject: [Engine-devel] RFE: Netwrok Main Tab
> > > 
> > > Hi all,
> > > 
> > > Please review the wiki and add your comments.
> > > 
> > > http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> > > 
> > > 
> > > 
> > > RFE:
> > > https://bugzilla.redhat.com/858742
> > > 
> > > Thanks,
> > > Alona.
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > 
> 


From ykaul at redhat.com  Sun Sep 23 15:01:09 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Sun, 23 Sep 2012 17:01:09 +0200
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <157698272.342581.1348412154540.JavaMail.root@redhat.com>
References: <157698272.342581.1348412154540.JavaMail.root@redhat.com>
Message-ID: <505F2435.3060801@redhat.com>

On 09/23/2012 04:55 PM, Alona Kaplan wrote:
>
> ----- Original Message -----
>> From: "Avi Tal" <atal at redhat.com>
>> To: "Alona Kaplan" <alkaplan at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Sunday, September 23, 2012 4:17:26 PM
>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>
>>
>>
>> ----- Original Message -----
>>> From: "Alona Kaplan" <alkaplan at redhat.com>
>>> To: "Avi Tal" <atal at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Sunday, September 23, 2012 1:31:32 PM
>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>>
>>> 1. What actions do you suggest to add?
>> Add, Edit and remove actions of each component.
>>
> Host:
> Add- What will add action on Networks->Hosts sub tab do? Choose an existing host and attach the network to one of its nics?
> How will it work? after choosing the host you will open setupNetworks window and just enable dragging of the selected network (in the main tab) to a nic? I think it is too much.
>
> Edit- same as add.
>
> Remove- What is the meaning of Remove host in network's context? The network will be detached from the host? I think it is confusing.
>
> Vm/Template:
> Add: What will it do? You will choose a vm and then add a vnic to the vm? Where will you see the vnic details?
> Edit: Same as add.
> Remove: You will remove all the vnics that use the selected network from the vm? Or do you mean to add a remove per vnic?

For all the above: yes.
We can certainly work on the small details, but having a main tab with 
little to no action whatsoever is kinda disappointing.
(example of 'small details' - yes, when clicking remove for a 
VM/template - it will remove all vNICs that use that network from the 
VM. A VM with more than one vNIC on the same network is not the common 
case anyway, and if you want to remove just a single vNIC, go to the 
VM). I think we can also live with Add/Remove only, and 'defer' the edit 
actions to the respectable objects (host/vm/template) edit dialog boxes 
that originates from their own main tab.

Y.

>>> 2. I don't understand your suggestion. You mean add SetupNetworks
>>> to
>>> the Networks main tab?
>> No.
>> Just Thinking of a nice display which will present logical network
>> and his connected host/vm configuration instead of passing through
>> dub-tabs
>> I should give it a deeper thought.
>>
>>
>>> ----- Original Message -----
>>>> From: "Avi Tal" <atal at redhat.com>
>>>> To: "Alona Kaplan" <alkaplan at redhat.com>
>>>> Cc: engine-devel at ovirt.org
>>>> Sent: Thursday, September 20, 2012 6:58:05 PM
>>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>>>
>>>> 1. Why not having Actions for Hosts, Vms and Templates ?
>>>> I believe we should have those actions instead of making the user
>>>> browse between 'network -> vms -> network (sub tab)' just in
>>>> order
>>>> to update VM's network.
>>>>
>>>> 2. Why having such a big window which most of the time will be
>>>> empty?
>>>>     How many logical network would be existed in the system?
>>>>     My suggestion is to go for more graphical window which will
>>>>     present:
>>>>     A. logical networks
>>>>     B. Multiple networks with the host configurations at the same
>>>>     window: very important because most of the System admins would
>>>>     like to have a bigger picture of the entire network
>>>>     configuration
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>> From: "Alona Kaplan" <alkaplan at redhat.com>
>>>> To: engine-devel at ovirt.org
>>>> Sent: Thursday, September 20, 2012 1:50:15 PM
>>>> Subject: [Engine-devel] RFE: Netwrok Main Tab
>>>>
>>>> Hi all,
>>>>
>>>> Please review the wiki and add your comments.
>>>>
>>>> http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
>>>>
>>>>
>>>>
>>>> RFE:
>>>> https://bugzilla.redhat.com/858742
>>>>
>>>> Thanks,
>>>> Alona.
>>>> _______________________________________________
>>>> Engine-devel mailing list
>>>> Engine-devel at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From abaron at redhat.com  Sun Sep 23 15:27:54 2012
From: abaron at redhat.com (Ayal Baron)
Date: Sun, 23 Sep 2012 11:27:54 -0400 (EDT)
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <1732039544.128498.1348401818357.JavaMail.root@redhat.com>
Message-ID: <1076165956.100805.1348414074541.JavaMail.root@redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Ayal Baron" <abaron at redhat.com>
> > To: "Allon Mureinik" <amureini at redhat.com>, "Michael Kublin"
> > <mkublin at redhat.com>
> > Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel"
> > <engine-devel at ovirt.org>, "Eduardo Warszawski"
> > <ewarszaw at redhat.com>
> > Sent: Sunday, September 23, 2012 1:10:07 PM
> > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > 
> > 
> > 
> > ----- Original Message -----
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Michael Kublin" <mkublin at redhat.com>
> > > > To: "Allon Mureinik" <amureini at redhat.com>
> > > > Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Liron Aravot"
> > > > <laravot at redhat.com>, "Maor Lipchuk"
> > > > <mlipchuk at redhat.com>, "engine-devel" <engine-devel at ovirt.org>
> > > > Sent: Sunday, September 23, 2012 12:41:05 PM
> > > > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > > > 
> > > > 
> > > > >>>>>>>>>> Hi guys,
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> As you may know the engine currently has the ability
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>> fire
> > > > >>>>>>>>>> an
> > > > >>>>>>>>>> SPM
> > > > >>>>>>>>>> task, and be asynchronously be "woken-up" when it
> > > > >>>>>>>>>> ends.
> > > > >>>>>>>>>> This is great, but we found the for the Live Storage
> > > > >>>>>>>>>> Migration
> > > > >>>>>>>>>> feature we need something a bit complex - the
> > > > >>>>>>>>>> ability
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>> have a
> > > > >>>>>>>>>> series of async tasks in a single control flow.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Here's my initial design for this, your comments and
> > > > >>>>>>>>>> criticism
> > > > >>>>>>>>>> would
> > > > >>>>>>>>>> be welcome:
> > > > >>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
> > > > >>>>>>>> -successful execution -
> > > > >>>>>>>> * "CommandBase iterates over its SPMAsyncTaskHandlers"
> > > > >>>>>>>> -
> > > > >>>>>>>> when?
> > > > >>>>>>> This is the new suggested format of executeCommand().
> > > > >>>>>>> I'll
> > > > >>>>>>> clarify
> > > > >>>>>>> this too.
> > > > >>>>>>>
> > > > >>>>>>>> * If the second task is an HSM command (vs. SPM
> > > > >>>>>>>> command),
> > > > >>>>>>>> I
> > > > >>>>>>>> think you
> > > > >>>>>>>> should explain in the design how to handle such flows
> > > > >>>>>>>> as
> > > > >>>>>>>> well.
> > > > >>>>>>> HSM commands do not create AsyncTasks, as they do today
> > > > >>>>>>> -
> > > > >>>>>>> I
> > > > >>>>>>> will
> > > > >>>>>>> clarify this.
> > > > >>>>>>>
> > > > >>>>>>>> * Why do we need before task? can you give a concrete
> > > > >>>>>>>> example
> > > > >>>>>>>> of what
> > > > >>>>>>>> would you do in such a method.
> > > > >>>>>>> Basically, /today/, command look like this:
> > > > >>>>>>> executeCommand() {
> > > > >>>>>>>     doStuffInTheDB();
> > > > >>>>>>>     runVdsCommand(someCommand);
> > > > >>>>>>> }
> > > > >>>>>>>
> > > > >>>>>>> endSuccessfully() {
> > > > >>>>>>>     doMoreStuffInTheDB();
> > > > >>>>>>> }
> > > > >>>>>>>
> > > > >>>>>>> endWithFailure() {
> > > > >>>>>>>     doMoreStuffForFailureInTheDB();
> > > > >>>>>>> }
> > > > >>>>>>>
> > > > >>>>>>> In the new design, the entire doStuffInTheDB() should
> > > > >>>>>>> be
> > > > >>>>>>> moved
> > > > >>>>>>> to a
> > > > >>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> - I see you added SPMAsyncTaskHandler, any reason not
> > > > >>>>>>>> to
> > > > >>>>>>>> use
> > > > >>>>>>>> SPMAsyncTasK to manage it own life-cycle?
> > > > >>>>>>> Conserving today's design - The SPMAsyncTaskHandler is
> > > > >>>>>>> the
> > > > >>>>>>> place to
> > > > >>>>>>> add additional, non-SPM, logic around the SPM task
> > > > >>>>>>> execution,
> > > > >>>>>>> like
> > > > >>>>>>> CommandBase allows today.
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> - In the life-cycle managed by the SPMAsyncTaskHandler
> > > > >>>>>>>> there
> > > > >>>>>>>> is a
> > > > >>>>>>>> step
> > > > >>>>>>>> 'createTask - how to create the async task' can you
> > > > >>>>>>>> please
> > > > >>>>>>>> elaborate
> > > > >>>>>>>> what are the options.
> > > > >>>>>>> new [any type of async task]
> > > > 
> > > > (I cleaned thread a little.)
> > > > The following design and it is implementation
> > > > http://gerrit.ovirt.org/#/c/7956/
> > > > is bad.
> > > > I don't see any reason for creating a new SPMAsyncTaskHandler
> > > > and
> > > > especially in the
> > > > way as it's done in patch.
> > > > The reason are following:
> > > > 1. Performance , increased memory footprint, created CYCLIC
> > > > REFERENCE.
> > > > 2. Readability and robust of code: the code which is written as
> > > > cyclic references is unreadable
> > > >    and difficult for debug.
> > > > 3. Why I need a generic implementation and changes all over
> > > > whole
> > > > project because of
> > > >    series of async commands, for me it is a private case?
> > 
> > What is the private case here exactly?
> > Every task can have multiple jobs.  We've identified several such
> > places (e.g. live storage migration, move disk, move vm) and I have
> > no doubt more will popup.
> > As Allon notes below, task handling is done at CommandBase, if you
> > think task management should be for storage only, you're welcome to
> > push it down to StorageHandlingCommandBase (or get rid of
> > inheritance here altogether).
> Interesting , regards cyclic reference no response, but who cares,
> it is difficult to answer , that's why better not to response?

There is no problem with cyclic references in general, GCs know how to deal with these just fine and in this case it's limited to the command and its handlers.
I did not reply, however, as I do not feel strongly about this.

> Regards private case:
> 1. We have command that not creating any task
> 2. We have command that will create a one task.
> 3. And we have 3 commands meanwhile which will create more than one
> task.
> I think that 3 is private case and not common? (In the future, I

once happens
twice is a coincidence
three times is a method

But if you insist on more then it's easy enough.  We've discussed many times in the past that we need to change many of the storage verbs to run asynchronously (e.g. createStorageDomain) once this happens then existing flows would have to run multiple async tasks serially.

> removed too many
> lines of code that were preparation for future that never come)

This is not in preparation for the future, it is for a feature we're working on right now (live storage migration) and for fixing move disk on which we have several bugs pending.

> The handling done at CommandBase it means that it is influence all
> system.

That is how the task management was done.  Again, if you feel it should only affect storage flows, feel free to push it down into StorageCommandHandlingBase and then only storage verbs will have task management.

> Now regards architecture why I need some handler which will be inside
> a command
> and will call for command methods? Please explain.

As opposed to what?

> 
>   
> > > This will occur all over the storage commands (which are the only
> > > usages of tasks nowadys).
> > > Moreover, async task handling is done at the Commandbase level
> > > (see
> > > the end* methods) - instead of hacking it in X different places
> > > whenever we need it, I'd prefer doing it once, properly.
> > > > 
> > > > 
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > 
> 


From masayag at redhat.com  Sun Sep 23 16:12:51 2012
From: masayag at redhat.com (Moti Asayag)
Date: Sun, 23 Sep 2012 18:12:51 +0200
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <505F2435.3060801@redhat.com>
References: <157698272.342581.1348412154540.JavaMail.root@redhat.com>
	<505F2435.3060801@redhat.com>
Message-ID: <505F3503.9010009@redhat.com>

On 09/23/2012 05:01 PM, Yaniv Kaul wrote:
> On 09/23/2012 04:55 PM, Alona Kaplan wrote:
>>
>> ----- Original Message -----
>>> From: "Avi Tal" <atal at redhat.com>
>>> To: "Alona Kaplan" <alkaplan at redhat.com>
>>> Cc: engine-devel at ovirt.org
>>> Sent: Sunday, September 23, 2012 4:17:26 PM
>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>>
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Alona Kaplan" <alkaplan at redhat.com>
>>>> To: "Avi Tal" <atal at redhat.com>
>>>> Cc: engine-devel at ovirt.org
>>>> Sent: Sunday, September 23, 2012 1:31:32 PM
>>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>>>
>>>> 1. What actions do you suggest to add?
>>> Add, Edit and remove actions of each component.
>>>
>> Host:
>> Add- What will add action on Networks->Hosts sub tab do? Choose an
>> existing host and attach the network to one of its nics?
>> How will it work? after choosing the host you will open setupNetworks
>> window and just enable dragging of the selected network (in the main
>> tab) to a nic? I think it is too much.
>>
>> Edit- same as add.
>>
>> Remove- What is the meaning of Remove host in network's context? The
>> network will be detached from the host? I think it is confusing.
>>
>> Vm/Template:
>> Add: What will it do? You will choose a vm and then add a vnic to the
>> vm? Where will you see the vnic details?
>> Edit: Same as add.
>> Remove: You will remove all the vnics that use the selected network
>> from the vm? Or do you mean to add a remove per vnic?
> 
> For all the above: yes.
> We can certainly work on the small details, but having a main tab with
> little to no action whatsoever is kinda disappointing.

IMO adding 'assign' action in the main tab might be handy in order to
assign a single network in a single dialog to several clusters.

However It is not clear to me how Add/Edit for a VM in a sub-tab should
look like. The VM should have a list of interfaces (represented as sub
collection/tree).
What will be the meaning of 'Add' in that context? Since the VM already
have a vNIC attached to that network. If adding another once, it should
be enabled on the record representing the VM itself which will confuse
the user.
Same goes for 'Edit' of a VM interface under a context of a specific
network: in the current 'Edit VM Interface' you can change the network.
Should the same dialog be used here as well?

The 'Remove' option is the clearer action on this context.


> (example of 'small details' - yes, when clicking remove for a
> VM/template - it will remove all vNICs that use that network from the
> VM. A VM with more than one vNIC on the same network is not the common
> case anyway, and if you want to remove just a single vNIC, go to the
> VM). I think we can also live with Add/Remove only, and 'defer' the edit
> actions to the respectable objects (host/vm/template) edit dialog boxes
> that originates from their own main tab.
> 
> Y.
> 
>>>> 2. I don't understand your suggestion. You mean add SetupNetworks
>>>> to
>>>> the Networks main tab?
>>> No.
>>> Just Thinking of a nice display which will present logical network
>>> and his connected host/vm configuration instead of passing through
>>> dub-tabs
>>> I should give it a deeper thought.
>>>
>>>
>>>> ----- Original Message -----
>>>>> From: "Avi Tal" <atal at redhat.com>
>>>>> To: "Alona Kaplan" <alkaplan at redhat.com>
>>>>> Cc: engine-devel at ovirt.org
>>>>> Sent: Thursday, September 20, 2012 6:58:05 PM
>>>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>>>>
>>>>> 1. Why not having Actions for Hosts, Vms and Templates ?
>>>>> I believe we should have those actions instead of making the user
>>>>> browse between 'network -> vms -> network (sub tab)' just in
>>>>> order
>>>>> to update VM's network.
>>>>>
>>>>> 2. Why having such a big window which most of the time will be
>>>>> empty?
>>>>>     How many logical network would be existed in the system?
>>>>>     My suggestion is to go for more graphical window which will
>>>>>     present:
>>>>>     A. logical networks
>>>>>     B. Multiple networks with the host configurations at the same
>>>>>     window: very important because most of the System admins would
>>>>>     like to have a bigger picture of the entire network
>>>>>     configuration
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>> From: "Alona Kaplan" <alkaplan at redhat.com>
>>>>> To: engine-devel at ovirt.org
>>>>> Sent: Thursday, September 20, 2012 1:50:15 PM
>>>>> Subject: [Engine-devel] RFE: Netwrok Main Tab
>>>>>
>>>>> Hi all,
>>>>>
>>>>> Please review the wiki and add your comments.
>>>>>
>>>>> http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
>>>>>
>>>>>
>>>>>
>>>>> RFE:
>>>>> https://bugzilla.redhat.com/858742
>>>>>
>>>>> Thanks,
>>>>> Alona.
>>>>> _______________________________________________
>>>>> Engine-devel mailing list
>>>>> Engine-devel at ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel



From ykaul at redhat.com  Sun Sep 23 16:16:47 2012
From: ykaul at redhat.com (Yaniv Kaul)
Date: Sun, 23 Sep 2012 18:16:47 +0200
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <505F3503.9010009@redhat.com>
References: <157698272.342581.1348412154540.JavaMail.root@redhat.com>
	<505F2435.3060801@redhat.com> <505F3503.9010009@redhat.com>
Message-ID: <505F35EF.1090309@redhat.com>

On 09/23/2012 06:12 PM, Moti Asayag wrote:
> On 09/23/2012 05:01 PM, Yaniv Kaul wrote:
>> On 09/23/2012 04:55 PM, Alona Kaplan wrote:
>>> ----- Original Message -----
>>>> From: "Avi Tal" <atal at redhat.com>
>>>> To: "Alona Kaplan" <alkaplan at redhat.com>
>>>> Cc: engine-devel at ovirt.org
>>>> Sent: Sunday, September 23, 2012 4:17:26 PM
>>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Alona Kaplan" <alkaplan at redhat.com>
>>>>> To: "Avi Tal" <atal at redhat.com>
>>>>> Cc: engine-devel at ovirt.org
>>>>> Sent: Sunday, September 23, 2012 1:31:32 PM
>>>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>>>>
>>>>> 1. What actions do you suggest to add?
>>>> Add, Edit and remove actions of each component.
>>>>
>>> Host:
>>> Add- What will add action on Networks->Hosts sub tab do? Choose an
>>> existing host and attach the network to one of its nics?
>>> How will it work? after choosing the host you will open setupNetworks
>>> window and just enable dragging of the selected network (in the main
>>> tab) to a nic? I think it is too much.
>>>
>>> Edit- same as add.
>>>
>>> Remove- What is the meaning of Remove host in network's context? The
>>> network will be detached from the host? I think it is confusing.
>>>
>>> Vm/Template:
>>> Add: What will it do? You will choose a vm and then add a vnic to the
>>> vm? Where will you see the vnic details?
>>> Edit: Same as add.
>>> Remove: You will remove all the vnics that use the selected network
>>> from the vm? Or do you mean to add a remove per vnic?
>> For all the above: yes.
>> We can certainly work on the small details, but having a main tab with
>> little to no action whatsoever is kinda disappointing.
> IMO adding 'assign' action in the main tab might be handy in order to
> assign a single network in a single dialog to several clusters.
>
> However It is not clear to me how Add/Edit for a VM in a sub-tab should
> look like. The VM should have a list of interfaces (represented as sub
> collection/tree).

I was thinking more of right-click on the network, selecting 'Add to VM' 
and then selecting single/mutliple VMs from a dialog with all the VMs 
(yes, filtered via search).
Y.

> What will be the meaning of 'Add' in that context? Since the VM already
> have a vNIC attached to that network. If adding another once, it should
> be enabled on the record representing the VM itself which will confuse
> the user.
> Same goes for 'Edit' of a VM interface under a context of a specific
> network: in the current 'Edit VM Interface' you can change the network.
> Should the same dialog be used here as well?
>
> The 'Remove' option is the clearer action on this context.
>
>
>> (example of 'small details' - yes, when clicking remove for a
>> VM/template - it will remove all vNICs that use that network from the
>> VM. A VM with more than one vNIC on the same network is not the common
>> case anyway, and if you want to remove just a single vNIC, go to the
>> VM). I think we can also live with Add/Remove only, and 'defer' the edit
>> actions to the respectable objects (host/vm/template) edit dialog boxes
>> that originates from their own main tab.
>>
>> Y.
>>
>>>>> 2. I don't understand your suggestion. You mean add SetupNetworks
>>>>> to
>>>>> the Networks main tab?
>>>> No.
>>>> Just Thinking of a nice display which will present logical network
>>>> and his connected host/vm configuration instead of passing through
>>>> dub-tabs
>>>> I should give it a deeper thought.
>>>>
>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Avi Tal" <atal at redhat.com>
>>>>>> To: "Alona Kaplan" <alkaplan at redhat.com>
>>>>>> Cc: engine-devel at ovirt.org
>>>>>> Sent: Thursday, September 20, 2012 6:58:05 PM
>>>>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
>>>>>>
>>>>>> 1. Why not having Actions for Hosts, Vms and Templates ?
>>>>>> I believe we should have those actions instead of making the user
>>>>>> browse between 'network -> vms -> network (sub tab)' just in
>>>>>> order
>>>>>> to update VM's network.
>>>>>>
>>>>>> 2. Why having such a big window which most of the time will be
>>>>>> empty?
>>>>>>      How many logical network would be existed in the system?
>>>>>>      My suggestion is to go for more graphical window which will
>>>>>>      present:
>>>>>>      A. logical networks
>>>>>>      B. Multiple networks with the host configurations at the same
>>>>>>      window: very important because most of the System admins would
>>>>>>      like to have a bigger picture of the entire network
>>>>>>      configuration
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>> From: "Alona Kaplan" <alkaplan at redhat.com>
>>>>>> To: engine-devel at ovirt.org
>>>>>> Sent: Thursday, September 20, 2012 1:50:15 PM
>>>>>> Subject: [Engine-devel] RFE: Netwrok Main Tab
>>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> Please review the wiki and add your comments.
>>>>>>
>>>>>> http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
>>>>>>
>>>>>>
>>>>>>
>>>>>> RFE:
>>>>>> https://bugzilla.redhat.com/858742
>>>>>>
>>>>>> Thanks,
>>>>>> Alona.
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel



From derekh at redhat.com  Sun Sep 23 21:36:48 2012
From: derekh at redhat.com (Derek Higgins)
Date: Sun, 23 Sep 2012 22:36:48 +0100
Subject: [Engine-devel] Splitting out the oVirt engine installer
Message-ID: <505F80F0.9010007@redhat.com>

Hi All,
    First time poster here so let me know if this is better directed
some where else

    I have been writing a installer for openstack and as a base for this
I've split out the generic parts of your engine-setup utility[1].
In order to do this I've created a generic installer[2] that is
basically all of the parts of the oVirt engine installer that I needed
for my openstack installer[3]

I'd now like to see if there is interest here to use this generic
installer for oVirt.

I may have stripped out some stuff from the installer that I should have
left in, if that is the case I'm open to recreating this repository with
anything generic I shouldn't have stripped out so that the original
history is maintained. Would anybody here like to look into the work
necessary to use this installer in oVirt? For openstack I've made
exclusive use of the plugin functionality and I suspect most changes
that would be needed for oVirt would be to move a lot of the code into a
similar pattern.

If you have any questions/comments just let me know, hopefully you will
find this useful and other projects could make use of the great work you
have done developing this setup utility

Thanks,
Derek.

[1]
https://github.com/derekhiggins/ovirt-engine/tree/master/packaging/fedora/setup
[2] https://github.com/derekhiggins/installer
[3] https://github.com/derekhiggins/os-installer



From mkublin at redhat.com  Mon Sep 24 08:10:08 2012
From: mkublin at redhat.com (Michael Kublin)
Date: Mon, 24 Sep 2012 04:10:08 -0400 (EDT)
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <1076165956.100805.1348414074541.JavaMail.root@redhat.com>
Message-ID: <905631359.493569.1348474208972.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Ayal Baron" <abaron at redhat.com>
> To: "Michael Kublin" <mkublin at redhat.com>
> Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel" <engine-devel at ovirt.org>, "Eduardo Warszawski"
> <ewarszaw at redhat.com>, "Allon Mureinik" <amureini at redhat.com>
> Sent: Sunday, September 23, 2012 5:27:54 PM
> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> 
> 
> 
> ----- Original Message -----
> > 
> > 
> > ----- Original Message -----
> > > From: "Ayal Baron" <abaron at redhat.com>
> > > To: "Allon Mureinik" <amureini at redhat.com>, "Michael Kublin"
> > > <mkublin at redhat.com>
> > > Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel"
> > > <engine-devel at ovirt.org>, "Eduardo Warszawski"
> > > <ewarszaw at redhat.com>
> > > Sent: Sunday, September 23, 2012 1:10:07 PM
> > > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > > 
> > > 
> > > 
> > > ----- Original Message -----
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > From: "Michael Kublin" <mkublin at redhat.com>
> > > > > To: "Allon Mureinik" <amureini at redhat.com>
> > > > > Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Liron
> > > > > Aravot"
> > > > > <laravot at redhat.com>, "Maor Lipchuk"
> > > > > <mlipchuk at redhat.com>, "engine-devel"
> > > > > <engine-devel at ovirt.org>
> > > > > Sent: Sunday, September 23, 2012 12:41:05 PM
> > > > > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > > > > 
> > > > > 
> > > > > >>>>>>>>>> Hi guys,
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> As you may know the engine currently has the
> > > > > >>>>>>>>>> ability
> > > > > >>>>>>>>>> to
> > > > > >>>>>>>>>> fire
> > > > > >>>>>>>>>> an
> > > > > >>>>>>>>>> SPM
> > > > > >>>>>>>>>> task, and be asynchronously be "woken-up" when it
> > > > > >>>>>>>>>> ends.
> > > > > >>>>>>>>>> This is great, but we found the for the Live
> > > > > >>>>>>>>>> Storage
> > > > > >>>>>>>>>> Migration
> > > > > >>>>>>>>>> feature we need something a bit complex - the
> > > > > >>>>>>>>>> ability
> > > > > >>>>>>>>>> to
> > > > > >>>>>>>>>> have a
> > > > > >>>>>>>>>> series of async tasks in a single control flow.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Here's my initial design for this, your comments
> > > > > >>>>>>>>>> and
> > > > > >>>>>>>>>> criticism
> > > > > >>>>>>>>>> would
> > > > > >>>>>>>>>> be welcome:
> > > > > >>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
> > > > > >>>>>>>> -successful execution -
> > > > > >>>>>>>> * "CommandBase iterates over its
> > > > > >>>>>>>> SPMAsyncTaskHandlers"
> > > > > >>>>>>>> -
> > > > > >>>>>>>> when?
> > > > > >>>>>>> This is the new suggested format of executeCommand().
> > > > > >>>>>>> I'll
> > > > > >>>>>>> clarify
> > > > > >>>>>>> this too.
> > > > > >>>>>>>
> > > > > >>>>>>>> * If the second task is an HSM command (vs. SPM
> > > > > >>>>>>>> command),
> > > > > >>>>>>>> I
> > > > > >>>>>>>> think you
> > > > > >>>>>>>> should explain in the design how to handle such
> > > > > >>>>>>>> flows
> > > > > >>>>>>>> as
> > > > > >>>>>>>> well.
> > > > > >>>>>>> HSM commands do not create AsyncTasks, as they do
> > > > > >>>>>>> today
> > > > > >>>>>>> -
> > > > > >>>>>>> I
> > > > > >>>>>>> will
> > > > > >>>>>>> clarify this.
> > > > > >>>>>>>
> > > > > >>>>>>>> * Why do we need before task? can you give a
> > > > > >>>>>>>> concrete
> > > > > >>>>>>>> example
> > > > > >>>>>>>> of what
> > > > > >>>>>>>> would you do in such a method.
> > > > > >>>>>>> Basically, /today/, command look like this:
> > > > > >>>>>>> executeCommand() {
> > > > > >>>>>>>     doStuffInTheDB();
> > > > > >>>>>>>     runVdsCommand(someCommand);
> > > > > >>>>>>> }
> > > > > >>>>>>>
> > > > > >>>>>>> endSuccessfully() {
> > > > > >>>>>>>     doMoreStuffInTheDB();
> > > > > >>>>>>> }
> > > > > >>>>>>>
> > > > > >>>>>>> endWithFailure() {
> > > > > >>>>>>>     doMoreStuffForFailureInTheDB();
> > > > > >>>>>>> }
> > > > > >>>>>>>
> > > > > >>>>>>> In the new design, the entire doStuffInTheDB() should
> > > > > >>>>>>> be
> > > > > >>>>>>> moved
> > > > > >>>>>>> to a
> > > > > >>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
> > > > > >>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> - I see you added SPMAsyncTaskHandler, any reason
> > > > > >>>>>>>> not
> > > > > >>>>>>>> to
> > > > > >>>>>>>> use
> > > > > >>>>>>>> SPMAsyncTasK to manage it own life-cycle?
> > > > > >>>>>>> Conserving today's design - The SPMAsyncTaskHandler
> > > > > >>>>>>> is
> > > > > >>>>>>> the
> > > > > >>>>>>> place to
> > > > > >>>>>>> add additional, non-SPM, logic around the SPM task
> > > > > >>>>>>> execution,
> > > > > >>>>>>> like
> > > > > >>>>>>> CommandBase allows today.
> > > > > >>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> - In the life-cycle managed by the
> > > > > >>>>>>>> SPMAsyncTaskHandler
> > > > > >>>>>>>> there
> > > > > >>>>>>>> is a
> > > > > >>>>>>>> step
> > > > > >>>>>>>> 'createTask - how to create the async task' can you
> > > > > >>>>>>>> please
> > > > > >>>>>>>> elaborate
> > > > > >>>>>>>> what are the options.
> > > > > >>>>>>> new [any type of async task]
> > > > > 
> > > > > (I cleaned thread a little.)
> > > > > The following design and it is implementation
> > > > > http://gerrit.ovirt.org/#/c/7956/
> > > > > is bad.
> > > > > I don't see any reason for creating a new SPMAsyncTaskHandler
> > > > > and
> > > > > especially in the
> > > > > way as it's done in patch.
> > > > > The reason are following:
> > > > > 1. Performance , increased memory footprint, created CYCLIC
> > > > > REFERENCE.
> > > > > 2. Readability and robust of code: the code which is written
> > > > > as
> > > > > cyclic references is unreadable
> > > > >    and difficult for debug.
> > > > > 3. Why I need a generic implementation and changes all over
> > > > > whole
> > > > > project because of
> > > > >    series of async commands, for me it is a private case?
> > > 
> > > What is the private case here exactly?
> > > Every task can have multiple jobs.  We've identified several such
> > > places (e.g. live storage migration, move disk, move vm) and I
> > > have
> > > no doubt more will popup.
> > > As Allon notes below, task handling is done at CommandBase, if
> > > you
> > > think task management should be for storage only, you're welcome
> > > to
> > > push it down to StorageHandlingCommandBase (or get rid of
> > > inheritance here altogether).
> > Interesting , regards cyclic reference no response, but who cares,
> > it is difficult to answer , that's why better not to response?
> 
> There is no problem with cyclic references in general, GCs know how
> to deal with these just fine and in this case it's limited to the
> command and its handlers.
> I did not reply, however, as I do not feel strongly about this.
> 
> > Regards private case:
> > 1. We have command that not creating any task
> > 2. We have command that will create a one task.
> > 3. And we have 3 commands meanwhile which will create more than one
> > task.
> > I think that 3 is private case and not common? (In the future, I
> 
> once happens
> twice is a coincidence
> three times is a method
> 
> But if you insist on more then it's easy enough.  We've discussed
> many times in the past that we need to change many of the storage
> verbs to run asynchronously (e.g. createStorageDomain) once this
> happens then existing flows would have to run multiple async tasks
> serially.
> 
> > removed too many
> > lines of code that were preparation for future that never come)
> 
> This is not in preparation for the future, it is for a feature we're
> working on right now (live storage migration) and for fixing move
> disk on which we have several bugs pending.
> 
> > The handling done at CommandBase it means that it is influence all
> > system.
> 
> That is how the task management was done.  Again, if you feel it
> should only affect storage flows, feel free to push it down into
> StorageCommandHandlingBase and then only storage verbs will have
> task management.
> 
> > Now regards architecture why I need some handler which will be
> > inside
> > a command
> > and will call for command methods? Please explain.
> 
> As opposed to what?
So, u think it is a good design where we are using a "Circular references" 
design pattern. If yes, please elaborate.

> > 
> >   
> > > > This will occur all over the storage commands (which are the
> > > > only
> > > > usages of tasks nowadys).
> > > > Moreover, async task handling is done at the Commandbase level
> > > > (see
> > > > the end* methods) - instead of hacking it in X different places
> > > > whenever we need it, I'd prefer doing it once, properly.
> > > > > 
> > > > > 
> > > > _______________________________________________
> > > > Engine-devel mailing list
> > > > Engine-devel at ovirt.org
> > > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > 
> > > 
> > 
> 


From atal at redhat.com  Mon Sep 24 08:13:52 2012
From: atal at redhat.com (Avi Tal)
Date: Mon, 24 Sep 2012 04:13:52 -0400 (EDT)
Subject: [Engine-devel] RFE: Netwrok Main Tab
In-Reply-To: <505F35EF.1090309@redhat.com>
Message-ID: <1814574.41.1348474429144.JavaMail.atal@atal-fedora.tlv.redhat.com>



----- Original Message -----
> From: "Yaniv Kaul" <ykaul at redhat.com>
> To: "Moti Asayag" <masayag at redhat.com>
> Cc: engine-devel at ovirt.org
> Sent: Sunday, September 23, 2012 6:16:47 PM
> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> 
> On 09/23/2012 06:12 PM, Moti Asayag wrote:
> > On 09/23/2012 05:01 PM, Yaniv Kaul wrote:
> >> On 09/23/2012 04:55 PM, Alona Kaplan wrote:
> >>> ----- Original Message -----
> >>>> From: "Avi Tal" <atal at redhat.com>
> >>>> To: "Alona Kaplan" <alkaplan at redhat.com>
> >>>> Cc: engine-devel at ovirt.org
> >>>> Sent: Sunday, September 23, 2012 4:17:26 PM
> >>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> >>>>
> >>>>
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Alona Kaplan" <alkaplan at redhat.com>
> >>>>> To: "Avi Tal" <atal at redhat.com>
> >>>>> Cc: engine-devel at ovirt.org
> >>>>> Sent: Sunday, September 23, 2012 1:31:32 PM
> >>>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> >>>>>
> >>>>> 1. What actions do you suggest to add?
> >>>> Add, Edit and remove actions of each component.
> >>>>
> >>> Host:
> >>> Add- What will add action on Networks->Hosts sub tab do? Choose
> >>> an
> >>> existing host and attach the network to one of its nics?
> >>> How will it work? after choosing the host you will open
> >>> setupNetworks
> >>> window and just enable dragging of the selected network (in the
> >>> main
> >>> tab) to a nic? I think it is too much.
> >>>
> >>> Edit- same as add.
> >>>
> >>> Remove- What is the meaning of Remove host in network's context?
> >>> The
> >>> network will be detached from the host? I think it is confusing.
> >>>
> >>> Vm/Template:
> >>> Add: What will it do? You will choose a vm and then add a vnic to
> >>> the
> >>> vm? Where will you see the vnic details?
> >>> Edit: Same as add.
> >>> Remove: You will remove all the vnics that use the selected
> >>> network
> >>> from the vm? Or do you mean to add a remove per vnic?
> >> For all the above: yes.
> >> We can certainly work on the small details, but having a main tab
> >> with
> >> little to no action whatsoever is kinda disappointing.
> > IMO adding 'assign' action in the main tab might be handy in order
> > to
> > assign a single network in a single dialog to several clusters.
> >
> > However It is not clear to me how Add/Edit for a VM in a sub-tab
> > should
> > look like. The VM should have a list of interfaces (represented as
> > sub
> > collection/tree).
> 
> I was thinking more of right-click on the network, selecting 'Add to
> VM'
> and then selecting single/mutliple VMs from a dialog with all the VMs
> (yes, filtered via search).
> Y.
> 

+1

> > What will be the meaning of 'Add' in that context? Since the VM
> > already
> > have a vNIC attached to that network. If adding another once, it
> > should
> > be enabled on the record representing the VM itself which will
> > confuse
> > the user.
> > Same goes for 'Edit' of a VM interface under a context of a
> > specific
> > network: in the current 'Edit VM Interface' you can change the
> > network.
> > Should the same dialog be used here as well?
> >
> > The 'Remove' option is the clearer action on this context.
> >
> >
> >> (example of 'small details' - yes, when clicking remove for a
> >> VM/template - it will remove all vNICs that use that network from
> >> the
> >> VM. A VM with more than one vNIC on the same network is not the
> >> common
> >> case anyway, and if you want to remove just a single vNIC, go to
> >> the
> >> VM). I think we can also live with Add/Remove only, and 'defer'
> >> the edit
> >> actions to the respectable objects (host/vm/template) edit dialog
> >> boxes
> >> that originates from their own main tab.
> >>
> >> Y.
> >>
> >>>>> 2. I don't understand your suggestion. You mean add
> >>>>> SetupNetworks
> >>>>> to
> >>>>> the Networks main tab?
> >>>> No.
> >>>> Just Thinking of a nice display which will present logical
> >>>> network
> >>>> and his connected host/vm configuration instead of passing
> >>>> through
> >>>> dub-tabs
> >>>> I should give it a deeper thought.
> >>>>
> >>>>
> >>>>> ----- Original Message -----
> >>>>>> From: "Avi Tal" <atal at redhat.com>
> >>>>>> To: "Alona Kaplan" <alkaplan at redhat.com>
> >>>>>> Cc: engine-devel at ovirt.org
> >>>>>> Sent: Thursday, September 20, 2012 6:58:05 PM
> >>>>>> Subject: Re: [Engine-devel] RFE: Netwrok Main Tab
> >>>>>>
> >>>>>> 1. Why not having Actions for Hosts, Vms and Templates ?
> >>>>>> I believe we should have those actions instead of making the
> >>>>>> user
> >>>>>> browse between 'network -> vms -> network (sub tab)' just in
> >>>>>> order
> >>>>>> to update VM's network.
> >>>>>>
> >>>>>> 2. Why having such a big window which most of the time will be
> >>>>>> empty?
> >>>>>>      How many logical network would be existed in the system?
> >>>>>>      My suggestion is to go for more graphical window which
> >>>>>>      will
> >>>>>>      present:
> >>>>>>      A. logical networks
> >>>>>>      B. Multiple networks with the host configurations at the
> >>>>>>      same
> >>>>>>      window: very important because most of the System admins
> >>>>>>      would
> >>>>>>      like to have a bigger picture of the entire network
> >>>>>>      configuration
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> ----- Original Message -----
> >>>>>> From: "Alona Kaplan" <alkaplan at redhat.com>
> >>>>>> To: engine-devel at ovirt.org
> >>>>>> Sent: Thursday, September 20, 2012 1:50:15 PM
> >>>>>> Subject: [Engine-devel] RFE: Netwrok Main Tab
> >>>>>>
> >>>>>> Hi all,
> >>>>>>
> >>>>>> Please review the wiki and add your comments.
> >>>>>>
> >>>>>> http://wiki.ovirt.org/wiki/Feature/NetworkMainTab
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> RFE:
> >>>>>> https://bugzilla.redhat.com/858742
> >>>>>>
> >>>>>> Thanks,
> >>>>>> Alona.
> >>>>>> _______________________________________________
> >>>>>> Engine-devel mailing list
> >>>>>> Engine-devel at ovirt.org
> >>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>>>
> >>> _______________________________________________
> >>> Engine-devel mailing list
> >>> Engine-devel at ovirt.org
> >>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >> _______________________________________________
> >> Engine-devel mailing list
> >> Engine-devel at ovirt.org
> >> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From rgolan at redhat.com  Mon Sep 24 14:51:37 2012
From: rgolan at redhat.com (Roy Golan)
Date: Mon, 24 Sep 2012 16:51:37 +0200
Subject: [Engine-devel] libosinfo integration
Message-ID: <50607379.7020102@redhat.com>

Here's a draft wiki of the libosinfo integration with the engine 
http://wiki.ovirt.org/wiki/Libosinfo

please share your comments on this thread and I'll update the wiki 
accordingly.

Thanks,
Roy


From djasa at redhat.com  Mon Sep 24 15:15:12 2012
From: djasa at redhat.com (David =?UTF-8?Q?Ja=C5=A1a?=)
Date: Mon, 24 Sep 2012 17:15:12 +0200
Subject: [Engine-devel] libosinfo integration
In-Reply-To: <50607379.7020102@redhat.com>
References: <50607379.7020102@redhat.com>
Message-ID: <1348499712.28489.9.camel@dhcp-29-7.brq.redhat.com>

Roy Golan p??e v Po 24. 09. 2012 v 16:51 +0200:
> Here's a draft wiki of the libosinfo integration with the engine 
> http://wiki.ovirt.org/wiki/Libosinfo
> 
> please share your comments on this thread and I'll update the wiki 
> accordingly.
> 
> Thanks,
> Roy
> 

If you use java bindings created by GObject Introspection, will the
program leave JVM when invoking them or not? If not, that would simplify
your choice by great deal.

David

-- 

David Ja?a, RHCE

SPICE QE based in Brno
GPG Key:     22C33E24 
Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24







From berrange at redhat.com  Mon Sep 24 15:24:03 2012
From: berrange at redhat.com (Daniel P. Berrange)
Date: Mon, 24 Sep 2012 16:24:03 +0100
Subject: [Engine-devel] libosinfo integration
In-Reply-To: <50607379.7020102@redhat.com>
References: <50607379.7020102@redhat.com>
Message-ID: <20120924152403.GI18952@redhat.com>

On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
> Here's a draft wiki of the libosinfo integration with the engine
> http://wiki.ovirt.org/wiki/Libosinfo
> 
> please share your comments on this thread and I'll update the wiki
> accordingly.

As a libosinfo maintainer I strongly discourage you from writing a custom
API directly against the XML. While the format is stable from the POV
that vendors/users can create new XML docs for new/customized OS variants
they have, we do not consider it something for applications to be using
directly. The libosinfo library API includes alot of logic beyond simply
being an convenient way to process the XML. In particular dealing with
the inheritance of data across OS, searching for data across a variety
of data sources, relying on GIO for URI resolving, live update of the
database, and more.

We're also not really interested in maintaining manually written bindings
for any languages. Rather than putting effort into such bindings for Java,
time is better spent on finishing the GOBject introspection mapping layer
for Java. There was a proof of concept, which was never finished / updated
to the final introspection spec, which can be used as a basis for ongoing
development.

 https://live.gnome.org/JGIR

This work will be applicable way beyond just libosinfo, to a wide
variety of useful APIs. Of possible interest to VDSM in the future will
be things like the new libvirt-designer & libvirt-builder APIs which
are going to provide a bridge between libvirt + libosinfo to simply
life for application developers constructing XML for provisioning
guests.


Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|


From jhernand at redhat.com  Mon Sep 24 16:30:43 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Mon, 24 Sep 2012 18:30:43 +0200
Subject: [Engine-devel] libosinfo integration
In-Reply-To: <20120924152403.GI18952@redhat.com>
References: <50607379.7020102@redhat.com> <20120924152403.GI18952@redhat.com>
Message-ID: <50608AB3.1010808@redhat.com>

On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
> On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
>> Here's a draft wiki of the libosinfo integration with the engine
>> http://wiki.ovirt.org/wiki/Libosinfo
>>
>> please share your comments on this thread and I'll update the wiki
>> accordingly.
> 
> As a libosinfo maintainer I strongly discourage you from writing a custom
> API directly against the XML. While the format is stable from the POV
> that vendors/users can create new XML docs for new/customized OS variants
> they have, we do not consider it something for applications to be using
> directly. The libosinfo library API includes alot of logic beyond simply
> being an convenient way to process the XML. In particular dealing with
> the inheritance of data across OS, searching for data across a variety
> of data sources, relying on GIO for URI resolving, live update of the
> database, and more.
> 
> We're also not really interested in maintaining manually written bindings
> for any languages. Rather than putting effort into such bindings for Java,
> time is better spent on finishing the GOBject introspection mapping layer
> for Java. There was a proof of concept, which was never finished / updated
> to the final introspection spec, which can be used as a basis for ongoing
> development.
> 
>  https://live.gnome.org/JGIR
> 
> This work will be applicable way beyond just libosinfo, to a wide
> variety of useful APIs. Of possible interest to VDSM in the future will
> be things like the new libvirt-designer & libvirt-builder APIs which
> are going to provide a bridge between libvirt + libosinfo to simply
> life for application developers constructing XML for provisioning
> guests.

I think that the use of JGIR has already been discussed as a way to
interface the engine with libvdsm.so:

http://lists.ovirt.org/pipermail/arch/2012-August/000760.html

If I remember correctly the outcome of that discussion was that it
shouldn't be used because it is considered dangerous to use native
libraries from Java and because JGIR itself is very outdated. I don't
100% agree with the first part, but the second part is completely true.

Anyhow, as it seems to be the right way to interface with libosinfo and
future libvirt capabilities, we may want to rethink and put some effort
to bring it up to date.

-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From alonbl at redhat.com  Mon Sep 24 18:59:33 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 24 Sep 2012 14:59:33 -0400 (EDT)
Subject: [Engine-devel] libosinfo integration
In-Reply-To: <50608AB3.1010808@redhat.com>
Message-ID: <418834811.224466.1348513173334.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: "Daniel P. Berrange" <berrange at redhat.com>
> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org
> Sent: Monday, September 24, 2012 6:30:43 PM
> Subject: Re: [Engine-devel] libosinfo integration
> 
> On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
> > On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
> >> Here's a draft wiki of the libosinfo integration with the engine
> >> http://wiki.ovirt.org/wiki/Libosinfo
> >>
> >> please share your comments on this thread and I'll update the wiki
> >> accordingly.
> > 
> > As a libosinfo maintainer I strongly discourage you from writing a
> > custom
> > API directly against the XML. While the format is stable from the
> > POV
> > that vendors/users can create new XML docs for new/customized OS
> > variants
> > they have, we do not consider it something for applications to be
> > using
> > directly. The libosinfo library API includes alot of logic beyond
> > simply
> > being an convenient way to process the XML. In particular dealing
> > with
> > the inheritance of data across OS, searching for data across a
> > variety
> > of data sources, relying on GIO for URI resolving, live update of
> > the
> > database, and more.
> > 
> > We're also not really interested in maintaining manually written
> > bindings
> > for any languages. Rather than putting effort into such bindings
> > for Java,
> > time is better spent on finishing the GOBject introspection mapping
> > layer
> > for Java. There was a proof of concept, which was never finished /
> > updated
> > to the final introspection spec, which can be used as a basis for
> > ongoing
> > development.
> > 
> >  https://live.gnome.org/JGIR
> > 
> > This work will be applicable way beyond just libosinfo, to a wide
> > variety of useful APIs. Of possible interest to VDSM in the future
> > will
> > be things like the new libvirt-designer & libvirt-builder APIs
> > which
> > are going to provide a bridge between libvirt + libosinfo to simply
> > life for application developers constructing XML for provisioning
> > guests.
> 
> I think that the use of JGIR has already been discussed as a way to
> interface the engine with libvdsm.so:
> 
> http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
> 
> If I remember correctly the outcome of that discussion was that it
> shouldn't be used because it is considered dangerous to use native
> libraries from Java and because JGIR itself is very outdated. I don't
> 100% agree with the first part, but the second part is completely
> true.
> 
> Anyhow, as it seems to be the right way to interface with libosinfo
> and
> future libvirt capabilities, we may want to rethink and put some
> effort
> to bring it up to date.

I don't think Java should go native to obtain data, using JNI or whatever native interface that is offered. It can totally break the JVM and cause leaks and stability issues JRE would like to avoid.

Having language binding for data access logic seems a bit strange for me, as it should be easy to stabilize a schema either using XML or a set of CSVs, for all to read and use.

I do understand why JNI is to be used when OS specific features are to be used, for example unix domain sockets, shared memory, security, but not for data access.

The alternatives would be:

1. Run a python program at ovirt-engine start-up, use the python API to fetch the data we need, and generate our own tiny XML of the subset or even load it to the database. We can use the output safely in Java. Disadvantage is that we need to stop/start service or have a "refresh" verb for reload after libosinfo update.

2. When data is requested, execute a python process with cmdline arguments and get result from stdout of the process. The disadvantage is the overhead of clone().

3. Read the XML directly on runtime, disadvantage is if XML is changed.

4. Use the libosinfo python API during build and distribute an update to the information in versions of ovirt, disadvantage is slow refresh of data, not sure how often this data is updated.

5. Provide a web service that has specific XML-RPC / RESTful API, that can provide data out of the libosinfo, and publish it at internet. Configure ovirt to use it, disadvantage is the need for internet connection.

6. Configure cgi-bin locally at apache where ovirt is running, which provide XML-RPC / RESTful API, and access it locally.

In any case I would avoid exiting the JVM to interact with data provider.

Regards,
Alon.


From jhernand at redhat.com  Mon Sep 24 19:33:38 2012
From: jhernand at redhat.com (Juan Hernandez)
Date: Mon, 24 Sep 2012 21:33:38 +0200
Subject: [Engine-devel] libosinfo integration
In-Reply-To: <418834811.224466.1348513173334.JavaMail.root@redhat.com>
References: <418834811.224466.1348513173334.JavaMail.root@redhat.com>
Message-ID: <5060B592.2080104@redhat.com>

On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
> 
> 
> ----- Original Message -----
>> From: "Juan Hernandez" <jhernand at redhat.com>
>> To: "Daniel P. Berrange" <berrange at redhat.com>
>> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org
>> Sent: Monday, September 24, 2012 6:30:43 PM
>> Subject: Re: [Engine-devel] libosinfo integration
>>
>> On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
>>> On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
>>>> Here's a draft wiki of the libosinfo integration with the engine
>>>> http://wiki.ovirt.org/wiki/Libosinfo
>>>>
>>>> please share your comments on this thread and I'll update the wiki
>>>> accordingly.
>>>
>>> As a libosinfo maintainer I strongly discourage you from writing a
>>> custom
>>> API directly against the XML. While the format is stable from the
>>> POV
>>> that vendors/users can create new XML docs for new/customized OS
>>> variants
>>> they have, we do not consider it something for applications to be
>>> using
>>> directly. The libosinfo library API includes alot of logic beyond
>>> simply
>>> being an convenient way to process the XML. In particular dealing
>>> with
>>> the inheritance of data across OS, searching for data across a
>>> variety
>>> of data sources, relying on GIO for URI resolving, live update of
>>> the
>>> database, and more.
>>>
>>> We're also not really interested in maintaining manually written
>>> bindings
>>> for any languages. Rather than putting effort into such bindings
>>> for Java,
>>> time is better spent on finishing the GOBject introspection mapping
>>> layer
>>> for Java. There was a proof of concept, which was never finished /
>>> updated
>>> to the final introspection spec, which can be used as a basis for
>>> ongoing
>>> development.
>>>
>>>  https://live.gnome.org/JGIR
>>>
>>> This work will be applicable way beyond just libosinfo, to a wide
>>> variety of useful APIs. Of possible interest to VDSM in the future
>>> will
>>> be things like the new libvirt-designer & libvirt-builder APIs
>>> which
>>> are going to provide a bridge between libvirt + libosinfo to simply
>>> life for application developers constructing XML for provisioning
>>> guests.
>>
>> I think that the use of JGIR has already been discussed as a way to
>> interface the engine with libvdsm.so:
>>
>> http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
>>
>> If I remember correctly the outcome of that discussion was that it
>> shouldn't be used because it is considered dangerous to use native
>> libraries from Java and because JGIR itself is very outdated. I don't
>> 100% agree with the first part, but the second part is completely
>> true.
>>
>> Anyhow, as it seems to be the right way to interface with libosinfo
>> and
>> future libvirt capabilities, we may want to rethink and put some
>> effort
>> to bring it up to date.
> 
> I don't think Java should go native to obtain data, using JNI or whatever native interface that is offered. It can totally break the JVM and cause leaks and stability issues JRE would like to avoid.

This is were I disagree, only a bit. It is true that using JNI *without
care* can totally break the JVM, generate leaks, and bring down all the
application. But if the JNI layer is developed carefully this should not
be more risky than any of the other multiple native libraries that you
use every day from Java without noticing.

In addition JGIR is based on JNA (which is well maintained and stable)
and doesn't include any "dangerous" native C code.

> Having language binding for data access logic seems a bit strange for me, as it should be easy to stabilize a schema either using XML or a set of CSVs, for all to read and use.
> 
> I do understand why JNI is to be used when OS specific features are to be used, for example unix domain sockets, shared memory, security, but not for data access.
> 
> The alternatives would be:
> 
> 1. Run a python program at ovirt-engine start-up, use the python API to fetch the data we need, and generate our own tiny XML of the subset or even load it to the database. We can use the output safely in Java. Disadvantage is that we need to stop/start service or have a "refresh" verb for reload after libosinfo update.
> 
> 2. When data is requested, execute a python process with cmdline arguments and get result from stdout of the process. The disadvantage is the overhead of clone().
> 
> 3. Read the XML directly on runtime, disadvantage is if XML is changed.
> 
> 4. Use the libosinfo python API during build and distribute an update to the information in versions of ovirt, disadvantage is slow refresh of data, not sure how often this data is updated.
> 
> 5. Provide a web service that has specific XML-RPC / RESTful API, that can provide data out of the libosinfo, and publish it at internet. Configure ovirt to use it, disadvantage is the need for internet connection.
> 
> 6. Configure cgi-bin locally at apache where ovirt is running, which provide XML-RPC / RESTful API, and access it locally.
> 
> In any case I would avoid exiting the JVM to interact with data provider.
> 
> Regards,
> Alon.
> 


-- 
Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
3?D, 28016 Madrid, Spain
Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat S.L.


From alonbl at redhat.com  Mon Sep 24 20:01:08 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Mon, 24 Sep 2012 16:01:08 -0400 (EDT)
Subject: [Engine-devel] libosinfo integration
In-Reply-To: <5060B592.2080104@redhat.com>
Message-ID: <1716804551.237454.1348516868577.JavaMail.root@redhat.com>



----- Original Message -----
> From: "Juan Hernandez" <jhernand at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org, "Daniel P. Berrange" <berrange at redhat.com>
> Sent: Monday, September 24, 2012 9:33:38 PM
> Subject: Re: [Engine-devel] libosinfo integration
> 
> On 09/24/2012 08:59 PM, Alon Bar-Lev wrote:
> > 
> > 
> > ----- Original Message -----
> >> From: "Juan Hernandez" <jhernand at redhat.com>
> >> To: "Daniel P. Berrange" <berrange at redhat.com>
> >> Cc: "arch" <arch at ovirt.org>, engine-devel at ovirt.org
> >> Sent: Monday, September 24, 2012 6:30:43 PM
> >> Subject: Re: [Engine-devel] libosinfo integration
> >>
> >> On 09/24/2012 05:24 PM, Daniel P. Berrange wrote:
> >>> On Mon, Sep 24, 2012 at 04:51:37PM +0200, Roy Golan wrote:
> >>>> Here's a draft wiki of the libosinfo integration with the engine
> >>>> http://wiki.ovirt.org/wiki/Libosinfo
> >>>>
> >>>> please share your comments on this thread and I'll update the
> >>>> wiki
> >>>> accordingly.
> >>>
> >>> As a libosinfo maintainer I strongly discourage you from writing
> >>> a
> >>> custom
> >>> API directly against the XML. While the format is stable from the
> >>> POV
> >>> that vendors/users can create new XML docs for new/customized OS
> >>> variants
> >>> they have, we do not consider it something for applications to be
> >>> using
> >>> directly. The libosinfo library API includes alot of logic beyond
> >>> simply
> >>> being an convenient way to process the XML. In particular dealing
> >>> with
> >>> the inheritance of data across OS, searching for data across a
> >>> variety
> >>> of data sources, relying on GIO for URI resolving, live update of
> >>> the
> >>> database, and more.
> >>>
> >>> We're also not really interested in maintaining manually written
> >>> bindings
> >>> for any languages. Rather than putting effort into such bindings
> >>> for Java,
> >>> time is better spent on finishing the GOBject introspection
> >>> mapping
> >>> layer
> >>> for Java. There was a proof of concept, which was never finished
> >>> /
> >>> updated
> >>> to the final introspection spec, which can be used as a basis for
> >>> ongoing
> >>> development.
> >>>
> >>>  https://live.gnome.org/JGIR
> >>>
> >>> This work will be applicable way beyond just libosinfo, to a wide
> >>> variety of useful APIs. Of possible interest to VDSM in the
> >>> future
> >>> will
> >>> be things like the new libvirt-designer & libvirt-builder APIs
> >>> which
> >>> are going to provide a bridge between libvirt + libosinfo to
> >>> simply
> >>> life for application developers constructing XML for provisioning
> >>> guests.
> >>
> >> I think that the use of JGIR has already been discussed as a way
> >> to
> >> interface the engine with libvdsm.so:
> >>
> >> http://lists.ovirt.org/pipermail/arch/2012-August/000760.html
> >>
> >> If I remember correctly the outcome of that discussion was that it
> >> shouldn't be used because it is considered dangerous to use native
> >> libraries from Java and because JGIR itself is very outdated. I
> >> don't
> >> 100% agree with the first part, but the second part is completely
> >> true.
> >>
> >> Anyhow, as it seems to be the right way to interface with
> >> libosinfo
> >> and
> >> future libvirt capabilities, we may want to rethink and put some
> >> effort
> >> to bring it up to date.
> > 
> > I don't think Java should go native to obtain data, using JNI or
> > whatever native interface that is offered. It can totally break
> > the JVM and cause leaks and stability issues JRE would like to
> > avoid.
> 
> This is were I disagree, only a bit. It is true that using JNI
> *without
> care* can totally break the JVM, generate leaks, and bring down all
> the
> application. But if the JNI layer is developed carefully this should
> not
> be more risky than any of the other multiple native libraries that
> you
> use every day from Java without noticing.
> In addition JGIR is based on JNA (which is well maintained and
> stable)
> and doesn't include any "dangerous" native C code.
>

Well, you can call me old fashioned...

But I don't think that any none critical task should be run out of process of the production component.

It is not GObject layer I am afraid of, it is what it eventually runs.

I believe that the data provided by libosinfo is not vital for our runtime, no problem in fetching it before start.

And again, I don't think data provider should be language binding... this concept alone is making me scared, as instead of using well defined data container people will start to use mesh of components... I don't see how you can stabilize an application this way.

For the native library that you use "every day", I don't know what you referring to, I've done lsof of engine and saw no exception... which other native library we use these days?
 
> > Having language binding for data access logic seems a bit strange
> > for me, as it should be easy to stabilize a schema either using
> > XML or a set of CSVs, for all to read and use.
> > 
> > I do understand why JNI is to be used when OS specific features are
> > to be used, for example unix domain sockets, shared memory,
> > security, but not for data access.
> > 
> > The alternatives would be:
> > 
> > 1. Run a python program at ovirt-engine start-up, use the python
> > API to fetch the data we need, and generate our own tiny XML of
> > the subset or even load it to the database. We can use the output
> > safely in Java. Disadvantage is that we need to stop/start service
> > or have a "refresh" verb for reload after libosinfo update.
> > 
> > 2. When data is requested, execute a python process with cmdline
> > arguments and get result from stdout of the process. The
> > disadvantage is the overhead of clone().
> > 
> > 3. Read the XML directly on runtime, disadvantage is if XML is
> > changed.
> > 
> > 4. Use the libosinfo python API during build and distribute an
> > update to the information in versions of ovirt, disadvantage is
> > slow refresh of data, not sure how often this data is updated.
> > 
> > 5. Provide a web service that has specific XML-RPC / RESTful API,
> > that can provide data out of the libosinfo, and publish it at
> > internet. Configure ovirt to use it, disadvantage is the need for
> > internet connection.
> > 
> > 6. Configure cgi-bin locally at apache where ovirt is running,
> > which provide XML-RPC / RESTful API, and access it locally.
> > 
> > In any case I would avoid exiting the JVM to interact with data
> > provider.
> > 
> > Regards,
> > Alon.
> > 
> 
> 
> --
> Direcci?n Comercial: C/Jose Bardasano Baos, 9, Edif. Gorbea 3, planta
> 3?D, 28016 Madrid, Spain
> Inscrita en el Reg. Mercantil de Madrid ? C.I.F. B82657941 - Red Hat
> S.L.
> 


From deadhorseconsulting at gmail.com  Mon Sep 24 21:31:22 2012
From: deadhorseconsulting at gmail.com (Dead Horse)
Date: Mon, 24 Sep 2012 16:31:22 -0500
Subject: [Engine-devel] engine build AD/LDAP error
Message-ID: <CAEWPe=oEjbtANvGsEnYMqG1kShPpdmjJWdT010KfjsH7+E8Bqw@mail.gmail.com>

I have built ovirt-engine from the latest GIT master over the past few
nights and am running into a MS AD/LDAP error.  This same MS AD setup works
fine with Released 3.1.

Current nightly builds from jenkins are not building all the GWT
permutations (Internet Explorer for example) so I have been attempting to
re-build the engine so I can test using Internet Explorer.
It would be nice if the nightly jenkins builds could be changed to do the
build with all GWT permutations enabled.

I am building in an FC17 environment with updates applied and current as of
today. Accordingly maven3 and all build dependencies are installed from the
FC17 repositories.
I tried the latest nightly from Jenkins and it does not exhibit the below
error. I am guessing I must not have the build recipe/env correct.

The AD error I am running into looks like:

2012-09-24 16:15:44,037 ERROR
[org.ovirt.engine.core.bll.adbroker.DirectorySearcher]
(ajp--127.0.0.1-8702-4) Failed ldap search server LDAP://
someserver.foo.com:389 due to Kerberos error. Please check log for further
details.. We should not try the next server
2012-09-24 16:15:44,039 ERROR
[org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy]
(ajp--127.0.0.1-8702-2) Error from Kerberos: java.lang.NullPointerException
    at
org.ovirt.engine.core.bll.adbroker.GSSAPICallbackHandler.handle(GSSAPICallbackHandler.java:47)
    at
javax.security.auth.login.LoginContext$SecureCallbackHandler$1.run(LoginContext.java:969)
    at
javax.security.auth.login.LoginContext$SecureCallbackHandler$1.run(LoginContext.java:966)
    at java.security.AccessController.doPrivileged(Native Method)
    at
javax.security.auth.login.LoginContext$SecureCallbackHandler.handle(LoginContext.java:965)
    at
com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:870)
    at
com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:715)
    at
com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:580)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
    at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698)
    at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696)
    at java.security.AccessController.doPrivileged(Native Method)
    at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695)
    at javax.security.auth.login.LoginContext.login(LoginContext.java:594)
    at
org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticateToKDC(GSSAPIDirContextAuthenticationStrategy.java:127)
    at
org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.explicitAuth(GSSAPIDirContextAuthenticationStrategy.java:119)
    at
org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticate(GSSAPIDirContextAuthenticationStrategy.java:111)
    at
org.ovirt.engine.core.bll.adbroker.GSSAPILdapTemplateWrapper.useAuthenticationStrategy(GSSAPILdapTemplateWrapper.java:86)
    at
org.ovirt.engine.core.bll.adbroker.PrepareLdapConnectionTask.call(PrepareLdapConnectionTask.java:56)
    at
org.ovirt.engine.core.bll.adbroker.DirectorySearcher.find(DirectorySearcher.java:103)
    at
org.ovirt.engine.core.bll.adbroker.DirectorySearcher.FindAll(DirectorySearcher.java:48)
    at
org.ovirt.engine.core.bll.adbroker.LdapSearchGroupsByQueryCommand.executeQuery(LdapSearchGroupsByQueryCommand.java:22)
    at
org.ovirt.engine.core.bll.adbroker.LdapBrokerCommandBase.execute(LdapBrokerCommandBase.java:69)
    at
org.ovirt.engine.core.bll.adbroker.LdapBrokerBase.RunAdAction(LdapBrokerBase.java:18)
    at org.ovirt.engine.core.bll.SearchQuery.adSearch(SearchQuery.java:194)
    at
org.ovirt.engine.core.bll.SearchQuery.searchAdGroups(SearchQuery.java:172)
    at
org.ovirt.engine.core.bll.SearchQuery.executeQueryCommand(SearchQuery.java:79)
    at
org.ovirt.engine.core.bll.QueriesCommandBase.ExecuteCommand(QueriesCommandBase.java:71)
    at
org.ovirt.engine.core.dal.VdcCommandBase.Execute(VdcCommandBase.java:41)
    at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:384)
    at org.ovirt.engine.core.bll.Backend.RunQuery(Backend.java:367)
    at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at
org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374)
    at
org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11)
    at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at
org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
    at
org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
    at
org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211)
    at
org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363)
    at
org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
    at
org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165)
    at
org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173)
    at
org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
    at
org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
    at
org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72)
    at
org.ovirt.engine.core.common.interfaces.BackendLocal$$$view9.RunQuery(Unknown
Source)
    at
org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.RunQuery(GenericApiGWTServiceImpl.java:51)
    at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196)
    at com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:161)
    at com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:222)
    at
com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
    at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
    at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
    at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
    at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
    at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489)
    at
org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
    at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
    at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
    at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
    at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
    at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
    at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505)
    at
org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445)
    at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
    at java.lang.Thread.run(Thread.java:722)

- DHC
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120924/da095b19/attachment.html>

From iheim at redhat.com  Mon Sep 24 21:58:16 2012
From: iheim at redhat.com (Itamar Heim)
Date: Mon, 24 Sep 2012 23:58:16 +0200
Subject: [Engine-devel] engine build AD/LDAP error
In-Reply-To: <CAEWPe=oEjbtANvGsEnYMqG1kShPpdmjJWdT010KfjsH7+E8Bqw@mail.gmail.com>
References: <CAEWPe=oEjbtANvGsEnYMqG1kShPpdmjJWdT010KfjsH7+E8Bqw@mail.gmail.com>
Message-ID: <5060D778.2000408@redhat.com>

On 09/24/2012 11:31 PM, Dead Horse wrote:
> I have built ovirt-engine from the latest GIT master over the past few
> nights and am running into a MS AD/LDAP error.  This same MS AD setup
> works fine with Released 3.1.
>
> Current nightly builds from jenkins are not building all the GWT
> permutations (Internet Explorer for example) so I have been attempting
> to re-build the engine so I can test using Internet Explorer.
> It would be nice if the nightly jenkins builds could be changed to do
> the build with all GWT permutations enabled.
>
> I am building in an FC17 environment with updates applied and current as
> of today. Accordingly maven3 and all build dependencies are installed
> from the FC17 repositories.
> I tried the latest nightly from Jenkins and it does not exhibit the
> below error. I am guessing I must not have the build recipe/env correct.

Bug 858769 - manage-domains: once the domain is added admin at internal 
can't search in that domain

I assume you are not using -addPermissions during the manage domains 
command - it should workaround it.
I know yair is working on analyzing this.

>
> The AD error I am running into looks like:
>
> 2012-09-24 16:15:44,037 ERROR
> [org.ovirt.engine.core.bll.adbroker.DirectorySearcher]
> (ajp--127.0.0.1-8702-4) Failed ldap search server
> LDAP://someserver.foo.com:389 <http://someserver.foo.com:389> due to
> Kerberos error. Please check log for further details.. We should not try
> the next server
> 2012-09-24 16:15:44,039 ERROR
> [org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy]
> (ajp--127.0.0.1-8702-2) Error from Kerberos: java.lang.NullPointerException
>      at
> org.ovirt.engine.core.bll.adbroker.GSSAPICallbackHandler.handle(GSSAPICallbackHandler.java:47)
>      at
> javax.security.auth.login.LoginContext$SecureCallbackHandler$1.run(LoginContext.java:969)
>      at
> javax.security.auth.login.LoginContext$SecureCallbackHandler$1.run(LoginContext.java:966)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at
> javax.security.auth.login.LoginContext$SecureCallbackHandler.handle(LoginContext.java:965)
>      at
> com.sun.security.auth.module.Krb5LoginModule.promptForPass(Krb5LoginModule.java:870)
>      at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:715)
>      at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:580)
>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>      at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>      at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>      at java.lang.reflect.Method.invoke(Method.java:601)
>      at javax.security.auth.login.LoginContext.invoke(LoginContext.java:784)
>      at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
>      at javax.security.auth.login.LoginContext$4.run(LoginContext.java:698)
>      at javax.security.auth.login.LoginContext$4.run(LoginContext.java:696)
>      at java.security.AccessController.doPrivileged(Native Method)
>      at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:695)
>      at javax.security.auth.login.LoginContext.login(LoginContext.java:594)
>      at
> org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticateToKDC(GSSAPIDirContextAuthenticationStrategy.java:127)
>      at
> org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.explicitAuth(GSSAPIDirContextAuthenticationStrategy.java:119)
>      at
> org.ovirt.engine.core.bll.adbroker.GSSAPIDirContextAuthenticationStrategy.authenticate(GSSAPIDirContextAuthenticationStrategy.java:111)
>      at
> org.ovirt.engine.core.bll.adbroker.GSSAPILdapTemplateWrapper.useAuthenticationStrategy(GSSAPILdapTemplateWrapper.java:86)
>      at
> org.ovirt.engine.core.bll.adbroker.PrepareLdapConnectionTask.call(PrepareLdapConnectionTask.java:56)
>      at
> org.ovirt.engine.core.bll.adbroker.DirectorySearcher.find(DirectorySearcher.java:103)
>      at
> org.ovirt.engine.core.bll.adbroker.DirectorySearcher.FindAll(DirectorySearcher.java:48)
>      at
> org.ovirt.engine.core.bll.adbroker.LdapSearchGroupsByQueryCommand.executeQuery(LdapSearchGroupsByQueryCommand.java:22)
>      at
> org.ovirt.engine.core.bll.adbroker.LdapBrokerCommandBase.execute(LdapBrokerCommandBase.java:69)
>      at
> org.ovirt.engine.core.bll.adbroker.LdapBrokerBase.RunAdAction(LdapBrokerBase.java:18)
>      at org.ovirt.engine.core.bll.SearchQuery.adSearch(SearchQuery.java:194)
>      at
> org.ovirt.engine.core.bll.SearchQuery.searchAdGroups(SearchQuery.java:172)
>      at
> org.ovirt.engine.core.bll.SearchQuery.executeQueryCommand(SearchQuery.java:79)
>      at
> org.ovirt.engine.core.bll.QueriesCommandBase.ExecuteCommand(QueriesCommandBase.java:71)
>      at
> org.ovirt.engine.core.dal.VdcCommandBase.Execute(VdcCommandBase.java:41)
>      at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:384)
>      at org.ovirt.engine.core.bll.Backend.RunQuery(Backend.java:367)
>      at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
>      at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>      at java.lang.reflect.Method.invoke(Method.java:601)
>      at
> org.jboss.as.ee.component.ManagedReferenceMethodInterceptorFactory$ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptorFactory.java:72)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:374)
>      at
> org.ovirt.engine.core.utils.ThreadLocalSessionCleanerInterceptor.injectWebContextToThreadLocal(ThreadLocalSessionCleanerInterceptor.java:11)
>      at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
>      at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>      at java.lang.reflect.Method.invoke(Method.java:601)
>      at
> org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptorFactory$ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptorFactory.java:123)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.invocation.WeavedInterceptor.processInvocation(WeavedInterceptor.java:53)
>      at
> org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:36)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:21)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>      at
> org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:53)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:211)
>      at
> org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:363)
>      at
> org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:194)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:59)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.as.ee.component.TCCLInterceptor.processInvocation(TCCLInterceptor.java:45)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>      at
> org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:165)
>      at
> org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:173)
>      at
> org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:288)
>      at
> org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61)
>      at
> org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:72)
>      at
> org.ovirt.engine.core.common.interfaces.BackendLocal$$$view9.RunQuery(Unknown
> Source)
>      at
> org.ovirt.engine.ui.frontend.server.gwt.GenericApiGWTServiceImpl.RunQuery(GenericApiGWTServiceImpl.java:51)
>      at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
>      at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>      at java.lang.reflect.Method.invoke(Method.java:601)
>      at com.google.gwt.rpc.server.RPC.invokeAndStreamResponse(RPC.java:196)
>      at
> com.google.gwt.rpc.server.RpcServlet.processCall(RpcServlet.java:161)
>      at
> com.google.gwt.rpc.server.RpcServlet.processPost(RpcServlet.java:222)
>      at
> com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
>      at javax.servlet.http.HttpServlet.service(HttpServlet.java:754)
>      at javax.servlet.http.HttpServlet.service(HttpServlet.java:847)
>      at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:329)
>      at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:248)
>      at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:275)
>      at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:161)
>      at
> org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:489)
>      at
> org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:153)
>      at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:155)
>      at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
>      at org.jboss.web.rewrite.RewriteValve.invoke(RewriteValve.java:466)
>      at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>      at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:368)
>      at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:505)
>      at
> org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:445)
>      at
> org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:930)
>      at java.lang.Thread.run(Thread.java:722)
>
> - DHC
>
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>




From deadhorseconsulting at gmail.com  Tue Sep 25 05:07:07 2012
From: deadhorseconsulting at gmail.com (Dead Horse)
Date: Tue, 25 Sep 2012 00:07:07 -0500
Subject: [Engine-devel] engine build AD/LDAP error
In-Reply-To: <5060D778.2000408@redhat.com>
References: <CAEWPe=oEjbtANvGsEnYMqG1kShPpdmjJWdT010KfjsH7+E8Bqw@mail.gmail.com>
	<5060D778.2000408@redhat.com>
Message-ID: <CAEWPe=qKrdd467KYatqSGg-5zrp2SKrXt4pi90S_=EAmPXN6Bw@mail.gmail.com>

Many thanks Itamar, I had not noticed that bug.
The -addPermissions flag indeed did the trick.

- DHC

On Mon, Sep 24, 2012 at 4:58 PM, Itamar Heim <iheim at redhat.com> wrote:

> On 09/24/2012 11:31 PM, Dead Horse wrote:
>
>> I have built ovirt-engine from the latest GIT master over the past few
>> nights and am running into a MS AD/LDAP error.  This same MS AD setup
>> works fine with Released 3.1.
>>
>> Current nightly builds from jenkins are not building all the GWT
>> permutations (Internet Explorer for example) so I have been attempting
>> to re-build the engine so I can test using Internet Explorer.
>> It would be nice if the nightly jenkins builds could be changed to do
>> the build with all GWT permutations enabled.
>>
>> I am building in an FC17 environment with updates applied and current as
>> of today. Accordingly maven3 and all build dependencies are installed
>> from the FC17 repositories.
>> I tried the latest nightly from Jenkins and it does not exhibit the
>> below error. I am guessing I must not have the build recipe/env correct.
>>
>
> Bug 858769 - manage-domains: once the domain is added admin at internalcan't search in that domain
>
> I assume you are not using -addPermissions during the manage domains
> command - it should workaround it.
> I know yair is working on analyzing this.
>
>
>> The AD error I am running into looks like:
>>
>> 2012-09-24 16:15:44,037 ERROR
>> [org.ovirt.engine.core.bll.**adbroker.DirectorySearcher]
>> (ajp--127.0.0.1-8702-4) Failed ldap search server
>> LDAP://someserver.foo.com:389 <http://someserver.foo.com:389**> due to
>>
>> Kerberos error. Please check log for further details.. We should not try
>> the next server
>> 2012-09-24 16:15:44,039 ERROR
>> [org.ovirt.engine.core.bll.**adbroker.**GSSAPIDirContextAuthentication**
>> Strategy]
>> (ajp--127.0.0.1-8702-2) Error from Kerberos:
>> java.lang.NullPointerException
>>      at
>> org.ovirt.engine.core.bll.**adbroker.**GSSAPICallbackHandler.handle(**
>> GSSAPICallbackHandler.java:47)
>>      at
>> javax.security.auth.login.**LoginContext$**SecureCallbackHandler$1.run(**
>> LoginContext.java:969)
>>      at
>> javax.security.auth.login.**LoginContext$**SecureCallbackHandler$1.run(**
>> LoginContext.java:966)
>>      at java.security.**AccessController.doPrivileged(**Native Method)
>>      at
>> javax.security.auth.login.**LoginContext$**SecureCallbackHandler.handle(*
>> *LoginContext.java:965)
>>      at
>> com.sun.security.auth.module.**Krb5LoginModule.promptForPass(**
>> Krb5LoginModule.java:870)
>>      at
>> com.sun.security.auth.module.**Krb5LoginModule.**attemptAuthentication(**
>> Krb5LoginModule.java:715)
>>      at
>> com.sun.security.auth.module.**Krb5LoginModule.login(**
>> Krb5LoginModule.java:580)
>>      at sun.reflect.**NativeMethodAccessorImpl.**invoke0(Native Method)
>>      at
>> sun.reflect.**NativeMethodAccessorImpl.**invoke(**
>> NativeMethodAccessorImpl.java:**57)
>>      at
>> sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
>> DelegatingMethodAccessorImpl.**java:43)
>>      at java.lang.reflect.Method.**invoke(Method.java:601)
>>      at javax.security.auth.login.**LoginContext.invoke(**
>> LoginContext.java:784)
>>      at
>> javax.security.auth.login.**LoginContext.access$000(**
>> LoginContext.java:203)
>>      at javax.security.auth.login.**LoginContext$4.run(**
>> LoginContext.java:698)
>>      at javax.security.auth.login.**LoginContext$4.run(**
>> LoginContext.java:696)
>>      at java.security.**AccessController.doPrivileged(**Native Method)
>>      at
>> javax.security.auth.login.**LoginContext.invokePriv(**
>> LoginContext.java:695)
>>      at javax.security.auth.login.**LoginContext.login(**
>> LoginContext.java:594)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.**GSSAPIDirContextAuthentication**
>> Strategy.authenticateToKDC(**GSSAPIDirContextAuthentication**
>> Strategy.java:127)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.**GSSAPIDirContextAuthentication**
>> Strategy.explicitAuth(**GSSAPIDirContextAuthentication**
>> Strategy.java:119)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.**GSSAPIDirContextAuthentication**
>> Strategy.authenticate(**GSSAPIDirContextAuthentication**
>> Strategy.java:111)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.**GSSAPILdapTemplateWrapper.**
>> useAuthenticationStrategy(**GSSAPILdapTemplateWrapper.**java:86)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.**PrepareLdapConnectionTask.**call(*
>> *PrepareLdapConnectionTask.**java:56)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.DirectorySearcher.**
>> find(DirectorySearcher.java:**103)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.DirectorySearcher.**
>> FindAll(DirectorySearcher.**java:48)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.**LdapSearchGroupsByQueryCommand**
>> .executeQuery(**LdapSearchGroupsByQueryCommand**.java:22)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.**LdapBrokerCommandBase.execute(**
>> LdapBrokerCommandBase.java:69)
>>      at
>> org.ovirt.engine.core.bll.**adbroker.LdapBrokerBase.**
>> RunAdAction(LdapBrokerBase.**java:18)
>>      at org.ovirt.engine.core.bll.**SearchQuery.adSearch(**
>> SearchQuery.java:194)
>>      at
>> org.ovirt.engine.core.bll.**SearchQuery.searchAdGroups(**
>> SearchQuery.java:172)
>>      at
>> org.ovirt.engine.core.bll.**SearchQuery.**executeQueryCommand(**
>> SearchQuery.java:79)
>>      at
>> org.ovirt.engine.core.bll.**QueriesCommandBase.**ExecuteCommand(**
>> QueriesCommandBase.java:71)
>>      at
>> org.ovirt.engine.core.dal.**VdcCommandBase.Execute(**
>> VdcCommandBase.java:41)
>>      at org.ovirt.engine.core.bll.**Backend.runQueryImpl(Backend.**
>> java:384)
>>      at org.ovirt.engine.core.bll.**Backend.RunQuery(Backend.java:**367)
>>      at sun.reflect.**GeneratedMethodAccessor12.**invoke(Unknown Source)
>>      at
>> sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
>> DelegatingMethodAccessorImpl.**java:43)
>>      at java.lang.reflect.Method.**invoke(Method.java:601)
>>      at
>> org.jboss.as.ee.component.**ManagedReferenceMethodIntercep**torFactory$**
>> ManagedReferenceMethodIntercep**tor.processInvocation(**
>> ManagedReferenceMethodIntercep**torFactory.java:72)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.invocation.**InterceptorContext$Invocation.**
>> proceed(InterceptorContext.**java:374)
>>      at
>> org.ovirt.engine.core.utils.**ThreadLocalSessionCleanerInter**ceptor.**
>> injectWebContextToThreadLocal(**ThreadLocalSessionCleanerInter**
>> ceptor.java:11)
>>      at sun.reflect.**GeneratedMethodAccessor7.**invoke(Unknown Source)
>>      at
>> sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
>> DelegatingMethodAccessorImpl.**java:43)
>>      at java.lang.reflect.Method.**invoke(Method.java:601)
>>      at
>> org.jboss.as.ee.component.**ManagedReferenceLifecycleMetho**
>> dInterceptorFactory$**ManagedReferenceLifecycleMetho**dInterceptor.**
>> processInvocation(**ManagedReferenceLifecycleMetho**
>> dInterceptorFactory.java:123)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.invocation.**WeavedInterceptor.**processInvocation(**
>> WeavedInterceptor.java:53)
>>      at
>> org.jboss.as.ee.component.**interceptors.**UserInterceptorFactory$1.**
>> processInvocation(**UserInterceptorFactory.java:**36)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.invocation.**InitialInterceptor.**processInvocation(**
>> InitialInterceptor.java:21)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.invocation.**ChainedInterceptor.**processInvocation(**
>> ChainedInterceptor.java:61)
>>      at
>> org.jboss.as.ee.component.**interceptors.**ComponentDispatcherInterceptor
>> **.processInvocation(**ComponentDispatcherInterceptor**.java:53)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.as.ejb3.component.**singleton.**SingletonComponentInstanceAsso*
>> *ciationInterceptor.**processInvocation(**SingletonComponentInstanceAsso*
>> *ciationInterceptor.java:53)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.as.ejb3.tx.**CMTTxInterceptor.invokeInNoTx(**
>> CMTTxInterceptor.java:211)
>>      at
>> org.jboss.as.ejb3.tx.**CMTTxInterceptor.supports(**
>> CMTTxInterceptor.java:363)
>>      at
>> org.jboss.as.ejb3.tx.**CMTTxInterceptor.**processInvocation(**
>> CMTTxInterceptor.java:194)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.as.ejb3.component.**interceptors.**
>> CurrentInvocationContextInterc**eptor.processInvocation(**
>> CurrentInvocationContextInterc**eptor.java:41)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.as.ejb3.component.**interceptors.**LoggingInterceptor.**
>> processInvocation(**LoggingInterceptor.java:59)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.as.ee.component.**NamespaceContextInterceptor.**
>> processInvocation(**NamespaceContextInterceptor.**java:50)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.as.ee.component.**TCCLInterceptor.**processInvocation(**
>> TCCLInterceptor.java:45)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.invocation.**ChainedInterceptor.**processInvocation(**
>> ChainedInterceptor.java:61)
>>      at
>> org.jboss.as.ee.component.**ViewService$View.invoke(**
>> ViewService.java:165)
>>      at
>> org.jboss.as.ee.component.**ViewDescription$1.**processInvocation(**
>> ViewDescription.java:173)
>>      at
>> org.jboss.invocation.**InterceptorContext.proceed(**
>> InterceptorContext.java:288)
>>      at
>> org.jboss.invocation.**ChainedInterceptor.**processInvocation(**
>> ChainedInterceptor.java:61)
>>      at
>> org.jboss.as.ee.component.**ProxyInvocationHandler.invoke(**
>> ProxyInvocationHandler.java:**72)
>>      at
>> org.ovirt.engine.core.common.**interfaces.BackendLocal$$$**
>> view9.RunQuery(Unknown
>> Source)
>>      at
>> org.ovirt.engine.ui.frontend.**server.gwt.**GenericApiGWTServiceImpl.**
>> RunQuery(**GenericApiGWTServiceImpl.java:**51)
>>      at sun.reflect.**GeneratedMethodAccessor15.**invoke(Unknown Source)
>>      at
>> sun.reflect.**DelegatingMethodAccessorImpl.**invoke(**
>> DelegatingMethodAccessorImpl.**java:43)
>>      at java.lang.reflect.Method.**invoke(Method.java:601)
>>      at com.google.gwt.rpc.server.RPC.**invokeAndStreamResponse(RPC.**
>> java:196)
>>      at
>> com.google.gwt.rpc.server.**RpcServlet.processCall(**RpcServlet.java:161)
>>      at
>> com.google.gwt.rpc.server.**RpcServlet.processPost(**RpcServlet.java:222)
>>      at
>> com.google.gwt.user.server.**rpc.**AbstractRemoteServiceServlet.**doPost(
>> **AbstractRemoteServiceServlet.**java:62)
>>      at javax.servlet.http.**HttpServlet.service(**HttpServlet.java:754)
>>      at javax.servlet.http.**HttpServlet.service(**HttpServlet.java:847)
>>      at
>> org.apache.catalina.core.**ApplicationFilterChain.**internalDoFilter(**
>> ApplicationFilterChain.java:**329)
>>      at
>> org.apache.catalina.core.**ApplicationFilterChain.**doFilter(**
>> ApplicationFilterChain.java:**248)
>>      at
>> org.apache.catalina.core.**StandardWrapperValve.invoke(**
>> StandardWrapperValve.java:275)
>>      at
>> org.apache.catalina.core.**StandardContextValve.invoke(**
>> StandardContextValve.java:161)
>>      at
>> org.apache.catalina.**authenticator.**AuthenticatorBase.invoke(**
>> AuthenticatorBase.java:489)
>>      at
>> org.jboss.as.web.security.**SecurityContextAssociationValv**e.invoke(**
>> SecurityContextAssociationValv**e.java:153)
>>      at
>> org.apache.catalina.core.**StandardHostValve.invoke(**
>> StandardHostValve.java:155)
>>      at
>> org.apache.catalina.valves.**ErrorReportValve.invoke(**
>> ErrorReportValve.java:102)
>>      at org.jboss.web.rewrite.**RewriteValve.invoke(**
>> RewriteValve.java:466)
>>      at
>> org.apache.catalina.core.**StandardEngineValve.invoke(**
>> StandardEngineValve.java:109)
>>      at
>> org.apache.catalina.connector.**CoyoteAdapter.service(**
>> CoyoteAdapter.java:368)
>>      at org.apache.coyote.ajp.**AjpProcessor.process(**
>> AjpProcessor.java:505)
>>      at
>> org.apache.coyote.ajp.**AjpProtocol$**AjpConnectionHandler.process(**
>> AjpProtocol.java:445)
>>      at
>> org.apache.tomcat.util.net.**JIoEndpoint$Worker.run(**
>> JIoEndpoint.java:930)
>>      at java.lang.Thread.run(Thread.**java:722)
>>
>> - DHC
>>
>>
>> ______________________________**_________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/**mailman/listinfo/engine-devel<http://lists.ovirt.org/mailman/listinfo/engine-devel>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/engine-devel/attachments/20120925/8d547957/attachment.html>

From yzaslavs at redhat.com  Tue Sep 25 05:51:27 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Tue, 25 Sep 2012 07:51:27 +0200
Subject: [Engine-devel] engine build AD/LDAP error
In-Reply-To: <CAEWPe=qKrdd467KYatqSGg-5zrp2SKrXt4pi90S_=EAmPXN6Bw@mail.gmail.com>
References: <CAEWPe=oEjbtANvGsEnYMqG1kShPpdmjJWdT010KfjsH7+E8Bqw@mail.gmail.com>
	<5060D778.2000408@redhat.com>
	<CAEWPe=qKrdd467KYatqSGg-5zrp2SKrXt4pi90S_=EAmPXN6Bw@mail.gmail.com>
Message-ID: <5061465F.4000703@redhat.com>

Sorry for late response on this,
Patch for soling this issue is available on gerrit -
http://gerrit.ovirt.org/#/c/8178/

Yair


On 09/25/2012 07:07 AM, Dead Horse wrote:
> Many thanks Itamar, I had not noticed that bug.
> The -addPermissions flag indeed did the trick.
>
> - DHC
>
> On Mon, Sep 24, 2012 at 4:58 PM, Itamar Heim <iheim at redhat.com
> <mailto:iheim at redhat.com>> wrote:
>
>     On 09/24/2012 11:31 PM, Dead Horse wrote:
>
>         I have built ovirt-engine from the latest GIT master over the
>         past few
>         nights and am running into a MS AD/LDAP error.  This same MS AD
>         setup
>         works fine with Released 3.1.
>
>         Current nightly builds from jenkins are not building all the GWT
>         permutations (Internet Explorer for example) so I have been
>         attempting
>         to re-build the engine so I can test using Internet Explorer.
>         It would be nice if the nightly jenkins builds could be changed
>         to do
>         the build with all GWT permutations enabled.
>
>         I am building in an FC17 environment with updates applied and
>         current as
>         of today. Accordingly maven3 and all build dependencies are
>         installed
>         from the FC17 repositories.
>         I tried the latest nightly from Jenkins and it does not exhibit the
>         below error. I am guessing I must not have the build recipe/env
>         correct.
>
>
>     Bug 858769 - manage-domains: once the domain is added admin at internal
>     can't search in that domain
>
>     I assume you are not using -addPermissions during the manage domains
>     command - it should workaround it.
>     I know yair is working on analyzing this.
>
>
>         The AD error I am running into looks like:
>
>         2012-09-24 16:15:44,037 ERROR
>         [org.ovirt.engine.core.bll.__adbroker.DirectorySearcher]
>         (ajp--127.0.0.1-8702-4) Failed ldap search server
>         LDAP://someserver.foo.com:389 <http://someserver.foo.com:389>
>         <http://someserver.foo.com:389__> due to
>
>         Kerberos error. Please check log for further details.. We should
>         not try
>         the next server
>         2012-09-24 16:15:44,039 ERROR
>         [org.ovirt.engine.core.bll.__adbroker.__GSSAPIDirContextAuthentication__Strategy]
>         (ajp--127.0.0.1-8702-2) Error from Kerberos:
>         java.lang.NullPointerException
>               at
>         org.ovirt.engine.core.bll.__adbroker.__GSSAPICallbackHandler.handle(__GSSAPICallbackHandler.java:47)
>               at
>         javax.security.auth.login.__LoginContext$__SecureCallbackHandler$1.run(__LoginContext.java:969)
>               at
>         javax.security.auth.login.__LoginContext$__SecureCallbackHandler$1.run(__LoginContext.java:966)
>               at java.security.__AccessController.doPrivileged(__Native
>         Method)
>               at
>         javax.security.auth.login.__LoginContext$__SecureCallbackHandler.handle(__LoginContext.java:965)
>               at
>         com.sun.security.auth.module.__Krb5LoginModule.promptForPass(__Krb5LoginModule.java:870)
>               at
>         com.sun.security.auth.module.__Krb5LoginModule.__attemptAuthentication(__Krb5LoginModule.java:715)
>               at
>         com.sun.security.auth.module.__Krb5LoginModule.login(__Krb5LoginModule.java:580)
>               at sun.reflect.__NativeMethodAccessorImpl.__invoke0(Native
>         Method)
>               at
>         sun.reflect.__NativeMethodAccessorImpl.__invoke(__NativeMethodAccessorImpl.java:__57)
>               at
>         sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)
>               at java.lang.reflect.Method.__invoke(Method.java:601)
>               at
>         javax.security.auth.login.__LoginContext.invoke(__LoginContext.java:784)
>               at
>         javax.security.auth.login.__LoginContext.access$000(__LoginContext.java:203)
>               at
>         javax.security.auth.login.__LoginContext$4.run(__LoginContext.java:698)
>               at
>         javax.security.auth.login.__LoginContext$4.run(__LoginContext.java:696)
>               at java.security.__AccessController.doPrivileged(__Native
>         Method)
>               at
>         javax.security.auth.login.__LoginContext.invokePriv(__LoginContext.java:695)
>               at
>         javax.security.auth.login.__LoginContext.login(__LoginContext.java:594)
>               at
>         org.ovirt.engine.core.bll.__adbroker.__GSSAPIDirContextAuthentication__Strategy.authenticateToKDC(__GSSAPIDirContextAuthentication__Strategy.java:127)
>               at
>         org.ovirt.engine.core.bll.__adbroker.__GSSAPIDirContextAuthentication__Strategy.explicitAuth(__GSSAPIDirContextAuthentication__Strategy.java:119)
>               at
>         org.ovirt.engine.core.bll.__adbroker.__GSSAPIDirContextAuthentication__Strategy.authenticate(__GSSAPIDirContextAuthentication__Strategy.java:111)
>               at
>         org.ovirt.engine.core.bll.__adbroker.__GSSAPILdapTemplateWrapper.__useAuthenticationStrategy(__GSSAPILdapTemplateWrapper.__java:86)
>               at
>         org.ovirt.engine.core.bll.__adbroker.__PrepareLdapConnectionTask.__call(__PrepareLdapConnectionTask.__java:56)
>               at
>         org.ovirt.engine.core.bll.__adbroker.DirectorySearcher.__find(DirectorySearcher.java:__103)
>               at
>         org.ovirt.engine.core.bll.__adbroker.DirectorySearcher.__FindAll(DirectorySearcher.__java:48)
>               at
>         org.ovirt.engine.core.bll.__adbroker.__LdapSearchGroupsByQueryCommand__.executeQuery(__LdapSearchGroupsByQueryCommand__.java:22)
>               at
>         org.ovirt.engine.core.bll.__adbroker.__LdapBrokerCommandBase.execute(__LdapBrokerCommandBase.java:69)
>               at
>         org.ovirt.engine.core.bll.__adbroker.LdapBrokerBase.__RunAdAction(LdapBrokerBase.__java:18)
>               at
>         org.ovirt.engine.core.bll.__SearchQuery.adSearch(__SearchQuery.java:194)
>               at
>         org.ovirt.engine.core.bll.__SearchQuery.searchAdGroups(__SearchQuery.java:172)
>               at
>         org.ovirt.engine.core.bll.__SearchQuery.__executeQueryCommand(__SearchQuery.java:79)
>               at
>         org.ovirt.engine.core.bll.__QueriesCommandBase.__ExecuteCommand(__QueriesCommandBase.java:71)
>               at
>         org.ovirt.engine.core.dal.__VdcCommandBase.Execute(__VdcCommandBase.java:41)
>               at
>         org.ovirt.engine.core.bll.__Backend.runQueryImpl(Backend.__java:384)
>               at
>         org.ovirt.engine.core.bll.__Backend.RunQuery(Backend.java:__367)
>               at
>         sun.reflect.__GeneratedMethodAccessor12.__invoke(Unknown Source)
>               at
>         sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)
>               at java.lang.reflect.Method.__invoke(Method.java:601)
>               at
>         org.jboss.as.ee.component.__ManagedReferenceMethodIntercep__torFactory$__ManagedReferenceMethodIntercep__tor.processInvocation(__ManagedReferenceMethodIntercep__torFactory.java:72)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.invocation.__InterceptorContext$Invocation.__proceed(InterceptorContext.__java:374)
>               at
>         org.ovirt.engine.core.utils.__ThreadLocalSessionCleanerInter__ceptor.__injectWebContextToThreadLocal(__ThreadLocalSessionCleanerInter__ceptor.java:11)
>               at sun.reflect.__GeneratedMethodAccessor7.__invoke(Unknown
>         Source)
>               at
>         sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)
>               at java.lang.reflect.Method.__invoke(Method.java:601)
>               at
>         org.jboss.as.ee.component.__ManagedReferenceLifecycleMetho__dInterceptorFactory$__ManagedReferenceLifecycleMetho__dInterceptor.__processInvocation(__ManagedReferenceLifecycleMetho__dInterceptorFactory.java:123)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.invocation.__WeavedInterceptor.__processInvocation(__WeavedInterceptor.java:53)
>               at
>         org.jboss.as.ee.component.__interceptors.__UserInterceptorFactory$1.__processInvocation(__UserInterceptorFactory.java:__36)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.invocation.__InitialInterceptor.__processInvocation(__InitialInterceptor.java:21)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.invocation.__ChainedInterceptor.__processInvocation(__ChainedInterceptor.java:61)
>               at
>         org.jboss.as.ee.component.__interceptors.__ComponentDispatcherInterceptor__.processInvocation(__ComponentDispatcherInterceptor__.java:53)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.as.ejb3.component.__singleton.__SingletonComponentInstanceAsso__ciationInterceptor.__processInvocation(__SingletonComponentInstanceAsso__ciationInterceptor.java:53)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.as.ejb3.tx.__CMTTxInterceptor.invokeInNoTx(__CMTTxInterceptor.java:211)
>               at
>         org.jboss.as.ejb3.tx.__CMTTxInterceptor.supports(__CMTTxInterceptor.java:363)
>               at
>         org.jboss.as.ejb3.tx.__CMTTxInterceptor.__processInvocation(__CMTTxInterceptor.java:194)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.as.ejb3.component.__interceptors.__CurrentInvocationContextInterc__eptor.processInvocation(__CurrentInvocationContextInterc__eptor.java:41)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.as.ejb3.component.__interceptors.__LoggingInterceptor.__processInvocation(__LoggingInterceptor.java:59)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.as.ee.component.__NamespaceContextInterceptor.__processInvocation(__NamespaceContextInterceptor.__java:50)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.as.ee.component.__TCCLInterceptor.__processInvocation(__TCCLInterceptor.java:45)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.invocation.__ChainedInterceptor.__processInvocation(__ChainedInterceptor.java:61)
>               at
>         org.jboss.as.ee.component.__ViewService$View.invoke(__ViewService.java:165)
>               at
>         org.jboss.as.ee.component.__ViewDescription$1.__processInvocation(__ViewDescription.java:173)
>               at
>         org.jboss.invocation.__InterceptorContext.proceed(__InterceptorContext.java:288)
>               at
>         org.jboss.invocation.__ChainedInterceptor.__processInvocation(__ChainedInterceptor.java:61)
>               at
>         org.jboss.as.ee.component.__ProxyInvocationHandler.invoke(__ProxyInvocationHandler.java:__72)
>               at
>         org.ovirt.engine.core.common.__interfaces.BackendLocal$$$__view9.RunQuery(Unknown
>         Source)
>               at
>         org.ovirt.engine.ui.frontend.__server.gwt.__GenericApiGWTServiceImpl.__RunQuery(__GenericApiGWTServiceImpl.java:__51)
>               at
>         sun.reflect.__GeneratedMethodAccessor15.__invoke(Unknown Source)
>               at
>         sun.reflect.__DelegatingMethodAccessorImpl.__invoke(__DelegatingMethodAccessorImpl.__java:43)
>               at java.lang.reflect.Method.__invoke(Method.java:601)
>               at
>         com.google.gwt.rpc.server.RPC.__invokeAndStreamResponse(RPC.__java:196)
>               at
>         com.google.gwt.rpc.server.__RpcServlet.processCall(__RpcServlet.java:161)
>               at
>         com.google.gwt.rpc.server.__RpcServlet.processPost(__RpcServlet.java:222)
>               at
>         com.google.gwt.user.server.__rpc.__AbstractRemoteServiceServlet.__doPost(__AbstractRemoteServiceServlet.__java:62)
>               at
>         javax.servlet.http.__HttpServlet.service(__HttpServlet.java:754)
>               at
>         javax.servlet.http.__HttpServlet.service(__HttpServlet.java:847)
>               at
>         org.apache.catalina.core.__ApplicationFilterChain.__internalDoFilter(__ApplicationFilterChain.java:__329)
>               at
>         org.apache.catalina.core.__ApplicationFilterChain.__doFilter(__ApplicationFilterChain.java:__248)
>               at
>         org.apache.catalina.core.__StandardWrapperValve.invoke(__StandardWrapperValve.java:275)
>               at
>         org.apache.catalina.core.__StandardContextValve.invoke(__StandardContextValve.java:161)
>               at
>         org.apache.catalina.__authenticator.__AuthenticatorBase.invoke(__AuthenticatorBase.java:489)
>               at
>         org.jboss.as.web.security.__SecurityContextAssociationValv__e.invoke(__SecurityContextAssociationValv__e.java:153)
>               at
>         org.apache.catalina.core.__StandardHostValve.invoke(__StandardHostValve.java:155)
>               at
>         org.apache.catalina.valves.__ErrorReportValve.invoke(__ErrorReportValve.java:102)
>               at
>         org.jboss.web.rewrite.__RewriteValve.invoke(__RewriteValve.java:466)
>               at
>         org.apache.catalina.core.__StandardEngineValve.invoke(__StandardEngineValve.java:109)
>               at
>         org.apache.catalina.connector.__CoyoteAdapter.service(__CoyoteAdapter.java:368)
>               at
>         org.apache.coyote.ajp.__AjpProcessor.process(__AjpProcessor.java:505)
>               at
>         org.apache.coyote.ajp.__AjpProtocol$__AjpConnectionHandler.process(__AjpProtocol.java:445)
>               at
>         org.apache.tomcat.util.net
>         <http://org.apache.tomcat.util.net>.__JIoEndpoint$Worker.run(__JIoEndpoint.java:930)
>               at java.lang.Thread.run(Thread.__java:722)
>
>         - DHC
>
>
>         _________________________________________________
>         Engine-devel mailing list
>         Engine-devel at ovirt.org <mailto:Engine-devel at ovirt.org>
>         http://lists.ovirt.org/__mailman/listinfo/engine-devel
>         <http://lists.ovirt.org/mailman/listinfo/engine-devel>
>
>
>
>


From snmishra at linux.vnet.ibm.com  Wed Sep 26 22:35:44 2012
From: snmishra at linux.vnet.ibm.com (snmishra at linux.vnet.ibm.com)
Date: Wed, 26 Sep 2012 15:35:44 -0700
Subject: [Engine-devel] Local build failing.
Message-ID: <20120926153544.Horde.TQ6K9pir309QY4NA5MXgVJA@imap.linux.ibm.com>


Hi,

     My local build is failing today. I cloned the ovirt-engine repo  
and ran 'mvn clean install". Failed with following error -

Failed to execute goal  
org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile  
(default-compile) on project vdsbroker: Compilation failure
[ERROR]  
/tmp/ovirt-engine/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/VdsManager.java:[604,44] illegal start of  
expression


   I noticed that Jenkins build succeeded after this change was merged.

-Sharad



From abaron at redhat.com  Wed Sep 26 23:11:02 2012
From: abaron at redhat.com (Ayal Baron)
Date: Wed, 26 Sep 2012 19:11:02 -0400 (EDT)
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <905631359.493569.1348474208972.JavaMail.root@redhat.com>
Message-ID: <354414032.2826856.1348701062700.JavaMail.root@redhat.com>



----- Original Message -----
> 
> 
> ----- Original Message -----
> > From: "Ayal Baron" <abaron at redhat.com>
> > To: "Michael Kublin" <mkublin at redhat.com>
> > Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel"
> > <engine-devel at ovirt.org>, "Eduardo Warszawski"
> > <ewarszaw at redhat.com>, "Allon Mureinik" <amureini at redhat.com>
> > Sent: Sunday, September 23, 2012 5:27:54 PM
> > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > 
> > 
> > 
> > ----- Original Message -----
> > > 
> > > 
> > > ----- Original Message -----
> > > > From: "Ayal Baron" <abaron at redhat.com>
> > > > To: "Allon Mureinik" <amureini at redhat.com>, "Michael Kublin"
> > > > <mkublin at redhat.com>
> > > > Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel"
> > > > <engine-devel at ovirt.org>, "Eduardo Warszawski"
> > > > <ewarszaw at redhat.com>
> > > > Sent: Sunday, September 23, 2012 1:10:07 PM
> > > > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > > > 
> > > > 
> > > > 
> > > > ----- Original Message -----
> > > > > 
> > > > > 
> > > > > ----- Original Message -----
> > > > > > From: "Michael Kublin" <mkublin at redhat.com>
> > > > > > To: "Allon Mureinik" <amureini at redhat.com>
> > > > > > Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Liron
> > > > > > Aravot"
> > > > > > <laravot at redhat.com>, "Maor Lipchuk"
> > > > > > <mlipchuk at redhat.com>, "engine-devel"
> > > > > > <engine-devel at ovirt.org>
> > > > > > Sent: Sunday, September 23, 2012 12:41:05 PM
> > > > > > Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> > > > > > 
> > > > > > 
> > > > > > >>>>>>>>>> Hi guys,
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> As you may know the engine currently has the
> > > > > > >>>>>>>>>> ability
> > > > > > >>>>>>>>>> to
> > > > > > >>>>>>>>>> fire
> > > > > > >>>>>>>>>> an
> > > > > > >>>>>>>>>> SPM
> > > > > > >>>>>>>>>> task, and be asynchronously be "woken-up" when
> > > > > > >>>>>>>>>> it
> > > > > > >>>>>>>>>> ends.
> > > > > > >>>>>>>>>> This is great, but we found the for the Live
> > > > > > >>>>>>>>>> Storage
> > > > > > >>>>>>>>>> Migration
> > > > > > >>>>>>>>>> feature we need something a bit complex - the
> > > > > > >>>>>>>>>> ability
> > > > > > >>>>>>>>>> to
> > > > > > >>>>>>>>>> have a
> > > > > > >>>>>>>>>> series of async tasks in a single control flow.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Here's my initial design for this, your comments
> > > > > > >>>>>>>>>> and
> > > > > > >>>>>>>>>> criticism
> > > > > > >>>>>>>>>> would
> > > > > > >>>>>>>>>> be welcome:
> > > > > > >>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
> > > > > > >>>>>>>> -successful execution -
> > > > > > >>>>>>>> * "CommandBase iterates over its
> > > > > > >>>>>>>> SPMAsyncTaskHandlers"
> > > > > > >>>>>>>> -
> > > > > > >>>>>>>> when?
> > > > > > >>>>>>> This is the new suggested format of
> > > > > > >>>>>>> executeCommand().
> > > > > > >>>>>>> I'll
> > > > > > >>>>>>> clarify
> > > > > > >>>>>>> this too.
> > > > > > >>>>>>>
> > > > > > >>>>>>>> * If the second task is an HSM command (vs. SPM
> > > > > > >>>>>>>> command),
> > > > > > >>>>>>>> I
> > > > > > >>>>>>>> think you
> > > > > > >>>>>>>> should explain in the design how to handle such
> > > > > > >>>>>>>> flows
> > > > > > >>>>>>>> as
> > > > > > >>>>>>>> well.
> > > > > > >>>>>>> HSM commands do not create AsyncTasks, as they do
> > > > > > >>>>>>> today
> > > > > > >>>>>>> -
> > > > > > >>>>>>> I
> > > > > > >>>>>>> will
> > > > > > >>>>>>> clarify this.
> > > > > > >>>>>>>
> > > > > > >>>>>>>> * Why do we need before task? can you give a
> > > > > > >>>>>>>> concrete
> > > > > > >>>>>>>> example
> > > > > > >>>>>>>> of what
> > > > > > >>>>>>>> would you do in such a method.
> > > > > > >>>>>>> Basically, /today/, command look like this:
> > > > > > >>>>>>> executeCommand() {
> > > > > > >>>>>>>     doStuffInTheDB();
> > > > > > >>>>>>>     runVdsCommand(someCommand);
> > > > > > >>>>>>> }
> > > > > > >>>>>>>
> > > > > > >>>>>>> endSuccessfully() {
> > > > > > >>>>>>>     doMoreStuffInTheDB();
> > > > > > >>>>>>> }
> > > > > > >>>>>>>
> > > > > > >>>>>>> endWithFailure() {
> > > > > > >>>>>>>     doMoreStuffForFailureInTheDB();
> > > > > > >>>>>>> }
> > > > > > >>>>>>>
> > > > > > >>>>>>> In the new design, the entire doStuffInTheDB()
> > > > > > >>>>>>> should
> > > > > > >>>>>>> be
> > > > > > >>>>>>> moved
> > > > > > >>>>>>> to a
> > > > > > >>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
> > > > > > >>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> - I see you added SPMAsyncTaskHandler, any reason
> > > > > > >>>>>>>> not
> > > > > > >>>>>>>> to
> > > > > > >>>>>>>> use
> > > > > > >>>>>>>> SPMAsyncTasK to manage it own life-cycle?
> > > > > > >>>>>>> Conserving today's design - The SPMAsyncTaskHandler
> > > > > > >>>>>>> is
> > > > > > >>>>>>> the
> > > > > > >>>>>>> place to
> > > > > > >>>>>>> add additional, non-SPM, logic around the SPM task
> > > > > > >>>>>>> execution,
> > > > > > >>>>>>> like
> > > > > > >>>>>>> CommandBase allows today.
> > > > > > >>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> - In the life-cycle managed by the
> > > > > > >>>>>>>> SPMAsyncTaskHandler
> > > > > > >>>>>>>> there
> > > > > > >>>>>>>> is a
> > > > > > >>>>>>>> step
> > > > > > >>>>>>>> 'createTask - how to create the async task' can
> > > > > > >>>>>>>> you
> > > > > > >>>>>>>> please
> > > > > > >>>>>>>> elaborate
> > > > > > >>>>>>>> what are the options.
> > > > > > >>>>>>> new [any type of async task]
> > > > > > 
> > > > > > (I cleaned thread a little.)
> > > > > > The following design and it is implementation
> > > > > > http://gerrit.ovirt.org/#/c/7956/
> > > > > > is bad.
> > > > > > I don't see any reason for creating a new
> > > > > > SPMAsyncTaskHandler
> > > > > > and
> > > > > > especially in the
> > > > > > way as it's done in patch.
> > > > > > The reason are following:
> > > > > > 1. Performance , increased memory footprint, created CYCLIC
> > > > > > REFERENCE.
> > > > > > 2. Readability and robust of code: the code which is
> > > > > > written
> > > > > > as
> > > > > > cyclic references is unreadable
> > > > > >    and difficult for debug.
> > > > > > 3. Why I need a generic implementation and changes all over
> > > > > > whole
> > > > > > project because of
> > > > > >    series of async commands, for me it is a private case?
> > > > 
> > > > What is the private case here exactly?
> > > > Every task can have multiple jobs.  We've identified several
> > > > such
> > > > places (e.g. live storage migration, move disk, move vm) and I
> > > > have
> > > > no doubt more will popup.
> > > > As Allon notes below, task handling is done at CommandBase, if
> > > > you
> > > > think task management should be for storage only, you're
> > > > welcome
> > > > to
> > > > push it down to StorageHandlingCommandBase (or get rid of
> > > > inheritance here altogether).
> > > Interesting , regards cyclic reference no response, but who
> > > cares,
> > > it is difficult to answer , that's why better not to response?
> > 
> > There is no problem with cyclic references in general, GCs know how
> > to deal with these just fine and in this case it's limited to the
> > command and its handlers.
> > I did not reply, however, as I do not feel strongly about this.
> > 
> > > Regards private case:
> > > 1. We have command that not creating any task
> > > 2. We have command that will create a one task.
> > > 3. And we have 3 commands meanwhile which will create more than
> > > one
> > > task.
> > > I think that 3 is private case and not common? (In the future, I
> > 
> > once happens
> > twice is a coincidence
> > three times is a method
> > 
> > But if you insist on more then it's easy enough.  We've discussed
> > many times in the past that we need to change many of the storage
> > verbs to run asynchronously (e.g. createStorageDomain) once this
> > happens then existing flows would have to run multiple async tasks
> > serially.
> > 
> > > removed too many
> > > lines of code that were preparation for future that never come)
> > 
> > This is not in preparation for the future, it is for a feature
> > we're
> > working on right now (live storage migration) and for fixing move
> > disk on which we have several bugs pending.
> > 
> > > The handling done at CommandBase it means that it is influence
> > > all
> > > system.
> > 
> > That is how the task management was done.  Again, if you feel it
> > should only affect storage flows, feel free to push it down into
> > StorageCommandHandlingBase and then only storage verbs will have
> > task management.
> > 
> > > Now regards architecture why I need some handler which will be
> > > inside
> > > a command
> > > and will call for command methods? Please explain.
> > 
> > As opposed to what?
> So, u think it is a good design where we are using a "Circular
> references"
> design pattern. If yes, please elaborate.

I'm saying nacking something without suggesting a better alternative is not good practice.

Besides, the design is classic callback pattern (http://en.wikipedia.org/wiki/Callback_%28computer_programming%29) which is quite common and accepted.
Regardless though, take a look at the new patches.

> 
> > > 
> > >   
> > > > > This will occur all over the storage commands (which are the
> > > > > only
> > > > > usages of tasks nowadys).
> > > > > Moreover, async task handling is done at the Commandbase
> > > > > level
> > > > > (see
> > > > > the end* methods) - instead of hacking it in X different
> > > > > places
> > > > > whenever we need it, I'd prefer doing it once, properly.
> > > > > > 
> > > > > > 
> > > > > _______________________________________________
> > > > > Engine-devel mailing list
> > > > > Engine-devel at ovirt.org
> > > > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > > > 
> > > > 
> > > 
> > 
> 


From kmayilsa at redhat.com  Thu Sep 27 04:22:25 2012
From: kmayilsa at redhat.com (Kanagaraj)
Date: Thu, 27 Sep 2012 09:52:25 +0530
Subject: [Engine-devel] Local build failing.
In-Reply-To: <20120926153544.Horde.TQ6K9pir309QY4NA5MXgVJA@imap.linux.ibm.com>
References: <20120926153544.Horde.TQ6K9pir309QY4NA5MXgVJA@imap.linux.ibm.com>
Message-ID: <5063D481.8000106@redhat.com>

Hi,

 The build goes through fine with Java 1.7.

Regards
Kanagaraj M

On Thursday 27 September 2012 04:05:44 AM IST, 
snmishra at linux.vnet.ibm.com wrote:
>
> Hi,
>
>     My local build is failing today. I cloned the ovirt-engine repo
> and ran 'mvn clean install". Failed with following error -
>
> Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile
> (default-compile) on project vdsbroker: Compilation failure
> [ERROR]
> /tmp/ovirt-engine/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/VdsManager.java:[604,44]
> illegal start of expression
>
>
>   I noticed that Jenkins build succeeded after this change was merged.
>
> -Sharad
>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>
>




From yzaslavs at redhat.com  Thu Sep 27 05:41:29 2012
From: yzaslavs at redhat.com (Yair Zaslavsky)
Date: Thu, 27 Sep 2012 07:41:29 +0200
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <354414032.2826856.1348701062700.JavaMail.root@redhat.com>
References: <354414032.2826856.1348701062700.JavaMail.root@redhat.com>
Message-ID: <5063E709.5070408@redhat.com>



On 09/27/2012 01:11 AM, Ayal Baron wrote:
>
>
> ----- Original Message -----
>>
>>
>> ----- Original Message -----
>>> From: "Ayal Baron" <abaron at redhat.com>
>>> To: "Michael Kublin" <mkublin at redhat.com>
>>> Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel"
>>> <engine-devel at ovirt.org>, "Eduardo Warszawski"
>>> <ewarszaw at redhat.com>, "Allon Mureinik" <amureini at redhat.com>
>>> Sent: Sunday, September 23, 2012 5:27:54 PM
>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
>>>
>>>
>>>
>>> ----- Original Message -----
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Ayal Baron" <abaron at redhat.com>
>>>>> To: "Allon Mureinik" <amureini at redhat.com>, "Michael Kublin"
>>>>> <mkublin at redhat.com>
>>>>> Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel"
>>>>> <engine-devel at ovirt.org>, "Eduardo Warszawski"
>>>>> <ewarszaw at redhat.com>
>>>>> Sent: Sunday, September 23, 2012 1:10:07 PM
>>>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Michael Kublin" <mkublin at redhat.com>
>>>>>>> To: "Allon Mureinik" <amureini at redhat.com>
>>>>>>> Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Liron
>>>>>>> Aravot"
>>>>>>> <laravot at redhat.com>, "Maor Lipchuk"
>>>>>>> <mlipchuk at redhat.com>, "engine-devel"
>>>>>>> <engine-devel at ovirt.org>
>>>>>>> Sent: Sunday, September 23, 2012 12:41:05 PM
>>>>>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
>>>>>>>
>>>>>>>
>>>>>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> As you may know the engine currently has the
>>>>>>>>>>>>>>>>> ability
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> fire
>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>> SPM
>>>>>>>>>>>>>>>>> task, and be asynchronously be "woken-up" when
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> ends.
>>>>>>>>>>>>>>>>> This is great, but we found the for the Live
>>>>>>>>>>>>>>>>> Storage
>>>>>>>>>>>>>>>>> Migration
>>>>>>>>>>>>>>>>> feature we need something a bit complex - the
>>>>>>>>>>>>>>>>> ability
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>> series of async tasks in a single control flow.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Here's my initial design for this, your comments
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> criticism
>>>>>>>>>>>>>>>>> would
>>>>>>>>>>>>>>>>> be welcome:
>>>>>>>>>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
>>>>>>>>>>>>>>> -successful execution -
>>>>>>>>>>>>>>> * "CommandBase iterates over its
>>>>>>>>>>>>>>> SPMAsyncTaskHandlers"
>>>>>>>>>>>>>>> -
>>>>>>>>>>>>>>> when?
>>>>>>>>>>>>>> This is the new suggested format of
>>>>>>>>>>>>>> executeCommand().
>>>>>>>>>>>>>> I'll
>>>>>>>>>>>>>> clarify
>>>>>>>>>>>>>> this too.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * If the second task is an HSM command (vs. SPM
>>>>>>>>>>>>>>> command),
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> think you
>>>>>>>>>>>>>>> should explain in the design how to handle such
>>>>>>>>>>>>>>> flows
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>> well.
>>>>>>>>>>>>>> HSM commands do not create AsyncTasks, as they do
>>>>>>>>>>>>>> today
>>>>>>>>>>>>>> -
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>> clarify this.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> * Why do we need before task? can you give a
>>>>>>>>>>>>>>> concrete
>>>>>>>>>>>>>>> example
>>>>>>>>>>>>>>> of what
>>>>>>>>>>>>>>> would you do in such a method.
>>>>>>>>>>>>>> Basically, /today/, command look like this:
>>>>>>>>>>>>>> executeCommand() {
>>>>>>>>>>>>>>      doStuffInTheDB();
>>>>>>>>>>>>>>      runVdsCommand(someCommand);
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> endSuccessfully() {
>>>>>>>>>>>>>>      doMoreStuffInTheDB();
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> endWithFailure() {
>>>>>>>>>>>>>>      doMoreStuffForFailureInTheDB();
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In the new design, the entire doStuffInTheDB()
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>> moved
>>>>>>>>>>>>>> to a
>>>>>>>>>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - I see you added SPMAsyncTaskHandler, any reason
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> use
>>>>>>>>>>>>>>> SPMAsyncTasK to manage it own life-cycle?
>>>>>>>>>>>>>> Conserving today's design - The SPMAsyncTaskHandler
>>>>>>>>>>>>>> is
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> place to
>>>>>>>>>>>>>> add additional, non-SPM, logic around the SPM task
>>>>>>>>>>>>>> execution,
>>>>>>>>>>>>>> like
>>>>>>>>>>>>>> CommandBase allows today.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - In the life-cycle managed by the
>>>>>>>>>>>>>>> SPMAsyncTaskHandler
>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>> is a
>>>>>>>>>>>>>>> step
>>>>>>>>>>>>>>> 'createTask - how to create the async task' can
>>>>>>>>>>>>>>> you
>>>>>>>>>>>>>>> please
>>>>>>>>>>>>>>> elaborate
>>>>>>>>>>>>>>> what are the options.
>>>>>>>>>>>>>> new [any type of async task]
>>>>>>>
>>>>>>> (I cleaned thread a little.)
>>>>>>> The following design and it is implementation
>>>>>>> http://gerrit.ovirt.org/#/c/7956/
>>>>>>> is bad.
>>>>>>> I don't see any reason for creating a new
>>>>>>> SPMAsyncTaskHandler
>>>>>>> and
>>>>>>> especially in the
>>>>>>> way as it's done in patch.
>>>>>>> The reason are following:
>>>>>>> 1. Performance , increased memory footprint, created CYCLIC
>>>>>>> REFERENCE.
>>>>>>> 2. Readability and robust of code: the code which is
>>>>>>> written
>>>>>>> as
>>>>>>> cyclic references is unreadable
>>>>>>>     and difficult for debug.
>>>>>>> 3. Why I need a generic implementation and changes all over
>>>>>>> whole
>>>>>>> project because of
>>>>>>>     series of async commands, for me it is a private case?
>>>>>
>>>>> What is the private case here exactly?
>>>>> Every task can have multiple jobs.  We've identified several
>>>>> such
>>>>> places (e.g. live storage migration, move disk, move vm) and I
>>>>> have
>>>>> no doubt more will popup.
>>>>> As Allon notes below, task handling is done at CommandBase, if
>>>>> you
>>>>> think task management should be for storage only, you're
>>>>> welcome
>>>>> to
>>>>> push it down to StorageHandlingCommandBase (or get rid of
>>>>> inheritance here altogether).
>>>> Interesting , regards cyclic reference no response, but who
>>>> cares,
>>>> it is difficult to answer , that's why better not to response?
>>>
>>> There is no problem with cyclic references in general, GCs know how
>>> to deal with these just fine and in this case it's limited to the
>>> command and its handlers.
>>> I did not reply, however, as I do not feel strongly about this.
>>>
>>>> Regards private case:
>>>> 1. We have command that not creating any task
>>>> 2. We have command that will create a one task.
>>>> 3. And we have 3 commands meanwhile which will create more than
>>>> one
>>>> task.
>>>> I think that 3 is private case and not common? (In the future, I
>>>
>>> once happens
>>> twice is a coincidence
>>> three times is a method
>>>
>>> But if you insist on more then it's easy enough.  We've discussed
>>> many times in the past that we need to change many of the storage
>>> verbs to run asynchronously (e.g. createStorageDomain) once this
>>> happens then existing flows would have to run multiple async tasks
>>> serially.
>>>
>>>> removed too many
>>>> lines of code that were preparation for future that never come)
>>>
>>> This is not in preparation for the future, it is for a feature
>>> we're
>>> working on right now (live storage migration) and for fixing move
>>> disk on which we have several bugs pending.
>>>
>>>> The handling done at CommandBase it means that it is influence
>>>> all
>>>> system.
IMHO, I think the fact ALL commands have the ability to create VDSM 
async tasks (even nowadays, regardless of the serial execution 
suggestion) - is bad.
IMHO, A command should use AsyncTaskManager to create/manage tasks
>>>
>>> That is how the task management was done.  Again, if you feel it
>>> should only affect storage flows, feel free to push it down into
>>> StorageCommandHandlingBase and then only storage verbs will have
>>> task management.

Who knows if storage will be the only case. See previous comment on 
where we should consider having task MGMT code.


>>>
>>>> Now regards architecture why I need some handler which will be
>>>> inside
>>>> a command
>>>> and will call for command methods? Please explain.
>>>
>>> As opposed to what?
>> So, u think it is a good design where we are using a "Circular
>> references"
>> design pattern. If yes, please elaborate.
>
> I'm saying nacking something without suggesting a better alternative is not good practice.
>
> Besides, the design is classic callback pattern (http://en.wikipedia.org/wiki/Callback_%28computer_programming%29) which is quite common and accepted.
> Regardless though, take a look at the new patches.
>
>>
>>>>
>>>>
>>>>>> This will occur all over the storage commands (which are the
>>>>>> only
>>>>>> usages of tasks nowadys).
>>>>>> Moreover, async task handling is done at the Commandbase
>>>>>> level
>>>>>> (see
>>>>>> the end* methods) - instead of hacking it in X different
>>>>>> places
>>>>>> whenever we need it, I'd prefer doing it once, properly.
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Engine-devel mailing list
>>>>>> Engine-devel at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>>>>
>>>>>
>>>>
>>>
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
>


From lhornyak at redhat.com  Thu Sep 27 07:06:04 2012
From: lhornyak at redhat.com (Laszlo Hornyak)
Date: Thu, 27 Sep 2012 03:06:04 -0400 (EDT)
Subject: [Engine-devel] Local build failing.
In-Reply-To: <20120926153544.Horde.TQ6K9pir309QY4NA5MXgVJA@imap.linux.ibm.com>
Message-ID: <1846799139.3113027.1348729564482.JavaMail.root@redhat.com>



----- Original Message -----
> From: snmishra at linux.vnet.ibm.com
> To: engine-devel at ovirt.org
> Sent: Thursday, September 27, 2012 12:35:44 AM
> Subject: [Engine-devel] Local build failing.
> 
> 
> Hi,
> 
>      My local build is failing today. I cloned the ovirt-engine repo
> and ran 'mvn clean install". Failed with following error -
> 
> Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile
> (default-compile) on project vdsbroker: Compilation failure
> [ERROR]
> /tmp/ovirt-engine/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/VdsManager.java:[604,44]

this file was last modified Wed Aug 15, looks like working. But that line should be fine even with java 1.6.

> illegal start of
> expression
> 
> 
>    I noticed that Jenkins build succeeded after this change was
>    merged.
> 
> -Sharad
> 
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 


From abaron at redhat.com  Thu Sep 27 07:17:17 2012
From: abaron at redhat.com (Ayal Baron)
Date: Thu, 27 Sep 2012 03:17:17 -0400 (EDT)
Subject: [Engine-devel] Serial Execution of Async Tasks
In-Reply-To: <5063E709.5070408@redhat.com>
Message-ID: <395810273.2915061.1348730237377.JavaMail.root@redhat.com>



----- Original Message -----
> 
> 
> On 09/27/2012 01:11 AM, Ayal Baron wrote:
> >
> >
> > ----- Original Message -----
> >>
> >>
> >> ----- Original Message -----
> >>> From: "Ayal Baron" <abaron at redhat.com>
> >>> To: "Michael Kublin" <mkublin at redhat.com>
> >>> Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel"
> >>> <engine-devel at ovirt.org>, "Eduardo Warszawski"
> >>> <ewarszaw at redhat.com>, "Allon Mureinik" <amureini at redhat.com>
> >>> Sent: Sunday, September 23, 2012 5:27:54 PM
> >>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> >>>
> >>>
> >>>
> >>> ----- Original Message -----
> >>>>
> >>>>
> >>>> ----- Original Message -----
> >>>>> From: "Ayal Baron" <abaron at redhat.com>
> >>>>> To: "Allon Mureinik" <amureini at redhat.com>, "Michael Kublin"
> >>>>> <mkublin at redhat.com>
> >>>>> Cc: "Liron Aravot" <laravot at redhat.com>, "engine-devel"
> >>>>> <engine-devel at ovirt.org>, "Eduardo Warszawski"
> >>>>> <ewarszaw at redhat.com>
> >>>>> Sent: Sunday, September 23, 2012 1:10:07 PM
> >>>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> >>>>>
> >>>>>
> >>>>>
> >>>>> ----- Original Message -----
> >>>>>>
> >>>>>>
> >>>>>> ----- Original Message -----
> >>>>>>> From: "Michael Kublin" <mkublin at redhat.com>
> >>>>>>> To: "Allon Mureinik" <amureini at redhat.com>
> >>>>>>> Cc: "Eduardo Warszawski" <ewarszaw at redhat.com>, "Liron
> >>>>>>> Aravot"
> >>>>>>> <laravot at redhat.com>, "Maor Lipchuk"
> >>>>>>> <mlipchuk at redhat.com>, "engine-devel"
> >>>>>>> <engine-devel at ovirt.org>
> >>>>>>> Sent: Sunday, September 23, 2012 12:41:05 PM
> >>>>>>> Subject: Re: [Engine-devel] Serial Execution of Async Tasks
> >>>>>>>
> >>>>>>>
> >>>>>>>>>>>>>>>>> Hi guys,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> As you may know the engine currently has the
> >>>>>>>>>>>>>>>>> ability
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> fire
> >>>>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>> SPM
> >>>>>>>>>>>>>>>>> task, and be asynchronously be "woken-up" when
> >>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>> ends.
> >>>>>>>>>>>>>>>>> This is great, but we found the for the Live
> >>>>>>>>>>>>>>>>> Storage
> >>>>>>>>>>>>>>>>> Migration
> >>>>>>>>>>>>>>>>> feature we need something a bit complex - the
> >>>>>>>>>>>>>>>>> ability
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>> series of async tasks in a single control flow.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Here's my initial design for this, your comments
> >>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>> criticism
> >>>>>>>>>>>>>>>>> would
> >>>>>>>>>>>>>>>>> be welcome:
> >>>>>>>>>>>>>>>>> http://wiki.ovirt.org/wiki/Features/Serial_Execution_of_Asynchronous_Tasks_Detailed_Design
> >>>>>>>>>>>>>>> -successful execution -
> >>>>>>>>>>>>>>> * "CommandBase iterates over its
> >>>>>>>>>>>>>>> SPMAsyncTaskHandlers"
> >>>>>>>>>>>>>>> -
> >>>>>>>>>>>>>>> when?
> >>>>>>>>>>>>>> This is the new suggested format of
> >>>>>>>>>>>>>> executeCommand().
> >>>>>>>>>>>>>> I'll
> >>>>>>>>>>>>>> clarify
> >>>>>>>>>>>>>> this too.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * If the second task is an HSM command (vs. SPM
> >>>>>>>>>>>>>>> command),
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>> think you
> >>>>>>>>>>>>>>> should explain in the design how to handle such
> >>>>>>>>>>>>>>> flows
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>> well.
> >>>>>>>>>>>>>> HSM commands do not create AsyncTasks, as they do
> >>>>>>>>>>>>>> today
> >>>>>>>>>>>>>> -
> >>>>>>>>>>>>>> I
> >>>>>>>>>>>>>> will
> >>>>>>>>>>>>>> clarify this.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> * Why do we need before task? can you give a
> >>>>>>>>>>>>>>> concrete
> >>>>>>>>>>>>>>> example
> >>>>>>>>>>>>>>> of what
> >>>>>>>>>>>>>>> would you do in such a method.
> >>>>>>>>>>>>>> Basically, /today/, command look like this:
> >>>>>>>>>>>>>> executeCommand() {
> >>>>>>>>>>>>>>      doStuffInTheDB();
> >>>>>>>>>>>>>>      runVdsCommand(someCommand);
> >>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> endSuccessfully() {
> >>>>>>>>>>>>>>      doMoreStuffInTheDB();
> >>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> endWithFailure() {
> >>>>>>>>>>>>>>      doMoreStuffForFailureInTheDB();
> >>>>>>>>>>>>>> }
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> In the new design, the entire doStuffInTheDB()
> >>>>>>>>>>>>>> should
> >>>>>>>>>>>>>> be
> >>>>>>>>>>>>>> moved
> >>>>>>>>>>>>>> to a
> >>>>>>>>>>>>>> breforeTask of the (only) SPMAsyncTaskHandler.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - I see you added SPMAsyncTaskHandler, any reason
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> use
> >>>>>>>>>>>>>>> SPMAsyncTasK to manage it own life-cycle?
> >>>>>>>>>>>>>> Conserving today's design - The SPMAsyncTaskHandler
> >>>>>>>>>>>>>> is
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>> place to
> >>>>>>>>>>>>>> add additional, non-SPM, logic around the SPM task
> >>>>>>>>>>>>>> execution,
> >>>>>>>>>>>>>> like
> >>>>>>>>>>>>>> CommandBase allows today.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - In the life-cycle managed by the
> >>>>>>>>>>>>>>> SPMAsyncTaskHandler
> >>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>> is a
> >>>>>>>>>>>>>>> step
> >>>>>>>>>>>>>>> 'createTask - how to create the async task' can
> >>>>>>>>>>>>>>> you
> >>>>>>>>>>>>>>> please
> >>>>>>>>>>>>>>> elaborate
> >>>>>>>>>>>>>>> what are the options.
> >>>>>>>>>>>>>> new [any type of async task]
> >>>>>>>
> >>>>>>> (I cleaned thread a little.)
> >>>>>>> The following design and it is implementation
> >>>>>>> http://gerrit.ovirt.org/#/c/7956/
> >>>>>>> is bad.
> >>>>>>> I don't see any reason for creating a new
> >>>>>>> SPMAsyncTaskHandler
> >>>>>>> and
> >>>>>>> especially in the
> >>>>>>> way as it's done in patch.
> >>>>>>> The reason are following:
> >>>>>>> 1. Performance , increased memory footprint, created CYCLIC
> >>>>>>> REFERENCE.
> >>>>>>> 2. Readability and robust of code: the code which is
> >>>>>>> written
> >>>>>>> as
> >>>>>>> cyclic references is unreadable
> >>>>>>>     and difficult for debug.
> >>>>>>> 3. Why I need a generic implementation and changes all over
> >>>>>>> whole
> >>>>>>> project because of
> >>>>>>>     series of async commands, for me it is a private case?
> >>>>>
> >>>>> What is the private case here exactly?
> >>>>> Every task can have multiple jobs.  We've identified several
> >>>>> such
> >>>>> places (e.g. live storage migration, move disk, move vm) and I
> >>>>> have
> >>>>> no doubt more will popup.
> >>>>> As Allon notes below, task handling is done at CommandBase, if
> >>>>> you
> >>>>> think task management should be for storage only, you're
> >>>>> welcome
> >>>>> to
> >>>>> push it down to StorageHandlingCommandBase (or get rid of
> >>>>> inheritance here altogether).
> >>>> Interesting , regards cyclic reference no response, but who
> >>>> cares,
> >>>> it is difficult to answer , that's why better not to response?
> >>>
> >>> There is no problem with cyclic references in general, GCs know
> >>> how
> >>> to deal with these just fine and in this case it's limited to the
> >>> command and its handlers.
> >>> I did not reply, however, as I do not feel strongly about this.
> >>>
> >>>> Regards private case:
> >>>> 1. We have command that not creating any task
> >>>> 2. We have command that will create a one task.
> >>>> 3. And we have 3 commands meanwhile which will create more than
> >>>> one
> >>>> task.
> >>>> I think that 3 is private case and not common? (In the future, I
> >>>
> >>> once happens
> >>> twice is a coincidence
> >>> three times is a method
> >>>
> >>> But if you insist on more then it's easy enough.  We've discussed
> >>> many times in the past that we need to change many of the storage
> >>> verbs to run asynchronously (e.g. createStorageDomain) once this
> >>> happens then existing flows would have to run multiple async
> >>> tasks
> >>> serially.
> >>>
> >>>> removed too many
> >>>> lines of code that were preparation for future that never come)
> >>>
> >>> This is not in preparation for the future, it is for a feature
> >>> we're
> >>> working on right now (live storage migration) and for fixing move
> >>> disk on which we have several bugs pending.
> >>>
> >>>> The handling done at CommandBase it means that it is influence
> >>>> all
> >>>> system.
> IMHO, I think the fact ALL commands have the ability to create VDSM
> async tasks (even nowadays, regardless of the serial execution
> suggestion) - is bad.
> IMHO, A command should use AsyncTaskManager to create/manage tasks

Fine by me, but that's a whole different ball game and has nothing to do with the current feature.

> >>>
> >>> That is how the task management was done.  Again, if you feel it
> >>> should only affect storage flows, feel free to push it down into
> >>> StorageCommandHandlingBase and then only storage verbs will have
> >>> task management.
> 
> Who knows if storage will be the only case. See previous comment on
> where we should consider having task MGMT code.

I agree it should be split out of the commands altogether, as most of the code in commands should (in fact I see no justification for the command pattern in most places in the code at all).
But that is a much bigger change and requires a ton of refactoring.

> 
> 
> >>>
> >>>> Now regards architecture why I need some handler which will be
> >>>> inside
> >>>> a command
> >>>> and will call for command methods? Please explain.
> >>>
> >>> As opposed to what?
> >> So, u think it is a good design where we are using a "Circular
> >> references"
> >> design pattern. If yes, please elaborate.
> >
> > I'm saying nacking something without suggesting a better
> > alternative is not good practice.
> >
> > Besides, the design is classic callback pattern
> > (http://en.wikipedia.org/wiki/Callback_%28computer_programming%29)
> > which is quite common and accepted.
> > Regardless though, take a look at the new patches.
> >
> >>
> >>>>
> >>>>
> >>>>>> This will occur all over the storage commands (which are the
> >>>>>> only
> >>>>>> usages of tasks nowadys).
> >>>>>> Moreover, async task handling is done at the Commandbase
> >>>>>> level
> >>>>>> (see
> >>>>>> the end* methods) - instead of hacking it in X different
> >>>>>> places
> >>>>>> whenever we need it, I'd prefer doing it once, properly.
> >>>>>>>
> >>>>>>>
> >>>>>> _______________________________________________
> >>>>>> Engine-devel mailing list
> >>>>>> Engine-devel at ovirt.org
> >>>>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> >
> 


From rgolan at redhat.com  Thu Sep 27 10:12:05 2012
From: rgolan at redhat.com (Roy Golan)
Date: Thu, 27 Sep 2012 12:12:05 +0200
Subject: [Engine-devel] Local build failing.
In-Reply-To: <1846799139.3113027.1348729564482.JavaMail.root@redhat.com>
References: <1846799139.3113027.1348729564482.JavaMail.root@redhat.com>
Message-ID: <50642675.9000303@redhat.com>

On 09/27/2012 09:06 AM, Laszlo Hornyak wrote:
>
> ----- Original Message -----
>> From: snmishra at linux.vnet.ibm.com
>> To: engine-devel at ovirt.org
>> Sent: Thursday, September 27, 2012 12:35:44 AM
>> Subject: [Engine-devel] Local build failing.
>>
>>
>> Hi,
>>
>>       My local build is failing today. I cloned the ovirt-engine repo
>> and ran 'mvn clean install". Failed with following error -
>>
>> Failed to execute goal
>> org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile
>> (default-compile) on project vdsbroker: Compilation failure
>> [ERROR]
>> /tmp/ovirt-engine/backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/VdsManager.java:[604,44]
> this file was last modified Wed Aug 15, looks like working. But that line should be fine even with java 1.6.
>
>> illegal start of
>> expression
>>
>>
>>     I noticed that Jenkins build succeeded after this change was
>>     merged.
>>
>> -Sharad
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
also reported by fsimonce
http://gerrit.ovirt.org/#/c/8235/


From alonbl at redhat.com  Sun Sep 30 08:27:00 2012
From: alonbl at redhat.com (Alon Bar-Lev)
Date: Sun, 30 Sep 2012 04:27:00 -0400 (EDT)
Subject: [Engine-devel] [ATTENTION] Engine now uses PKCS#12 format to store
	private keys
In-Reply-To: <1123520528.1172148.1348992937576.JavaMail.root@redhat.com>
Message-ID: <630488619.1172228.1348993620855.JavaMail.root@redhat.com>

Hello All,

We committed a change in the method engine uses to store private keys.

So far the engine used Java proprietary JKS format, this format enabled only Java applications to access the keys, and made it hard to manipulate them using external programs.

>From now the engine is using the standard PKCS#12 format to store keys and associated certificate chain.

PKCS#12 format is standard and supported by many applications, and it allowed simpler enrollment procedure.

We also issue different certificate and key to be used as engine authentication (SSH, VDSM), and to be used for engine web interface (HTTPS). This change has two reasons:

1. Allow simpler migration to 3rd party certificate for the web interface.

2. Separate between different private key usages (signature and key exchange).

engine-upgrade script has been modified to upgrade the environment to the new state.

Please CC me for every issue you may experience.

Regards,
Alon.


From amureini at redhat.com  Sun Sep 30 08:47:34 2012
From: amureini at redhat.com (Allon Mureinik)
Date: Sun, 30 Sep 2012 04:47:34 -0400 (EDT)
Subject: [Engine-devel] Running test suites on other OSs
In-Reply-To: <862901001.3786928.1348994425585.JavaMail.root@redhat.com>
Message-ID: <761126070.3787482.1348994854627.JavaMail.root@redhat.com>

Hi guys,

After some on-and-off work over the last week (thanks Daniel!), as of this morning, engine's test suite runs successfully on a couple of non-fedora/rhel OSs[1].
The easier to get/build/test the code, the more likely we'll be to get more developers on board, so let's try and keep it this way.

How to achieve this, in one sentence: "Assumption is the mother of all screw-ups"

Some slightly more detailed pointers:
1. File separators: Remember File.separatorChar? good.
2. TimeZones: If your test implicitly assumes a time zone (i.e., has anything to do with date formatting, DST changes, etc.), set it explicitly - don't assume that everyone's env shares your timezone
3. org.junit.Assume is your friend


Thanks,
Allon

[1] Don't ask, don't tell: http://gerrit.ovirt.org/#/c/8192/5