[Engine-devel] Migrating engine-setup to otopi
Hi All Recent development of the otopi [1] framework allows us to migrate the engine-setup, upgrade and cleanup (and potentially other) utilities to implementation as an otopi plugin. Potential benefits of such a move are: 1. Be able to port engine to other distributions. 2. Be able to install engine in a development mode. 3. Be able to customize installation easily. 4. Share installation of components (reports, dwh). 5. Modular implementation, reduce maintenance costs. 6. Code reuse of installer code for multiple purposes (host-deploy, enigne-setup). Currently we are in the process of creating a 'setup' plugin for the otopi, and the progress can be monitored at [2]. The current roadmap for the feature is as follows: 1. Recreate the configuration utilities as plugins in otopi. 2. Support side-by side installation using both the old and the new utilities. 3. Switch to the new utility when the confidence that it is on-par with an old one is high. Our goal is to have the new utilities ready for 3.3 release (at least for the step 2 in the roadmap). We'd like to hear as much feedback as possible, so we could address it as soon as possible. Thanks! [1] http://gerrit.ovirt.org/#/q/project:otopi,n,z [2] http://www.ovirt.org/Features/Otopi_Infra_Migration
On Thu, 14 Mar 2013 06:13:39 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
Hi All
Recent development of the otopi [1] framework allows us to migrate the engine-setup, upgrade and cleanup (and potentially other) utilities to implementation as an otopi plugin.
Potential benefits of such a move are:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts... http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh... Anyway, everything what is not RPM/YUM specific and more portable is good way... jbelka
----- Original Message -----
From: "Jiri Belka" <jbelka@redhat.com> To: "Alex Lourie" <alourie@redhat.com> Cc: engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 12:38:46 PM Subject: Re: [Users] Migrating engine-setup to otopi
On Thu, 14 Mar 2013 06:13:39 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
Hi All
Recent development of the otopi [1] framework allows us to migrate the engine-setup, upgrade and cleanup (and potentially other) utilities to implementation as an otopi plugin.
Potential benefits of such a move are:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts...
This is going away soon.
http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh...
Anyway, everything what is not RPM/YUM specific and more portable is good way...
jbelka _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Jiri ----- Original Message -----
From: "Jiri Belka" <jbelka@redhat.com> To: "Alex Lourie" <alourie@redhat.com> Cc: engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 12:38:46 PM Subject: Re: [Users] Migrating engine-setup to otopi
On Thu, 14 Mar 2013 06:13:39 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
Hi All
Recent development of the otopi [1] framework allows us to migrate the engine-setup, upgrade and cleanup (and potentially other) utilities to implementation as an otopi plugin.
Potential benefits of such a move are:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts...
http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh...
These usernames are not hard-coded. There are default values present which are kept for local installations, but with remote DB setup the user is prompted to provide a username of her/his own.
Anyway, everything what is not RPM/YUM specific and more portable is good way...
Thanks!
jbelka
-- Best regards, Alex Lourie Software Developer in Integration Red Hat
On Thu, 14 Mar 2013 07:06:04 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts...
http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh...
These usernames are not hard-coded. There are default values present which are kept for local installations, but with remote DB setup the user is prompted to provide a username of her/his own.
Not everywhere are postgresql dirs owned by postgres, on some BSDs it is _postgresql. jbelka
----- Original Message -----
From: "Jiri Belka" <jbelka@redhat.com> To: "Alex Lourie" <alourie@redhat.com> Cc: engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 1:47:12 PM Subject: Re: [Users] Migrating engine-setup to otopi
On Thu, 14 Mar 2013 07:06:04 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts...
http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh...
These usernames are not hard-coded. There are default values present which are kept for local installations, but with remote DB setup the user is prompted to provide a username of her/his own.
Not everywhere are postgresql dirs owned by postgres, on some BSDs it is _postgresql.
Right, as I said this is going away. I am porting this first to Gentoo, which is the most complex, then I will be able to provide debian based. For the postgres issue, I am against assuming local database and the configuration of the database it-self (hba, etc). Like in other products, the dba will create a user and a database with the user as an owner and provide us the user/password and database name, this method does not require privileged database user for product installation and working locally or remotely, and is portable. We will keep the functionality of system provisioning as an optional component exists in some distribution. Regards, Alon Bar-Lev
On Thu, Mar 14, 2013 at 07:52:31AM -0400, Alon Bar-Lev wrote:
----- Original Message -----
From: "Jiri Belka" <jbelka@redhat.com> To: "Alex Lourie" <alourie@redhat.com> Cc: engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 1:47:12 PM Subject: Re: [Users] Migrating engine-setup to otopi
On Thu, 14 Mar 2013 07:06:04 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts...
http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh...
These usernames are not hard-coded. There are default values present which are kept for local installations, but with remote DB setup the user is prompted to provide a username of her/his own.
Not everywhere are postgresql dirs owned by postgres, on some BSDs it is _postgresql.
Right, as I said this is going away.
I am porting this first to Gentoo, which is the most complex, then I will be able to provide debian based.
Would it be useful to start to provide an ovirt-overlay for this? I already started https://github.com/ekohl/ovirt-overlay where I packaged ovirt-shell and its dependencies (including ovirt-sdk-python). I'd be happy to extend this with more packages based on the info available on http://wiki.gentoo.org/wiki/OVirt.
----- Original Message -----
From: "Ewoud Kohl van Wijngaarden" <ewoud+ovirt@kohlvanwijngaarden.nl> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "Jiri Belka" <jbelka@redhat.com>, engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 2:06:01 PM Subject: Re: [Engine-devel] [Users] Migrating engine-setup to otopi
On Thu, Mar 14, 2013 at 07:52:31AM -0400, Alon Bar-Lev wrote:
----- Original Message -----
From: "Jiri Belka" <jbelka@redhat.com> To: "Alex Lourie" <alourie@redhat.com> Cc: engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 1:47:12 PM Subject: Re: [Users] Migrating engine-setup to otopi
On Thu, 14 Mar 2013 07:06:04 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts...
http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh...
These usernames are not hard-coded. There are default values present which are kept for local installations, but with remote DB setup the user is prompted to provide a username of her/his own.
Not everywhere are postgresql dirs owned by postgres, on some BSDs it is _postgresql.
Right, as I said this is going away.
I am porting this first to Gentoo, which is the most complex, then I will be able to provide debian based.
Would it be useful to start to provide an ovirt-overlay for this? I already started https://github.com/ekohl/ovirt-overlay where I packaged ovirt-shell and its dependencies (including ovirt-sdk-python). I'd be happy to extend this with more packages based on the info available on http://wiki.gentoo.org/wiki/OVirt.
Working on this[1] for some time, until I realized we need to rewrite the whole packaging... I will update you when I have something working. The gentoo wiki instruction should simply go aware, as it is manual installation. [1] https://github.com/alonbl/ovirt-overlay
On Thu, 14 Mar 2013 08:08:52 -0400 (EDT) Alon Bar-Lev <alonbl@redhat.com> wrote:
Working on this[1] for some time, until I realized we need to rewrite the whole packaging...
I will update you when I have something working. The gentoo wiki instruction should simply go aware, as it is manual installation.
Please remove absolute symlinks too, it's totally stupid. Some packaging tools scream a lot about symlinks pointing outside of fake root during packaging. I had to do stupid kung-fu for my WIP OpenBSD port[1] :D # make symlinks relative cd ${WRKDIST}/usr/local && \ for link in `find . -type l`; do \ dest=`stat -f %Y $${link}`; \ depth=`dirname $${link} | \ perl -p -e 's|^./||;s|[^/]+|..|g;'`; \ newdest=`echo $${dest} | \ perl -pe "s#/usr/local#$${depth}#;"`; \ ln -sf $${newdest} $${link}; \ done [1] https://github.com/jirib/openbsd-mystuff/blob/master/sysutils/ovirt/engine/M...
----- Original Message -----
From: "Jiri Belka" <jbelka@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 2:46:44 PM Subject: Re: [Engine-devel] [Users] Migrating engine-setup to otopi
On Thu, 14 Mar 2013 08:08:52 -0400 (EDT) Alon Bar-Lev <alonbl@redhat.com> wrote:
Working on this[1] for some time, until I realized we need to rewrite the whole packaging...
I will update you when I have something working. The gentoo wiki instruction should simply go aware, as it is manual installation.
Please remove absolute symlinks too, it's totally stupid. Some packaging tools scream a lot about symlinks pointing outside of fake root during packaging.
I had to do stupid kung-fu for my WIP OpenBSD port[1] :D
Not sure I agree... switching to relative symlinks is not doing a great work either, especially if we take the location from 3rd party, such as systemd or cron.
On 03/14/2013 01:52 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Jiri Belka" <jbelka@redhat.com> To: "Alex Lourie" <alourie@redhat.com> Cc: engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 1:47:12 PM Subject: Re: [Users] Migrating engine-setup to otopi
On Thu, 14 Mar 2013 07:06:04 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts...
http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh...
These usernames are not hard-coded. There are default values present which are kept for local installations, but with remote DB setup the user is prompted to provide a username of her/his own.
Not everywhere are postgresql dirs owned by postgres, on some BSDs it is _postgresql.
Right, as I said this is going away.
I am porting this first to Gentoo, which is the most complex, then I will be able to provide debian based.
For the postgres issue, I am against assuming local database and the configuration of the database it-self (hba, etc).
Like in other products, the dba will create a user and a database with the user as an owner and provide us the user/password and database name, this method does not require privileged database user for product installation and working locally or remotely, and is portable.
actually, I'm against assuming we need a dba for a local install. we need to keep deployment easy, not assume user should worry about the db at all (other than providing the password for it, since it is needed later). can you pleas explain the concern, and the suggested solution on how it will look to run engine-setup/engine-upgrade in your approach? Thanks, Itamar
We will keep the functionality of system provisioning as an optional component exists in some distribution.
Regards, Alon Bar-Lev _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "Jiri Belka" <jbelka@redhat.com>, engine-devel@ovirt.org, users@ovirt.org Sent: Friday, March 15, 2013 1:24:25 PM Subject: Re: [Engine-devel] [Users] Migrating engine-setup to otopi
On 03/14/2013 01:52 PM, Alon Bar-Lev wrote:
----- Original Message -----
From: "Jiri Belka" <jbelka@redhat.com> To: "Alex Lourie" <alourie@redhat.com> Cc: engine-devel@ovirt.org, users@ovirt.org Sent: Thursday, March 14, 2013 1:47:12 PM Subject: Re: [Users] Migrating engine-setup to otopi
On Thu, 14 Mar 2013 07:06:04 -0400 (EDT) Alex Lourie <alourie@redhat.com> wrote:
1. Be able to port engine to other distributions.
Really? Beside this topic I see hardcoded usernames in scripts...
http://gerrit.ovirt.org/#/c/12551/2/backend/manager/dbscripts/dbfunctions.sh...
These usernames are not hard-coded. There are default values present which are kept for local installations, but with remote DB setup the user is prompted to provide a username of her/his own.
Not everywhere are postgresql dirs owned by postgres, on some BSDs it is _postgresql.
Right, as I said this is going away.
I am porting this first to Gentoo, which is the most complex, then I will be able to provide debian based.
For the postgres issue, I am against assuming local database and the configuration of the database it-self (hba, etc).
Like in other products, the dba will create a user and a database with the user as an owner and provide us the user/password and database name, this method does not require privileged database user for product installation and working locally or remotely, and is portable.
actually, I'm against assuming we need a dba for a local install. we need to keep deployment easy, not assume user should worry about the db at all (other than providing the password for it, since it is needed later).
can you pleas explain the concern, and the suggested solution on how it will look to run engine-setup/engine-upgrade in your approach?
Let's start from the end... there will be no change (accept of maybe be one/two more prompts during installation) at rhel/centos/fedora. Now... People who wrote the current installation confused between host provisioning and database interaction. Another confusion out there is the wrong assumption that ovirt-engine owns the system it installed on, and can perform changes to shared system components such as apache and postgres. Both of these should not leak into other distributions as we port, oh well, as much as we can, in our current state. Host provisioning: 1. Check if postgres is installed on host. 2. Check if postgres already initialized its database and optionally perform initdb. 3. Start postgres service. 4. Mark postgres service to start at boot. 5. Open network access to postgres. 6. Set network authentication (pg_hba). 7. Set the dba (postgres) user password. 8. Create user for engine database. 9. Create database for engine. Database interaction: 1. Accept application user/password. Either: a2. Create schema. a3. Import data. Or: b2: Upgrade database. Now, let's assume we are going to support oracle or db2, will we do the host provisioning as part of our setup? well, we can provide a script to do that, but most likely the dba already know how to install and configure his database to be ready for the database interaction phase. Another issue is that in case of remote database, we are unable to perform the host provisioning anyway, making the local vs remote installation procedure different, so installation documentation becomes even more complex. And of course there is the issue of permissions, we cannot assume the user that is installing ovirt-engine have dba permissions to the database, especially if the database is remote. Well behaved database application will skip the host provisioning and perform the database interaction only. While the user will configure database and create the database for the product. We are going to be well behaved, this is mandatory for porting. There can be host provisioning plugin to optionally perform the provisioning phase, this is distribution dependent plugin, which will run only on supported distributions and the database is installed locally. Currently we will support the currently supported distributions, I am not sure it is wroth the effort to port this module. Example for user visible change if provisioning module is enabled: --- Database host [localhost]: localhost You have selected local database, setup can configure the database for ovirt-engine use, this includes system configuration and database creation. Do you wish installer to configure database, or you prefer to do so manually (configure, manual) [configure]: manual Please configure database to accept network password connections at pg_hba.conf: host all all 127.0.0.1/32 password host all all ::1/128 password Then create user and database, example: create user engine password 'engine'; create database engine owner engine; Database user: Database user password: Database name: --- Of course the provisioning module can be enforced to "configure", skipping this question, this can be done by installing configuration file as part of the branding. Comments? Alon
Hi all, The patch introducing the new implementation is now under review: http://gerrit.ovirt.org/14612 Il 14/03/2013 11:13, Alex Lourie ha scritto:
Hi All
Recent development of the otopi [1] framework allows us to migrate the engine-setup, upgrade and cleanup (and potentially other) utilities to implementation as an otopi plugin.
Potential benefits of such a move are:
1. Be able to port engine to other distributions. 2. Be able to install engine in a development mode. 3. Be able to customize installation easily. 4. Share installation of components (reports, dwh). 5. Modular implementation, reduce maintenance costs. 6. Code reuse of installer code for multiple purposes (host-deploy, enigne-setup).
Currently we are in the process of creating a 'setup' plugin for the otopi, and the progress can be monitored at [2]. The current roadmap for the feature is as follows:
1. Recreate the configuration utilities as plugins in otopi. 2. Support side-by side installation using both the old and the new utilities. 3. Switch to the new utility when the confidence that it is on-par with an old one is high.
Our goal is to have the new utilities ready for 3.3 release (at least for the step 2 in the roadmap).
We'd like to hear as much feedback as possible, so we could address it as soon as possible.
Thanks!
[1] http://gerrit.ovirt.org/#/q/project:otopi,n,z [2] http://www.ovirt.org/Features/Otopi_Infra_Migration
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
-- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com
participants (6)
-
Alex Lourie -
Alon Bar-Lev -
Ewoud Kohl van Wijngaarden -
Itamar Heim -
Jiri Belka -
Sandro Bonazzola