[Engine-devel] [ANN] New development environment for ovirt-engine

Tue May 14 10:32:39 UTC 2013

----- Original Message -----
> From: "Alon Bar-Lev" <alonbl at redhat.com>
> To: "Eli Mesika" <emesika at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>, "arch" <arch at ovirt.org>
> Sent: Tuesday, May 14, 2013 9:18:44 AM
> Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
> 
> 
> 
> ----- Original Message -----
> > From: "Eli Mesika" <emesika at redhat.com>
> > To: "Alon Bar-Lev" <alonbl at redhat.com>
> > Cc: "engine-devel" <engine-devel at ovirt.org>, "arch" <arch at ovirt.org>
> > Sent: Tuesday, May 14, 2013 3:45:41 AM
> > Subject: Re: [Engine-devel] [ANN] New development environment for
> > ovirt-engine
> > 
> > 
> > 
> > ----- Original Message -----
> > > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > > To: "engine-devel" <engine-devel at ovirt.org>
> > > Cc: "arch" <arch at ovirt.org>, "Sharad Mishra" <snmishra at us.ibm.com>,
> > > "Limor
> > > Gavish" <lgavish at gmail.com>
> > > Sent: Sunday, May 12, 2013 2:52:51 PM
> > > Subject: [Engine-devel] [ANN] New development environment for
> > > ovirt-engine
> > > 
> > > Hello all ovirt-engine developers,
> > > 
> > > When I first joined the ovirt project, it took me about two weeks to
> > > setup
> > > a
> > > development environment, I needed to work on a bug related to host-deploy
> > > so
> > > I needed an environment that could use the ssh, PKI, vdsm-bootstrap and
> > > communicate with vdsm using SSL, this was virtually impossible to do so
> > > without tweaking the product in a way that it is so different from
> > > production use, that I cannot guarantee that whatever tested in
> > > development
> > > will actually work in production.
> > > 
> > > I peeked at the installation script in a hope that I can create partial
> > > environment similar to production, but I found that the packaging
> > > implementation makes to much assumption and is very difficult to adopt.
> > > The
> > > fact that I do not use fedora/rhel for my development made it even worse.
> > > 
> > > I had no other option than to create rpms after each of my changes and
> > > test
> > > each in real production like setup.
> > > 
> > > It was obvious to me that the manual customization of developers to
> > > achieve
> > > working product will eventually break as product grow and move away from
> > > being developer friendly to production friendly. For example, product
> > > defaults cannot be these which serve developers, but these which serve
> > > production the best, or having a valid PKI setup cannot be optional any
> > > more
> > > as components do need to use it. Same for location of files and
> > > configuration, for example, if we write a pluggable infrastructure for
> > > branding, we cannot damage the interface just because developers runs the
> > > product in their own manual customization.
> > > 
> > > I took the opportunity handed to me to port the ovirt-engine to other
> > > distributions in order to provide a development environment that is
> > > similar
> > > to production setup. Together with Sandro Bonazzola and Alex Lourie we
> > > re-wrote the whole installation of the product which can also be used to
> > > setup the desired development environment.
> > > 
> > > Within this environment the product is set up using the same tools and
> > > configuration as in production, while the process does not require
> > > special
> > > privileges nor changes the state of the developer machine.
> > > 
> > > A complete documentation is available[1], I preferred to use README
> > > within
> > > the source tree as wiki tend to quickly become obsolete, while
> > > documentation
> > > within source tree can be modified by the commit that introduces a
> > > change.
> > > I
> > > will redirect to this file from the current wiki once the site will be
> > > up.
> > > 
> > > In a nut shell, after installing prerequisites, build and install the
> > > product
> > > using:
> > > 
> > > $ make clean install-dev PREFIX=$HOME/ovirt-engine
> > > 
> > > This will run maven and create product installation at $HOME/ovirt-engine
> > > Next, a setup phase is required just like in production, to initialize
> > > configuration and database:
> > > 
> > > $ $HOME/ovirt-engine/bin/engine-setup-2
> > > 
> > > You have now fully functional product, including PKI, SSL, host-deploy,
> > > tools.
> > > No manual database updates are required, no lose of functionality.
> > > 
> > > All that is left is to start the engine service:
> > > 
> > > $ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
> > > 
> > > Access to application:
> > >    http://localhost:8080
> > >    https://localhost:8443
> > > Debugging port is opened at port 8787.
> > > 
> > > Farther information exists in the documentation[1].
> > > 
> > > There are several inherit benefits of the new environment, the major one
> > > is
> > > the ability to manage several environments in parallel on the same host.
> > > For
> > > example, if we develop two separate features on two branches we can
> > > install
> > > the product into $HOME/ovirt-engine-feature1 and
> > > $HOME/ovirt-engine-feature-2 and have a separate database for each, if we
> > > modify the ports jboss is listening to we can run two instances of engine
> > > at
> > > the same time!
> > 
> > It is not clear to me why working on 2 bugs needs 2 installations of the
> > development environment.
> > If you have 2 different git branches and a separate database for each, its
> > enough , am I missing something ?
> 
> You have two different git branches, so you have two different binaries out
> of the build, right?
> These binaries should reside some where.
> Currently you are probably use maven deploy or something similar into single
> jboss, without proper environment, overriding each time the other branch's
> binaries.
> What if there is a change in configuration between the two? more
> prerequisites, different service configuration? different database schema?
> 
> To make it easy, you can just duplicate the development environment, and be
> sure you are up and running.
> 
> If you still want to take the chance you can... just make install-dev
> PREFIX=$HOME/same-place and you will have the behavior of overriding the
> same environment, and hoping for the best.

That's good , thanks

> 
> > I was used to create a git branch with the name of the BZ# and use
> > create_db.sh script to create a new database with the BZ# name.
> > Is this possible in the new method?
> 
> Yes.
> You create an empty database manually, let's say engine-branch1.
> Then you install the environment to own location, let say make install-dev
> PREFIX=$HOME/ovirt-engine-branch1
> When running engine-setup-2 you instruct the setup to use engine-branch1
> database.
> This database will be created out of the new branch's create_db.sh.
> create_db.sh should not run manually now.
> 
> However, if you insist of creating a database manually and modify an existing
> environment to use the new database, it is possible as well.
> Edit $PREFIX/etc/ovirt-engine/engine.conf.d/10-setup-database.conf, and
> specify where database is, then start the engine service.

Great, thanks

> 
> > Also, does this mean that I will have to create/configure a new workspace
> > for
> > eclipse each time I am starting to work on a new bug?
> 
> This I do not know. How does it work now when you switch branches?

I will check that, maybe this is unrelated ...

Thanks again , will test that ASAP
> 
> > 
> > 
> > Thanks
> > 
> > 
> > > 
> > > We will be happy to work with all developers to assist in porting into
> > > the
> > > new development environment, the simplest is to create a new database for
> > > this effort. Moti has a sequence of converting the existing database
> > > owned
> > > by postgres to be owned by the engine, Moti, can you please share that?
> > > 
> > > We are sure there are missing bits, we will be happy to know these so we
> > > can
> > > improve.
> > > 
> > > I am aware that developers (especially java) are conservative, but I ask
> > > you
> > > to give us a chance, so that we make it easy for developers to join the
> > > project, and to allow us to drop the parallel effort of packaging to
> > > production and fixing the broken development environment.
> > > 
> > > A special thanks to developers who took the time to test and provide
> > > feedback
> > > before the merged:
> > > - Yaniv Bronheim
> > > - Moti Asayag
> > > - Limor Gavish
> > > - Sharad Mishra
> > > - Ofer Schreiber
> > > 
> > > We are hoping that after migration you will be find this environment
> > > useful
> > > and friendly,
> > > 
> > > Sandro Bonazzola,
> > > Alex Lourie,
> > > Alon Bar-Lev.
> > > 
> > > [1]
> > > http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;hb=HEAD
> > > _______________________________________________
> > > Engine-devel mailing list
> > > Engine-devel at ovirt.org
> > > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > > 
> > 
> 