[Engine-devel] [ANN] New development environment for ovirt-engine

[Alon Bar-Lev]

----- Original Message -----
> From: "Eli Mesika" <emesika at redhat.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "engine-devel" <engine-devel at ovirt.org>, "arch" <arch at ovirt.org>
> Sent: Tuesday, May 14, 2013 3:45:41 AM
> Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
> 
> 
> 
> ----- Original Message -----
> > From: "Alon Bar-Lev" <alonbl at redhat.com>
> > To: "engine-devel" <engine-devel at ovirt.org>
> > Cc: "arch" <arch at ovirt.org>, "Sharad Mishra" <snmishra at us.ibm.com>, "Limor
> > Gavish" <lgavish at gmail.com>
> > Sent: Sunday, May 12, 2013 2:52:51 PM
> > Subject: [Engine-devel] [ANN] New development environment for ovirt-engine
> > 
> > Hello all ovirt-engine developers,
> > 
> > When I first joined the ovirt project, it took me about two weeks to setup
> > a
> > development environment, I needed to work on a bug related to host-deploy
> > so
> > I needed an environment that could use the ssh, PKI, vdsm-bootstrap and
> > communicate with vdsm using SSL, this was virtually impossible to do so
> > without tweaking the product in a way that it is so different from
> > production use, that I cannot guarantee that whatever tested in development
> > will actually work in production.
> > 
> > I peeked at the installation script in a hope that I can create partial
> > environment similar to production, but I found that the packaging
> > implementation makes to much assumption and is very difficult to adopt. The
> > fact that I do not use fedora/rhel for my development made it even worse.
> > 
> > I had no other option than to create rpms after each of my changes and test
> > each in real production like setup.
> > 
> > It was obvious to me that the manual customization of developers to achieve
> > working product will eventually break as product grow and move away from
> > being developer friendly to production friendly. For example, product
> > defaults cannot be these which serve developers, but these which serve
> > production the best, or having a valid PKI setup cannot be optional any
> > more
> > as components do need to use it. Same for location of files and
> > configuration, for example, if we write a pluggable infrastructure for
> > branding, we cannot damage the interface just because developers runs the
> > product in their own manual customization.
> > 
> > I took the opportunity handed to me to port the ovirt-engine to other
> > distributions in order to provide a development environment that is similar
> > to production setup. Together with Sandro Bonazzola and Alex Lourie we
> > re-wrote the whole installation of the product which can also be used to
> > setup the desired development environment.
> > 
> > Within this environment the product is set up using the same tools and
> > configuration as in production, while the process does not require special
> > privileges nor changes the state of the developer machine.
> > 
> > A complete documentation is available[1], I preferred to use README within
> > the source tree as wiki tend to quickly become obsolete, while
> > documentation
> > within source tree can be modified by the commit that introduces a change.
> > I
> > will redirect to this file from the current wiki once the site will be up.
> > 
> > In a nut shell, after installing prerequisites, build and install the
> > product
> > using:
> > 
> > $ make clean install-dev PREFIX=$HOME/ovirt-engine
> > 
> > This will run maven and create product installation at $HOME/ovirt-engine
> > Next, a setup phase is required just like in production, to initialize
> > configuration and database:
> > 
> > $ $HOME/ovirt-engine/bin/engine-setup-2
> > 
> > You have now fully functional product, including PKI, SSL, host-deploy,
> > tools.
> > No manual database updates are required, no lose of functionality.
> > 
> > All that is left is to start the engine service:
> > 
> > $ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
> > 
> > Access to application:
> >    http://localhost:8080
> >    https://localhost:8443
> > Debugging port is opened at port 8787.
> > 
> > Farther information exists in the documentation[1].
> > 
> > There are several inherit benefits of the new environment, the major one is
> > the ability to manage several environments in parallel on the same host.
> > For
> > example, if we develop two separate features on two branches we can install
> > the product into $HOME/ovirt-engine-feature1 and
> > $HOME/ovirt-engine-feature-2 and have a separate database for each, if we
> > modify the ports jboss is listening to we can run two instances of engine
> > at
> > the same time!
> 
> It is not clear to me why working on 2 bugs needs 2 installations of the
> development environment.
> If you have 2 different git branches and a separate database for each, its
> enough , am I missing something ?

You have two different git branches, so you have two different binaries out of the build, right?
These binaries should reside some where.
Currently you are probably use maven deploy or something similar into single jboss, without proper environment, overriding each time the other branch's binaries.
What if there is a change in configuration between the two? more prerequisites, different service configuration? different database schema?

To make it easy, you can just duplicate the development environment, and be sure you are up and running.

If you still want to take the chance you can... just make install-dev PREFIX=$HOME/same-place and you will have the behavior of overriding the same environment, and hoping for the best.

> I was used to create a git branch with the name of the BZ# and use
> create_db.sh script to create a new database with the BZ# name.
> Is this possible in the new method?

Yes.
You create an empty database manually, let's say engine-branch1.
Then you install the environment to own location, let say make install-dev PREFIX=$HOME/ovirt-engine-branch1
When running engine-setup-2 you instruct the setup to use engine-branch1 database.
This database will be created out of the new branch's create_db.sh.
create_db.sh should not run manually now.

However, if you insist of creating a database manually and modify an existing environment to use the new database, it is possible as well.
Edit $PREFIX/etc/ovirt-engine/engine.conf.d/10-setup-database.conf, and specify where database is, then start the engine service.

> Also, does this mean that I will have to create/configure a new workspace for
> eclipse each time I am starting to work on a new bug?

This I do not know. How does it work now when you switch branches?

> 
> 
> Thanks
> 
> 
> > 
> > We will be happy to work with all developers to assist in porting into the
> > new development environment, the simplest is to create a new database for
> > this effort. Moti has a sequence of converting the existing database owned
> > by postgres to be owned by the engine, Moti, can you please share that?
> > 
> > We are sure there are missing bits, we will be happy to know these so we
> > can
> > improve.
> > 
> > I am aware that developers (especially java) are conservative, but I ask
> > you
> > to give us a chance, so that we make it easy for developers to join the
> > project, and to allow us to drop the parallel effort of packaging to
> > production and fixing the broken development environment.
> > 
> > A special thanks to developers who took the time to test and provide
> > feedback
> > before the merged:
> > - Yaniv Bronheim
> > - Moti Asayag
> > - Limor Gavish
> > - Sharad Mishra
> > - Ofer Schreiber
> > 
> > We are hoping that after migration you will be find this environment useful
> > and friendly,
> > 
> > Sandro Bonazzola,
> > Alex Lourie,
> > Alon Bar-Lev.
> > 
> > [1]
> > http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;hb=HEAD
> > _______________________________________________
> > Engine-devel mailing list
> > Engine-devel at ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/engine-devel
> > 
>