[Engine-devel] [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers, When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production. I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse. I had no other option than to create rpms after each of my changes and test each in real production like setup. It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization. I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment. Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine. A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up. In a nut shell, after installing prerequisites, build and install the product using: $ make clean install-dev PREFIX=$HOME/ovirt-engine This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database: $ $HOME/ovirt-engine/bin/engine-setup-2 You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality. All that is left is to start the engine service: $ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787. Farther information exists in the documentation[1]. There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time! We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that? We are sure there are missing bits, we will be happy to know these so we can improve. I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment. A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber We are hoping that after migration you will be find this environment useful and friendly, Sandro Bonazzola, Alex Lourie, Alon Bar-Lev. [1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;...
Hello, As promised, I updated the wiki pages of engine developer environment to refer to this[1] single new page, I hope in time we can merge all non-trivial contributions into the README.developer. Feel free to contribute/fix as you experience issues. Regards, Alon Bar-Lev. [1] http://www.ovirt.org/OVirt_Engine_Development_Environment ----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "Yaniv Bronheim" <ybronhei@redhat.com>, "Moti Asayag" <masayag@redhat.com>, "Limor Gavish" <lgavish@gmail.com>, "Sharad Mishra" <snmishra@us.ibm.com>, "Alex Lourie" <alourie@redhat.com>, "Sandro Bonazzola" <sbonazzo@redhat.com>, "arch" <arch@ovirt.org>, "Ofer Schreiber" <oschreib@redhat.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;...
Good work guys, Thanks Barak Azulay On May 12, 2013, at 14:52, Alon Bar-Lev <alonbl@redhat.com> wrote:
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Arch mailing list Arch@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch
Has anyone tried using this setup with eclipse and running jboss server from eclipse? Thanks Sharad Mishra Quoting Barak Azulay <bazulay@redhat.com>:
Good work guys,
Thanks Barak Azulay
On May 12, 2013, at 14:52, Alon Bar-Lev <alonbl@redhat.com> wrote:
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Arch mailing list Arch@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: snmishra@linux.vnet.ibm.com To: engine-devel@ovirt.org Sent: Wednesday, May 15, 2013 11:28:31 PM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
Has anyone tried using this setup with eclipse and running jboss server from eclipse?
You probably don't need that. You can have eclipse connect to the running service using debug port at localhost:8787
Thanks Sharad Mishra
Quoting Barak Azulay <bazulay@redhat.com>:
Good work guys,
Thanks Barak Azulay
On May 12, 2013, at 14:52, Alon Bar-Lev <alonbl@redhat.com> wrote:
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Arch mailing list Arch@ovirt.org http://lists.ovirt.org/mailman/listinfo/arch
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "arch" <arch@ovirt.org>, "Sharad Mishra" <snmishra@us.ibm.com>, "Limor Gavish" <lgavish@gmail.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [Engine-devel] [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
It is not clear to me why working on 2 bugs needs 2 installations of the development environment. If you have 2 different git branches and a separate database for each, its enough , am I missing something ? I was used to create a git branch with the name of the BZ# and use create_db.sh script to create a new database with the BZ# name. Is this possible in the new method? Also, does this mean that I will have to create/configure a new workspace for eclipse each time I am starting to work on a new bug? Thanks
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Alon, I have FC17, and followed the steps at the wiki , i defined the ovirt nightly repo [ovirt-nightly] name=ovirt-nightly baseurl=http://resources.ovirt.org/releases/nightly/rpm/Fedora/17/ enabled=1 gpgcheck=0 priority=1 protect=1 And performed yum install according to your guidelines. It fails to find python-m2crypto Can you please advise on the matter? Many thanks, Yair ----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 3:45:41 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "arch" <arch@ovirt.org>, "Sharad Mishra" <snmishra@us.ibm.com>, "Limor Gavish" <lgavish@gmail.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [Engine-devel] [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
It is not clear to me why working on 2 bugs needs 2 installations of the development environment. If you have 2 different git branches and a separate database for each, its enough , am I missing something ? I was used to create a git branch with the name of the BZ# and use create_db.sh script to create a new database with the BZ# name. Is this possible in the new method? Also, does this mean that I will have to create/configure a new workspace for eclipse each time I am starting to work on a new bug?
Thanks
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Alon Bar-Lev" <alonbl@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 5:39:19 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
Alon, I have FC17, and followed the steps at the wiki , i defined the ovirt nightly repo
[ovirt-nightly] name=ovirt-nightly baseurl=http://resources.ovirt.org/releases/nightly/rpm/Fedora/17/ enabled=1 gpgcheck=0 priority=1 protect=1
And performed yum install according to your guidelines. It fails to find python-m2crypto
Has nothing to do with ovirt :) Try m2crypto please.
Can you please advise on the matter?
Many thanks, Yair
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 3:45:41 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "arch" <arch@ovirt.org>, "Sharad Mishra" <snmishra@us.ibm.com>, "Limor Gavish" <lgavish@gmail.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [Engine-devel] [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
It is not clear to me why working on 2 bugs needs 2 installations of the development environment. If you have 2 different git branches and a separate database for each, its enough , am I missing something ? I was used to create a git branch with the name of the BZ# and use create_db.sh script to create a new database with the BZ# name. Is this possible in the new method? Also, does this mean that I will have to create/configure a new workspace for eclipse each time I am starting to work on a new bug?
Thanks
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Eli Mesika" <emesika@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 8:58:06 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Alon Bar-Lev" <alonbl@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 5:39:19 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
Alon, I have FC17, and followed the steps at the wiki , i defined the ovirt nightly repo
[ovirt-nightly] name=ovirt-nightly baseurl=http://resources.ovirt.org/releases/nightly/rpm/Fedora/17/ enabled=1 gpgcheck=0 priority=1 protect=1
And performed yum install according to your guidelines. It fails to find python-m2crypto
Has nothing to do with ovirt :) Try m2crypto please.
Worked ! I would suggest updating the WIKI However, now I fail at webadmin - commit hash cda607c80a19dd08585fc0271ea7d57e03f9a43f [INFO] javac option: -s [INFO] javac option: /home/yzaslavs/work/ovirt_git/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/annotations [INFO] diagnostic /home/yzaslavs/work/ovirt_git/ovirt-engine/frontend/webadmin/modules/gwt-common/target/gwt-common-3.3.0-SNAPSHOT.jar(org/ovirt/engine/ui/common/presenter/AbstractPopupPresenterWidget.java):27: error: method getCloseButton() is already defined in interface org.ovirt.engine.ui.common.presenter.AbstractPopupPresenterWidget.ViewDef HasClickHandlers getCloseButton(); ^ [INFO] diagnostic /home/yzaslavs/work/ovirt_git/ovirt-engine/frontend/webadmin/modules/gwt-common/target/gwt-common-3.3.0-SNAPSHOT.jar(org/ovirt/engine/ui/common/presenter/AbstractPopupPresenterWidget.java):32: error: method getCloseIconButton() is already defined in interface org.ovirt.engine.ui.common.presenter.AbstractPopupPresenterWidget.ViewDef HasClickHandlers getCloseIconButton(); Anyone got a clue?
Can you please advise on the matter?
Many thanks, Yair
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 3:45:41 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "arch" <arch@ovirt.org>, "Sharad Mishra" <snmishra@us.ibm.com>, "Limor Gavish" <lgavish@gmail.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [Engine-devel] [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
It is not clear to me why working on 2 bugs needs 2 installations of the development environment. If you have 2 different git branches and a separate database for each, its enough , am I missing something ? I was used to create a git branch with the name of the BZ# and use create_db.sh script to create a new database with the BZ# name. Is this possible in the new method? Also, does this mean that I will have to create/configure a new workspace for eclipse each time I am starting to work on a new bug?
Thanks
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com>, "Daniel Erez" <derez@redhat.com>, "Gilad Chaplik" <gchaplik@redhat.com> Cc: "Eli Mesika" <emesika@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 9:55:52 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Eli Mesika" <emesika@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 8:58:06 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Alon Bar-Lev" <alonbl@redhat.com>, "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 5:39:19 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
Alon, I have FC17, and followed the steps at the wiki , i defined the ovirt nightly repo
[ovirt-nightly] name=ovirt-nightly baseurl=http://resources.ovirt.org/releases/nightly/rpm/Fedora/17/ enabled=1 gpgcheck=0 priority=1 protect=1
And performed yum install according to your guidelines. It fails to find python-m2crypto
Has nothing to do with ovirt :) Try m2crypto please.
Worked ! I would suggest updating the WIKI However, now I fail at webadmin - commit hash cda607c80a19dd08585fc0271ea7d57e03f9a43f
Please open a new thread, this is unrelated.
[INFO] javac option: -s [INFO] javac option: /home/yzaslavs/work/ovirt_git/ovirt-engine/frontend/webadmin/modules/webadmin/target/generated-sources/annotations [INFO] diagnostic /home/yzaslavs/work/ovirt_git/ovirt-engine/frontend/webadmin/modules/gwt-common/target/gwt-common-3.3.0-SNAPSHOT.jar(org/ovirt/engine/ui/common/presenter/AbstractPopupPresenterWidget.java):27: error: method getCloseButton() is already defined in interface org.ovirt.engine.ui.common.presenter.AbstractPopupPresenterWidget.ViewDef HasClickHandlers getCloseButton(); ^ [INFO] diagnostic /home/yzaslavs/work/ovirt_git/ovirt-engine/frontend/webadmin/modules/gwt-common/target/gwt-common-3.3.0-SNAPSHOT.jar(org/ovirt/engine/ui/common/presenter/AbstractPopupPresenterWidget.java):32: error: method getCloseIconButton() is already defined in interface org.ovirt.engine.ui.common.presenter.AbstractPopupPresenterWidget.ViewDef HasClickHandlers getCloseIconButton();
Anyone got a clue?
Can you please advise on the matter?
Many thanks, Yair
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 3:45:41 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "arch" <arch@ovirt.org>, "Sharad Mishra" <snmishra@us.ibm.com>, "Limor Gavish" <lgavish@gmail.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [Engine-devel] [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
It is not clear to me why working on 2 bugs needs 2 installations of the development environment. If you have 2 different git branches and a separate database for each, its enough , am I missing something ? I was used to create a git branch with the name of the BZ# and use create_db.sh script to create a new database with the BZ# name. Is this possible in the new method? Also, does this mean that I will have to create/configure a new workspace for eclipse each time I am starting to work on a new bug?
Thanks
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 3:45:41 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "arch" <arch@ovirt.org>, "Sharad Mishra" <snmishra@us.ibm.com>, "Limor Gavish" <lgavish@gmail.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [Engine-devel] [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
It is not clear to me why working on 2 bugs needs 2 installations of the development environment. If you have 2 different git branches and a separate database for each, its enough , am I missing something ?
You have two different git branches, so you have two different binaries out of the build, right? These binaries should reside some where. Currently you are probably use maven deploy or something similar into single jboss, without proper environment, overriding each time the other branch's binaries. What if there is a change in configuration between the two? more prerequisites, different service configuration? different database schema? To make it easy, you can just duplicate the development environment, and be sure you are up and running. If you still want to take the chance you can... just make install-dev PREFIX=$HOME/same-place and you will have the behavior of overriding the same environment, and hoping for the best.
I was used to create a git branch with the name of the BZ# and use create_db.sh script to create a new database with the BZ# name. Is this possible in the new method?
Yes. You create an empty database manually, let's say engine-branch1. Then you install the environment to own location, let say make install-dev PREFIX=$HOME/ovirt-engine-branch1 When running engine-setup-2 you instruct the setup to use engine-branch1 database. This database will be created out of the new branch's create_db.sh. create_db.sh should not run manually now. However, if you insist of creating a database manually and modify an existing environment to use the new database, it is possible as well. Edit $PREFIX/etc/ovirt-engine/engine.conf.d/10-setup-database.conf, and specify where database is, then start the engine service.
Also, does this mean that I will have to create/configure a new workspace for eclipse each time I am starting to work on a new bug?
This I do not know. How does it work now when you switch branches?
Thanks
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 9:18:44 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Alon Bar-Lev" <alonbl@redhat.com> Cc: "engine-devel" <engine-devel@ovirt.org>, "arch" <arch@ovirt.org> Sent: Tuesday, May 14, 2013 3:45:41 AM Subject: Re: [Engine-devel] [ANN] New development environment for ovirt-engine
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "arch" <arch@ovirt.org>, "Sharad Mishra" <snmishra@us.ibm.com>, "Limor Gavish" <lgavish@gmail.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [Engine-devel] [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
It is not clear to me why working on 2 bugs needs 2 installations of the development environment. If you have 2 different git branches and a separate database for each, its enough , am I missing something ?
You have two different git branches, so you have two different binaries out of the build, right? These binaries should reside some where. Currently you are probably use maven deploy or something similar into single jboss, without proper environment, overriding each time the other branch's binaries. What if there is a change in configuration between the two? more prerequisites, different service configuration? different database schema?
To make it easy, you can just duplicate the development environment, and be sure you are up and running.
If you still want to take the chance you can... just make install-dev PREFIX=$HOME/same-place and you will have the behavior of overriding the same environment, and hoping for the best.
That's good , thanks
I was used to create a git branch with the name of the BZ# and use create_db.sh script to create a new database with the BZ# name. Is this possible in the new method?
Yes. You create an empty database manually, let's say engine-branch1. Then you install the environment to own location, let say make install-dev PREFIX=$HOME/ovirt-engine-branch1 When running engine-setup-2 you instruct the setup to use engine-branch1 database. This database will be created out of the new branch's create_db.sh. create_db.sh should not run manually now.
However, if you insist of creating a database manually and modify an existing environment to use the new database, it is possible as well. Edit $PREFIX/etc/ovirt-engine/engine.conf.d/10-setup-database.conf, and specify where database is, then start the engine service.
Great, thanks
Also, does this mean that I will have to create/configure a new workspace for eclipse each time I am starting to work on a new bug?
This I do not know. How does it work now when you switch branches?
I will check that, maybe this is unrelated ... Thanks again , will test that ASAP
Thanks
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;... _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "Yaniv Bronheim" <ybronhei@redhat.com>, "Moti Asayag" <masayag@redhat.com>, "Limor Gavish" <lgavish@gmail.com>, "Sharad Mishra" <snmishra@us.ibm.com>, "Alex Lourie" <alourie@redhat.com>, "Sandro Bonazzola" <sbonazzo@redhat.com>, "arch" <arch@ovirt.org>, "Ofer Schreiber" <oschreib@redhat.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
Reusing an existing DB schema requires a bit more work since the installation of dev-env advise using database as a user and not a superuser like 'postgres' user used to create the database originally. Therefore if wishes to use user 'engine' as in the instructions, there is a need to change the current schema owner and also the ownership of all of its objects. The easiest path I found for that purpose is: 1. Create a dump of the database using the script in [1]. 2. Rename the owner in the dump file to the new owner (s/OWNER TO postgres/OWNER TO engine/g). 3. Import the dump file to the new DB owned by the engine user using [2] (provide -r flag to drop the former db). [1] ovirt-engine/backend/manager/dbscripts/backup.sh [2] ovirt-engine/backend/manager/dbscripts/restore.sh
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1] http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;...
On Tue, May 14, 2013 at 1:54 PM, Moti Asayag <masayag@redhat.com> wrote:
----- Original Message -----
From: "Alon Bar-Lev" <alonbl@redhat.com> To: "engine-devel" <engine-devel@ovirt.org> Cc: "Yaniv Bronheim" <ybronhei@redhat.com>, "Moti Asayag" <masayag@redhat.com>, "Limor Gavish" <lgavish@gmail.com>, "Sharad Mishra" <snmishra@us.ibm.com>, "Alex Lourie" <alourie@redhat.com>, "Sandro Bonazzola" <sbonazzo@redhat.com>, "arch" <arch@ovirt.org>, "Ofer Schreiber" <oschreib@redhat.com> Sent: Sunday, May 12, 2013 2:52:51 PM Subject: [ANN] New development environment for ovirt-engine
Hello all ovirt-engine developers,
When I first joined the ovirt project, it took me about two weeks to setup a development environment, I needed to work on a bug related to host-deploy so I needed an environment that could use the ssh, PKI, vdsm-bootstrap and communicate with vdsm using SSL, this was virtually impossible to do so without tweaking the product in a way that it is so different from production use, that I cannot guarantee that whatever tested in development will actually work in production.
I peeked at the installation script in a hope that I can create partial environment similar to production, but I found that the packaging implementation makes to much assumption and is very difficult to adopt. The fact that I do not use fedora/rhel for my development made it even worse.
I had no other option than to create rpms after each of my changes and test each in real production like setup.
It was obvious to me that the manual customization of developers to achieve working product will eventually break as product grow and move away from being developer friendly to production friendly. For example, product defaults cannot be these which serve developers, but these which serve production the best, or having a valid PKI setup cannot be optional any more as components do need to use it. Same for location of files and configuration, for example, if we write a pluggable infrastructure for branding, we cannot damage the interface just because developers runs the product in their own manual customization.
I took the opportunity handed to me to port the ovirt-engine to other distributions in order to provide a development environment that is similar to production setup. Together with Sandro Bonazzola and Alex Lourie we re-wrote the whole installation of the product which can also be used to setup the desired development environment.
Within this environment the product is set up using the same tools and configuration as in production, while the process does not require special privileges nor changes the state of the developer machine.
A complete documentation is available[1], I preferred to use README within the source tree as wiki tend to quickly become obsolete, while documentation within source tree can be modified by the commit that introduces a change. I will redirect to this file from the current wiki once the site will be up.
In a nut shell, after installing prerequisites, build and install the product using:
$ make clean install-dev PREFIX=$HOME/ovirt-engine
This will run maven and create product installation at $HOME/ovirt-engine Next, a setup phase is required just like in production, to initialize configuration and database:
$ $HOME/ovirt-engine/bin/engine-setup-2
You have now fully functional product, including PKI, SSL, host-deploy, tools. No manual database updates are required, no lose of functionality.
All that is left is to start the engine service:
$ $HOME/ovirt-engine/share/ovirt-engine/services/ovirt-engine.py start
Access to application: http://localhost:8080 https://localhost:8443 Debugging port is opened at port 8787.
Farther information exists in the documentation[1].
There are several inherit benefits of the new environment, the major one is the ability to manage several environments in parallel on the same host. For example, if we develop two separate features on two branches we can install the product into $HOME/ovirt-engine-feature1 and $HOME/ovirt-engine-feature-2 and have a separate database for each, if we modify the ports jboss is listening to we can run two instances of engine at the same time!
We will be happy to work with all developers to assist in porting into the new development environment, the simplest is to create a new database for this effort. Moti has a sequence of converting the existing database owned by postgres to be owned by the engine, Moti, can you please share that?
Reusing an existing DB schema requires a bit more work since the installation of dev-env advise using database as a user and not a superuser like 'postgres' user used to create the database originally.
We no longer (since 3.1) create the DB in production with user postgres. All the DB operations are done with application user (engine by default), especially with remote DB installations. The application user doesn't have superuser privileges, only the ones required for DB creation and removal.
Therefore if wishes to use user 'engine' as in the instructions, there is a need to change the current schema owner and also the ownership of all of its objects.
The easiest path I found for that purpose is: 1. Create a dump of the database using the script in [1]. 2. Rename the owner in the dump file to the new owner (s/OWNER TO postgres/OWNER TO engine/g). 3. Import the dump file to the new DB owned by the engine user using [2] (provide -r flag to drop the former db).
[1] ovirt-engine/backend/manager/dbscripts/backup.sh [2] ovirt-engine/backend/manager/dbscripts/restore.sh
We are sure there are missing bits, we will be happy to know these so we can improve.
I am aware that developers (especially java) are conservative, but I ask you to give us a chance, so that we make it easy for developers to join the project, and to allow us to drop the parallel effort of packaging to production and fixing the broken development environment.
A special thanks to developers who took the time to test and provide feedback before the merged: - Yaniv Bronheim - Moti Asayag - Limor Gavish - Sharad Mishra - Ofer Schreiber
We are hoping that after migration you will be find this environment useful and friendly,
Sandro Bonazzola, Alex Lourie, Alon Bar-Lev.
[1]
http://gerrit.ovirt.org/gitweb?p=ovirt-engine.git;a=blob;f=README.developer;...
participants (7)
-
Alex Lourie -
Alon Bar-Lev -
Barak Azulay -
Eli Mesika -
Moti Asayag -
snmishraï¼ linux.vnet.ibm.com -
Yair Zaslavsky