From mburns at redhat.com Fri Feb 3 19:30:32 2012 From: mburns at redhat.com (Mike Burns) Date: Fri, 03 Feb 2012 14:30:32 -0500 Subject: [node-devel] ovirt-node 2.2.2-2 is available Message-ID: <1328297432.2587.4.camel@mburns-laptop.usersys.redhat.com> ovirt-node-2.2.2-2 RPMs and image are available on ovirt.org. This build includes a single patch for installs on UEFI machines. http://ovirt.org/releases/nightly/binary/ovirt-node-image-2.2.2-2.1.fc16.iso http://ovirt.org/releases/nightly/src/ovirt-node-2.2.2.tar.gz http://ovirt.org/releases/nightly/fedora/16/ovirt-node-2.2.2-2.fc16.noarch.rpm http://ovirt.org/releases/nightly/fedora/16/ovirt-node-tools-2.2.2-2.fc16.noarch.rpm Mike From oschreib at redhat.com Tue Feb 7 11:05:49 2012 From: oschreib at redhat.com (Ofer Schreiber) Date: Tue, 07 Feb 2012 06:05:49 -0500 (EST) Subject: [node-devel] oVirt's First Release go/no go meeting Message-ID: <1d8aeedf-04a5-438e-9658-b5a7e30df8a0@zmail14.collab.prod.int.phx2.redhat.com> The following is a new meeting request: Subject: oVirt's First Release go/no go meeting Organizer: "Ofer Schreiber" Location: #ovirt on oftc Time: Tuesday, February 7, 2012, 5:00:00 PM - 6:00:00 PM GMT +02:00 Jerusalem Invitees: engine-devel at ovirt.org; node-devel at ovirt.org; board at ovirt.org *~*~*~*~*~*~*~*~*~* This is the official go/no go meeting for Ovirt's first release. Useful resources: 1. http://www.ovirt.org/wiki/Releases/First_Release_Blockers 2. http://www.ovirt.org/wiki/Releases/First_Release -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: meeting.ics Type: text/calendar Size: 2789 bytes Desc: not available URL: From mburns at redhat.com Tue Feb 7 14:43:25 2012 From: mburns at redhat.com (Mike Burns) Date: Tue, 07 Feb 2012 09:43:25 -0500 Subject: [node-devel] oVirt Node Weekly Meeting Summary -- 2012-02-07 Message-ID: <1328625805.5199.3.camel@beelzebub.mburnsfire.net> Minutes: http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-07-14.00.html Minutes (text): http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-07-14.00.txt Log: http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-07-14.00.log.html Action items, by person 1. jboggs 1. jboggs to followup with dnaori again 2. mburns 1. mburns to incorporate archipel-node stateless design into ovirt-node stateless plan From taget at linux.vnet.ibm.com Thu Feb 9 04:50:41 2012 From: taget at linux.vnet.ibm.com (taget at linux.vnet.ibm.com) Date: Thu, 9 Feb 2012 12:50:41 +0800 Subject: [node-devel] [PATCH v2] Add network configure before dropping into emergency shell Message-ID: <1328763041-19720-1-git-send-email-taget@linux.vnet.ibm.com> From: Eli Configure network regardless of storage_auto status. --- scripts/ovirt-auto-install.py | 19 ++++++++++++------- 1 files changed, 12 insertions(+), 7 deletions(-) diff --git a/scripts/ovirt-auto-install.py b/scripts/ovirt-auto-install.py index bfd0a12..ceadcd1 100755 --- a/scripts/ovirt-auto-install.py +++ b/scripts/ovirt-auto-install.py @@ -30,7 +30,19 @@ from ovirtnode.snmp import * from ovirt_config_setup.collectd import * print "Performing automatic disk partitioning" +storage_auto_succ = 0 if storage_auto(): + stroage_auto_succ = 1 + +# network configuration +print "Configuring Network" +if OVIRT_VARS["OVIRT_BOOTIF"] != "": + network_auto() + +if OVIRT_VARS.has_key("OVIRT_HOSTNAME"): + system("hostname %s" % OVIRT_VARS["OVIRT_HOSTNAME"]) + +if stroage_auto_succ == 1: print "Completed automatic disk partitioning" # store /etc/shadow if adminpw/rootpw are set, handled already in ovirt-early file = open("/proc/cmdline") @@ -40,13 +52,6 @@ if storage_auto(): ovirt_store_config("/etc/passwd") ovirt_store_config("/etc/shadow") file.close() - # network configuration - print "Configuring Network" - if OVIRT_VARS["OVIRT_BOOTIF"] != "": - network_auto() - - if OVIRT_VARS.has_key("OVIRT_HOSTNAME"): - system("hostname %s" % OVIRT_VARS["OVIRT_HOSTNAME"]) #set ssh_passwd_auth if OVIRT_VARS.has_key("OVIRT_SSH_PWAUTH"): if self.ssh_passwd_status.value() == 1: -- 1.7.4.4 From mburns at redhat.com Thu Feb 9 12:38:21 2012 From: mburns at redhat.com (Mike Burns) Date: Thu, 09 Feb 2012 07:38:21 -0500 Subject: [node-devel] [PATCH v2] Add network configure before dropping into emergency shell In-Reply-To: <1328763041-19720-1-git-send-email-taget@linux.vnet.ibm.com> References: <1328763041-19720-1-git-send-email-taget@linux.vnet.ibm.com> Message-ID: <1328791101.2553.26.camel@beelzebub.mburnsfire.net> Hi Eli, Just a couple things. 1. Thanks for the patch! 2. White space errors: when applying the patch, I get whitespace errors on line 25. Not a big issue, but something to watch for. 3. We use gerrit[1] for patches in ovirt-node. Please use it for future patch submissions. I've submitted this patch (with whitespace fixed) to gerrit already[2]. Mike [1] http://ovirt.org/wiki/Working_with_gerrit.ovirt.org [2] http://gerrit.ovirt.org/1759 On Thu, 2012-02-09 at 12:50 +0800, taget at linux.vnet.ibm.com wrote: > From: Eli > > Configure network regardless of storage_auto status. > --- > scripts/ovirt-auto-install.py | 19 ++++++++++++------- > 1 files changed, 12 insertions(+), 7 deletions(-) > > diff --git a/scripts/ovirt-auto-install.py b/scripts/ovirt-auto-install.py > index bfd0a12..ceadcd1 100755 > --- a/scripts/ovirt-auto-install.py > +++ b/scripts/ovirt-auto-install.py > @@ -30,7 +30,19 @@ from ovirtnode.snmp import * > from ovirt_config_setup.collectd import * > > print "Performing automatic disk partitioning" > +storage_auto_succ = 0 > if storage_auto(): > + stroage_auto_succ = 1 > + > +# network configuration > +print "Configuring Network" > +if OVIRT_VARS["OVIRT_BOOTIF"] != "": > + network_auto() > + > +if OVIRT_VARS.has_key("OVIRT_HOSTNAME"): > + system("hostname %s" % OVIRT_VARS["OVIRT_HOSTNAME"]) > + > +if stroage_auto_succ == 1: > print "Completed automatic disk partitioning" > # store /etc/shadow if adminpw/rootpw are set, handled already in ovirt-early > file = open("/proc/cmdline") > @@ -40,13 +52,6 @@ if storage_auto(): > ovirt_store_config("/etc/passwd") > ovirt_store_config("/etc/shadow") > file.close() > - # network configuration > - print "Configuring Network" > - if OVIRT_VARS["OVIRT_BOOTIF"] != "": > - network_auto() > - > - if OVIRT_VARS.has_key("OVIRT_HOSTNAME"): > - system("hostname %s" % OVIRT_VARS["OVIRT_HOSTNAME"]) > #set ssh_passwd_auth > if OVIRT_VARS.has_key("OVIRT_SSH_PWAUTH"): > if self.ssh_passwd_status.value() == 1: From taget at linux.vnet.ibm.com Fri Feb 10 01:47:58 2012 From: taget at linux.vnet.ibm.com (eli) Date: Fri, 10 Feb 2012 09:47:58 +0800 Subject: [node-devel] [PATCH v2] Add network configure before dropping into emergency shell In-Reply-To: <1328791101.2553.26.camel@beelzebub.mburnsfire.net> References: <1328763041-19720-1-git-send-email-taget@linux.vnet.ibm.com> <1328791101.2553.26.camel@beelzebub.mburnsfire.net> Message-ID: <4F34774E.4060408@linux.vnet.ibm.com> ? 2012?02?09? 20:38, Mike Burns ??: > Hi Eli, > > Just a couple things. > > 1. Thanks for the patch! > 2. White space errors: when applying the patch, I get whitespace > errors on line 25. Not a big issue, but something to watch for. > 3. We use gerrit[1] for patches in ovirt-node. Please use it for > future patch submissions. I've submitted this patch (with whitespace > fixed) to gerrit already[2]. > > Mike thanks Mike, I will pay more attention for it. > [1] http://ovirt.org/wiki/Working_with_gerrit.ovirt.org > [2] http://gerrit.ovirt.org/1759 > > On Thu, 2012-02-09 at 12:50 +0800, taget at linux.vnet.ibm.com wrote: >> From: Eli >> >> Configure network regardless of storage_auto status. >> --- >> scripts/ovirt-auto-install.py | 19 ++++++++++++------- >> 1 files changed, 12 insertions(+), 7 deletions(-) >> >> diff --git a/scripts/ovirt-auto-install.py b/scripts/ovirt-auto-install.py >> index bfd0a12..ceadcd1 100755 >> --- a/scripts/ovirt-auto-install.py >> +++ b/scripts/ovirt-auto-install.py >> @@ -30,7 +30,19 @@ from ovirtnode.snmp import * >> from ovirt_config_setup.collectd import * >> >> print "Performing automatic disk partitioning" >> +storage_auto_succ = 0 >> if storage_auto(): >> + stroage_auto_succ = 1 >> + >> +# network configuration >> +print "Configuring Network" >> +if OVIRT_VARS["OVIRT_BOOTIF"] != "": >> + network_auto() >> + >> +if OVIRT_VARS.has_key("OVIRT_HOSTNAME"): >> + system("hostname %s" % OVIRT_VARS["OVIRT_HOSTNAME"]) >> + >> +if stroage_auto_succ == 1: >> print "Completed automatic disk partitioning" >> # store /etc/shadow if adminpw/rootpw are set, handled already in ovirt-early >> file = open("/proc/cmdline") >> @@ -40,13 +52,6 @@ if storage_auto(): >> ovirt_store_config("/etc/passwd") >> ovirt_store_config("/etc/shadow") >> file.close() >> - # network configuration >> - print "Configuring Network" >> - if OVIRT_VARS["OVIRT_BOOTIF"] != "": >> - network_auto() >> - >> - if OVIRT_VARS.has_key("OVIRT_HOSTNAME"): >> - system("hostname %s" % OVIRT_VARS["OVIRT_HOSTNAME"]) >> #set ssh_passwd_auth >> if OVIRT_VARS.has_key("OVIRT_SSH_PWAUTH"): >> if self.ssh_passwd_status.value() == 1: > From mburns at redhat.com Tue Feb 14 14:22:24 2012 From: mburns at redhat.com (Mike Burns) Date: Tue, 14 Feb 2012 09:22:24 -0500 Subject: [node-devel] oVirt Node Weekly Sync Meeting Minutes 2012-02-14 Message-ID: <1329229344.23440.12.camel@beelzebub.mburnsfire.net> Minutes: http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-14-14.00.html Minutes (text): http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-14-14.00.txt Log: http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-14-14.00.log.html Action items, by person 1. _jboggs 1. _jboggs to review gerrit patches 2. mburns 1. mburns to update stateless wiki 2. mburns to release 2.2.3 on 15-Feb 3. mburns to review/update workshop presentation for jbrooks to get translated for workshop in Beijing From fabian.deutsch at gmx.de Wed Feb 15 20:27:34 2012 From: fabian.deutsch at gmx.de (Fabian Deutsch) Date: Wed, 15 Feb 2012 21:27:34 +0100 Subject: [node-devel] Introduction and migration of the ovirt-node SysV init scripts to systemd Message-ID: <1329337654.2003.54.camel@apu.local> Hey, I'm new to this list a just want to quickly introduce myself. My name is Fabian Deutsch (fabiand), I'm 29yrs of age and live near Bremen in Germany. Currently I'm maintaining a couple of packages and using libvirt and friends within our working group to maintain a couple of virtual guests. Coming to ovirt-node I saw that the init scripts are in the SysV/LSB format, but it makes sense to migrate them - like the init scripts of all daemons - to systemd. Because the scripts ain't trivial I would like to discuss the migration before. First it seems if the affected files are ovirt and ovirt-awake, ovirt-early, and ovirt-firstboot in the scripts directory. Besides fulfilling the SysV requirements by providing start(), stop() etc. those files also contain much logic to setup and check ovirt specific things. But it's essential to pull the logic out of the init scripts, to allow a migration to systemd service files, which usually just call an appropriate daemon or - like in our case - oneshot scripts. Therefor I'm suggesting to split the SysV stuff from the logic in a first step while keeping SysV init files. After this step we would have reduced SysV init scripts, containing just the relevant SysV stuff and a bunch of scripts residing in e.g. /usr/libexec/ovirt which provide the logic and are used by the init scripts to do the magic. The second step would then be to replace the reduced init scripts by appropriate systemd service files. A last step could be migrating the init scripts to python, in order to consolidate some code. One problem I could imagine is, that the scripts currently export some global variables and I don't to whom they will visible when using systemd files - or if there is no change in the behavior at all. Does the migration and the suggested steps make sense? Or am I missing something? Greetings fabian From mburns at redhat.com Wed Feb 15 20:44:45 2012 From: mburns at redhat.com (Mike Burns) Date: Wed, 15 Feb 2012 15:44:45 -0500 Subject: [node-devel] Introduction and migration of the ovirt-node SysV init scripts to systemd In-Reply-To: <1329337654.2003.54.camel@apu.local> References: <1329337654.2003.54.camel@apu.local> Message-ID: <1329338685.9399.20.camel@beelzebub.mburnsfire.net> On Wed, 2012-02-15 at 21:27 +0100, Fabian Deutsch wrote: > Hey, > > I'm new to this list a just want to quickly introduce myself. My name is > Fabian Deutsch (fabiand), I'm 29yrs of age and live near Bremen in > Germany. Currently I'm maintaining a couple of packages and using > libvirt and friends within our working group to maintain a couple of > virtual guests. > > Coming to ovirt-node I saw that the init scripts are in the SysV/LSB > format, but it makes sense to migrate them - like the init scripts of > all daemons - to systemd. > Because the scripts ain't trivial I would like to discuss the migration > before. > > First it seems if the affected files are ovirt and ovirt-awake, > ovirt-early, and ovirt-firstboot in the scripts directory. > Besides fulfilling the SysV requirements by providing start(), stop() > etc. those files also contain much logic to setup and check ovirt > specific things. > But it's essential to pull the logic out of the init scripts, to allow a > migration to systemd service files, which usually just call an > appropriate daemon or - like in our case - oneshot scripts. > > Therefor I'm suggesting to split the SysV stuff from the logic in a > first step while keeping SysV init files. > After this step we would have reduced SysV init scripts, containing just > the relevant SysV stuff and a bunch of scripts residing in > e.g. /usr/libexec/ovirt which provide the logic and are used by the init > scripts to do the magic. > > The second step would then be to replace the reduced init scripts by > appropriate systemd service files. > > A last step could be migrating the init scripts to python, in order to > consolidate some code. Definitely important. We currently have a number of functions in ovirt-functions (bash) and ovirtfunctions.py (python) that do the same thing. Since the vast majority of the scripts are in python now, getting the init scripts to be python would allow us to deprecate the ovirt-functions script. > > One problem I could imagine is, that the scripts currently export some > global variables and I don't to whom they will visible when using > systemd files - or if there is no change in the behavior at all. I don't know of anywhere that we rely on the global variables. In truth, any variables should be set in the /etc/default/ovirt file directly and sourced from there in other scripts. > > Does the migration and the suggested steps make sense? > Or am I missing something? The migration makes sense to me. One thing to keep in mind on this is the fact that we have to continue to support RHEL which hasn't moved to systemd yet. So having both the systemd and sysv init scripts around and running the same code in the background is also an important step. Mike > > Greetings > fabian > > _______________________________________________ > node-devel mailing list > node-devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/node-devel From fabian.deutsch at gmx.de Wed Feb 15 21:12:14 2012 From: fabian.deutsch at gmx.de (Fabian Deutsch) Date: Wed, 15 Feb 2012 22:12:14 +0100 Subject: [node-devel] Introduction and migration of the ovirt-node SysV init scripts to systemd In-Reply-To: <1329338685.9399.20.camel@beelzebub.mburnsfire.net> References: <1329337654.2003.54.camel@apu.local> <1329338685.9399.20.camel@beelzebub.mburnsfire.net> Message-ID: <1329340334.2003.60.camel@apu.local> Am Mittwoch, den 15.02.2012, 15:44 -0500 schrieb Mike Burns: > On Wed, 2012-02-15 at 21:27 +0100, Fabian Deutsch wrote: > > A last step could be migrating the init scripts to python, in order to > > consolidate some code. > > Definitely important. We currently have a number of functions in > ovirt-functions (bash) and ovirtfunctions.py (python) that do the same > thing. Since the vast majority of the scripts are in python now, > getting the init scripts to be python would allow us to deprecate the > ovirt-functions script. Good to know, I did not yet take such a close look, but this sounds good and is one more reason. > > > > One problem I could imagine is, that the scripts currently export some > > global variables and I don't to whom they will visible when using > > systemd files - or if there is no change in the behavior at all. > > I don't know of anywhere that we rely on the global variables. In > truth, any variables should be set in the /etc/default/ovirt file > directly and sourced from there in other scripts. Right. > > > > Does the migration and the suggested steps make sense? > > Or am I missing something? > > The migration makes sense to me. One thing to keep in mind on this is > the fact that we have to continue to support RHEL which hasn't moved to > systemd yet. So having both the systemd and sysv init scripts around > and running the same code in the background is also an important step. Yes, true. my suggestion would be to keep the "reduced" SysV init scripts around which in turn call the appropriate bash or python scripts providing the required functionality. Thanks fabian From fabian.deutsch at gmx.de Wed Feb 15 22:10:51 2012 From: fabian.deutsch at gmx.de (Fabian Deutsch) Date: Wed, 15 Feb 2012 23:10:51 +0100 Subject: [node-devel] I18N support in oVirt Node Message-ID: <1329343851.2003.92.camel@apu.local> Hey, to open oVirt Node to a wider and non-english speaking audience it might help to add some I18N support to node. Currently it seems as if it wasn't the highest priority adding i18n to node. I couldn't find traces in the init scripts or the tui sources. But both parts require user interaction and should therefor be at least translated to give non-english speakers a chance to use node. I'd propose to start with adding some gettext infrastructure to the present python sources and identify and extract the relevant strings in a next step. The init scripts, which are currently plain bash scripts, can be left aside right now, because it might make more sense to add gettext while migrating them to python [1]. Any thoughts on this? And volunteers to translate some strings? :) I can provide some de_DE strings. Greetings fabian -- 1 http://lists.ovirt.org/pipermail/node-devel/2012-February/000127.html From acathrow at redhat.com Wed Feb 15 22:50:51 2012 From: acathrow at redhat.com (Andrew Cathrow) Date: Wed, 15 Feb 2012 17:50:51 -0500 (EST) Subject: [node-devel] I18N support in oVirt Node In-Reply-To: <1329343851.2003.92.camel@apu.local> Message-ID: <08813f48-3589-411c-b72c-1806ba58580d@zmail07.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > From: "Fabian Deutsch" > To: "node-devel" > Sent: Wednesday, February 15, 2012 5:10:51 PM > Subject: [node-devel] I18N support in oVirt Node > > Hey, > > to open oVirt Node to a wider and non-english speaking audience it > might > help to add some I18N support to node. > > Currently it seems as if it wasn't the highest priority adding i18n > to > node. I couldn't find traces in the init scripts or the tui sources. > But both parts require user interaction and should therefor be at > least > translated to give non-english speakers a chance to use node. > > I'd propose to start with adding some gettext infrastructure to the > present python sources and identify and extract the relevant strings > in > a next step. > The init scripts, which are currently plain bash scripts, can be left > aside right now, because it might make more sense to add gettext > while > migrating them to python [1]. > > Any thoughts on this? And volunteers to translate some strings? :) > I can provide some de_DE strings. The other thing to consider is keyboard support - making sure that we support different keyboard layouts in the node. Downstream bug here https://bugzilla.redhat.com/show_bug.cgi?id=742041 > > Greetings > fabian > > -- > 1 > http://lists.ovirt.org/pipermail/node-devel/2012-February/000127.html > > _______________________________________________ > node-devel mailing list > node-devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/node-devel > From fabian.deutsch at gmx.de Thu Feb 16 07:55:44 2012 From: fabian.deutsch at gmx.de (Fabian Deutsch) Date: Thu, 16 Feb 2012 08:55:44 +0100 Subject: [node-devel] I18N support in oVirt Node In-Reply-To: <08813f48-3589-411c-b72c-1806ba58580d@zmail07.collab.prod.int.phx2.redhat.com> References: <08813f48-3589-411c-b72c-1806ba58580d@zmail07.collab.prod.int.phx2.redhat.com> Message-ID: <1329378944.2059.2.camel@apu.local> Am Mittwoch, den 15.02.2012, 17:50 -0500 schrieb Andrew Cathrow: > ----- Original Message ----- > > From: "Fabian Deutsch" > > To: "node-devel" > > Sent: Wednesday, February 15, 2012 5:10:51 PM > > Subject: [node-devel] I18N support in oVirt Node > > > > Any thoughts on this? And volunteers to translate some strings? :) > > I can provide some de_DE strings. > > The other thing to consider is keyboard support - making sure that we support different keyboard layouts in the node. > Downstream bug here https://bugzilla.redhat.com/show_bug.cgi?id=742041 I can't access that bug, but you are right. That's definetly something that should also be taken care off. fabian From pmyers at redhat.com Thu Feb 16 13:48:56 2012 From: pmyers at redhat.com (Perry Myers) Date: Thu, 16 Feb 2012 08:48:56 -0500 Subject: [node-devel] I18N support in oVirt Node In-Reply-To: <1329378944.2059.2.camel@apu.local> References: <08813f48-3589-411c-b72c-1806ba58580d@zmail07.collab.prod.int.phx2.redhat.com> <1329378944.2059.2.camel@apu.local> Message-ID: <4F3D0948.3020007@redhat.com> On 02/16/2012 02:55 AM, Fabian Deutsch wrote: > Am Mittwoch, den 15.02.2012, 17:50 -0500 schrieb Andrew Cathrow: >> ----- Original Message ----- >>> From: "Fabian Deutsch" >>> To: "node-devel" >>> Sent: Wednesday, February 15, 2012 5:10:51 PM >>> Subject: [node-devel] I18N support in oVirt Node >>> >>> Any thoughts on this? And volunteers to translate some strings? :) >>> I can provide some de_DE strings. >> >> The other thing to consider is keyboard support - making sure that we support different keyboard layouts in the node. >> Downstream bug here https://bugzilla.redhat.com/show_bug.cgi?id=742041 > > I can't access that bug, but you are right. That's definetly something that should also be taken care off. That one was internal to Red Hat. I've cloned it to upstream here: https://bugzilla.redhat.com/show_bug.cgi?id=791202 And targeted it to our 2.3.0 release of oVirt Node. Perry From fabian.deutsch at gmx.de Thu Feb 16 14:45:23 2012 From: fabian.deutsch at gmx.de (Fabian Deutsch) Date: Thu, 16 Feb 2012 15:45:23 +0100 Subject: [node-devel] I18N support in oVirt Node Message-ID: <20120216144523.155500@gmx.net> > On 02/16/2012 02:55 AM, Fabian Deutsch wrote: > > Am Mittwoch, den 15.02.2012, 17:50 -0500 schrieb Andrew Cathrow: > >> ----- Original Message ----- > >>> From: "Fabian Deutsch" > >>> To: "node-devel" > >>> Sent: Wednesday, February 15, 2012 5:10:51 PM > >>> Subject: [node-devel] I18N support in oVirt Node > >>> > >>> Any thoughts on this? And volunteers to translate some strings? :) > >>> I can provide some de_DE strings. > >> > >> The other thing to consider is keyboard support - making sure that we support different keyboard layouts in the node. > >> Downstream bug here https://bugzilla.redhat.com/show_bug.cgi?id=742041 > > > > I can't access that bug, but you are right. That's definetly something that should also be taken care off. > > That one was internal to Red Hat. > > I've cloned it to upstream here: > https://bugzilla.redhat.com/show_bug.cgi?id=791202 Thanks. As said this is also needed, but because it affects different parts of ovirt (the tui and maybe also the boot scripts) opposed to the simple translation of other parts, I'd see the integration of a language selection dialog as a separate task. Well, it's obvious that a translated tui just makes sense if we are able to change the language :) As said in the bug above, we can have a look at anaconda and see what parts we can reuse to bake the language selection dialog. > And targeted it to our 2.3.0 release of oVirt Node. I actually don't know when this release is scheduled and I'm also not yet that deep into the sources to give a good estimate, but if the feature freeze is not within a couple of days this could be done I'd guess. Greetings fabian From fabian.deutsch at gmx.de Thu Feb 16 15:10:10 2012 From: fabian.deutsch at gmx.de (Fabian Deutsch) Date: Thu, 16 Feb 2012 16:10:10 +0100 Subject: [node-devel] Introduction and migration of the ovirt-node SysV init scripts to systemd Message-ID: <20120216151010.155530@gmx.net> > Coming to ovirt-node I saw that the init scripts are in the SysV/LSB > format, but it makes sense to migrate them - like the init scripts of > all daemons - to systemd. > Because the scripts ain't trivial I would like to discuss the migration > before. Just for the sake of completness I'd like to point to the corresponding bug https://bugzilla.redhat.com/show_bug.cgi?id=751856 fabian From mburns at redhat.com Tue Feb 21 13:34:08 2012 From: mburns at redhat.com (Mike Burns) Date: Tue, 21 Feb 2012 08:34:08 -0500 Subject: [node-devel] oVirt Node 2.2.3 Release Message-ID: <1329831248.28523.19.camel@mburns-laptop.usersys.redhat.com> oVirt Node 2.2.3 is now available on ovirt.org [1]. Changes in this release * Add kickstarts for F18 * Configuring networking even if autoinstall fails * Small build enhancements * Added archipel configuration * Limited stateless support introduced * Various cleanup fixes related to stateless changes Thanks The Node Team [1] http://ovirt.org/wiki/Category:Node#Current_Release From mburns at redhat.com Tue Feb 21 14:56:10 2012 From: mburns at redhat.com (Mike Burns) Date: Tue, 21 Feb 2012 09:56:10 -0500 Subject: [node-devel] oVirt Node Weekly Sync Minutes 2012-02-21 Message-ID: <1329836170.20337.2.camel@mburns-laptop.usersys.redhat.com> Minutes: http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-21-14.03.html Minutes (text): http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-21-14.03.txt Log: http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-21-14.03.log.html Action items, by person 1. _jboggs 1. _jboggs to follow up with dnaori on install issues 2. _jboggs to create initial plugin to change admin password only 2. mburns 1. mburns to get on top of gerrit patches 2. mburns to follow up on jenkins this week 3. mburns to make ovirt-node-iso builds available by EOW From mburns at redhat.com Wed Feb 22 14:57:38 2012 From: mburns at redhat.com (Mike Burns) Date: Wed, 22 Feb 2012 09:57:38 -0500 Subject: [node-devel] Support for stateless nodes Message-ID: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> There has been a lot of interest in being able to run stateless Nodes with ovirt-engine. ovirt-node has designed a way [1] to achieve this on the node side, but we need input from the engine and vdsm teams to see if we're missing some requirement or if there needs to be changes on the engine/vdsm side to achieve this. As it currently stands, every time you reboot an ovirt-node that is stateless, it would require manually removing the host in engine, then re-registering/approving it again in engine. Any thoughts, concerns, input on how to solve this? Thanks Mike [1] http://ovirt.org/wiki/Node_Stateless From dfediuck at redhat.com Wed Feb 22 15:33:15 2012 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 22 Feb 2012 17:33:15 +0200 Subject: [node-devel] Support for stateless nodes In-Reply-To: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> Message-ID: <4F450ABB.30009@redhat.com> On 22/02/12 16:57, Mike Burns wrote: > There has been a lot of interest in being able to run stateless Nodes > with ovirt-engine. ovirt-node has designed a way [1] to achieve this on > the node side, but we need input from the engine and vdsm teams to see > if we're missing some requirement or if there needs to be changes on the > engine/vdsm side to achieve this. > > As it currently stands, every time you reboot an ovirt-node that is > stateless, it would require manually removing the host in engine, then > re-registering/approving it again in engine. > > Any thoughts, concerns, input on how to solve this? > > Thanks > > Mike > > [1] http://ovirt.org/wiki/Node_Stateless > Some points need to be considered; - Installation issues * Just stating the obvious, which is users need to remove-add the host on every reboot. This will not make this feature a lovable one from user's point of view. * During initial boot, vdsm-reg configures the networking and creates a management network bridge. This is a very delicate process which may fail due to networking issues such as resolution, routing, etc. So re-doing this on every boot increases the chances of loosing a node due to network problems. * CA pollution; generating a certificate on each reboot for each node will create a huge number of certificates in the engine side, which eventually may damage the CA. (Unsure if there's a limitation to certificates number, but having hundreds of junk cert's can't be good). * Today there's a supported flow that for nodes with password, the user is allowed to use the "add host" scenario. For stateless, it means re-configuring a password on every boot... - Other issues * Local storage; so far we were able to define a local storage in ovirt node. Stateless will block this ability. * Node upgrade; currently it's possible to upgrade a node from the engine. In stateless it will error, since no where to d/l the iso file to. * Collecting information; core dumps and logging may not be available due to lack of space? Or will it cause kernel panic if all space is consumed? -- /d "Hi, my name is Any Key. Please don't hit me!" From mburns at redhat.com Wed Feb 22 15:58:48 2012 From: mburns at redhat.com (Mike Burns) Date: Wed, 22 Feb 2012 10:58:48 -0500 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F450ABB.30009@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> Message-ID: <1329926328.6140.30.camel@beelzebub.mburnsfire.net> On Wed, 2012-02-22 at 17:33 +0200, Doron Fediuck wrote: > On 22/02/12 16:57, Mike Burns wrote: > > There has been a lot of interest in being able to run stateless Nodes > > with ovirt-engine. ovirt-node has designed a way [1] to achieve this on > > the node side, but we need input from the engine and vdsm teams to see > > if we're missing some requirement or if there needs to be changes on the > > engine/vdsm side to achieve this. > > > > As it currently stands, every time you reboot an ovirt-node that is > > stateless, it would require manually removing the host in engine, then > > re-registering/approving it again in engine. > > > > Any thoughts, concerns, input on how to solve this? > > > > Thanks > > > > Mike > > > > [1] http://ovirt.org/wiki/Node_Stateless > > > > Some points need to be considered; > > - Installation issues > > * Just stating the obvious, which is users need > to remove-add the host on every reboot. This will > not make this feature a lovable one from user's point of view. Yes, this is something that will cause this to be a non-starter. We'd need to change something in the engine/vdsm to make it smoother. Perhaps, a flag in engine on the host saying that it's stateless. Then if a host comes up with the same information, but no certs, etc, it would validate some other embedded key (TPM, key embedded in the node itself), and auto-approve it to be the same state as the previous boot > > * During initial boot, vdsm-reg configures the networking > and creates a management network bridge. This is a very > delicate process which may fail due to networking issues > such as resolution, routing, etc. So re-doing this on > every boot increases the chances of loosing a node due > to network problems. vdsm-reg runs on *every* boot anyway and renames the bridge. This is something that was debated previously, but it was decided to re-run it every boot. > > * CA pollution; generating a certificate on each reboot > for each node will create a huge number of certificates > in the engine side, which eventually may damage the CA. > (Unsure if there's a limitation to certificates number, > but having hundreds of junk cert's can't be good). We could have vdsm/engine store the certs on the engine side, and on boot, after validating the host (however that is done), it will load the certs onto the node machine. > > * Today there's a supported flow that for nodes with > password, the user is allowed to use the "add host" > scenario. For stateless, it means re-configuring a password > on every boot... Stateless is really targeted for a PXE environment. There is a supported kernel param that can be set that will set this password. Also, if we follow the design mentioned ^^, then it's not an issue since the host will auto-approve itself when it connects > > - Other issues > > * Local storage; so far we were able to define a local > storage in ovirt node. Stateless will block this ability. Yes, this would be unavailable if you're running stateless. I think that's a fine tradeoff since people want the host to be diskless. > > * Node upgrade; currently it's possible to upgrade a node > from the engine. In stateless it will error, since no where > to d/l the iso file to. Upgrade is handled easily by rebooting the host after updating the pxe server > > * Collecting information; core dumps and logging may not > be available due to lack of space? Or will it cause kernel > panic if all space is consumed? A valid concern, but a stateless environment would likely have collectd/rsyslog/netconsole servers running elsewhere that will collect the logs. kdumps can be configured to dump remotely as well. > Another concern raised is swap and overcommit. First version would likely disable swap completely. This would disable overcommit as well. Future versions could enable a local disk to be used completely for swap, but that is another tradeoff that people would need to evaluate when choosing between stateless and stateful installs. Mike From dfediuck at redhat.com Wed Feb 22 15:59:54 2012 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 22 Feb 2012 17:59:54 +0200 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F450ABB.30009@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> Message-ID: <4F4510FA.4050107@redhat.com> On 22/02/12 17:33, Doron Fediuck wrote: > On 22/02/12 16:57, Mike Burns wrote: >> There has been a lot of interest in being able to run stateless Nodes >> with ovirt-engine. ovirt-node has designed a way [1] to achieve this on >> the node side, but we need input from the engine and vdsm teams to see >> if we're missing some requirement or if there needs to be changes on the >> engine/vdsm side to achieve this. >> >> As it currently stands, every time you reboot an ovirt-node that is >> stateless, it would require manually removing the host in engine, then >> re-registering/approving it again in engine. >> >> Any thoughts, concerns, input on how to solve this? >> >> Thanks >> >> Mike >> >> [1] http://ovirt.org/wiki/Node_Stateless >> > > Some points need to be considered; > > - Installation issues > > * Just stating the obvious, which is users need > to remove-add the host on every reboot. This will > not make this feature a lovable one from user's point of view. > > * During initial boot, vdsm-reg configures the networking > and creates a management network bridge. This is a very > delicate process which may fail due to networking issues > such as resolution, routing, etc. So re-doing this on > every boot increases the chances of loosing a node due > to network problems. > > * CA pollution; generating a certificate on each reboot > for each node will create a huge number of certificates > in the engine side, which eventually may damage the CA. > (Unsure if there's a limitation to certificates number, > but having hundreds of junk cert's can't be good). > > * Today there's a supported flow that for nodes with > password, the user is allowed to use the "add host" > scenario. For stateless, it means re-configuring a password > on every boot... > > - Other issues > > * Local storage; so far we were able to define a local > storage in ovirt node. Stateless will block this ability. > > * Node upgrade; currently it's possible to upgrade a node > from the engine. In stateless it will error, since no where > to d/l the iso file to. > > * Collecting information; core dumps and logging may not > be available due to lack of space? Or will it cause kernel > panic if all space is consumed? > One more question / thing to consider; Currently when you manually install a node, you need to configure the management-server's address. Will I need to re-do it on every boot of a stateless node? -- /d "Air conditioned environment - Do NOT open Windows!" From pmyers at redhat.com Wed Feb 22 16:06:40 2012 From: pmyers at redhat.com (Perry Myers) Date: Wed, 22 Feb 2012 11:06:40 -0500 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F450ABB.30009@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> Message-ID: <4F451290.3090103@redhat.com> > * Just stating the obvious, which is users need > to remove-add the host on every reboot. This will > not make this feature a lovable one from user's point of view. I think the point mburns is trying to make in his initial email is that we're going to need to do some joint work between node and vdsm teams to change the registration process so that this is no longer necessary. It will require some redesigning of the registration process > * During initial boot, vdsm-reg configures the networking > and creates a management network bridge. This is a very > delicate process which may fail due to networking issues > such as resolution, routing, etc. So re-doing this on > every boot increases the chances of loosing a node due > to network problems. Well, if the network is busted which leads to the bridge rename failing, wouldn't the fact that the network is broken cause other problems anyhow? So I don't see this as a problem. If your network doesn't work properly, don't expect hosts in the network to subsequently work properly. As an aside, why is reverse DNS lookup a requirement? If we remove that it makes things a lot easier, no? > * CA pollution; generating a certificate on each reboot > for each node will create a huge number of certificates > in the engine side, which eventually may damage the CA. > (Unsure if there's a limitation to certificates number, > but having hundreds of junk cert's can't be good). I don't think we should regenerate a new certificate on each boot. I think we need a way for 'an already registered host to retrieve it's certificate from the oVirt Engine server' Using an embedded encryption key (if you trust your mgmt network or are booting from embedded flash), or for the paranoid a key stored in TPM can be used to have vdsm safely retrieve this from the oVirt Engine server on each boot so that it's not required to regenerate/reregister on each boot > * Today there's a supported flow that for nodes with > password, the user is allowed to use the "add host" > scenario. For stateless, it means re-configuring a password > on every boot... This flow would still be applicable. We are going to allow setting of the admin password embedded in the core ISO via an offline process. Once vdsm is fixed to use a non-root account for installation flow, this is no longer a problem Also, if we (as described above) make registrations persistent across reboots by changing the registration flow a bit, then the install user password only need be set for the initial boot anyhow. Therefore I think as a requirement for stateless oVirt Node, we must have as a prerequsite removing root account usage for registration/installation > - Other issues > > * Local storage; so far we were able to define a local > storage in ovirt node. Stateless will block this ability. It shouldn't. The Node should be able to automatically scan locally attached disks to look for a well defined VG or partition label and based on that automatically activate/mount Stateless doesn't imply diskless. It is a requirement even for stateless node usage to be able to leverage locally attached disks both for VM storage and also for Swap. > * Node upgrade; currently it's possible to upgrade a node > from the engine. In stateless it will error, since no where > to d/l the iso file to. Upgrades are no longer needed with stateless. To upgrade a stateless node all you need to do is 'reboot from a newer image'. i.e. all upgrades would be done via PXE server image replacement. So the flow of 'upload ISO to running oVirt Node' is no longer even necessary > * Collecting information; core dumps and logging may not > be available due to lack of space? Or will it cause kernel > panic if all space is consumed? We already provide ability to send kdumps to remote ssh/NFS location and already provide the ability to use both collectd and rsyslogs to pipe logs/stats to remote server(s). Local logs can be set to logrotate to a reasonable size so that local RAM FS always contains recent log information for quick triage, but long term historical logging would be maintained on the rsyslog server Perry From mburns at redhat.com Wed Feb 22 16:08:12 2012 From: mburns at redhat.com (Mike Burns) Date: Wed, 22 Feb 2012 11:08:12 -0500 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F4510FA.4050107@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <4F4510FA.4050107@redhat.com> Message-ID: <1329926892.6140.39.camel@beelzebub.mburnsfire.net> On Wed, 2012-02-22 at 17:59 +0200, Doron Fediuck wrote: > On 22/02/12 17:33, Doron Fediuck wrote: > > On 22/02/12 16:57, Mike Burns wrote: > >> There has been a lot of interest in being able to run stateless Nodes > >> with ovirt-engine. ovirt-node has designed a way [1] to achieve this on > >> the node side, but we need input from the engine and vdsm teams to see > >> if we're missing some requirement or if there needs to be changes on the > >> engine/vdsm side to achieve this. > >> > >> As it currently stands, every time you reboot an ovirt-node that is > >> stateless, it would require manually removing the host in engine, then > >> re-registering/approving it again in engine. > >> > >> Any thoughts, concerns, input on how to solve this? > >> > >> Thanks > >> > >> Mike > >> > >> [1] http://ovirt.org/wiki/Node_Stateless > >> > > > > Some points need to be considered; > > > > - Installation issues > > > > * Just stating the obvious, which is users need > > to remove-add the host on every reboot. This will > > not make this feature a lovable one from user's point of view. > > > > * During initial boot, vdsm-reg configures the networking > > and creates a management network bridge. This is a very > > delicate process which may fail due to networking issues > > such as resolution, routing, etc. So re-doing this on > > every boot increases the chances of loosing a node due > > to network problems. > > > > * CA pollution; generating a certificate on each reboot > > for each node will create a huge number of certificates > > in the engine side, which eventually may damage the CA. > > (Unsure if there's a limitation to certificates number, > > but having hundreds of junk cert's can't be good). > > > > * Today there's a supported flow that for nodes with > > password, the user is allowed to use the "add host" > > scenario. For stateless, it means re-configuring a password > > on every boot... > > > > - Other issues > > > > * Local storage; so far we were able to define a local > > storage in ovirt node. Stateless will block this ability. > > > > * Node upgrade; currently it's possible to upgrade a node > > from the engine. In stateless it will error, since no where > > to d/l the iso file to. > > > > * Collecting information; core dumps and logging may not > > be available due to lack of space? Or will it cause kernel > > panic if all space is consumed? > > > > One more question / thing to consider; > Currently when you manually install a node, > you need to configure the management-server's address. > Will I need to re-do it on every boot of a stateless node? As answered in the other response, there are kernel command line parameters to set the management_server. Since this will likely be in a pxe environment, setting the pxe profile to include management_server= should be fine. Another solution could be to setup a specific DNS SRV record that points to the ovirt-engine and have node automatically query that for the location. Mike From dfediuck at redhat.com Wed Feb 22 16:10:50 2012 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 22 Feb 2012 18:10:50 +0200 Subject: [node-devel] Support for stateless nodes In-Reply-To: <1329926328.6140.30.camel@beelzebub.mburnsfire.net> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <1329926328.6140.30.camel@beelzebub.mburnsfire.net> Message-ID: <4F45138A.9060407@redhat.com> On 22/02/12 17:58, Mike Burns wrote: > On Wed, 2012-02-22 at 17:33 +0200, Doron Fediuck wrote: >> On 22/02/12 16:57, Mike Burns wrote: >>> There has been a lot of interest in being able to run stateless Nodes >>> with ovirt-engine. ovirt-node has designed a way [1] to achieve this on >>> the node side, but we need input from the engine and vdsm teams to see >>> if we're missing some requirement or if there needs to be changes on the >>> engine/vdsm side to achieve this. >>> >>> As it currently stands, every time you reboot an ovirt-node that is >>> stateless, it would require manually removing the host in engine, then >>> re-registering/approving it again in engine. >>> >>> Any thoughts, concerns, input on how to solve this? >>> >>> Thanks >>> >>> Mike >>> >>> [1] http://ovirt.org/wiki/Node_Stateless >>> >> >> Some points need to be considered; >> >> - Installation issues >> >> * Just stating the obvious, which is users need >> to remove-add the host on every reboot. This will >> not make this feature a lovable one from user's point of view. > > Yes, this is something that will cause this to be a non-starter. We'd > need to change something in the engine/vdsm to make it smoother. > Perhaps, a flag in engine on the host saying that it's stateless. Then > if a host comes up with the same information, but no certs, etc, it > would validate some other embedded key (TPM, key embedded in the node > itself), and auto-approve it to be the same state as the previous boot > This will require some thinking. >> >> * During initial boot, vdsm-reg configures the networking >> and creates a management network bridge. This is a very >> delicate process which may fail due to networking issues >> such as resolution, routing, etc. So re-doing this on >> every boot increases the chances of loosing a node due >> to network problems. > > vdsm-reg runs on *every* boot anyway and renames the bridge. This is > something that was debated previously, but it was decided to re-run it > every boot. > Close, but not exactly; vdsm-reg will run on every boot, but if the relevant bridge is found, then networking is unchanged. >> >> * CA pollution; generating a certificate on each reboot >> for each node will create a huge number of certificates >> in the engine side, which eventually may damage the CA. >> (Unsure if there's a limitation to certificates number, >> but having hundreds of junk cert's can't be good). > > We could have vdsm/engine store the certs on the engine side, and on > boot, after validating the host (however that is done), it will load the > certs onto the node machine. > This is a security issue, since the key pair should be generated on the node. This will lead us back to your TPM suggestion, but (although I like it, ) will cause us to be tpm-dependent, not to mention a non-trivial implementation. >> >> * Today there's a supported flow that for nodes with >> password, the user is allowed to use the "add host" >> scenario. For stateless, it means re-configuring a password >> on every boot... > > Stateless is really targeted for a PXE environment. There is a > supported kernel param that can be set that will set this password. > Also, if we follow the design mentioned ^^, then it's not an issue since > the host will auto-approve itself when it connects > >> >> - Other issues >> >> * Local storage; so far we were able to define a local >> storage in ovirt node. Stateless will block this ability. > > Yes, this would be unavailable if you're running stateless. I think > that's a fine tradeoff since people want the host to be diskless. >> >> * Node upgrade; currently it's possible to upgrade a node >> from the engine. In stateless it will error, since no where >> to d/l the iso file to. > > Upgrade is handled easily by rebooting the host after updating the pxe > server > >> >> * Collecting information; core dumps and logging may not >> be available due to lack of space? Or will it cause kernel >> panic if all space is consumed? > > A valid concern, but a stateless environment would likely have > collectd/rsyslog/netconsole servers running elsewhere that will collect > the logs. kdumps can be configured to dump remotely as well. This will also need some work on the vdsm side. >> > > Another concern raised is swap and overcommit. First version would > likely disable swap completely. This would disable overcommit as well. > Future versions could enable a local disk to be used completely for > swap, but that is another tradeoff that people would need to evaluate > when choosing between stateless and stateful installs. Indeed so- completely forgot about swap... > > Mike > -- /d ?Funny,? he intoned funereally, ?how just when you think life can't possibly get any worse it suddenly does.? --Douglas Adams, The Hitchhiker's Guide to the Galaxy From pmyers at redhat.com Wed Feb 22 16:21:49 2012 From: pmyers at redhat.com (Perry Myers) Date: Wed, 22 Feb 2012 11:21:49 -0500 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F45138A.9060407@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <1329926328.6140.30.camel@beelzebub.mburnsfire.net> <4F45138A.9060407@redhat.com> Message-ID: <4F45161D.6010208@redhat.com> >>> >>> * CA pollution; generating a certificate on each reboot >>> for each node will create a huge number of certificates >>> in the engine side, which eventually may damage the CA. >>> (Unsure if there's a limitation to certificates number, >>> but having hundreds of junk cert's can't be good). >> >> We could have vdsm/engine store the certs on the engine side, and on >> boot, after validating the host (however that is done), it will load the >> certs onto the node machine. >> > This is a security issue, since the key pair should be > generated on the node. This will lead us back to your TPM > suggestion, but (although I like it, ) will cause us > to be tpm-dependent, not to mention a non-trivial implementation. Not necessarily 1. generate cert on oVirt Node 2. generate symmetric key and embed in TPM or use embedded symmetric key (for secured network model) 3. encrypt certs w/ symmetric key 4. push encryted cert to oVirt Engine On reboot 1. download encrypted cert from OE 2. use either embedded symmetric key or retrieve TPM based symmetric key and use to decrypt cert So no dependency on TPM, but the security is definitely much better if you have it. Use cases like this are one of the fundamental reasons why TPM exists :) From mburns at redhat.com Wed Feb 22 16:25:36 2012 From: mburns at redhat.com (Mike Burns) Date: Wed, 22 Feb 2012 11:25:36 -0500 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F45138A.9060407@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <1329926328.6140.30.camel@beelzebub.mburnsfire.net> <4F45138A.9060407@redhat.com> Message-ID: <1329927936.6140.46.camel@beelzebub.mburnsfire.net> On Wed, 2012-02-22 at 18:10 +0200, Doron Fediuck wrote: > On 22/02/12 17:58, Mike Burns wrote: > > On Wed, 2012-02-22 at 17:33 +0200, Doron Fediuck wrote: > >> On 22/02/12 16:57, Mike Burns wrote: > > > > vdsm-reg runs on *every* boot anyway and renames the bridge. This is > > something that was debated previously, but it was decided to re-run it > > every boot. > > > Close, but not exactly; vdsm-reg will run on every boot, but > if the relevant bridge is found, then networking is unchanged. Yes, that's true, but vdsm-reg doesn't persist the changes it makes. So on the next boot, it will never find the management bridge it's looking for. So while the condition is there to skip it, it will actually never find the bridge and will run the rename every boot. From dfediuck at redhat.com Wed Feb 22 16:40:59 2012 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 22 Feb 2012 18:40:59 +0200 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F45161D.6010208@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <1329926328.6140.30.camel@beelzebub.mburnsfire.net> <4F45138A.9060407@redhat.com> <4F45161D.6010208@redhat.com> Message-ID: <4F451A9B.9090303@redhat.com> On 22/02/12 18:21, Perry Myers wrote: >>>> >>>> * CA pollution; generating a certificate on each reboot >>>> for each node will create a huge number of certificates >>>> in the engine side, which eventually may damage the CA. >>>> (Unsure if there's a limitation to certificates number, >>>> but having hundreds of junk cert's can't be good). >>> >>> We could have vdsm/engine store the certs on the engine side, and on >>> boot, after validating the host (however that is done), it will load the >>> certs onto the node machine. >>> >> This is a security issue, since the key pair should be >> generated on the node. This will lead us back to your TPM >> suggestion, but (although I like it, ) will cause us >> to be tpm-dependent, not to mention a non-trivial implementation. > > Not necessarily > > 1. generate cert on oVirt Node > 2. generate symmetric key and embed in TPM or use embedded symmetric > key (for secured network model) IIUC in this step you're using TPM. What if there is no TPM (at all)? > 3. encrypt certs w/ symmetric key > 4. push encryted cert to oVirt Engine > > On reboot > > 1. download encrypted cert from OE > 2. use either embedded symmetric key or retrieve TPM based symmetric > key and use to decrypt cert > > So no dependency on TPM, but the security is definitely much better if > you have it. Use cases like this are one of the fundamental reasons why > TPM exists :) > _______________________________________________ > node-devel mailing list > node-devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/node-devel -- /d "Ford," he said, "you're turning into a penguin. Stop it." --Douglas Adams, The Hitchhiker's Guide to the Galaxy From pmyers at redhat.com Wed Feb 22 16:42:50 2012 From: pmyers at redhat.com (Perry Myers) Date: Wed, 22 Feb 2012 11:42:50 -0500 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F451A9B.9090303@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <1329926328.6140.30.camel@beelzebub.mburnsfire.net> <4F45138A.9060407@redhat.com> <4F45161D.6010208@redhat.com> <4F451A9B.9090303@redhat.com> Message-ID: <4F451B0A.3000300@redhat.com> On 02/22/2012 11:40 AM, Doron Fediuck wrote: > On 22/02/12 18:21, Perry Myers wrote: >>>>> >>>>> * CA pollution; generating a certificate on each reboot >>>>> for each node will create a huge number of certificates >>>>> in the engine side, which eventually may damage the CA. >>>>> (Unsure if there's a limitation to certificates number, >>>>> but having hundreds of junk cert's can't be good). >>>> >>>> We could have vdsm/engine store the certs on the engine side, and on >>>> boot, after validating the host (however that is done), it will load the >>>> certs onto the node machine. >>>> >>> This is a security issue, since the key pair should be >>> generated on the node. This will lead us back to your TPM >>> suggestion, but (although I like it, ) will cause us >>> to be tpm-dependent, not to mention a non-trivial implementation. >> >> Not necessarily >> >> 1. generate cert on oVirt Node >> 2. generate symmetric key and embed in TPM or use embedded symmetric >> key (for secured network model) > IIUC in this step you're using TPM. > What if there is no TPM (at all)? That statement had an 'or' in it. Either you use TPM with a self generated key 'or' you use a key that is preembedded in the image on either a node by node basis or per site. >> 3. encrypt certs w/ symmetric key >> 4. push encryted cert to oVirt Engine >> >> On reboot >> >> 1. download encrypted cert from OE >> 2. use either embedded symmetric key or retrieve TPM based symmetric >> key and use to decrypt cert >> >> So no dependency on TPM, but the security is definitely much better if >> you have it. Use cases like this are one of the fundamental reasons why >> TPM exists :) >> _______________________________________________ >> node-devel mailing list >> node-devel at ovirt.org >> http://lists.ovirt.org/mailman/listinfo/node-devel > > From dfediuck at redhat.com Wed Feb 22 16:47:31 2012 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 22 Feb 2012 18:47:31 +0200 Subject: [node-devel] Support for stateless nodes In-Reply-To: <1329927936.6140.46.camel@beelzebub.mburnsfire.net> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <1329926328.6140.30.camel@beelzebub.mburnsfire.net> <4F45138A.9060407@redhat.com> <1329927936.6140.46.camel@beelzebub.mburnsfire.net> Message-ID: <4F451C23.4070908@redhat.com> On 22/02/12 18:25, Mike Burns wrote: > On Wed, 2012-02-22 at 18:10 +0200, Doron Fediuck wrote: >> On 22/02/12 17:58, Mike Burns wrote: >>> On Wed, 2012-02-22 at 17:33 +0200, Doron Fediuck wrote: >>>> On 22/02/12 16:57, Mike Burns wrote: > > > >>> >>> vdsm-reg runs on *every* boot anyway and renames the bridge. This is >>> something that was debated previously, but it was decided to re-run it >>> every boot. >>> >> Close, but not exactly; vdsm-reg will run on every boot, but >> if the relevant bridge is found, then networking is unchanged. > > Yes, that's true, but vdsm-reg doesn't persist the changes it makes. So > on the next boot, it will never find the management bridge it's looking > for. So while the condition is there to skip it, it will actually never > find the bridge and will run the rename every boot. > Sounds like a bug to me. Last time I handled it, network configuration was persisted. You can see in line 102 here: http://gerrit.ovirt.org/gitweb?p=vdsm.git;a=blob;f=vdsm_reg/vdsm-reg-setup.in;h=c51f40c53f5303cfb447cedf5bc0c16228cb876d;hb=HEAD -- /d "Who's General Failure and why's he reading my disk?" From dfediuck at redhat.com Wed Feb 22 16:50:15 2012 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 22 Feb 2012 18:50:15 +0200 Subject: [node-devel] Support for stateless nodes In-Reply-To: <1329926892.6140.39.camel@beelzebub.mburnsfire.net> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <4F4510FA.4050107@redhat.com> <1329926892.6140.39.camel@beelzebub.mburnsfire.net> Message-ID: <4F451CC7.4050909@redhat.com> On 22/02/12 18:08, Mike Burns wrote: > On Wed, 2012-02-22 at 17:59 +0200, Doron Fediuck wrote: >> On 22/02/12 17:33, Doron Fediuck wrote: >>> On 22/02/12 16:57, Mike Burns wrote: >>>> There has been a lot of interest in being able to run stateless Nodes >>>> with ovirt-engine. ovirt-node has designed a way [1] to achieve this on >>>> the node side, but we need input from the engine and vdsm teams to see >>>> if we're missing some requirement or if there needs to be changes on the >>>> engine/vdsm side to achieve this. >>>> >>>> As it currently stands, every time you reboot an ovirt-node that is >>>> stateless, it would require manually removing the host in engine, then >>>> re-registering/approving it again in engine. >>>> >>>> Any thoughts, concerns, input on how to solve this? >>>> >>>> Thanks >>>> >>>> Mike >>>> >>>> [1] http://ovirt.org/wiki/Node_Stateless >>>> >>> >>> Some points need to be considered; >>> >>> - Installation issues >>> >>> * Just stating the obvious, which is users need >>> to remove-add the host on every reboot. This will >>> not make this feature a lovable one from user's point of view. >>> >>> * During initial boot, vdsm-reg configures the networking >>> and creates a management network bridge. This is a very >>> delicate process which may fail due to networking issues >>> such as resolution, routing, etc. So re-doing this on >>> every boot increases the chances of loosing a node due >>> to network problems. >>> >>> * CA pollution; generating a certificate on each reboot >>> for each node will create a huge number of certificates >>> in the engine side, which eventually may damage the CA. >>> (Unsure if there's a limitation to certificates number, >>> but having hundreds of junk cert's can't be good). >>> >>> * Today there's a supported flow that for nodes with >>> password, the user is allowed to use the "add host" >>> scenario. For stateless, it means re-configuring a password >>> on every boot... >>> >>> - Other issues >>> >>> * Local storage; so far we were able to define a local >>> storage in ovirt node. Stateless will block this ability. >>> >>> * Node upgrade; currently it's possible to upgrade a node >>> from the engine. In stateless it will error, since no where >>> to d/l the iso file to. >>> >>> * Collecting information; core dumps and logging may not >>> be available due to lack of space? Or will it cause kernel >>> panic if all space is consumed? >>> >> >> One more question / thing to consider; >> Currently when you manually install a node, >> you need to configure the management-server's address. >> Will I need to re-do it on every boot of a stateless node? > > As answered in the other response, there are kernel command line > parameters to set the management_server. Since this will likely be in a > pxe environment, setting the pxe profile to include > management_server= should be fine. > I agree it's a valid solution as long as you assume this is relevant for PXE only use case. > Another solution could be to setup a specific DNS SRV record that points > to the ovirt-engine and have node automatically query that for the > location. This was discussed in the past and for some reason not implemented. > > Mike > > _______________________________________________ > node-devel mailing list > node-devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/node-devel -- /d "Ford," he said, "you're turning into a penguin. Stop it." --Douglas Adams, The Hitchhiker's Guide to the Galaxy From pmyers at redhat.com Wed Feb 22 16:54:05 2012 From: pmyers at redhat.com (Perry Myers) Date: Wed, 22 Feb 2012 11:54:05 -0500 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F451CC7.4050909@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <4F4510FA.4050107@redhat.com> <1329926892.6140.39.camel@beelzebub.mburnsfire.net> <4F451CC7.4050909@redhat.com> Message-ID: <4F451DAD.9090706@redhat.com> >> As answered in the other response, there are kernel command line >> parameters to set the management_server. Since this will likely be in a >> pxe environment, setting the pxe profile to include >> management_server= should be fine. >> > I agree it's a valid solution as long as you assume this is relevant > for PXE only use case. Not necessarily... Take the ISO/USB Stick and you can embed the kargs into the ISO/USB itself so that it always boots with that mgmt server arg This actually also enables use of 'stateless' combined with static IP addressing as well. As you can create a USB Stick and embed the kargs for the NIC configuration, rsyslog config, etc, etc. >> Another solution could be to setup a specific DNS SRV record that points >> to the ovirt-engine and have node automatically query that for the >> location. > This was discussed in the past and for some reason not implemented. Concerns about security, iirc. Assumption that someone could hijack the DNS SRV record and provide a man-in-the-middle oVirt Engine server. If you're paranoid about security, don't use DNS SRV of course, instead use hardcoded kargs as described above. But for some DNS SRV might be an ok option From dfediuck at redhat.com Wed Feb 22 17:10:37 2012 From: dfediuck at redhat.com (Doron Fediuck) Date: Wed, 22 Feb 2012 19:10:37 +0200 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F451290.3090103@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <4F451290.3090103@redhat.com> Message-ID: <4F45218D.1020205@redhat.com> On 22/02/12 18:06, Perry Myers wrote: >> * Just stating the obvious, which is users need >> to remove-add the host on every reboot. This will >> not make this feature a lovable one from user's point of view. > > I think the point mburns is trying to make in his initial email is that > we're going to need to do some joint work between node and vdsm teams to > change the registration process so that this is no longer necessary. > > It will require some redesigning of the registration process > I'm aware of it, and that's why I'm raising my concerns, so we'll have a (partial) task list ;) >> * During initial boot, vdsm-reg configures the networking >> and creates a management network bridge. This is a very >> delicate process which may fail due to networking issues >> such as resolution, routing, etc. So re-doing this on >> every boot increases the chances of loosing a node due >> to network problems. > > Well, if the network is busted which leads to the bridge rename failing, > wouldn't the fact that the network is broken cause other problems anyhow? > Perry, my point is that we're increasing the chances to get into these holes. Network is not busted most of the time, but occasionally there's a glitch and we'd like to stay away from it. I'm sure you know what I'm talking about. > So I don't see this as a problem. If your network doesn't work > properly, don't expect hosts in the network to subsequently work properly. See previous answer. > As an aside, why is reverse DNS lookup a requirement? If we remove that > it makes things a lot easier, no? > Not sure I'm the right guy to defend it, but in order to drop reverse-dns, you need to consider dropping SSL, LDAP and some other important shortcuts... >> * CA pollution; generating a certificate on each reboot >> for each node will create a huge number of certificates >> in the engine side, which eventually may damage the CA. >> (Unsure if there's a limitation to certificates number, >> but having hundreds of junk cert's can't be good). > > I don't think we should regenerate a new certificate on each boot. I > think we need a way for 'an already registered host to retrieve it's > certificate from the oVirt Engine server' > > Using an embedded encryption key (if you trust your mgmt network or are > booting from embedded flash), or for the paranoid a key stored in TPM > can be used to have vdsm safely retrieve this from the oVirt Engine > server on each boot so that it's not required to regenerate/reregister > on each boot > Thoughtful redesign needed here... >> * Today there's a supported flow that for nodes with >> password, the user is allowed to use the "add host" >> scenario. For stateless, it means re-configuring a password >> on every boot... > > This flow would still be applicable. We are going to allow setting of > the admin password embedded in the core ISO via an offline process. > Once vdsm is fixed to use a non-root account for installation flow, this > is no longer a problem This is not exactly vdsm. More like vdsm-bootstrap. > > Also, if we (as described above) make registrations persistent across > reboots by changing the registration flow a bit, then the install user > password only need be set for the initial boot anyhow. > > Therefore I think as a requirement for stateless oVirt Node, we must > have as a prerequsite removing root account usage for > registration/installation > This is both for vdsm and engine, and I'm not sure it's that trivial. >> - Other issues >> >> * Local storage; so far we were able to define a local >> storage in ovirt node. Stateless will block this ability. > > It shouldn't. The Node should be able to automatically scan locally > attached disks to look for a well defined VG or partition label and > based on that automatically activate/mount > > Stateless doesn't imply diskless. It is a requirement even for > stateless node usage to be able to leverage locally attached disks both > for VM storage and also for Swap. > Still, in a pure disk-less setup you will not have local storage. See also Mike's answer. >> * Node upgrade; currently it's possible to upgrade a node >> from the engine. In stateless it will error, since no where >> to d/l the iso file to. > > Upgrades are no longer needed with stateless. To upgrade a stateless > node all you need to do is 'reboot from a newer image'. i.e. all > upgrades would be done via PXE server image replacement. So the flow of > 'upload ISO to running oVirt Node' is no longer even necessary > This is assuming PXE only use-case. I'm not sure it's the only one. >> * Collecting information; core dumps and logging may not >> be available due to lack of space? Or will it cause kernel >> panic if all space is consumed? > > We already provide ability to send kdumps to remote ssh/NFS location and > already provide the ability to use both collectd and rsyslogs to pipe > logs/stats to remote server(s). Local logs can be set to logrotate to a > reasonable size so that local RAM FS always contains recent log > information for quick triage, but long term historical logging would be > maintained on the rsyslog server > This needs to be co-ordinated with log-collection, as well as the bootstrapping code. > Perry -- /d "Willyoupleasehelpmefixmykeyboard?Thespacebarisbroken!" From pmyers at redhat.com Wed Feb 22 17:23:36 2012 From: pmyers at redhat.com (Perry Myers) Date: Wed, 22 Feb 2012 12:23:36 -0500 Subject: [node-devel] Support for stateless nodes In-Reply-To: <4F45218D.1020205@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <4F451290.3090103@redhat.com> <4F45218D.1020205@redhat.com> Message-ID: <4F452498.7040105@redhat.com> >> Well, if the network is busted which leads to the bridge rename failing, >> wouldn't the fact that the network is broken cause other problems anyhow? >> > Perry, my point is that we're increasing the chances to get > into these holes. Network is not busted most of the time, but occasionally > there's a glitch and we'd like to stay away from it. I'm sure > you know what I'm talking about. What if oVirt Node creates ifcfg-ovirt (instead of ifcfg-br0) by default as part of bringing up the network on each boot (via either DHCP or kernel args)? Then vdsm would never need to do this. This particular step could be something that is turned on/off only if vdsm is installed so that it doesn't affect any non-oVirt usages of oVirt Node (Archipel, etc) >>> * Today there's a supported flow that for nodes with >>> password, the user is allowed to use the "add host" >>> scenario. For stateless, it means re-configuring a password >>> on every boot... >> >> This flow would still be applicable. We are going to allow setting of >> the admin password embedded in the core ISO via an offline process. >> Once vdsm is fixed to use a non-root account for installation flow, this >> is no longer a problem > This is not exactly vdsm. More like vdsm-bootstrap. ack >> >> Also, if we (as described above) make registrations persistent across >> reboots by changing the registration flow a bit, then the install user >> password only need be set for the initial boot anyhow. >> >> Therefore I think as a requirement for stateless oVirt Node, we must >> have as a prerequsite removing root account usage for >> registration/installation >> > This is both for vdsm and engine, and I'm not sure it's that trivial. Understood, but it's a requirement for other things. There are security considerations for requiring remote root ssh access as part of your core infrastructure. So this needs to be dealt with regardless. >>> - Other issues >>> >>> * Local storage; so far we were able to define a local >>> storage in ovirt node. Stateless will block this ability. >> >> It shouldn't. The Node should be able to automatically scan locally >> attached disks to look for a well defined VG or partition label and >> based on that automatically activate/mount >> >> Stateless doesn't imply diskless. It is a requirement even for >> stateless node usage to be able to leverage locally attached disks both >> for VM storage and also for Swap. >> > Still, in a pure disk-less setup you will not have local storage. > See also Mike's answer. Sure. If you want diskless specifically and then complain about lack of swap or local storage for VMs... then you might not be getting the point :) That has no bearing on the stateless discussion, except that the first pass of stateless might not allow config of local disk/swap to start with. We might do it incrementally >>> * Node upgrade; currently it's possible to upgrade a node >>> from the engine. In stateless it will error, since no where >>> to d/l the iso file to. >> >> Upgrades are no longer needed with stateless. To upgrade a stateless >> node all you need to do is 'reboot from a newer image'. i.e. all >> upgrades would be done via PXE server image replacement. So the flow of >> 'upload ISO to running oVirt Node' is no longer even necessary >> > This is assuming PXE only use-case. I'm not sure it's the only one. Nope... copy oVirt Node 2.2.3 to a USB stick (via ovirt-iso-to-usb-disk) boot a host with it Later... copy oVirt Node 2.2.4 to same USB stick (via ovirt-iso-to-usb-disk) boot the host with it Yes, it requires you to touch the USB stick. If you specifically want stateless (implying no 'installation' of the Node) and you won't be using PXE to run, then it involves legwork. But again, we're not planning to eliminate the current 'install' methods. Stateless is in addition to installing to disk, and using the 'iso upload' upgrade method >>> * Collecting information; core dumps and logging may not >>> be available due to lack of space? Or will it cause kernel >>> panic if all space is consumed? >> >> We already provide ability to send kdumps to remote ssh/NFS location and >> already provide the ability to use both collectd and rsyslogs to pipe >> logs/stats to remote server(s). Local logs can be set to logrotate to a >> reasonable size so that local RAM FS always contains recent log >> information for quick triage, but long term historical logging would be >> maintained on the rsyslog server >> > This needs to be co-ordinated with log-collection, as well as the bootstrapping > code. Yep. Lots of stuff for vdsm/oVirt Engine team to do in order to meet this requirement :) In contrast, making oVirt Node stateless is quite trivial. Most of the work here is actually for vdsm and other related utilities (like log collector) Perry From iheim at redhat.com Wed Feb 22 20:38:01 2012 From: iheim at redhat.com (Itamar Heim) Date: Wed, 22 Feb 2012 22:38:01 +0200 Subject: [node-devel] [Engine-devel] Support for stateless nodes In-Reply-To: <4F452498.7040105@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <4F451290.3090103@redhat.com> <4F45218D.1020205@redhat.com> <4F452498.7040105@redhat.com> Message-ID: <4F455229.2030604@redhat.com> On 02/22/2012 07:23 PM, Perry Myers wrote: >>> Well, if the network is busted which leads to the bridge rename failing, >>> wouldn't the fact that the network is broken cause other problems anyhow? >>> >> Perry, my point is that we're increasing the chances to get >> into these holes. Network is not busted most of the time, but occasionally >> there's a glitch and we'd like to stay away from it. I'm sure >> you know what I'm talking about. > > What if oVirt Node creates ifcfg-ovirt (instead of ifcfg-br0) by default > as part of bringing up the network on each boot (via either DHCP or > kernel args)? what if admin wants this bonded, or bridgeless, or jumbo frames? stateless doesn't mean you can't save configuration somewhere on the network, right (could be on engine, could be just on some nfs or http location). if you have a tpm, just encrypt all the data to make sure no one tampered with your config, or if you don't care, just download your config (well, you trust the network to download your image without encryption, so no need to be fanatic i guess). so in short: 1. pxe boot the image 2. download from known location (kernerl param) the rest of the config you care about, certificates, etc. 3. use tpm for private key (or get the password to keystore in config via kernel parameter if you don't want/have tpm. I guess my main question is why does stateless implies no saved config for all the various issues > > Then vdsm would never need to do this. This particular step could be > something that is turned on/off only if vdsm is installed so that it > doesn't affect any non-oVirt usages of oVirt Node (Archipel, etc) > >>>> * Today there's a supported flow that for nodes with >>>> password, the user is allowed to use the "add host" >>>> scenario. For stateless, it means re-configuring a password >>>> on every boot... >>> >>> This flow would still be applicable. We are going to allow setting of >>> the admin password embedded in the core ISO via an offline process. >>> Once vdsm is fixed to use a non-root account for installation flow, this >>> is no longer a problem >> This is not exactly vdsm. More like vdsm-bootstrap. > > ack > >>> >>> Also, if we (as described above) make registrations persistent across >>> reboots by changing the registration flow a bit, then the install user >>> password only need be set for the initial boot anyhow. >>> >>> Therefore I think as a requirement for stateless oVirt Node, we must >>> have as a prerequsite removing root account usage for >>> registration/installation >>> >> This is both for vdsm and engine, and I'm not sure it's that trivial. > > Understood, but it's a requirement for other things. There are security > considerations for requiring remote root ssh access as part of your core > infrastructure. So this needs to be dealt with regardless. > >>>> - Other issues >>>> >>>> * Local storage; so far we were able to define a local >>>> storage in ovirt node. Stateless will block this ability. >>> >>> It shouldn't. The Node should be able to automatically scan locally >>> attached disks to look for a well defined VG or partition label and >>> based on that automatically activate/mount >>> >>> Stateless doesn't imply diskless. It is a requirement even for >>> stateless node usage to be able to leverage locally attached disks both >>> for VM storage and also for Swap. >>> >> Still, in a pure disk-less setup you will not have local storage. >> See also Mike's answer. > > Sure. If you want diskless specifically and then complain about lack of > swap or local storage for VMs... then you might not be getting the point :) > > That has no bearing on the stateless discussion, except that the first > pass of stateless might not allow config of local disk/swap to start > with. We might do it incrementally > >>>> * Node upgrade; currently it's possible to upgrade a node >>>> from the engine. In stateless it will error, since no where >>>> to d/l the iso file to. >>> >>> Upgrades are no longer needed with stateless. To upgrade a stateless >>> node all you need to do is 'reboot from a newer image'. i.e. all >>> upgrades would be done via PXE server image replacement. So the flow of >>> 'upload ISO to running oVirt Node' is no longer even necessary >>> >> This is assuming PXE only use-case. I'm not sure it's the only one. > > Nope... > > copy oVirt Node 2.2.3 to a USB stick (via ovirt-iso-to-usb-disk) > boot a host with it > > Later... > > copy oVirt Node 2.2.4 to same USB stick (via ovirt-iso-to-usb-disk) > boot the host with it > > Yes, it requires you to touch the USB stick. If you specifically want > stateless (implying no 'installation' of the Node) and you won't be > using PXE to run, then it involves legwork. > > But again, we're not planning to eliminate the current 'install' > methods. Stateless is in addition to installing to disk, and using the > 'iso upload' upgrade method > >>>> * Collecting information; core dumps and logging may not >>>> be available due to lack of space? Or will it cause kernel >>>> panic if all space is consumed? >>> >>> We already provide ability to send kdumps to remote ssh/NFS location and >>> already provide the ability to use both collectd and rsyslogs to pipe >>> logs/stats to remote server(s). Local logs can be set to logrotate to a >>> reasonable size so that local RAM FS always contains recent log >>> information for quick triage, but long term historical logging would be >>> maintained on the rsyslog server >>> >> This needs to be co-ordinated with log-collection, as well as the bootstrapping >> code. > > Yep. Lots of stuff for vdsm/oVirt Engine team to do in order to meet > this requirement :) > > In contrast, making oVirt Node stateless is quite trivial. Most of the > work here is actually for vdsm and other related utilities (like log > collector) > > Perry > _______________________________________________ > Engine-devel mailing list > Engine-devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel From pmyers at redhat.com Wed Feb 22 20:54:01 2012 From: pmyers at redhat.com (Perry Myers) Date: Wed, 22 Feb 2012 15:54:01 -0500 Subject: [node-devel] [Engine-devel] Support for stateless nodes In-Reply-To: <4F455229.2030604@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <4F451290.3090103@redhat.com> <4F45218D.1020205@redhat.com> <4F452498.7040105@redhat.com> <4F455229.2030604@redhat.com> Message-ID: <4F4555E9.8000908@redhat.com> On 02/22/2012 03:38 PM, Itamar Heim wrote: > On 02/22/2012 07:23 PM, Perry Myers wrote: >>>> Well, if the network is busted which leads to the bridge rename >>>> failing, >>>> wouldn't the fact that the network is broken cause other problems >>>> anyhow? >>>> >>> Perry, my point is that we're increasing the chances to get >>> into these holes. Network is not busted most of the time, but >>> occasionally >>> there's a glitch and we'd like to stay away from it. I'm sure >>> you know what I'm talking about. >> >> What if oVirt Node creates ifcfg-ovirt (instead of ifcfg-br0) by default >> as part of bringing up the network on each boot (via either DHCP or >> kernel args)? > > what if admin wants this bonded, or bridgeless, or jumbo frames? Right now those aren't things you can configure via oVirt Node, they're things you configure via oVirt Engine/vdsm. If we want to add things like this to oVirt Node, we can (just file some RFEs for us) and then we'll expose this sort of stuff via the kernel cmd line args as we do with all of our other config params > stateless doesn't mean you can't save configuration somewhere on the > network, right (could be on engine, could be just on some nfs or http > location). Correct. Our present thinking is that oVirt Node specific stuff us saved via kernel cmd line args persisted either on PXE server or directly on boot media. Anything else (oVirt Engine specific for example) would be stored in the mgmt server (OE) itself > if you have a tpm, just encrypt all the data to make sure no one > tampered with your config, or if you don't care, just download your > config (well, you trust the network to download your image without > encryption, so no need to be fanatic i guess). > so in short: > 1. pxe boot the image > 2. download from known location (kernerl param) the rest of the config > you care about, certificates, etc. > 3. use tpm for private key (or get the password to keystore in config > via kernel parameter if you don't want/have tpm. Agreed, though w/ vdsm you can simplify by just encrypting the certs 1. boot up, network and basics are configured via kernel cmd args or DNS SRV 2. vdsm retrieves cert from OE and decrypts via locally stored key (TPM or embedded in ISO) 3. vdsm now can securely communicate with OE, so it retrieves config for network/storage from OE and applies that config > I guess my main question is why does stateless implies no saved config > for all the various issues It doesn't imply no saved config, it implies no saved config on the local node itself. Config absolutely will need to be retrieved from a remote server. Right now, oVirt Node config (outside of vdsm) consists of: * network config (dhcp vs. static) * logging/rsyslog config/collectd * mgmt server config (oVirt Engine IP/port) * kdump remote config * auth info (admin password, etc) All of that can be handled through some combination of: * DNS SRV * kernel command line args * embedding things in the ISO pre-deployment (passwords for example) The remaining config is what vdsm does after the node has the above configured, so things like bonding, add'l interfaces, storage, should all be configured by vdsm on each boot by retrieving the configuration details from oVirt Engine and applying them on vdsm startup Perry From djasa at redhat.com Thu Feb 23 11:56:59 2012 From: djasa at redhat.com (David =?UTF-8?Q?Ja=C5=A1a?=) Date: Thu, 23 Feb 2012 12:56:59 +0100 Subject: [node-devel] [Engine-devel] Support for stateless nodes In-Reply-To: <4F451DAD.9090706@redhat.com> References: <1329922658.6140.14.camel@beelzebub.mburnsfire.net> <4F450ABB.30009@redhat.com> <4F4510FA.4050107@redhat.com> <1329926892.6140.39.camel@beelzebub.mburnsfire.net> <4F451CC7.4050909@redhat.com> <4F451DAD.9090706@redhat.com> Message-ID: <1329998220.4319.133.camel@dhcp-29-7.brq.redhat.com> Perry Myers p??e v St 22. 02. 2012 v 11:54 -0500: > >> As answered in the other response, there are kernel command line > >> parameters to set the management_server. Since this will likely be in a > >> pxe environment, setting the pxe profile to include > >> management_server= should be fine. > >> > > I agree it's a valid solution as long as you assume this is relevant > > for PXE only use case. > > Not necessarily... > > Take the ISO/USB Stick and you can embed the kargs into the ISO/USB > itself so that it always boots with that mgmt server arg > > This actually also enables use of 'stateless' combined with static IP > addressing as well. As you can create a USB Stick and embed the kargs > for the NIC configuration, rsyslog config, etc, etc. > > >> Another solution could be to setup a specific DNS SRV record that points > >> to the ovirt-engine and have node automatically query that for the > >> location. > > This was discussed in the past and for some reason not implemented. > > Concerns about security, iirc. Assumption that someone could hijack the > DNS SRV record and provide a man-in-the-middle oVirt Engine server. > What about DNSSEC validation for DNS records in node? David > If you're paranoid about security, don't use DNS SRV of course, instead > use hardcoded kargs as described above. But for some DNS SRV might be > an ok option > _______________________________________________ > Engine-devel mailing list > Engine-devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/engine-devel -- David Ja?a, RHCE SPICE QE based in Brno GPG Key: 22C33E24 Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24 From mburns at redhat.com Thu Feb 23 21:37:01 2012 From: mburns at redhat.com (Mike Burns) Date: Thu, 23 Feb 2012 16:37:01 -0500 Subject: [node-devel] oVirt Node is now building in jenkins Message-ID: <1330033021.6140.64.camel@beelzebub.mburnsfire.net> For those of you anxiously awaiting jenkins builds (or those that don't want to have to keep building your own iso images), Jenkins is now running daily ovirt-node builds. It will also trigger a new build if a new commit is pushed to the master ovirt-node branch. Still to come: Jenkins/Gerrit integration so that proposed patches get built automatically without waiting for an official commit/merge. http://jenkins.ovirt.org/job/ovirt-node/ Mike From abaron at redhat.com Mon Feb 27 10:06:12 2012 From: abaron at redhat.com (Ayal Baron) Date: Mon, 27 Feb 2012 05:06:12 -0500 (EST) Subject: [node-devel] [vdsm] [Engine-devel] Support for stateless nodes In-Reply-To: <1329998220.4319.133.camel@dhcp-29-7.brq.redhat.com> Message-ID: <749d6385-4ad6-4751-b5fa-2db43fb2d3e2@zmail13.collab.prod.int.phx2.redhat.com> ----- Original Message ----- > Perry Myers p??e v St 22. 02. 2012 v 11:54 -0500: > > >> As answered in the other response, there are kernel command line > > >> parameters to set the management_server. Since this will likely > > >> be in a > > >> pxe environment, setting the pxe profile to include > > >> management_server= should be fine. > > >> > > > I agree it's a valid solution as long as you assume this is > > > relevant > > > for PXE only use case. > > > > Not necessarily... > > > > Take the ISO/USB Stick and you can embed the kargs into the ISO/USB > > itself so that it always boots with that mgmt server arg > > > > This actually also enables use of 'stateless' combined with static > > IP > > addressing as well. As you can create a USB Stick and embed the > > kargs > > for the NIC configuration, rsyslog config, etc, etc. > > > > >> Another solution could be to setup a specific DNS SRV record > > >> that points > > >> to the ovirt-engine and have node automatically query that for > > >> the > > >> location. > > > This was discussed in the past and for some reason not > > > implemented. > > > > Concerns about security, iirc. Assumption that someone could > > hijack the > > DNS SRV record and provide a man-in-the-middle oVirt Engine server. > > > > What about DNSSEC validation for DNS records in node? This will require more than just changes to the registration process and it's quite difficult to track the required changes here on email. Let's setup a call to discuss this and try to capture the list of issues we already know about (I'm sure we'll discover more once we actually try to do this). To play devil's advocate though, I know there is interest, but I really don't understand the incentive. What is the *problem* you're trying to solve here (stateless is a solution) > > David > > > If you're paranoid about security, don't use DNS SRV of course, > > instead > > use hardcoded kargs as described above. But for some DNS SRV might > > be > > an ok option > > _______________________________________________ > > Engine-devel mailing list > > Engine-devel at ovirt.org > > http://lists.ovirt.org/mailman/listinfo/engine-devel > > -- > > David Ja?a, RHCE > > SPICE QE based in Brno > GPG Key: 22C33E24 > Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24 > > > > _______________________________________________ > vdsm-devel mailing list > vdsm-devel at lists.fedorahosted.org > https://fedorahosted.org/mailman/listinfo/vdsm-devel > From pmyers at redhat.com Mon Feb 27 19:38:44 2012 From: pmyers at redhat.com (Perry Myers) Date: Mon, 27 Feb 2012 14:38:44 -0500 Subject: [node-devel] [vdsm] [Engine-devel] Support for stateless nodes In-Reply-To: <749d6385-4ad6-4751-b5fa-2db43fb2d3e2@zmail13.collab.prod.int.phx2.redhat.com> References: <749d6385-4ad6-4751-b5fa-2db43fb2d3e2@zmail13.collab.prod.int.phx2.redhat.com> Message-ID: <4F4BDBC4.3020103@redhat.com> On 02/27/2012 05:06 AM, Ayal Baron wrote: > > > ----- Original Message ----- >> Perry Myers p??e v St 22. 02. 2012 v 11:54 -0500: >>>>> As answered in the other response, there are kernel command line >>>>> parameters to set the management_server. Since this will likely >>>>> be in a >>>>> pxe environment, setting the pxe profile to include >>>>> management_server= should be fine. >>>>> >>>> I agree it's a valid solution as long as you assume this is >>>> relevant >>>> for PXE only use case. >>> >>> Not necessarily... >>> >>> Take the ISO/USB Stick and you can embed the kargs into the ISO/USB >>> itself so that it always boots with that mgmt server arg >>> >>> This actually also enables use of 'stateless' combined with static >>> IP >>> addressing as well. As you can create a USB Stick and embed the >>> kargs >>> for the NIC configuration, rsyslog config, etc, etc. >>> >>>>> Another solution could be to setup a specific DNS SRV record >>>>> that points >>>>> to the ovirt-engine and have node automatically query that for >>>>> the >>>>> location. >>>> This was discussed in the past and for some reason not >>>> implemented. >>> >>> Concerns about security, iirc. Assumption that someone could >>> hijack the >>> DNS SRV record and provide a man-in-the-middle oVirt Engine server. >>> >> >> What about DNSSEC validation for DNS records in node? > > This will require more than just changes to the registration process > and it's quite difficult to track the required changes here on email. > Let's setup a call to discuss this and try to capture the list of > issues we already know about (I'm sure we'll discover more once we > actually try to do this). I'm fine with setting up a call as long as we publish the dial in info on list so that folks in the community can join in if they are interested. Also, stateless design has been a topic on the oVirt Node weekly IRC meeting for the last few weeks as we flesh out design issues, etc. I'd be happy for folks from other teams to join in there > To play devil's advocate though, I know there is interest, but I > really don't understand the incentive. > What is the *problem* you're trying to solve here (stateless is a solution) The primary motivation is diskless nodes. Being able to purchase mass quantities of servers and not needing to put any sort of storage in them. Since disks have much lower MTBF than other components (aside from fans probably), diskless servers require less maintenance. As mentioned earlier in this thread, diskless would mean no swap and no local storage. Both of which are suitable if you're using SAN (FC/iSCSI) based storage for VM images and if memory overcommit is not allowed. The argument for 'stateless without diskless' is a little more tenuous, however, if you're going to do stateless to get support for diskless, adding in support for local VM image storage and swap is fairly trivial. Andy, I think you've got some opinions here, care to weigh in? Otherwise if there is continued pushback on this feature, I'm happy to not waste effort on it. There are other things we can work on :) Perry From mburns at redhat.com Tue Feb 28 15:19:38 2012 From: mburns at redhat.com (Mike Burns) Date: Tue, 28 Feb 2012 10:19:38 -0500 Subject: [node-devel] oVirt Node Weekly Meeting Minutes Message-ID: <1330442378.6140.136.camel@beelzebub.mburnsfire.net> Minutes: http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-28-14.01.html Minutes (text): http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-28-14.01.txt Log: http://ovirt.org/meetings/ovirt/2012/ovirt.2012-02-28-14.01.log.html From mburns at redhat.com Tue Feb 28 21:10:07 2012 From: mburns at redhat.com (Mike Burns) Date: Tue, 28 Feb 2012 16:10:07 -0500 Subject: [node-devel] CIM support in oVirt Node Message-ID: <1330463407.6140.144.camel@beelzebub.mburnsfire.net> Based on discussions with various people, we decided to go forward with sblim-sfcb over tog-pegasus as the cim provider in ovirt-node. I'm working on setting the patches to add this support to ovirt-node and I've already hit the first issue. libvirt-cim requires tog-pegasus. Is this actually a hard requirement? Can this be changed to some generic "Requires cim" that both tog-pegasus and sblim-sfcb could provide? If not, are there any issues with us blacklisting tog-pegasus? Thanks Mike From mburns at redhat.com Tue Feb 28 21:17:43 2012 From: mburns at redhat.com (Mike Burns) Date: Tue, 28 Feb 2012 16:17:43 -0500 Subject: [node-devel] CIM support in oVirt Node In-Reply-To: <1330463407.6140.144.camel@beelzebub.mburnsfire.net> References: <1330463407.6140.144.camel@beelzebub.mburnsfire.net> Message-ID: <1330463863.6140.146.camel@beelzebub.mburnsfire.net> Adding others who have commented on cim previously. On Tue, 2012-02-28 at 16:10 -0500, Mike Burns wrote: > Based on discussions with various people, we decided to go forward with > sblim-sfcb over tog-pegasus as the cim provider in ovirt-node. I'm > working on setting the patches to add this support to ovirt-node and > I've already hit the first issue. > > libvirt-cim requires tog-pegasus. Is this actually a hard requirement? > Can this be changed to some generic "Requires cim" that both tog-pegasus > and sblim-sfcb could provide? If not, are there any issues with us > blacklisting tog-pegasus? > > Thanks > > Mike > > _______________________________________________ > node-devel mailing list > node-devel at ovirt.org > http://lists.ovirt.org/mailman/listinfo/node-devel From veillard at redhat.com Wed Feb 29 12:27:21 2012 From: veillard at redhat.com (Daniel Veillard) Date: Wed, 29 Feb 2012 20:27:21 +0800 Subject: [node-devel] CIM support in oVirt Node In-Reply-To: <1330463863.6140.146.camel@beelzebub.mburnsfire.net> References: <1330463407.6140.144.camel@beelzebub.mburnsfire.net> <1330463863.6140.146.camel@beelzebub.mburnsfire.net> Message-ID: <20120229122720.GJ4691@redhat.com> On Tue, Feb 28, 2012 at 04:17:43PM -0500, Mike Burns wrote: > Adding others who have commented on cim previously. > > On Tue, 2012-02-28 at 16:10 -0500, Mike Burns wrote: > > Based on discussions with various people, we decided to go forward with > > sblim-sfcb over tog-pegasus as the cim provider in ovirt-node. I'm > > working on setting the patches to add this support to ovirt-node and > > I've already hit the first issue. > > > > libvirt-cim requires tog-pegasus. Is this actually a hard requirement? > > Can this be changed to some generic "Requires cim" that both tog-pegasus > > and sblim-sfcb could provide? If not, are there any issues with us > > blacklisting tog-pegasus? I'm pretty sure I heard/read Chip say in the past that they used sblim in regression testing :-) I think the Require is the result of RHEL packaging where tog-pegasus was the only one present. Also I don't know (but might exist) any construct allowing Requires: tog-pegasus or sblim-sfcb as you said the only way in rpm is to define a generic provide (maybe more specific than just 'cim' what about 'cim-server') then Requires: cim-server in the libvirt-cim and add the Provides: cim-server in all server packages. The only problem with that is that we ultimately need to "fix" all packages. The first thing would be to make bug requests against tog-pegasus sblim-sfcb to have that added. Unfortunately I don't think we can get this in RHEL-6.3 at this point... Daniel -- Daniel Veillard | libxml Gnome XML XSLT toolkit http://xmlsoft.org/ daniel at veillard.com | Rpmfind RPM search engine http://rpmfind.net/ http://veillard.com/ | virtualization library http://libvirt.org/ From mvanhorssen at vluchtelingenwerk.nl Wed Feb 29 14:54:28 2012 From: mvanhorssen at vluchtelingenwerk.nl (Michel van Horssen) Date: Wed, 29 Feb 2012 15:54:28 +0100 (CET) Subject: [node-devel] Can not get Node to connect properly In-Reply-To: <66e9fe39-86bc-48d2-bb32-f7fcf41b328e@mailstore1.vluchtelingenwerk.nl> Message-ID: <56fee533-0d0d-4ecd-9227-02bd8f6e7617@mailstore1.vluchtelingenwerk.nl> Hi, I was fighting with the node to connect to my engine. mburns told me on IRC that you can only configure 1 network interface in the node tui and had to do the other one in the management interface on the engine server. Thing is: a. I have a management network and a separate iSCSI network b. I'm working with a DNS in an AD using .local as a domain extension Having installed my first host with a minimal FC16 and a vdsmd I learned that the iSCSI network needs to be connected to the node/host which connects to the engine. And first install from the management interface needs the connecting host/node to be able to resolve the engines name. For that to take place I needed a "search domainname.local" in my /etc/resolve.com If the "a" is not needed then at least "b" is, but when I put a host name of host.domainame.local in the node tui I get a warning telling me it is an "invalid hostname". And is "Non operational" in the management interface. Any help? Thnx, Michel From cvincent at linux.vnet.ibm.com Wed Feb 29 17:22:26 2012 From: cvincent at linux.vnet.ibm.com (Chip Vincent) Date: Wed, 29 Feb 2012 12:22:26 -0500 Subject: [node-devel] CIM support in oVirt Node In-Reply-To: <20120229122720.GJ4691@redhat.com> References: <1330463407.6140.144.camel@beelzebub.mburnsfire.net> <1330463863.6140.146.camel@beelzebub.mburnsfire.net> <20120229122720.GJ4691@redhat.com> Message-ID: <4F4E5ED2.3050800@linux.vnet.ibm.com> On 02/29/2012 07:27 AM, Daniel Veillard wrote: > On Tue, Feb 28, 2012 at 04:17:43PM -0500, Mike Burns wrote: >> Adding others who have commented on cim previously. >> >> On Tue, 2012-02-28 at 16:10 -0500, Mike Burns wrote: >>> Based on discussions with various people, we decided to go forward with >>> sblim-sfcb over tog-pegasus as the cim provider in ovirt-node. I'm >>> working on setting the patches to add this support to ovirt-node and >>> I've already hit the first issue. >>> >>> libvirt-cim requires tog-pegasus. Is this actually a hard requirement? >>> Can this be changed to some generic "Requires cim" that both tog-pegasus >>> and sblim-sfcb could provide? If not, are there any issues with us >>> blacklisting tog-pegasus? > I'm pretty sure I heard/read Chip say in the past that they used sblim > in regression testing :-) I think the Require is the result of RHEL > packaging where tog-pegasus was the only one present. Also I don't > know (but might exist) any construct allowing You're correct that libvirt-cim should work with SFCB, but it's been quite a while since any real testing has been done with it since almost all of our clients today use tog-pegasus. It's safe to assume there will be some bugs integrating the latest libvirt-cim with the latest SFCB CIMOM. > Requires: tog-pegasus or sblim-sfcb > > as you said the only way in rpm is to define a generic provide > (maybe more specific than just 'cim' what about 'cim-server') then > Requires: cim-server > in the libvirt-cim and add the Provides: cim-server in all server > packages. The only problem with that is that we ultimately need to > "fix" all packages. > > The first thing would be to make bug requests against tog-pegasus > sblim-sfcb to have that added. Unfortunately I don't think we can > get this in RHEL-6.3 at this point... > > Daniel > -- Chip Vincent Open Virtualization IBM Linux Technology Center cvincent at linux.vnet.ibm.com