* Just stating the obvious, which is users need to remove-add the host on every reboot. This will not make this feature a lovable one from user's point of view.
I think the point mburns is trying to make in his initial email is that we're going to need to do some joint work between node and vdsm teams to change the registration process so that this is no longer necessary. It will require some redesigning of the registration process
* During initial boot, vdsm-reg configures the networking and creates a management network bridge. This is a very delicate process which may fail due to networking issues such as resolution, routing, etc. So re-doing this on every boot increases the chances of loosing a node due to network problems.
Well, if the network is busted which leads to the bridge rename failing, wouldn't the fact that the network is broken cause other problems anyhow? So I don't see this as a problem. If your network doesn't work properly, don't expect hosts in the network to subsequently work properly. As an aside, why is reverse DNS lookup a requirement? If we remove that it makes things a lot easier, no?
* CA pollution; generating a certificate on each reboot for each node will create a huge number of certificates in the engine side, which eventually may damage the CA. (Unsure if there's a limitation to certificates number, but having hundreds of junk cert's can't be good).
I don't think we should regenerate a new certificate on each boot. I think we need a way for 'an already registered host to retrieve it's certificate from the oVirt Engine server' Using an embedded encryption key (if you trust your mgmt network or are booting from embedded flash), or for the paranoid a key stored in TPM can be used to have vdsm safely retrieve this from the oVirt Engine server on each boot so that it's not required to regenerate/reregister on each boot
* Today there's a supported flow that for nodes with password, the user is allowed to use the "add host" scenario. For stateless, it means re-configuring a password on every boot...
This flow would still be applicable. We are going to allow setting of the admin password embedded in the core ISO via an offline process. Once vdsm is fixed to use a non-root account for installation flow, this is no longer a problem Also, if we (as described above) make registrations persistent across reboots by changing the registration flow a bit, then the install user password only need be set for the initial boot anyhow. Therefore I think as a requirement for stateless oVirt Node, we must have as a prerequsite removing root account usage for registration/installation
- Other issues
* Local storage; so far we were able to define a local storage in ovirt node. Stateless will block this ability.
It shouldn't. The Node should be able to automatically scan locally attached disks to look for a well defined VG or partition label and based on that automatically activate/mount Stateless doesn't imply diskless. It is a requirement even for stateless node usage to be able to leverage locally attached disks both for VM storage and also for Swap.
* Node upgrade; currently it's possible to upgrade a node from the engine. In stateless it will error, since no where to d/l the iso file to.
Upgrades are no longer needed with stateless. To upgrade a stateless node all you need to do is 'reboot from a newer image'. i.e. all upgrades would be done via PXE server image replacement. So the flow of 'upload ISO to running oVirt Node' is no longer even necessary
* Collecting information; core dumps and logging may not be available due to lack of space? Or will it cause kernel panic if all space is consumed?
We already provide ability to send kdumps to remote ssh/NFS location and already provide the ability to use both collectd and rsyslogs to pipe logs/stats to remote server(s). Local logs can be set to logrotate to a reasonable size so that local RAM FS always contains recent log information for quick triage, but long term historical logging would be maintained on the rsyslog server Perry