[Engine-devel] Cloud-Init integration

[Dan Kenigsberg]

On Sun, Jun 16, 2013 at 05:50:38PM +0300, Greg Padgett wrote:
> On 06/16/2013 05:08 PM, Mike Kolesnik wrote:
> >----- Original Message -----
> >>On Thu, Mar 28, 2013 at 07:35:23PM -0400, Greg Padgett wrote:
> >>>Hi Everyone,
> >>>
> >>>I'd like to propose a feature we've been doing some investigation
> >>>into, which is to integrate cloud-init support into oVirt.
> >>>
> >>>Cloud-init is used to help provision new Linux systems by setting
> >>>the hostname, ip, ssh keys, timezone, injecting files, and more.
> >>>It's used by OpenStack (amongst others) now, and has a lot of
> >>>features that may be helpful to our users.
> >>>
> >>>Details are still evolving, but for more info please see the wiki page:
> >>>
> >>>http://www.ovirt.org/Features/Cloud-Init_Integration
> >>>
> >>>All feedback is welcome!
> >>
> >>All feedback? Even if it's 3 months too late?
> >>
> 
> I'll have to be more careful next time to specify a time limit :)
> But really, thank you both for the comments.  Feel free to add them
> to a "nice to have" list on the wiki page, or I can add it.  Replies
> inline...
> 
> >>If so, then here's a few comments:
> >>
> >>1. oVirt-3.2 completely supports IPv6 within guests. It would be nice to
> >>    carry this on to cloud-init, by allowing to set the IPv6 address of
> >>    the guest. (or are we happy with the auto configured ipv6 addresses?)
> >>
> 
> It would, but unfortunately cloud-init doesn't yet translate ipv6 fields
> from /etc/network/interfaces (its chosen networking input format) to
> ifcfg files.  Now that you mention it, it doesn't add IPV6INIT=yes,
> either.

ifcfg files? What's that? Those easily-edited text files that are being
deprecated by NetworkManager? Does cloud-init play well with the latter?
(we found a couple of pitfalls, the hard way).

> 
> These are things we can add--to both oVirt and cloud-init, I guess at a
> later time.
> 
> >>2. I think that the GUI used for setting IP addresses should be
> >>    immitated here. It allows Static/DHCP/None, and disables the
> >>    irrelvant fields when DHCP/None is selected.
> >
> 
> It's close: there is a checkbox for dhcp, and if selected it will
> hide the non-relevant fields.

I hate surprises, so I'm in favor of having the same thing, as well as
the "keep recent config in memory when bootproto is moved to None"
semantics.

This applies more strongly to the REST api. Host level configuration
http://www.ovirt.org/Features/Design/Network/SetupNetworks#Scheme has
   <host_nic>
     <name>em1</name>
     <ip address="10.35.1.247" netmask="255.255.254.0" gateway="10.35.1.254"/>
     <boot_protocol>dhcp</boot_protocol>
   </host_nic>

And for reporting guest configuration
http://www.ovirt.org/Feature/ReportingVnicImplementation#API_Changes has
  <network_device>
     <name>p1p2</name>
     <description>guest reported data</description>
     <ips>
         <ip version="v4" address="10.35.1.177"/>
         <ip version="v6" address="fe80::21a:4aff:fe16:151"/>
     </ips>
  </network_device>

which we should strive to maintain with this feature. Even if cloud-init is not
currently capable of setting multiple addresses, let the API allow for it.

> 
> >Additionally, you should consider showing the vNIC name and not "eth0" etc.
> >IIUC udev rules are rather unpredictable in this regard and could give your
> >vNIC a different name on different VM instances (probably by MAC address and/or
> >PCI address).
> >Either way, I think it's less confusing to refer to the vNIC itself.
> >
> 
> Similarly to the above, I'm not sure how much cloud-init supports
> with respect to vNICs.  The name is used as the
> /etc/network/interfaces adapter name and the ifcfg-* suffix (per
> distro), so I guess if you plumbed everything around that in the
> image and could finish the setup with just the interfaces/ifcfg
> config file then it would work as-is.  If that isn't adequate (I
> haven't looked or tested) then we may need to consider submitting a
> patch to add this to cloud-init.

Either way, RHEV should not expose in its own interface the "eth0" names
that we cannot enforce within the guest. E.g., my Fedora has funny
interface names such as em1 and wlp3s0, nothing like the good old eth*.

> 
> >>3. Is "Support custom volume label for vm payloads" still on the TODO
> >>    list? Note: F19's dosfstools has renamed mkfs.msdos to mkfs.fat.
> >>    This means that creating a virtual floppy with a payload currently
> >>    does not work there (it's a tiny fix, for sure).
> >>
> 
> It's implemented.  Of course, it still uses mkfs.msdos.

Implemented but not yet posted, I presume? Because upstream vdsm's does
not use mkfs.msdos's -n.

> 
> >>4. I do not see ovirt-guest-agent mentioned in the feaure page. It is
> >>    the obvious channel to report that cloud-init is Done to Engine.
> >>
> 
> You're right, and this is one part of the feature that hasn't been
> done. It may also require some work on cloud-init, or for us to use
> a different input format (i.e. a mime-formatted sequence instead of
> vanilla config-drive-2).  It would be great to add this, though my
> time to do that now is a bit limited.

What is the target oVirt version for this feature, by the way? It is not listed
in http://www.ovirt.org/OVirt_3.3_release-management, so I presume it's for
3.4?

> 
> >>5. When we come to implement auto-generate of system ssh key, we may
> >>    want to install a virtio-rng device in our VMs, to ensure that the
> >>    keys are not too easy to guess. (or create the key in the host and
> >>    inject it to the guest)
> >
> 
> Good idea.  I'm not too familiar with virtio-rng, but if the image
> can be configured to use it then the regeneration should follow
> suit.  So, not a limitation right now but not as easy as it could
> be.  It's another case of needing either a cloud-init patch for
> support or (I'm thinking more likely in this case) some scripting
> and using a mime-formatted input to cloud-init.

I do not think any user-space changes are needed in the guest. The guest
kernel should know how to suck entropy from the virtio-rng device and
expose it for guest applications in /dev/random.

> 
> >6. Is this going to be supported on template/instance type level?
> >Probably static IP is not wise on a template, but the other options seem
> >like they would be the same for most VMs from the same template/instance.
> >
> 
> Today it's just on "Run VM Once" but once we can persist the
> cloud-init options I don't see why we wouldn't want to allow
> persisting at least at least some of the fields for
> templates/instance types.
> 
> >7. Don't forget backwards compatibility considerations for the engine-VDSM
> >communication, if you're using APIs that aren't available in older VDSM versions.
> >
> 
> Thanks, it does rely on some new vdsm API features.

Would you document the suggested API changes in the feature page?

Dan.