[Engine-devel] Cloud-Init integration

Mon Jun 17 10:40:44 UTC 2013

On Jun 16, 2013, at 18:04 , Dan Kenigsberg <danken at redhat.com> wrote:

> On Sun, Jun 16, 2013 at 05:50:38PM +0300, Greg Padgett wrote:
>> On 06/16/2013 05:08 PM, Mike Kolesnik wrote:
>>> ----- Original Message -----
>>>> On Thu, Mar 28, 2013 at 07:35:23PM -0400, Greg Padgett wrote:
>>>>> Hi Everyone,
>>>>> 
>>>>> I'd like to propose a feature we've been doing some investigation
>>>>> into, which is to integrate cloud-init support into oVirt.
>>>>> 
>>>>> Cloud-init is used to help provision new Linux systems by setting
>>>>> the hostname, ip, ssh keys, timezone, injecting files, and more.
>>>>> It's used by OpenStack (amongst others) now, and has a lot of
>>>>> features that may be helpful to our users.
>>>>> 
>>>>> Details are still evolving, but for more info please see the wiki page:
>>>>> 
>>>>> http://www.ovirt.org/Features/Cloud-Init_Integration
>>>>> 
>>>>> All feedback is welcome!
>>>> 
>>>> All feedback? Even if it's 3 months too late?
>>>> 
>> 
>> I'll have to be more careful next time to specify a time limit :)
>> But really, thank you both for the comments.  Feel free to add them
>> to a "nice to have" list on the wiki page, or I can add it.  Replies
>> inline...
>> 
>>>> If so, then here's a few comments:
>>>> 
>>>> 1. oVirt-3.2 completely supports IPv6 within guests. It would be nice to
>>>>   carry this on to cloud-init, by allowing to set the IPv6 address of
>>>>   the guest. (or are we happy with the auto configured ipv6 addresses?)
>>>> 
>> 
>> It would, but unfortunately cloud-init doesn't yet translate ipv6 fields
>> from /etc/network/interfaces (its chosen networking input format) to
>> ifcfg files.  Now that you mention it, it doesn't add IPV6INIT=yes,
>> either.
> 
> ifcfg files? What's that? Those easily-edited text files that are being
> deprecated by NetworkManager? Does cloud-init play well with the latter?
> (we found a couple of pitfalls, the hard way).
> 
>> 
>> These are things we can add--to both oVirt and cloud-init, I guess at a
>> later time.
>> 
>>>> 2. I think that the GUI used for setting IP addresses should be
>>>>   immitated here. It allows Static/DHCP/None, and disables the
>>>>   irrelvant fields when DHCP/None is selected.
>>> 
>> 
>> It's close: there is a checkbox for dhcp, and if selected it will
>> hide the non-relevant fields.
> 
> I hate surprises, so I'm in favor of having the same thing, as well as
> the "keep recent config in memory when bootproto is moved to None"
> semantics.
> 
> This applies more strongly to the REST api. Host level configuration
> http://www.ovirt.org/Features/Design/Network/SetupNetworks#Scheme has
>   <host_nic>
>     <name>em1</name>
>     <ip address="10.35.1.247" netmask="255.255.254.0" gateway="10.35.1.254"/>
>     <boot_protocol>dhcp</boot_protocol>
>   </host_nic>
> 
> And for reporting guest configuration
> http://www.ovirt.org/Feature/ReportingVnicImplementation#API_Changes has
>  <network_device>
>     <name>p1p2</name>
>     <description>guest reported data</description>
>     <ips>
>         <ip version="v4" address="10.35.1.177"/>
>         <ip version="v6" address="fe80::21a:4aff:fe16:151"/>
>     </ips>
>  </network_device>
> 
> which we should strive to maintain with this feature. Even if cloud-init is not
> currently capable of setting multiple addresses, let the API allow for it.
> 
>> 
>>> Additionally, you should consider showing the vNIC name and not "eth0" etc.
>>> IIUC udev rules are rather unpredictable in this regard and could give your
>>> vNIC a different name on different VM instances (probably by MAC address and/or
>>> PCI address).
>>> Either way, I think it's less confusing to refer to the vNIC itself.
>>> 
>> 
>> Similarly to the above, I'm not sure how much cloud-init supports
>> with respect to vNICs.  The name is used as the
>> /etc/network/interfaces adapter name and the ifcfg-* suffix (per
>> distro), so I guess if you plumbed everything around that in the
>> image and could finish the setup with just the interfaces/ifcfg
>> config file then it would work as-is.  If that isn't adequate (I
>> haven't looked or tested) then we may need to consider submitting a
>> patch to add this to cloud-init.
> 
> Either way, RHEV should not expose in its own interface the "eth0" names
> that we cannot enforce within the guest. E.g., my Fedora has funny
> interface names such as em1 and wlp3s0, nothing like the good old eth*.
> 
>> 
>>>> 3. Is "Support custom volume label for vm payloads" still on the TODO
>>>>   list? Note: F19's dosfstools has renamed mkfs.msdos to mkfs.fat.
>>>>   This means that creating a virtual floppy with a payload currently
>>>>   does not work there (it's a tiny fix, for sure).
>>>> 
>> 
>> It's implemented.  Of course, it still uses mkfs.msdos.
> 
> Implemented but not yet posted, I presume? Because upstream vdsm's does
> not use mkfs.msdos's -n.
> 
>> 
>>>> 4. I do not see ovirt-guest-agent mentioned in the feaure page. It is
>>>>   the obvious channel to report that cloud-init is Done to Engine.
>>>> 
>> 
>> You're right, and this is one part of the feature that hasn't been
>> done. It may also require some work on cloud-init, or for us to use
>> a different input format (i.e. a mime-formatted sequence instead of
>> vanilla config-drive-2).  It would be great to add this, though my
>> time to do that now is a bit limited.
> 
> What is the target oVirt version for this feature, by the way? It is not listed
> in http://www.ovirt.org/OVirt_3.3_release-management, so I presume it's for
> 3.4?

no, should be 3.3. I think it's an omission. The goal is not to implement all network options we can ever have but rather an initial integration with cloud-init, whatever it does support now. And then go from there…

> 
>> 
>>>> 5. When we come to implement auto-generate of system ssh key, we may
>>>>   want to install a virtio-rng device in our VMs, to ensure that the
>>>>   keys are not too easy to guess. (or create the key in the host and
>>>>   inject it to the guest)
>>> 
>> 
>> Good idea.  I'm not too familiar with virtio-rng, but if the image
>> can be configured to use it then the regeneration should follow
>> suit.  So, not a limitation right now but not as easy as it could
>> be.  It's another case of needing either a cloud-init patch for
>> support or (I'm thinking more likely in this case) some scripting
>> and using a mime-formatted input to cloud-init.
> 
> I do not think any user-space changes are needed in the guest. The guest
> kernel should know how to suck entropy from the virtio-rng device and
> expose it for guest applications in /dev/random.

I suppose we want to generate it on the host and maintain/manage the ssh keys in the engine. The latter for sure so it doesn't seem much worth implementing virtio-rng in guest, but I may be wrong.

> 
>> 
>>> 6. Is this going to be supported on template/instance type level?
>>> Probably static IP is not wise on a template, but the other options seem
>>> like they would be the same for most VMs from the same template/instance.
>>> 
>> 
>> Today it's just on "Run VM Once" but once we can persist the
>> cloud-init options I don't see why we wouldn't want to allow
>> persisting at least at least some of the fields for
>> templates/instance types.
>> 
>>> 7. Don't forget backwards compatibility considerations for the engine-VDSM
>>> communication, if you're using APIs that aren't available in older VDSM versions.
>>> 
>> 
>> Thanks, it does rely on some new vdsm API features.
> 
> Would you document the suggested API changes in the feature page?
> 
> Dan.
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel