[ovirt-devel] oVirt Cloud-init
Juan Hernández
jhernand at redhat.com
Sat Feb 18 12:58:37 UTC 2017
On 02/17/2017 08:11 PM, Marc Young wrote:
> Per the script, i'll change that, probably just wasnt paying attention
> to where i ran chkconfig
>
> Per cloud-init YAML, not sure. It was in a previous reply so i gave it a
> shot. But i can't find that usage anywhere either so i gave up on it.
>
> Per troubleshooting:
>
> Spinning up a VM with dns_servers and dns_search:
>
> 66: vm_configuration[:initialization][:dns_servers] =
> iface_options[:dns_servers] unless iface_options[:dns_servers].nil?
> 67: vm_configuration[:initialization][:dns_search] =
> iface_options[:dns_search] unless iface_options[:dns_search].nil?
> 68: require 'pry'
> 69: binding.pry
> 70:
> => 71: machine.start(
> 72: use_cloud_init: true,
> 73: vm: vm_configuration
> 74: )
> 75:
> 76: @app.call(env)
>
> [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)>
> vm_configuration
> => {:initialization=>
> {:host_name=>"test",
> :nic_configurations=>[{:name=>"eth0", :on_boot=>true,
> :boot_protocol=>"static", :ip=>{:version=>"v4",
> :address=>"192.168.2.200", :gateway=>"192.168.2.1",
> :netmask=>"255.255.255.0"}}],
> :custom_script=>
> "manage-resolv-conf: true\n\nresolv_conf:\n nameservers:
> ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com
> <http://foo.example.com>\n - bar.example.com
> <http://bar.example.com>\n domain: example.com
> <http://example.com>\n options:\n rotate: true\n timeout: 1\n",
> :dns_servers=>"192.168.2.1",
> :dns_search=>"test.local"}}
> [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)>
> ==> default: Waiting for VM to get an IP address...
> ==> default: Machine is booted and ready for use!
> ==> default: Rsyncing folder:
> /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant
> ==> default: Setting hostname...
> default:
> default: Vagrant insecure key detected. Vagrant will
> automatically replace
> default: this with a newly generated keypair for better security.
> default:
> default: Inserting generated public key within guest...
> default: Removing insecure key from the guest if it's present...
> default: Key inserted! Disconnecting and reconnecting using new
> SSH key...
>
>
> now ill ssh in and look around
>
> myoung dev ~ repos github vagrant-ovirt4
> feature/gh65 ✎ 5❓ 1 $ vagrant ssh
> You appear to be running Vagrant outside of the official installers.
> Note that the installers are what ensure that Vagrant has all required
> dependencies, and Vagrant assumes that these dependencies exist. By
> running outside of the installer environment, Vagrant may not function
> properly. To remove this warning, install Vagrant using one of the
> official packages from vagrantup.com <http://vagrantup.com>.
>
> Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198
> [vagrant at test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt
> #cloud-config
> output:
> all: '>> /var/log/cloud-init-output.log'
> disable_root: 0
> runcmd:
> - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo
> ''datasource_list:
> ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg'
> ssh_pwauth: true
> chpasswd:
> expire: false
> user: root
> manage-resolv-conf: true
>
> resolv_conf:
> nameservers: ['8.8.4.4', '8.8.8.8']
> searchdomains:
> - foo.example.com <http://foo.example.com>
> - bar.example.com <http://bar.example.com>
> domain: example.com <http://example.com>
> options:
> rotate: true
> timeout: 1
Apparently you are passing in the 'custom_script' some DNS
configuration, which may conflict with the DNS configuration that you
are passing in the 'dns_servers' and 'dns_search' options. It could be
that cloud-init ignores one of them. I'd suggest that you try to remove
completely the 'custom_script' parameter.
> [vagrant at test ~]$ sudo cat /etc/resolv.conf
> # Generated by NetworkManager
> search localdomain
>
> [vagrant at test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
> NM_CONTROLLED=no
> NETMASK=255.255.255.0
> BOOTPROTO=static
> DEVICE=eth0
> IPADDR=192.168.2.200
> GATEWAY=192.168.2.1
> ONBOOT=yes
>
> [vagrant at test ~]$ sudo cat /var/log/cloud-init* | grep -i dns
> [vagrant at test ~]$ sudo cat /var/log/cloud-init* | grep -i local
> Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running
> 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds.
> Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running
> 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds.
> Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017
> 03:45:01 +0000. Up 1310.61 seconds.
> Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017
> 05:16:45 +0000. Up 11.00 seconds.
> Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017
> 05:03:35 +0000. Up 5.23 seconds.
> Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000.
> Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1]. Up
> 15.37 seconds
>
>
>
> On the vm host running that VM:
>
>
> [myoung at ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test
> qemu 12456 1 9 05:03 ? 00:00:19
> /usr/libexec/qemu-kvm -name ...snipped
> [myoung at ovirt ~]$ sudo cp
> /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img
> .
> [myoung at ovirt ~]$ sudo mount -o loop,ro
> c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img
> /mnt
> [myoung at ovirt ~]$ find /mnt -type f
> find: ‘/mnt’: Permission denied
> [myoung at ovirt ~]$ sudo find /mnt -type f
> /mnt/openstack/content/0000
> /mnt/openstack/latest/meta_data.json
> /mnt/openstack/latest/user_data
> [myoung at ovirt ~]$ sudo cat /mnt/openstack/content/0000
> auto eth0
> iface eth0 inet static
> address 192.168.2.200
> netmask 255.255.255.0
> gateway 192.168.2.1
> dns-nameservers 192.168.2.1
> dns-search test.local
> [myoung at ovirt ~]$ sudo umount
> c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img
>
> Cloud-init definitely isn't working right away with resolv.conf, but I'm
> definitely passing it correctly to the API, it shows up on the floppy as
> you described but it's not making its way to
> /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other settings
> for sure are (such as address, netmask,)
>
> If i add that information to /etc/sysconfig/network-scripts/ifcfg-eth0
> and bounce network it all works:
>
> [vagrant at test ~]$ ping -c 3 www.google.com <http://www.google.com>
> ping: www.google.com <http://www.google.com>: Name or service not known
> [vagrant at test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' |
> sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null
> [vagrant at test ~]$ ping -c 3 www.google.com <http://www.google.com>
> ping: www.google.com <http://www.google.com>: Name or service not known
> [vagrant at test ~]$ sudo service network restart
> Restarting network (via systemctl): [ OK ]
> [vagrant at test ~]$ ping -c 3 www.google.com <http://www.google.com>
> PING www.google.com <http://www.google.com> (216.58.217.4) 56(84)
> bytes of data.
> 64 bytes from den03s09-in-f4.1e100.net
> <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=1 ttl=54
> time=55.8 ms
> 64 bytes from den03s09-in-f4.1e100.net
> <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=2 ttl=54
> time=43.9 ms
> ^C
> --- www.google.com <http://www.google.com> ping statistics ---
> 2 packets transmitted, 2 received, 0% packet loss, time 1001ms
> rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms
> [vagrant at test ~]$ sudo cat /etc/resolv.conf
> # Generated by NetworkManager
> search localdomain
> nameserver 192.168.2.113
> nameserver 192.168.2.1
>
>
>
>
> On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand at redhat.com
> <mailto:jhernand at redhat.com>> wrote:
>
> On 02/17/2017 06:00 AM, Marc Young wrote:
> > I'm apparently really bad at email, I replied only to Shahar, not the
> > whole thread.
> >
> > Vinzenz your email slipped first, so to answer your question:
> >
> > It's the latest Centos 7 with these installed:
> >
> > cloud-init-0.7.5-10.el7.centos.1
> > kernel-3.10.0-514 <tel:3.10.0-514> <tel:3.10.0-514
> <tel:3.10.0-514>>.el7
> > ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1.el7.centos
> >
> > The setup script I use to create a template is here:
> > https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_for_box.sh
> <https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_for_box.sh>
> >
> <https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_for_box.sh
> <https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_for_box.sh>>
> >
>
> In that script you run "chkconfig cloud-init on" *before* installing the
> cloud-init package. That is irrelevant, as the cloud-init services are
> enabled by default when the package is installed. But worth changing.
>
> > The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos
> > The ruby SDK i'm working with is 4.1.2
> >
> > Halfway through I realized that it's actually supported in the API:
> >
> > custom_script String
> > dns_search String
> > dns_servers String
> >
> >
> > It also shows usage here:
> >
> https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/start_vm_with_cloud_init.rb
> <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/start_vm_with_cloud_init.rb>
> >
> <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/start_vm_with_cloud_init.rb
> <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/start_vm_with_cloud_init.rb>>
> >
> > Here's some verification:
> >
> > 66:
> vm_configuration[:initialization][:dns_servers] =
> > iface_options[:dns_servers] unless
> iface_options[:dns_servers].nil?
> > 67: vm_configuration[:initialization][:dns_search] =
> > iface_options[:dns_search] unless iface_options[:dns_search].nil?
> > 68: require 'pry'
> > 69: binding.pry
> > 70:
> > => 71: machine.start(
> > 72: use_cloud_init: true,
> > 73: vm: vm_configuration
> > 74: )
> > 75:
> > 76: @app.call(env)
> >
> > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)>
> > vm_configuration
> > => {:initialization=>
> > {:host_name=>"test",
> > :nic_configurations=>[{:name=>"eth0", :on_boot=>true,
> > :boot_protocol=>"static", :ip=>{:version=>"v4",
> > :address=>"192.168.2.200", :gateway=>"192.168.2.1",
> > :netmask=>"255.255.255.0"}}],
> > :custom_script=>nil,
> > :dns_servers=>"192.168.2.1",
> > :dns_search=>"test.local"}}
> >
> >
> > But it didn't do anything:
> >
> > [vagrant at test ~]$ cat /etc/resolv.conf
> > # Generated by NetworkManager
> > search localdomain
> >
> > [vagrant at test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
> > NM_CONTROLLED=no
> > NETMASK=255.255.255.0
> > BOOTPROTO=static
> > DEVICE=eth0
> > IPADDR=192.168.2.200
> > GATEWAY=192.168.2.1
> > ONBOOT=yes
> >
> >
> > The same is also true using cloud_init:
> >
> > ovirt.cloud_init =<<EOF
> > write_files:
> > - content: |
> > wat
> > path: /tmp/something.txt
> > permissions: '0644'
> > network-interfaces: |
> > auto eth0
> > iface eth0 inet static
> > address 192.168.2.201
> > network 192.168.2.0
> > netmask 255.255.255.0
> > gateway 192.168.2.1
> > dns-nameservers 192.168.2.113 192.168.2.1
> > EOF
> >
>
> Is this ^ supposed to work in cloud-init? I didn't find it in the
> documentation. I thought that the only way to provide network interface
> configuration is via the 'openstack/content/whatever' file within the
> generated floppy.
>
> >
> > and inspection:
> >
> > 66:
> vm_configuration[:initialization][:dns_servers] =
> > iface_options[:dns_servers] unless
> iface_options[:dns_servers].nil?
> > 67: vm_configuration[:initialization][:dns_search] =
> > iface_options[:dns_search] unless iface_options[:dns_search].nil?
> > 68: require 'pry'
> > 69: binding.pry
> > 70:
> > => 71: machine.start(
> > 72: use_cloud_init: true,
> > 73: vm: vm_configuration
> > 74: )
> > 75:
> > 76: @app.call(env)
> > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)>
> > vm_configuration
> > => {:initialization=>
> > {:host_name=>"test",
> > :nic_configurations=>[{:name=>"eth0", :on_boot=>true,
> > :boot_protocol=>"static", :ip=>{:version=>"v4",
> > :address=>"192.168.2.200", :gateway=>"192.168.2.1",
> > :netmask=>"255.255.255.0"}}],
> > :custom_script=>
> > "write_files:\n - content: |\n wat\n path:
> > /tmp/something.txt\n permissions:
> '0644'\nnetwork-interfaces: |\n
> > auto eth0\n iface eth0 inet static\n address 192.168.2.201\n
> > network 192.168.2.0\n netmask 255.255.255.0\n gateway
> > 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n",
> > :dns_servers=>"192.168.2.1",
> > :dns_search=>"test.local"}}
> >
> >
> > And here's my debugging after it comes up:
> >
> > [vagrant at test ~]$ cat /etc/resolv.conf
> > # Generated by NetworkManager
> > search localdomain
> > [vagrant at test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
> > NM_CONTROLLED=no
> > NETMASK=255.255.255.0
> > BOOTPROTO=static
> > DEVICE=eth0
> > IPADDR=192.168.2.200
> > GATEWAY=192.168.2.1
> > ONBOOT=yes
> > [vagrant at test ~]$ cat /tmp/something.txt
> > wat
> > [vagrant at test ~]$ ping -c 3 www.google.com <http://www.google.com>
> > <http://www.google.com/>ping: www.google.com
> <http://www.google.com>
> > <http://www.google.com/>: Name or service not known
> > [vagrant at test ~]$ echo $'nameserver 192.168.2.113\nnameserver
> > 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null
> > [vagrant at test ~]$ ping -c 3 www.google.com <http://www.google.com>
> > <http://www.google.com/>PING www.google.com
> <http://www.google.com>
> > <http://www.google.com/> (216.58.217.4) 56(84) bytes of data.
> > 64 bytes from den03s09-in-f4.1e100.net
> <http://den03s09-in-f4.1e100.net>
> > <http://den03s09-in-f4.1e100.net/
> <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=1
> > ttl=54 time=47.5 ms
> > 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net>
> > <http://den03s09-in-f4.1e100.net/
> <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=2
> > ttl=54 time=46.9 ms
> > 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net>
> > <http://den03s09-in-f4.1e100.net/
> <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=3
> > ttl=54 time=44.3 ms
> > --- www.google.com <http://www.google.com>
> <http://www.google.com/> ping statistics ---
> > 3 packets transmitted, 3 received, 0% packet loss, time 2002ms
> > rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms
> >
>
> Not sure if you did that already, but is very useful to check the
> content of the floppy that the oVirt engine creates and attaches to the
> virtual machine. To do so run the virtual machine, then go to the
> hypervisor where it is running and locate the corresponding 'qemu-kvm'
> process:
>
> # ps -ef | grep qemu-kvm | grep myvm
>
> That will have a very long command line, and should have a 'payload'
> option that points to the generated floppy, something like this:
>
> -drive file=/var/run/vdsm/payload/ae3a9cd4-....img
>
> Copy that file somewhere, and mount it:
>
> # mount -o loop,ro that_file.img /mnt
>
> Then explore the content:
>
> # find /mnt -type f
> /mnt/openstack/content/0000
> /mnt/openstack/latest/meta_data.json
> /mnt/openstack/latest/user_data
>
> The 'openstack/content/0000' file should contain your network
> configuration. Is it what you expected?
>
> The 'openstack/latest/user_data' file should contain the cloud-init
> ocnfiguration generated by oVirt, which your custom script appended.
> Does it look correct?
>
> (Remember to "umount /mnt" when done.)
>
>
>
>
More information about the Devel
mailing list