Re: [ovirt-devel] oVirt Cloud-init

On 02/17/2017 08:11 PM, Marc Young wrote: > Per the script, i'll change that, probably just wasnt paying attention > to where i ran chkconfig > > Per cloud-init YAML, not sure. It was in a previous reply so i gave it a > shot. But i can't find that usage anywhere either so i gave up on it. > > Per troubleshooting: > > Spinning up a VM with dns_servers and dns_search: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search] = > iface_options[:dns_search] unless iface_options[:dns_search].nil? > 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=> > "manage-resolv-conf: true\n\nresolv_conf:\n nameservers: > ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com > <http://foo.example.com>\n - bar.example.com > <http://bar.example.com>\n domain: example.com > <http://example.com>\n options:\n rotate: true\n timeout: 1\n", > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > ==> default: Waiting for VM to get an IP address... > ==> default: Machine is booted and ready for use! > ==> default: Rsyncing folder: > /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant > ==> default: Setting hostname... > default: > default: Vagrant insecure key detected. Vagrant will > automatically replace > default: this with a newly generated keypair for better security. > default: > default: Inserting generated public key within guest... > default: Removing insecure key from the guest if it's present... > default: Key inserted! Disconnecting and reconnecting using new > SSH key... > > > now ill ssh in and look around > > myoung  dev    ~  repos  github  vagrant-ovirt4  > feature/gh65  ✎  5❓  1  $  vagrant ssh > You appear to be running Vagrant outside of the official installers. > Note that the installers are what ensure that Vagrant has all required > dependencies, and Vagrant assumes that these dependencies exist. By > running outside of the installer environment, Vagrant may not function > properly. To remove this warning, install Vagrant using one of the > official packages from vagrantup.com <http://vagrantup.com>;. > > Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198 > [vagrant@test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt > #cloud-config > output: > all: '>> /var/log/cloud-init-output.log' > disable_root: 0 > runcmd: > - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo > ''datasource_list: > ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' > ssh_pwauth: true > chpasswd: > expire: false > user: root > manage-resolv-conf: true > > resolv_conf: > nameservers: ['8.8.4.4', '8.8.8.8'] > searchdomains: > - foo.example.com <http://foo.example.com> > - bar.example.com <http://bar.example.com> > domain: example.com <http://example.com> > options: > rotate: true > timeout: 1 Apparently you are passing in the 'custom_script' some DNS configuration, which may conflict with the DNS configuration that you are passing in the 'dns_servers' and 'dns_search' options. It could be that cloud-init ignores one of them. I'd suggest that you try to remove completely the 'custom_script' parameter. > [vagrant@test ~]$ sudo cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 > NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > > [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i dns > [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i local > Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running > 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. > Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running > 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. > Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017 > 03:45:01 +0000. Up 1310.61 seconds. > Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 > 05:16:45 +0000. Up 11.00 seconds. > Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 > 05:03:35 +0000. Up 5.23 seconds. > Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000. > Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1]. Up > 15.37 seconds > > > > On the vm host running that VM: > > > [myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test > qemu 12456 1 9 05:03 ? 00:00:19 > /usr/libexec/qemu-kvm -name ...snipped > [myoung@ovirt ~]$ sudo cp > /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846. 20fc2db0517e8c06579d7719d8f3fb35.img > . > [myoung@ovirt ~]$ sudo mount -o loop,ro > c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77 19d8f3fb35.img > /mnt > [myoung@ovirt ~]$ find /mnt -type f > find: ‘/mnt’: Permission denied > [myoung@ovirt ~]$ sudo find /mnt -type f > /mnt/openstack/content/0000 > /mnt/openstack/latest/meta_data.json > /mnt/openstack/latest/user_data > [myoung@ovirt ~]$ sudo cat /mnt/openstack/content/0000 > auto eth0 > iface eth0 inet static > address 192.168.2.200 > netmask 255.255.255.0 > gateway 192.168.2.1 > dns-nameservers 192.168.2.1 > dns-search test.local > [myoung@ovirt ~]$ sudo umount > c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77 19d8f3fb35.img > > Cloud-init definitely isn't working right away with resolv.conf, but I'm > definitely passing it correctly to the API, it shows up on the floppy as > you described but it's not making its way to > /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other settings > for sure are (such as address, netmask,) > > If i add that information to /etc/sysconfig/network-scripts/ifcfg-eth0 > and bounce network it all works: > > [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> > ping: www.google.com <http://www.google.com>;: Name or service not known > [vagrant@test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' | > sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null > [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> > ping: www.google.com <http://www.google.com>;: Name or service not known > [vagrant@test ~]$ sudo service network restart > Restarting network (via systemctl): [ OK ] > [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> > PING www.google.com <http://www.google.com> (216.58.217.4) 56(84) > bytes of data. > 64 bytes from den03s09-in-f4.1e100.net > <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=1 ttl=54 > time=55.8 ms > 64 bytes from den03s09-in-f4.1e100.net > <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=2 ttl=54 > time=43.9 ms > ^C > --- www.google.com <http://www.google.com> ping statistics --- > 2 packets transmitted, 2 received, 0% packet loss, time 1001ms > rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms > [vagrant@test ~]$ sudo cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > nameserver 192.168.2.113 > nameserver 192.168.2.1 > > > > > On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand(a)redhat.com > <mailto:jhernand@redhat.com>> wrote: > > On 02/17/2017 06:00 AM, Marc Young wrote: > > I'm apparently really bad at email, I replied only to Shahar, not the > > whole thread. > > > > Vinzenz your email slipped first, so to answer your question: > > > > It's the latest Centos 7 with these installed: > > > > cloud-init-0.7.5-10.el7.centos.1 > > kernel-3.10.0-514 <tel:3.10.0-514> <tel:3.10.0-514 > <tel:3.10.0-514>>.el7 > > ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1. el7.centos > > > > The setup script I use to create a template is here: > > https://github.com/myoung34/vagrant-ovirt4/blob/master/tools /prepare_redhat_for_box.sh > <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh> > > > <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh > <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh>> > > > > In that script you run "chkconfig cloud-init on" *before* installing the > cloud-init package. That is irrelevant, as the cloud-init services are > enabled by default when the package is installed. But worth changing. > > > The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos > > The ruby SDK i'm working with is 4.1.2 > > > > Halfway through I realized that it's actually supported in the API: > > > > custom_script String > > dns_search String > > dns_servers String > > > > > > It also shows usage here: > > > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/ sdk/examples/start_vm_with_cloud_init.rb > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb> > > > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb>> > > > > Here's some verification: > > > > 66: > vm_configuration[:initialization][:dns_servers] = > > iface_options[:dns_servers] unless > iface_options[:dns_servers].nil? > > 67: vm_configuration[:initialization][:dns_search] = > > iface_options[:dns_search] unless iface_options[:dns_search].nil? > > 68: require 'pry' > > 69: binding.pry > > 70: > > => 71: machine.start( > > 72: use_cloud_init: true, > > 73: vm: vm_configuration > > 74: ) > > 75: > > 76: @app.call(env) > > > > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > > vm_configuration > > => {:initialization=> > > {:host_name=>"test", > > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > > :boot_protocol=>"static", :ip=>{:version=>"v4", > > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > > :netmask=>"255.255.255.0"}}], > > :custom_script=>nil, > > :dns_servers=>"192.168.2.1", > > :dns_search=>"test.local"}} > > > > > > But it didn't do anything: > > > > [vagrant@test ~]$ cat /etc/resolv.conf > > # Generated by NetworkManager > > search localdomain > > > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts /ifcfg-eth0 > > NM_CONTROLLED=no > > NETMASK=255.255.255.0 > > BOOTPROTO=static > > DEVICE=eth0 > > IPADDR=192.168.2.200 > > GATEWAY=192.168.2.1 > > ONBOOT=yes > > > > > > The same is also true using cloud_init: > > > > ovirt.cloud_init =<<EOF > > write_files: > > - content: | > > wat > > path: /tmp/something.txt > > permissions: '0644' > > network-interfaces: | > > auto eth0 > > iface eth0 inet static > > address 192.168.2.201 > > network 192.168.2.0 > > netmask 255.255.255.0 > > gateway 192.168.2.1 > > dns-nameservers 192.168.2.113 192.168.2.1 > > EOF > > > > Is this ^ supposed to work in cloud-init? I didn't find it in the > documentation. I thought that the only way to provide network interface > configuration is via the 'openstack/content/whatever' file within the > generated floppy. > > > > > and inspection: > > > > 66: > vm_configuration[:initialization][:dns_servers] = > > iface_options[:dns_servers] unless > iface_options[:dns_servers].nil? > > 67: vm_configuration[:initialization][:dns_search] = > > iface_options[:dns_search] unless iface_options[:dns_search].nil? > > 68: require 'pry' > > 69: binding.pry > > 70: > > => 71: machine.start( > > 72: use_cloud_init: true, > > 73: vm: vm_configuration > > 74: ) > > 75: > > 76: @app.call(env) > > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > > vm_configuration > > => {:initialization=> > > {:host_name=>"test", > > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > > :boot_protocol=>"static", :ip=>{:version=>"v4", > > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > > :netmask=>"255.255.255.0"}}], > > :custom_script=> > > "write_files:\n - content: |\n wat\n path: > > /tmp/something.txt\n permissions: > '0644'\nnetwork-interfaces: |\n > > auto eth0\n iface eth0 inet static\n address 192.168.2.201\n > > network 192.168.2.0\n netmask 255.255.255.0\n gateway > > 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n", > > :dns_servers=>"192.168.2.1", > > :dns_search=>"test.local"}} > > > > > > And here's my debugging after it comes up: > > > > [vagrant@test ~]$ cat /etc/resolv.conf > > # Generated by NetworkManager > > search localdomain > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts /ifcfg-eth0 > > NM_CONTROLLED=no > > NETMASK=255.255.255.0 > > BOOTPROTO=static > > DEVICE=eth0 > > IPADDR=192.168.2.200 > > GATEWAY=192.168.2.1 > > ONBOOT=yes > > [vagrant@test ~]$ cat /tmp/something.txt > > wat > > [vagrant@test ~]$ ping -c 3 www.google.com < http://www.google.com> > > <http://www.google.com/>ping: www.google.com > <http://www.google.com> > > <http://www.google.com/>;: Name or service not known > > [vagrant@test ~]$ echo $'nameserver 192.168.2.113\nnameserver > > 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null > > [vagrant@test ~]$ ping -c 3 www.google.com < http://www.google.com> > > <http://www.google.com/>PING www.google.com > <http://www.google.com> > > <http://www.google.com/> (216.58.217.4) 56(84) bytes of data. > > 64 bytes from den03s09-in-f4.1e100.net > <http://den03s09-in-f4.1e100.net> > > <http://den03s09-in-f4.1e100.net/ > <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=1 > > ttl=54 time=47.5 ms > > 64 bytes from den03s09-in-f4.1e100.net < http://den03s09-in-f4.1e100.net> > > <http://den03s09-in-f4.1e100.net/ > <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=2 > > ttl=54 time=46.9 ms > > 64 bytes from den03s09-in-f4.1e100.net < http://den03s09-in-f4.1e100.net> > > <http://den03s09-in-f4.1e100.net/ > <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=3 > > ttl=54 time=44.3 ms > > --- www.google.com <http://www.google.com> > <http://www.google.com/> ping statistics --- > > 3 packets transmitted, 3 received, 0% packet loss, time 2002ms > > rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms > > > > Not sure if you did that already, but is very useful to check the > content of the floppy that the oVirt engine creates and attaches to the > virtual machine. To do so run the virtual machine, then go to the > hypervisor where it is running and locate the corresponding 'qemu-kvm' > process: > > # ps -ef | grep qemu-kvm | grep myvm > > That will have a very long command line, and should have a 'payload' > option that points to the generated floppy, something like this: > > -drive file=/var/run/vdsm/payload/ae3a9cd4-....img > > Copy that file somewhere, and mount it: > > # mount -o loop,ro that_file.img /mnt > > Then explore the content: > > # find /mnt -type f > /mnt/openstack/content/0000 > /mnt/openstack/latest/meta_data.json > /mnt/openstack/latest/user_data > > The 'openstack/content/0000' file should contain your network > configuration. Is it what you expected? > > The 'openstack/latest/user_data' file should contain the cloud-init > ocnfiguration generated by oVirt, which your custom script appended. > Does it look correct? > > (Remember to "umount /mnt" when done.) > > > >