Are there limitations to cloud-init and oVirt? I wouldn't think so, but i"m having a weird issue. If i use the cloud-init yaml from the ovirt4 ruby sdk examples (ignore formatting, gmail is messing with it): write_files: - content: | Hello, world! path: /tmp/greeting.txt permissions: '0644' it works, that file exists, and it shows in the /var/lib/cloud where i'd expect $ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root write_files: - content: | Hello, world! path: /tmp/greeting.txt permissions: '0644' If i use this: manage-resolv-conf: true resolv_conf: nameservers: ['192.168.2.113'] searchdomains: - blindrage.local - bar.example.com resolv_conf does not get modified. It looks as expected in /var/lib/cloud: $ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true resolv_conf: nameservers: ['192.168.2.113'] searchdomains: - foo.local - bar.example.com I also don't see anything in /var/log/cloud-init.log or /var/log/cloud-init-output.log on either run even though the write_files yaml worked.
On Feb 15, 2017, at 11:51 PM, Marc Young <3vilpenguin@gmail.com> wrote:
Are there limitations to cloud-init and oVirt? I wouldn't think so, but i"m having a weird issue.
What’s the guest OS?
If i use the cloud-init yaml from the ovirt4 ruby sdk examples (ignore formatting, gmail is messing with it):
write_files: - content: | Hello, world! path: /tmp/greeting.txt permissions: '0644'
it works, that file exists, and it shows in the /var/lib/cloud where i'd expect
$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root write_files: - content: | Hello, world! path: /tmp/greeting.txt permissions: '0644'
If i use this:
manage-resolv-conf: true resolv_conf: nameservers: ['192.168.2.113'] searchdomains: - blindrage.local - bar.example.com
resolv_conf does not get modified. It looks as expected in /var/lib/cloud:
$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true resolv_conf: nameservers: ['192.168.2.113'] searchdomains: - foo.local - bar.example.com
I also don't see anything in /var/log/cloud-init.log or /var/log/cloud-init-output.log on either run even though the write_files yaml worked. _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
Hi Marc, As I see it oVirt pass all your input to cloud-init package. I think the input is wrong as far as cloud-init parser goes, looking at the file: backend/manager/modules/vdsbroker/src/main/java/org/ovirt/engine/core/vdsbroker/vdsbroker/CloudInitHandler.java you need to set at-least one interface and the search domains should be under. you are using the CustomScript so you need to set the yaml as the package expected. something like: iface eth0 inet address x.x.x.x netmast x.x.x.x gateway x.x.x.x dns-nameservers x.x.x.x this is how oVirt set the interface (not via CustomScript but via setting each interface and dns servers to search) On Thu, Feb 16, 2017 at 8:39 AM, Vinzenz Feenstra <vfeenstr@redhat.com> wrote:
On Feb 15, 2017, at 11:51 PM, Marc Young <3vilpenguin@gmail.com> wrote:
Are there limitations to cloud-init and oVirt? I wouldn't think so, but i"m having a weird issue.
What’s the guest OS?
If i use the cloud-init yaml from the ovirt4 ruby sdk examples (ignore formatting, gmail is messing with it):
write_files: - content: | Hello, world! path: /tmp/greeting.txt permissions: '0644'
it works, that file exists, and it shows in the /var/lib/cloud where i'd
expect
$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root write_files: - content: | Hello, world! path: /tmp/greeting.txt permissions: '0644'
If i use this:
manage-resolv-conf: true resolv_conf: nameservers: ['192.168.2.113'] searchdomains: - blindrage.local - bar.example.com
resolv_conf does not get modified. It looks as expected in
/var/lib/cloud:
$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true resolv_conf: nameservers: ['192.168.2.113'] searchdomains: - foo.local - bar.example.com
I also don't see anything in /var/log/cloud-init.log or /var/log/cloud-init-output.log on either run even though the write_files yaml worked. _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
http://ask.projectatomic.io/en/question/3572/how-do-i-set-up-a-static-networ... On Thu, Feb 16, 2017 at 12:05 PM, Shahar Havivi <shavivi@redhat.com> wrote:
Hi Marc, As I see it oVirt pass all your input to cloud-init package. I think the input is wrong as far as cloud-init parser goes, looking at the file: backend/manager/modules/vdsbroker/src/main/java/org/ ovirt/engine/core/vdsbroker/vdsbroker/CloudInitHandler.java you need to set at-least one interface and the search domains should be under.
you are using the CustomScript so you need to set the yaml as the package expected. something like: iface eth0 inet address x.x.x.x netmast x.x.x.x gateway x.x.x.x dns-nameservers x.x.x.x
this is how oVirt set the interface (not via CustomScript but via setting each interface and dns servers to search)
On Thu, Feb 16, 2017 at 8:39 AM, Vinzenz Feenstra <vfeenstr@redhat.com> wrote:
On Feb 15, 2017, at 11:51 PM, Marc Young <3vilpenguin@gmail.com> wrote:
Are there limitations to cloud-init and oVirt? I wouldn't think so, but i"m having a weird issue.
What’s the guest OS?
If i use the cloud-init yaml from the ovirt4 ruby sdk examples (ignore formatting, gmail is messing with it):
write_files: - content: | Hello, world! path: /tmp/greeting.txt permissions: '0644'
it works, that file exists, and it shows in the /var/lib/cloud where
i'd expect
$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root write_files: - content: | Hello, world! path: /tmp/greeting.txt permissions: '0644'
If i use this:
manage-resolv-conf: true resolv_conf: nameservers: ['192.168.2.113'] searchdomains: - blindrage.local - bar.example.com
resolv_conf does not get modified. It looks as expected in
/var/lib/cloud:
$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true resolv_conf: nameservers: ['192.168.2.113'] searchdomains: - foo.local - bar.example.com
I also don't see anything in /var/log/cloud-init.log or /var/log/cloud-init-output.log on either run even though the write_files yaml worked. _______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
I'm apparently really bad at email, I replied only to Shahar, not the whole thread. Vinzenz your email slipped first, so to answer your question: It's the latest Centos 7 with these installed: cloud-init-0.7.5-10.el7.centos.1 kernel-3.10.0-514.el7 ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1.el7.centos The setup script I use to create a template is here: https://github.com/myoung34/vagrant-ovirt4/blob/master/ tools/prepare_redhat_for_box.sh The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos The ruby SDK i'm working with is 4.1.2 Halfway through I realized that it's actually supported in the API: custom_script String dns_search String dns_servers String It also shows usage here: https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/ master/sdk/examples/start_vm_with_cloud_init.rb Here's some verification: 66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env) [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=>nil, :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} But it didn't do anything: [vagrant@test ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes The same is also true using cloud_init: ovirt.cloud_init =<<EOF write_files: - content: | wat path: /tmp/something.txt permissions: '0644' network-interfaces: | auto eth0 iface eth0 inet static address 192.168.2.201 network 192.168.2.0 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.113 192.168.2.1 EOF and inspection: 66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env) [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "write_files:\n - content: |\n wat\n path: /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n auto eth0\n iface eth0 inet static\n address 192.168.2.201\n network 192.168.2.0\n netmask 255.255.255.0\n gateway 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} And here's my debugging after it comes up: [vagrant@test ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes [vagrant@test ~]$ cat /tmp/something.txt wat [vagrant@test ~]$ ping -c 3 www.google.com ping: www.google.com: Name or service not known [vagrant@test ~]$ echo $'nameserver 192.168.2.113\nnameserver 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com PING www.google.com (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net (216.58.217.4): icmp_seq=1 ttl=54 time=47.5 ms 64 bytes from den03s09-in-f4.1e100.net (216.58.217.4): icmp_seq=2 ttl=54 time=46.9 ms 64 bytes from den03s09-in-f4.1e100.net (216.58.217.4): icmp_seq=3 ttl=54 time=44.3 ms --- www.google.com ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms
On 02/17/2017 06:00 AM, Marc Young wrote:
I'm apparently really bad at email, I replied only to Shahar, not the whole thread.
Vinzenz your email slipped first, so to answer your question:
It's the latest Centos 7 with these installed:
cloud-init-0.7.5-10.el7.centos.1 kernel-3.10.0-514 <tel:3.10.0-514>.el7 ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1.el7.centos
The setup script I use to create a template is here: https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_... <https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_for_box.sh>
In that script you run "chkconfig cloud-init on" *before* installing the cloud-init package. That is irrelevant, as the cloud-init services are enabled by default when the package is installed. But worth changing.
The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos The ruby SDK i'm working with is 4.1.2
Halfway through I realized that it's actually supported in the API:
custom_script String dns_search String dns_servers String
It also shows usage here: https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/star... <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/start_vm_with_cloud_init.rb>
Here's some verification:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=>nil, :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}}
But it didn't do anything:
[vagrant@test ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain
[vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes
The same is also true using cloud_init:
ovirt.cloud_init =<<EOF write_files: - content: | wat path: /tmp/something.txt permissions: '0644' network-interfaces: | auto eth0 iface eth0 inet static address 192.168.2.201 network 192.168.2.0 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.113 192.168.2.1 EOF
Is this ^ supposed to work in cloud-init? I didn't find it in the documentation. I thought that the only way to provide network interface configuration is via the 'openstack/content/whatever' file within the generated floppy.
and inspection:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env) [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "write_files:\n - content: |\n wat\n path: /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n auto eth0\n iface eth0 inet static\n address 192.168.2.201\n network 192.168.2.0\n netmask 255.255.255.0\n gateway 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}}
And here's my debugging after it comes up:
[vagrant@test ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes [vagrant@test ~]$ cat /tmp/something.txt wat [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com/>ping: www.google.com <http://www.google.com/>: Name or service not known [vagrant@test ~]$ echo $'nameserver 192.168.2.113\nnameserver 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com/>PING www.google.com <http://www.google.com/> (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net/> (216.58.217.4): icmp_seq=1 ttl=54 time=47.5 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net/> (216.58.217.4): icmp_seq=2 ttl=54 time=46.9 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net/> (216.58.217.4): icmp_seq=3 ttl=54 time=44.3 ms --- www.google.com <http://www.google.com/> ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms
Not sure if you did that already, but is very useful to check the content of the floppy that the oVirt engine creates and attaches to the virtual machine. To do so run the virtual machine, then go to the hypervisor where it is running and locate the corresponding 'qemu-kvm' process: # ps -ef | grep qemu-kvm | grep myvm That will have a very long command line, and should have a 'payload' option that points to the generated floppy, something like this: -drive file=/var/run/vdsm/payload/ae3a9cd4-....img Copy that file somewhere, and mount it: # mount -o loop,ro that_file.img /mnt Then explore the content: # find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data The 'openstack/content/0000' file should contain your network configuration. Is it what you expected? The 'openstack/latest/user_data' file should contain the cloud-init ocnfiguration generated by oVirt, which your custom script appended. Does it look correct? (Remember to "umount /mnt" when done.)
Per the script, i'll change that, probably just wasnt paying attention to where i ran chkconfig Per cloud-init YAML, not sure. It was in a previous reply so i gave it a shot. But i can't find that usage anywhere either so i gave up on it. Per troubleshooting: Spinning up a VM with dns_servers and dns_search: 66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env) [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "manage-resolv-conf: true\n\nresolv_conf:\n nameservers: ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com\n - bar.example.com\n domain: example.com\n options:\n rotate: true\n timeout: 1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... now ill ssh in and look around myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ 5❓ 1 $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com. Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198 [vagrant@test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true resolv_conf: nameservers: ['8.8.4.4', '8.8.8.8'] searchdomains: - foo.example.com - bar.example.com domain: example.com options: rotate: true timeout: 1 [vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i dns [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i local Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017 03:45:01 +0000. Up 1310.61 seconds. Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000. Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1]. Up 15.37 seconds On the vm host running that VM: [myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test qemu 12456 1 9 05:03 ? 00:00:19 /usr/libexec/qemu-kvm -name ...snipped [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img . [myoung@ovirt ~]$ sudo mount -o loop,ro c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img /mnt [myoung@ovirt ~]$ find /mnt -type f find: ‘/mnt’: Permission denied [myoung@ovirt ~]$ sudo find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data [myoung@ovirt ~]$ sudo cat /mnt/openstack/content/0000 auto eth0 iface eth0 inet static address 192.168.2.200 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img Cloud-init definitely isn't working right away with resolv.conf, but I'm definitely passing it correctly to the API, it shows up on the floppy as you described but it's not making its way to /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other settings for sure are (such as address, netmask,) If i add that information to /etc/sysconfig/network-scripts/ifcfg-eth0 and bounce network it all works: [vagrant@test ~]$ ping -c 3 www.google.com ping: www.google.com: Name or service not known [vagrant@test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com ping: www.google.com: Name or service not known [vagrant@test ~]$ sudo service network restart Restarting network (via systemctl): [ OK ] [vagrant@test ~]$ ping -c 3 www.google.com PING www.google.com (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net (216.58.217.4): icmp_seq=1 ttl=54 time=55.8 ms 64 bytes from den03s09-in-f4.1e100.net (216.58.217.4): icmp_seq=2 ttl=54 time=43.9 ms ^C --- www.google.com ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms [vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain nameserver 192.168.2.113 nameserver 192.168.2.1 On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand@redhat.com> wrote:
On 02/17/2017 06:00 AM, Marc Young wrote:
I'm apparently really bad at email, I replied only to Shahar, not the whole thread.
Vinzenz your email slipped first, so to answer your question:
It's the latest Centos 7 with these installed:
cloud-init-0.7.5-10.el7.centos.1 kernel-3.10.0-514 <tel:3.10.0-514>.el7 ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1.el7.centos
The setup script I use to create a template is here: https://github.com/myoung34/vagrant-ovirt4/blob/master/ tools/prepare_redhat_for_box.sh <https://github.com/myoung34/vagrant-ovirt4/blob/master/ tools/prepare_redhat_for_box.sh>
In that script you run "chkconfig cloud-init on" *before* installing the cloud-init package. That is irrelevant, as the cloud-init services are enabled by default when the package is installed. But worth changing.
The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos The ruby SDK i'm working with is 4.1.2
Halfway through I realized that it's actually supported in the API:
custom_script String dns_search String dns_servers String
It also shows usage here: https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/ master/sdk/examples/start_vm_with_cloud_init.rb <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/ master/sdk/examples/start_vm_with_cloud_init.rb>
Here's some verification:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=>nil, :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}}
But it didn't do anything:
[vagrant@test ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain
[vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes
The same is also true using cloud_init:
ovirt.cloud_init =<<EOF write_files: - content: | wat path: /tmp/something.txt permissions: '0644' network-interfaces: | auto eth0 iface eth0 inet static address 192.168.2.201 network 192.168.2.0 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.113 192.168.2.1 EOF
Is this ^ supposed to work in cloud-init? I didn't find it in the documentation. I thought that the only way to provide network interface configuration is via the 'openstack/content/whatever' file within the generated floppy.
and inspection:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env) [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "write_files:\n - content: |\n wat\n path: /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n auto eth0\n iface eth0 inet static\n address 192.168.2.201\n network 192.168.2.0\n netmask 255.255.255.0\n gateway 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}}
And here's my debugging after it comes up:
[vagrant@test ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes [vagrant@test ~]$ cat /tmp/something.txt wat [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com/>ping: www.google.com <http://www.google.com/>: Name or service not known [vagrant@test ~]$ echo $'nameserver 192.168.2.113\nnameserver 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com/>PING www.google.com <http://www.google.com/> (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net/> (216.58.217.4): icmp_seq=1 ttl=54 time=47.5 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net/> (216.58.217.4): icmp_seq=2 ttl=54 time=46.9 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net/> (216.58.217.4): icmp_seq=3 ttl=54 time=44.3 ms --- www.google.com <http://www.google.com/> ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms
Not sure if you did that already, but is very useful to check the content of the floppy that the oVirt engine creates and attaches to the virtual machine. To do so run the virtual machine, then go to the hypervisor where it is running and locate the corresponding 'qemu-kvm' process:
# ps -ef | grep qemu-kvm | grep myvm
That will have a very long command line, and should have a 'payload' option that points to the generated floppy, something like this:
-drive file=/var/run/vdsm/payload/ae3a9cd4-....img
Copy that file somewhere, and mount it:
# mount -o loop,ro that_file.img /mnt
Then explore the content:
# find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data
The 'openstack/content/0000' file should contain your network configuration. Is it what you expected?
The 'openstack/latest/user_data' file should contain the cloud-init ocnfiguration generated by oVirt, which your custom script appended. Does it look correct?
(Remember to "umount /mnt" when done.)
On 02/17/2017 08:11 PM, Marc Young wrote:
Per the script, i'll change that, probably just wasnt paying attention to where i ran chkconfig
Per cloud-init YAML, not sure. It was in a previous reply so i gave it a shot. But i can't find that usage anywhere either so i gave up on it.
Per troubleshooting:
Spinning up a VM with dns_servers and dns_search:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "manage-resolv-conf: true\n\nresolv_conf:\n nameservers: ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com <http://foo.example.com>\n - bar.example.com <http://bar.example.com>\n domain: example.com <http://example.com>\n options:\n rotate: true\n timeout: 1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key...
now ill ssh in and look around
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ 5❓ 1 $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com <http://vagrantup.com>.
Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198 [vagrant@test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true
resolv_conf: nameservers: ['8.8.4.4', '8.8.8.8'] searchdomains: - foo.example.com <http://foo.example.com> - bar.example.com <http://bar.example.com> domain: example.com <http://example.com> options: rotate: true timeout: 1
Apparently you are passing in the 'custom_script' some DNS configuration, which may conflict with the DNS configuration that you are passing in the 'dns_servers' and 'dns_search' options. It could be that cloud-init ignores one of them. I'd suggest that you try to remove completely the 'custom_script' parameter.
[vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain
[vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes
[vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i dns [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i local Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017 03:45:01 +0000. Up 1310.61 seconds. Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000. Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1]. Up 15.37 seconds
On the vm host running that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test qemu 12456 1 9 05:03 ? 00:00:19 /usr/libexec/qemu-kvm -name ...snipped [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img . [myoung@ovirt ~]$ sudo mount -o loop,ro c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img /mnt [myoung@ovirt ~]$ find /mnt -type f find: ‘/mnt’: Permission denied [myoung@ovirt ~]$ sudo find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data [myoung@ovirt ~]$ sudo cat /mnt/openstack/content/0000 auto eth0 iface eth0 inet static address 192.168.2.200 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d7719d8f3fb35.img
Cloud-init definitely isn't working right away with resolv.conf, but I'm definitely passing it correctly to the API, it shows up on the floppy as you described but it's not making its way to /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other settings for sure are (such as address, netmask,)
If i add that information to /etc/sysconfig/network-scripts/ifcfg-eth0 and bounce network it all works:
[vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ sudo service network restart Restarting network (via systemctl): [ OK ] [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> PING www.google.com <http://www.google.com> (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=1 ttl=54 time=55.8 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=2 ttl=54 time=43.9 ms ^C --- www.google.com <http://www.google.com> ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms [vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain nameserver 192.168.2.113 nameserver 192.168.2.1
On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand@redhat.com <mailto:jhernand@redhat.com>> wrote:
On 02/17/2017 06:00 AM, Marc Young wrote: > I'm apparently really bad at email, I replied only to Shahar, not the > whole thread. > > Vinzenz your email slipped first, so to answer your question: > > It's the latest Centos 7 with these installed: > > cloud-init-0.7.5-10.el7.centos.1 > kernel-3.10.0-514 <tel:3.10.0-514> <tel:3.10.0-514 <tel:3.10.0-514>>.el7 > ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1.el7.centos > > The setup script I use to create a template is here: > https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_... <https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_for_box.sh> > <https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_... <https://github.com/myoung34/vagrant-ovirt4/blob/master/tools/prepare_redhat_for_box.sh>> >
In that script you run "chkconfig cloud-init on" *before* installing the cloud-init package. That is irrelevant, as the cloud-init services are enabled by default when the package is installed. But worth changing.
> The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos > The ruby SDK i'm working with is 4.1.2 > > Halfway through I realized that it's actually supported in the API: > > custom_script String > dns_search String > dns_servers String > > > It also shows usage here: > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/star... <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/start_vm_with_cloud_init.rb> > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/star... <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/sdk/examples/start_vm_with_cloud_init.rb>> > > Here's some verification: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search] = > iface_options[:dns_search] unless iface_options[:dns_search].nil? > 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=>nil, > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > But it didn't do anything: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 > NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > > > The same is also true using cloud_init: > > ovirt.cloud_init =<<EOF > write_files: > - content: | > wat > path: /tmp/something.txt > permissions: '0644' > network-interfaces: | > auto eth0 > iface eth0 inet static > address 192.168.2.201 > network 192.168.2.0 > netmask 255.255.255.0 > gateway 192.168.2.1 > dns-nameservers 192.168.2.113 192.168.2.1 > EOF >
Is this ^ supposed to work in cloud-init? I didn't find it in the documentation. I thought that the only way to provide network interface configuration is via the 'openstack/content/whatever' file within the generated floppy.
> > and inspection: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search] = > iface_options[:dns_search] unless iface_options[:dns_search].nil? > 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=> > "write_files:\n - content: |\n wat\n path: > /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n > auto eth0\n iface eth0 inet static\n address 192.168.2.201\n > network 192.168.2.0\n netmask 255.255.255.0\n gateway > 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n", > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > And here's my debugging after it comes up: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 > NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > [vagrant@test ~]$ cat /tmp/something.txt > wat > [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> > <http://www.google.com/>ping: www.google.com <http://www.google.com> > <http://www.google.com/>: Name or service not known > [vagrant@test ~]$ echo $'nameserver 192.168.2.113\nnameserver > 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null > [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> > <http://www.google.com/>PING www.google.com <http://www.google.com> > <http://www.google.com/> (216.58.217.4) 56(84) bytes of data. > 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=1 > ttl=54 time=47.5 ms > 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=2 > ttl=54 time=46.9 ms > 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=3 > ttl=54 time=44.3 ms > --- www.google.com <http://www.google.com> <http://www.google.com/> ping statistics --- > 3 packets transmitted, 3 received, 0% packet loss, time 2002ms > rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms >
Not sure if you did that already, but is very useful to check the content of the floppy that the oVirt engine creates and attaches to the virtual machine. To do so run the virtual machine, then go to the hypervisor where it is running and locate the corresponding 'qemu-kvm' process:
# ps -ef | grep qemu-kvm | grep myvm
That will have a very long command line, and should have a 'payload' option that points to the generated floppy, something like this:
-drive file=/var/run/vdsm/payload/ae3a9cd4-....img
Copy that file somewhere, and mount it:
# mount -o loop,ro that_file.img /mnt
Then explore the content:
# find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data
The 'openstack/content/0000' file should contain your network configuration. Is it what you expected?
The 'openstack/latest/user_data' file should contain the cloud-init ocnfiguration generated by oVirt, which your custom script appended. Does it look correct?
(Remember to "umount /mnt" when done.)
Still no change... myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant up You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com. Bringing machine 'default' up with 'ovirt4' provider... ==> default: Creating VM with the following settings... ==> default: -- Name: testing ==> default: -- Cluster: Default ==> default: -- Template: vagrant-centos7 ==> default: -- Console Type: vnc ==> default: -- Memory: ==> default: ---- Memory: 2048 MB ==> default: ---- Maximum: 2048 MB ==> default: ---- Guaranteed: 2048 MB ==> default: -- Cpu: ==> default: ---- Cores: 2 ==> default: ---- Sockets: 1 ==> default: ---- Threads: 1 ==> default: -- Cloud-Init: false ==> default: Waiting for VM to become "ready" to start... ==> default: Starting VM. From: /home/myoung/repos/github/vagrant-ovirt4/lib/vagrant-ovirt4/action/start_vm.rb @ line 79 VagrantPlugins::OVirtProvider::Action::StartVM#call: 74: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 75: 76: require 'pry' 77: binding.pry 78: => 79: machine.start( 80: use_cloud_init: true, 81: vm: vm_configuration 82: ) 83: 84: @app.call(env) [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"testing", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.120", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=>nil, :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com. Last login: Wed Feb 15 00:23:35 2017 from 192.168.2.32 [vagrant@testing ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@testing ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.120 GATEWAY=192.168.2.1 ONBOOT=yes If i check out the floppy attached to that VM: [myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test [sudo] password for myoung: qemu 24311 1 87 01:38 ? 00:00:18 ..... [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/9964609e-8d85-4a5c-9416-8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img . [myoung@ovirt ~]$ mkdir a; sudo mount -o loop,ro 9964609e-8d85-4a5c-9416-8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img a [myoung@ovirt ~]$ sudo cat $(sudo find a -name 0000 -type f) auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount a What's the next step? Is there a way to verify on the upstream that dns-search and dns-nameservers actually populate the ifcfg-eth0 file ? On Sat, Feb 18, 2017 at 6:58 AM, Juan Hernández <jhernand@redhat.com> wrote:
On 02/17/2017 08:11 PM, Marc Young wrote:
Per the script, i'll change that, probably just wasnt paying attention to where i ran chkconfig
Per cloud-init YAML, not sure. It was in a previous reply so i gave it a shot. But i can't find that usage anywhere either so i gave up on it.
Per troubleshooting:
Spinning up a VM with dns_servers and dns_search:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "manage-resolv-conf: true\n\nresolv_conf:\n nameservers: ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com <http://foo.example.com>\n - bar.example.com <http://bar.example.com>\n domain: example.com <http://example.com>\n options:\n rotate: true\n timeout: 1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key...
now ill ssh in and look around
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ 5❓ 1 $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com <http://vagrantup.com>.
Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198 [vagrant@test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true
resolv_conf: nameservers: ['8.8.4.4', '8.8.8.8'] searchdomains: - foo.example.com <http://foo.example.com> - bar.example.com <http://bar.example.com> domain: example.com <http://example.com> options: rotate: true timeout: 1
Apparently you are passing in the 'custom_script' some DNS configuration, which may conflict with the DNS configuration that you are passing in the 'dns_servers' and 'dns_search' options. It could be that cloud-init ignores one of them. I'd suggest that you try to remove completely the 'custom_script' parameter.
[vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain
[vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes
[vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i dns [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i local Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017 03:45:01 +0000. Up 1310.61 seconds. Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000. Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1]. Up 15.37 seconds
On the vm host running that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test qemu 12456 1 9 05:03 ? 00:00:19 /usr/libexec/qemu-kvm -name ...snipped [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846.
20fc2db0517e8c06579d7719d8f3fb35.img
. [myoung@ovirt ~]$ sudo mount -o loop,ro c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
/mnt [myoung@ovirt ~]$ find /mnt -type f find: ‘/mnt’: Permission denied [myoung@ovirt ~]$ sudo find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data [myoung@ovirt ~]$ sudo cat /mnt/openstack/content/0000 auto eth0 iface eth0 inet static address 192.168.2.200 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
Cloud-init definitely isn't working right away with resolv.conf, but I'm definitely passing it correctly to the API, it shows up on the floppy as you described but it's not making its way to /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other settings for sure are (such as address, netmask,)
If i add that information to /etc/sysconfig/network-scripts/ifcfg-eth0 and bounce network it all works:
[vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not
known
[vagrant@test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not
known
[vagrant@test ~]$ sudo service network restart Restarting network (via systemctl): [ OK ] [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> PING www.google.com <http://www.google.com> (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=1 ttl=54 time=55.8 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=2 ttl=54 time=43.9 ms ^C --- www.google.com <http://www.google.com> ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms [vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain nameserver 192.168.2.113 nameserver 192.168.2.1
On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand@redhat.com <mailto:jhernand@redhat.com>> wrote:
On 02/17/2017 06:00 AM, Marc Young wrote: > I'm apparently really bad at email, I replied only to Shahar, not
the
> whole thread. > > Vinzenz your email slipped first, so to answer your question: > > It's the latest Centos 7 with these installed: > > cloud-init-0.7.5-10.el7.centos.1 > kernel-3.10.0-514 <tel:3.10.0-514> <tel:3.10.0-514 <tel:3.10.0-514>>.el7 > ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1.
el7.centos
> > The setup script I use to create a template is here: > https://github.com/myoung34/vagrant-ovirt4/blob/master/tools
/prepare_redhat_for_box.sh
ls/prepare_redhat_for_box.sh>
> <https://github.com/myoung34/vagrant-ovirt4/blob/master/too
ls/prepare_redhat_for_box.sh
ls/prepare_redhat_for_box.sh>>
>
In that script you run "chkconfig cloud-init on" *before* installing
the
cloud-init package. That is irrelevant, as the cloud-init services
are
enabled by default when the package is installed. But worth changing.
> The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos > The ruby SDK i'm working with is 4.1.2 > > Halfway through I realized that it's actually supported in the API: > > custom_script String > dns_search String > dns_servers String > > > It also shows usage here: > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/
sdk/examples/start_vm_with_cloud_init.rb
/sdk/examples/start_vm_with_cloud_init.rb>
> <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master
/sdk/examples/start_vm_with_cloud_init.rb
/sdk/examples/start_vm_with_cloud_init.rb>>
> > Here's some verification: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search]
=
> iface_options[:dns_search] unless
iface_options[:dns_search].nil?
> 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=>nil, > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > But it didn't do anything: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts
/ifcfg-eth0
> NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > > > The same is also true using cloud_init: > > ovirt.cloud_init =<<EOF > write_files: > - content: | > wat > path: /tmp/something.txt > permissions: '0644' > network-interfaces: | > auto eth0 > iface eth0 inet static > address 192.168.2.201 > network 192.168.2.0 > netmask 255.255.255.0 > gateway 192.168.2.1 > dns-nameservers 192.168.2.113 192.168.2.1 > EOF >
Is this ^ supposed to work in cloud-init? I didn't find it in the documentation. I thought that the only way to provide network
interface
configuration is via the 'openstack/content/whatever' file within the generated floppy.
> > and inspection: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search]
=
> iface_options[:dns_search] unless
iface_options[:dns_search].nil?
> 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=> > "write_files:\n - content: |\n wat\n path: > /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n > auto eth0\n iface eth0 inet static\n address
192.168.2.201\n
> network 192.168.2.0\n netmask 255.255.255.0\n gateway > 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n", > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > And here's my debugging after it comes up: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts
/ifcfg-eth0
> NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > [vagrant@test ~]$ cat /tmp/something.txt > wat > [vagrant@test ~]$ ping -c 3 www.google.com <
> <http://www.google.com/>ping: www.google.com <http://www.google.com> > <http://www.google.com/>: Name or service not known > [vagrant@test ~]$ echo $'nameserver 192.168.2.113\nnameserver > 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null > [vagrant@test ~]$ ping -c 3 www.google.com <
> <http://www.google.com/>PING www.google.com <http://www.google.com> > <http://www.google.com/> (216.58.217.4) 56(84) bytes of data. > 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=1 > ttl=54 time=47.5 ms > 64 bytes from den03s09-in-f4.1e100.net <
http://den03s09-in-f4.1e100.net>
> <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=2 > ttl=54 time=46.9 ms > 64 bytes from den03s09-in-f4.1e100.net <
http://den03s09-in-f4.1e100.net>
> <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=3 > ttl=54 time=44.3 ms > --- www.google.com <http://www.google.com> <http://www.google.com/> ping statistics --- > 3 packets transmitted, 3 received, 0% packet loss, time 2002ms > rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms >
Not sure if you did that already, but is very useful to check the content of the floppy that the oVirt engine creates and attaches to
the
virtual machine. To do so run the virtual machine, then go to the hypervisor where it is running and locate the corresponding
'qemu-kvm'
process:
# ps -ef | grep qemu-kvm | grep myvm
That will have a very long command line, and should have a 'payload' option that points to the generated floppy, something like this:
-drive file=/var/run/vdsm/payload/ae3a9cd4-....img
Copy that file somewhere, and mount it:
# mount -o loop,ro that_file.img /mnt
Then explore the content:
# find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data
The 'openstack/content/0000' file should contain your network configuration. Is it what you expected?
The 'openstack/latest/user_data' file should contain the cloud-init ocnfiguration generated by oVirt, which your custom script appended. Does it look correct?
(Remember to "umount /mnt" when done.)
I started poking around in the cloud-init code. it's running v0.7.5 and the network code (forked) for rhel.py is here: https://github.com/number5/cloud-init/blob/0.7.5/cloudinit/distros/rhel.py#L... I used journalctl /usr/bin/cloud-init to get debug output and got this: Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] rhel.py[DEBUG]: Translated ubuntu style network settings auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local into {'eth0': {'dns-search': ['test.local'], 'auto': True, 'dns-nameservers': ['192.168.2.1'], 'netmask': '255.255.255.0', 'bootproto': 'static', 'address': '192.168.2.120', 'gateway': '192.168.2.1'}} Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network-scripts/ifcfg-eth0 (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 17 bytes from /etc/sysconfig/network-scripts/ifcfg-eth0 Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network-scripts/ifcfg-eth0 - wb: [420] 120 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/resolv.conf (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 86 bytes from /etc/resolv.conf Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/resolv.conf - wb: [420] 109 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/resolv.conf (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/resolv.conf (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 53 bytes from /etc/sysconfig/network Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network - wb: [420] 68 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False) It definitely feels like something related to cloud-init or my setup . I'll keep looking, perhaps the write to /etc/resolv.conf is failing silently. I can verify that any changes made to /etc/resolv.conf survive a service network restart so i'm not sure if it's not happening at all or what, but i don't think it's being overridden. On Mon, Feb 20, 2017 at 9:36 AM, Marc Young <3vilpenguin@gmail.com> wrote:
Still no change...
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant up You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com.
Bringing machine 'default' up with 'ovirt4' provider... ==> default: Creating VM with the following settings... ==> default: -- Name: testing ==> default: -- Cluster: Default ==> default: -- Template: vagrant-centos7 ==> default: -- Console Type: vnc ==> default: -- Memory: ==> default: ---- Memory: 2048 MB ==> default: ---- Maximum: 2048 MB ==> default: ---- Guaranteed: 2048 MB ==> default: -- Cpu: ==> default: ---- Cores: 2 ==> default: ---- Sockets: 1 ==> default: ---- Threads: 1 ==> default: -- Cloud-Init: false ==> default: Waiting for VM to become "ready" to start... ==> default: Starting VM.
From: /home/myoung/repos/github/vagrant-ovirt4/lib/vagrant-ovirt4/action/start_vm.rb @ line 79 VagrantPlugins::OVirtProvider::Action::StartVM#call:
74: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 75: 76: require 'pry' 77: binding.pry 78: => 79: machine.start( 80: use_cloud_init: true, 81: vm: vm_configuration 82: ) 83: 84: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"testing", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.120", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=>nil, :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com.
Last login: Wed Feb 15 00:23:35 2017 from 192.168.2.32 [vagrant@testing ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@testing ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.120 GATEWAY=192.168.2.1 ONBOOT=yes
If i check out the floppy attached to that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test [sudo] password for myoung: qemu 24311 1 87 01:38 ? 00:00:18 ..... [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/9964609e-8d85-4a5c-9416- 8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img . [myoung@ovirt ~]$ mkdir a; sudo mount -o loop,ro 9964609e-8d85-4a5c-9416- 8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img a [myoung@ovirt ~]$ sudo cat $(sudo find a -name 0000 -type f) auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount a
What's the next step? Is there a way to verify on the upstream that dns-search and dns-nameservers actually populate the ifcfg-eth0 file ?
On Sat, Feb 18, 2017 at 6:58 AM, Juan Hernández <jhernand@redhat.com> wrote:
On 02/17/2017 08:11 PM, Marc Young wrote:
Per the script, i'll change that, probably just wasnt paying attention to where i ran chkconfig
Per cloud-init YAML, not sure. It was in a previous reply so i gave it a shot. But i can't find that usage anywhere either so i gave up on it.
Per troubleshooting:
Spinning up a VM with dns_servers and dns_search:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].nil? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "manage-resolv-conf: true\n\nresolv_conf:\n nameservers: ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com <http://foo.example.com>\n - bar.example.com <http://bar.example.com>\n domain: example.com <http://example.com>\n options:\n rotate: true\n timeout: 1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key...
now ill ssh in and look around
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ 5❓ 1 $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com <http://vagrantup.com>.
Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198 [vagrant@test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true
resolv_conf: nameservers: ['8.8.4.4', '8.8.8.8'] searchdomains: - foo.example.com <http://foo.example.com> - bar.example.com <http://bar.example.com> domain: example.com <http://example.com> options: rotate: true timeout: 1
Apparently you are passing in the 'custom_script' some DNS configuration, which may conflict with the DNS configuration that you are passing in the 'dns_servers' and 'dns_search' options. It could be that cloud-init ignores one of them. I'd suggest that you try to remove completely the 'custom_script' parameter.
[vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain
[vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes
[vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i dns [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i local Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017 03:45:01 +0000. Up 1310.61 seconds. Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000. Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1].
Up
15.37 seconds
On the vm host running that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test qemu 12456 1 9 05:03 ? 00:00:19 /usr/libexec/qemu-kvm -name ...snipped [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846.
20fc2db0517e8c06579d7719d8f3fb35.img
. [myoung@ovirt ~]$ sudo mount -o loop,ro c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
/mnt [myoung@ovirt ~]$ find /mnt -type f find: ‘/mnt’: Permission denied [myoung@ovirt ~]$ sudo find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data [myoung@ovirt ~]$ sudo cat /mnt/openstack/content/0000 auto eth0 iface eth0 inet static address 192.168.2.200 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
Cloud-init definitely isn't working right away with resolv.conf, but I'm definitely passing it correctly to the API, it shows up on the floppy as you described but it's not making its way to /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other
for sure are (such as address, netmask,)
If i add that information to /etc/sysconfig/network-scripts/ifcfg-eth0 and bounce network it all works:
[vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ sudo service network restart Restarting network (via systemctl): [ OK ] [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> PING www.google.com <http://www.google.com> (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=1 ttl=54 time=55.8 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=2 ttl=54 time=43.9 ms ^C --- www.google.com <http://www.google.com> ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms [vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain nameserver 192.168.2.113 nameserver 192.168.2.1
On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand@redhat.com <mailto:jhernand@redhat.com>> wrote:
On 02/17/2017 06:00 AM, Marc Young wrote: > I'm apparently really bad at email, I replied only to Shahar, not
settings the
> whole thread. > > Vinzenz your email slipped first, so to answer your question: > > It's the latest Centos 7 with these installed: > > cloud-init-0.7.5-10.el7.centos.1 > kernel-3.10.0-514 <tel:3.10.0-514> <tel:3.10.0-514 <tel:3.10.0-514>>.el7 > ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1.
el7.centos
> > The setup script I use to create a template is here: > https://github.com/myoung34/vagrant-ovirt4/blob/master/tools
/prepare_redhat_for_box.sh
ls/prepare_redhat_for_box.sh>
> <https://github.com/myoung34/vagrant-ovirt4/blob/master/too
ls/prepare_redhat_for_box.sh
ls/prepare_redhat_for_box.sh>>
>
In that script you run "chkconfig cloud-init on" *before*
installing the
cloud-init package. That is irrelevant, as the cloud-init services
are
enabled by default when the package is installed. But worth
changing.
> The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos > The ruby SDK i'm working with is 4.1.2 > > Halfway through I realized that it's actually supported in the
API:
> > custom_script String > dns_search String > dns_servers String > > > It also shows usage here: > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/
sdk/examples/start_vm_with_cloud_init.rb
/sdk/examples/start_vm_with_cloud_init.rb>
> <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master
/sdk/examples/start_vm_with_cloud_init.rb
/sdk/examples/start_vm_with_cloud_init.rb>>
> > Here's some verification: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search]
=
> iface_options[:dns_search] unless
iface_options[:dns_search].nil?
> 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=>nil, > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > But it didn't do anything: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts
/ifcfg-eth0
> NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > > > The same is also true using cloud_init: > > ovirt.cloud_init =<<EOF > write_files: > - content: | > wat > path: /tmp/something.txt > permissions: '0644' > network-interfaces: | > auto eth0 > iface eth0 inet static > address 192.168.2.201 > network 192.168.2.0 > netmask 255.255.255.0 > gateway 192.168.2.1 > dns-nameservers 192.168.2.113 192.168.2.1 > EOF >
Is this ^ supposed to work in cloud-init? I didn't find it in the documentation. I thought that the only way to provide network
interface
configuration is via the 'openstack/content/whatever' file within
the
generated floppy.
> > and inspection: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search]
=
> iface_options[:dns_search] unless
iface_options[:dns_search].nil?
> 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=> > "write_files:\n - content: |\n wat\n path: > /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n > auto eth0\n iface eth0 inet static\n address
192.168.2.201\n
> network 192.168.2.0\n netmask 255.255.255.0\n gateway > 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n", > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > And here's my debugging after it comes up: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts
/ifcfg-eth0
> NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > [vagrant@test ~]$ cat /tmp/something.txt > wat > [vagrant@test ~]$ ping -c 3 www.google.com <
> <http://www.google.com/>ping: www.google.com <http://www.google.com> > <http://www.google.com/>: Name or service not known > [vagrant@test ~]$ echo $'nameserver 192.168.2.113\nnameserver > 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null > [vagrant@test ~]$ ping -c 3 www.google.com <
> <http://www.google.com/>PING www.google.com <http://www.google.com> > <http://www.google.com/> (216.58.217.4) 56(84) bytes of data. > 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=1 > ttl=54 time=47.5 ms > 64 bytes from den03s09-in-f4.1e100.net <
http://den03s09-in-f4.1e100.net>
> <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=2 > ttl=54 time=46.9 ms > 64 bytes from den03s09-in-f4.1e100.net <
http://den03s09-in-f4.1e100.net>
> <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=3 > ttl=54 time=44.3 ms > --- www.google.com <http://www.google.com> <http://www.google.com/> ping statistics --- > 3 packets transmitted, 3 received, 0% packet loss, time 2002ms > rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms >
Not sure if you did that already, but is very useful to check the content of the floppy that the oVirt engine creates and attaches to
the
virtual machine. To do so run the virtual machine, then go to the hypervisor where it is running and locate the corresponding
'qemu-kvm'
process:
# ps -ef | grep qemu-kvm | grep myvm
That will have a very long command line, and should have a 'payload' option that points to the generated floppy, something like this:
-drive file=/var/run/vdsm/payload/ae3a9cd4-....img
Copy that file somewhere, and mount it:
# mount -o loop,ro that_file.img /mnt
Then explore the content:
# find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data
The 'openstack/content/0000' file should contain your network configuration. Is it what you expected?
The 'openstack/latest/user_data' file should contain the cloud-init ocnfiguration generated by oVirt, which your custom script appended. Does it look correct?
(Remember to "umount /mnt" when done.)
FWIW I think the problem was cloud.cfg not having the resolv.conf module set up for management. I'm investigating now but it's taking longer than I like due to VPN issues. I'll update when I finally have the resolution On Mon, Feb 20, 2017 at 10:01 AM, Marc Young <3vilpenguin@gmail.com> wrote:
I started poking around in the cloud-init code. it's running v0.7.5 and the network code (forked) for rhel.py is here: https://github.com/ number5/cloud-init/blob/0.7.5/cloudinit/distros/rhel.py#L87
I used journalctl /usr/bin/cloud-init to get debug output and got this:
Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] rhel.py[DEBUG]: Translated ubuntu style network settings auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local into {'eth0': {'dns-search': ['test.local'], 'auto': True, 'dns-nameservers': ['192.168.2.1'], 'netmask': '255.255.255.0', 'bootproto': 'static', 'address': '192.168.2.120', 'gateway': '192.168.2.1'}} Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network-scripts/ifcfg-eth0 (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 17 bytes from /etc/sysconfig/network-scripts/ifcfg-eth0 Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network-scripts/ifcfg-eth0 - wb: [420] 120 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/resolv.conf (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 86 bytes from /etc/resolv.conf Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/resolv.conf - wb: [420] 109 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/resolv.conf (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/resolv.conf (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 53 bytes from /etc/sysconfig/network Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network - wb: [420] 68 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False)
It definitely feels like something related to cloud-init or my setup . I'll keep looking, perhaps the write to /etc/resolv.conf is failing silently. I can verify that any changes made to /etc/resolv.conf survive a service network restart so i'm not sure if it's not happening at all or what, but i don't think it's being overridden.
On Mon, Feb 20, 2017 at 9:36 AM, Marc Young <3vilpenguin@gmail.com> wrote:
Still no change...
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant up You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com.
Bringing machine 'default' up with 'ovirt4' provider... ==> default: Creating VM with the following settings... ==> default: -- Name: testing ==> default: -- Cluster: Default ==> default: -- Template: vagrant-centos7 ==> default: -- Console Type: vnc ==> default: -- Memory: ==> default: ---- Memory: 2048 MB ==> default: ---- Maximum: 2048 MB ==> default: ---- Guaranteed: 2048 MB ==> default: -- Cpu: ==> default: ---- Cores: 2 ==> default: ---- Sockets: 1 ==> default: ---- Threads: 1 ==> default: -- Cloud-Init: false ==> default: Waiting for VM to become "ready" to start... ==> default: Starting VM.
From: /home/myoung/repos/github/vagrant-ovirt4/lib/vagrant-ovirt4/action/start_vm.rb @ line 79 VagrantPlugins::OVirtProvider::Action::StartVM#call:
74: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 75: 76: require 'pry' 77: binding.pry 78: => 79: machine.start( 80: use_cloud_init: true, 81: vm: vm_configuration 82: ) 83: 84: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"testing", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.120", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=>nil, :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com.
Last login: Wed Feb 15 00:23:35 2017 from 192.168.2.32 [vagrant@testing ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@testing ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.120 GATEWAY=192.168.2.1 ONBOOT=yes
If i check out the floppy attached to that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test [sudo] password for myoung: qemu 24311 1 87 01:38 ? 00:00:18 ..... [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/9964609e -8d85-4a5c-9416-8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img . [myoung@ovirt ~]$ mkdir a; sudo mount -o loop,ro 9964609e-8d85-4a5c-9416-8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img a [myoung@ovirt ~]$ sudo cat $(sudo find a -name 0000 -type f) auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount a
What's the next step? Is there a way to verify on the upstream that dns-search and dns-nameservers actually populate the ifcfg-eth0 file ?
On Sat, Feb 18, 2017 at 6:58 AM, Juan Hernández <jhernand@redhat.com> wrote:
Per the script, i'll change that, probably just wasnt paying attention to where i ran chkconfig
Per cloud-init YAML, not sure. It was in a previous reply so i gave it a shot. But i can't find that usage anywhere either so i gave up on it.
Per troubleshooting:
Spinning up a VM with dns_servers and dns_search:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].ni l? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "manage-resolv-conf: true\n\nresolv_conf:\n nameservers: ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com <http://foo.example.com>\n - bar.example.com <http://bar.example.com>\n domain: example.com <http://example.com>\n options:\n rotate: true\n timeout: 1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's
On 02/17/2017 08:11 PM, Marc Young wrote: present...
default: Key inserted! Disconnecting and reconnecting using new SSH key...
now ill ssh in and look around
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ 5❓ 1 $ vagrant ssh You appear to be running Vagrant outside of the official
installers.
Note that the installers are what ensure that Vagrant has all
required
dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not
function
properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com <http://vagrantup.com>.
Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198 [vagrant@test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true
resolv_conf: nameservers: ['8.8.4.4', '8.8.8.8'] searchdomains: - foo.example.com <http://foo.example.com> - bar.example.com <http://bar.example.com> domain: example.com <http://example.com> options: rotate: true timeout: 1
Apparently you are passing in the 'custom_script' some DNS configuration, which may conflict with the DNS configuration that you are passing in the 'dns_servers' and 'dns_search' options. It could be that cloud-init ignores one of them. I'd suggest that you try to remove completely the 'custom_script' parameter.
[vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain
[vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes
[vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i dns [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i local Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017 03:45:01 +0000. Up 1310.61 seconds. Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000. Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1].
Up
15.37 seconds
On the vm host running that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test qemu 12456 1 9 05:03 ? 00:00:19 /usr/libexec/qemu-kvm -name ...snipped [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846.
20fc2db0517e8c06579d7719d8f3fb35.img
. [myoung@ovirt ~]$ sudo mount -o loop,ro c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
/mnt [myoung@ovirt ~]$ find /mnt -type f find: ‘/mnt’: Permission denied [myoung@ovirt ~]$ sudo find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data [myoung@ovirt ~]$ sudo cat /mnt/openstack/content/0000 auto eth0 iface eth0 inet static address 192.168.2.200 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
Cloud-init definitely isn't working right away with resolv.conf, but
definitely passing it correctly to the API, it shows up on the floppy as you described but it's not making its way to /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other settings for sure are (such as address, netmask,)
If i add that information to /etc/sysconfig/network-scripts/ifcfg-eth0 and bounce network it all works:
[vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ sudo service network restart Restarting network (via systemctl): [ OK ] [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com> PING www.google.com <http://www.google.com> (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=1 ttl=54 time=55.8 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=2 ttl=54 time=43.9 ms ^C --- www.google.com <http://www.google.com> ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms [vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain nameserver 192.168.2.113 nameserver 192.168.2.1
On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand@redhat.com <mailto:jhernand@redhat.com>> wrote:
On 02/17/2017 06:00 AM, Marc Young wrote: > I'm apparently really bad at email, I replied only to Shahar, not the > whole thread. > > Vinzenz your email slipped first, so to answer your question: > > It's the latest Centos 7 with these installed: > > cloud-init-0.7.5-10.el7.centos.1 > kernel-3.10.0-514 <tel:3.10.0-514> <tel:3.10.0-514 <tel:3.10.0-514>>.el7 > ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1. el7.centos > > The setup script I use to create a template is here: > https://github.com/myoung34/vagrant-ovirt4/blob/master/tools /prepare_redhat_for_box.sh <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh> > <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh>> >
In that script you run "chkconfig cloud-init on" *before* installing the cloud-init package. That is irrelevant, as the cloud-init services are enabled by default when the package is installed. But worth changing.
> The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos > The ruby SDK i'm working with is 4.1.2 > > Halfway through I realized that it's actually supported in the API: > > custom_script String > dns_search String > dns_servers String > > > It also shows usage here: > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/ sdk/examples/start_vm_with_cloud_init.rb <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb> > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb>> > > Here's some verification: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search] = > iface_options[:dns_search] unless iface_options[:dns_search].nil? > 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=>nil, > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > But it didn't do anything: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts /ifcfg-eth0 > NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > > > The same is also true using cloud_init: > > ovirt.cloud_init =<<EOF > write_files: > - content: | > wat > path: /tmp/something.txt > permissions: '0644' > network-interfaces: | > auto eth0 > iface eth0 inet static > address 192.168.2.201 > network 192.168.2.0 > netmask 255.255.255.0 > gateway 192.168.2.1 > dns-nameservers 192.168.2.113 192.168.2.1 > EOF >
Is this ^ supposed to work in cloud-init? I didn't find it in the documentation. I thought that the only way to provide network interface configuration is via the 'openstack/content/whatever' file within
I'm the
generated floppy.
> > and inspection: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search]
=
> iface_options[:dns_search] unless
iface_options[:dns_search].nil?
> 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=> > "write_files:\n - content: |\n wat\n path: > /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n > auto eth0\n iface eth0 inet static\n address
192.168.2.201\n
> network 192.168.2.0\n netmask 255.255.255.0\n gateway > 192.168.2.1\n dns-nameservers 192.168.2.113
192.168.2.1\n",
> :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > And here's my debugging after it comes up: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts
/ifcfg-eth0
> NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > [vagrant@test ~]$ cat /tmp/something.txt > wat > [vagrant@test ~]$ ping -c 3 www.google.com <
> <http://www.google.com/>ping: www.google.com <http://www.google.com> > <http://www.google.com/>: Name or service not known > [vagrant@test ~]$ echo $'nameserver
192.168.2.113\nnameserver
> 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null > [vagrant@test ~]$ ping -c 3 www.google.com <
> <http://www.google.com/>PING www.google.com <http://www.google.com> > <http://www.google.com/> (216.58.217.4) 56(84) bytes of
data.
> 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=1 > ttl=54 time=47.5 ms > 64 bytes from den03s09-in-f4.1e100.net <
http://den03s09-in-f4.1e100.net>
> <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=2 > ttl=54 time=46.9 ms > 64 bytes from den03s09-in-f4.1e100.net <
http://den03s09-in-f4.1e100.net>
> <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=3 > ttl=54 time=44.3 ms > --- www.google.com <http://www.google.com> <http://www.google.com/> ping statistics --- > 3 packets transmitted, 3 received, 0% packet loss, time
2002ms
> rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms >
Not sure if you did that already, but is very useful to check the content of the floppy that the oVirt engine creates and attaches
to the
virtual machine. To do so run the virtual machine, then go to the hypervisor where it is running and locate the corresponding
'qemu-kvm'
process:
# ps -ef | grep qemu-kvm | grep myvm
That will have a very long command line, and should have a
'payload'
option that points to the generated floppy, something like this:
-drive file=/var/run/vdsm/payload/ae3a9cd4-....img
Copy that file somewhere, and mount it:
# mount -o loop,ro that_file.img /mnt
Then explore the content:
# find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data
The 'openstack/content/0000' file should contain your network configuration. Is it what you expected?
The 'openstack/latest/user_data' file should contain the cloud-init ocnfiguration generated by oVirt, which your custom script
appended.
Does it look correct?
(Remember to "umount /mnt" when done.)
All good. Turns out the fix was -echo $'NM_CONTROLLED=no' > /etc/sysconfig/network-scripts/ifcfg-eth0 +echo $'' > /etc/sysconfig/network-scripts/ifcfg-eth0 + +CLOUD_CONFIG=/etc/cloud/cloud.cfg +grep -q ' - resolv-conf' $CLOUD_CONFIG || sed -i -e 's/ - timezone/&\n - resolv-conf/' $CLOUD_CONFIG +chkconfig NetworkManager off On Mon, Feb 20, 2017 at 1:12 PM, Marc Young <3vilpenguin@gmail.com> wrote:
FWIW I think the problem was cloud.cfg not having the resolv.conf module set up for management. I'm investigating now but it's taking longer than I like due to VPN issues. I'll update when I finally have the resolution
On Mon, Feb 20, 2017 at 10:01 AM, Marc Young <3vilpenguin@gmail.com> wrote:
I started poking around in the cloud-init code. it's running v0.7.5 and the network code (forked) for rhel.py is here: https://github.com/numbe r5/cloud-init/blob/0.7.5/cloudinit/distros/rhel.py#L87
I used journalctl /usr/bin/cloud-init to get debug output and got this:
Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] rhel.py[DEBUG]: Translated ubuntu style network settings auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local into {'eth0': {'dns-search': ['test.local'], 'auto': True, 'dns-nameservers': ['192.168.2.1'], 'netmask': '255.255.255.0', 'bootproto': 'static', 'address': '192.168.2.120', 'gateway': '192.168.2.1'}} Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network-scripts/ifcfg-eth0 (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 17 bytes from /etc/sysconfig/network-scripts/ifcfg-eth0 Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network-scripts/ifcfg-eth0 - wb: [420] 120 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/resolv.conf (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 86 bytes from /etc/resolv.conf Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/resolv.conf - wb: [420] 109 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/resolv.conf (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/resolv.conf (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 53 bytes from /etc/sysconfig/network Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network - wb: [420] 68 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False)
It definitely feels like something related to cloud-init or my setup . I'll keep looking, perhaps the write to /etc/resolv.conf is failing silently. I can verify that any changes made to /etc/resolv.conf survive a service network restart so i'm not sure if it's not happening at all or what, but i don't think it's being overridden.
On Mon, Feb 20, 2017 at 9:36 AM, Marc Young <3vilpenguin@gmail.com> wrote:
Still no change...
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant up You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com.
Bringing machine 'default' up with 'ovirt4' provider... ==> default: Creating VM with the following settings... ==> default: -- Name: testing ==> default: -- Cluster: Default ==> default: -- Template: vagrant-centos7 ==> default: -- Console Type: vnc ==> default: -- Memory: ==> default: ---- Memory: 2048 MB ==> default: ---- Maximum: 2048 MB ==> default: ---- Guaranteed: 2048 MB ==> default: -- Cpu: ==> default: ---- Cores: 2 ==> default: ---- Sockets: 1 ==> default: ---- Threads: 1 ==> default: -- Cloud-Init: false ==> default: Waiting for VM to become "ready" to start... ==> default: Starting VM.
From: /home/myoung/repos/github/vagrant-ovirt4/lib/vagrant-ovirt4/action/start_vm.rb @ line 79 VagrantPlugins::OVirtProvider::Action::StartVM#call:
74: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 75: 76: require 'pry' 77: binding.pry 78: => 79: machine.start( 80: use_cloud_init: true, 81: vm: vm_configuration 82: ) 83: 84: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"testing", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.120", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=>nil, :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com.
Last login: Wed Feb 15 00:23:35 2017 from 192.168.2.32 [vagrant@testing ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@testing ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.120 GATEWAY=192.168.2.1 ONBOOT=yes
If i check out the floppy attached to that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test [sudo] password for myoung: qemu 24311 1 87 01:38 ? 00:00:18 ..... [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/9964609e -8d85-4a5c-9416-8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img . [myoung@ovirt ~]$ mkdir a; sudo mount -o loop,ro 9964609e-8d85-4a5c-9416-8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img a [myoung@ovirt ~]$ sudo cat $(sudo find a -name 0000 -type f) auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount a
What's the next step? Is there a way to verify on the upstream that dns-search and dns-nameservers actually populate the ifcfg-eth0 file ?
On Sat, Feb 18, 2017 at 6:58 AM, Juan Hernández <jhernand@redhat.com> wrote:
Per the script, i'll change that, probably just wasnt paying attention to where i ran chkconfig
Per cloud-init YAML, not sure. It was in a previous reply so i gave it a shot. But i can't find that usage anywhere either so i gave up on it.
Per troubleshooting:
Spinning up a VM with dns_servers and dns_search:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].ni l? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "manage-resolv-conf: true\n\nresolv_conf:\n nameservers: ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com <http://foo.example.com>\n - bar.example.com <http://bar.example.com>\n domain: example.com <http://example.com>\n options:\n rotate: true\n timeout: 1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's
On 02/17/2017 08:11 PM, Marc Young wrote: present...
default: Key inserted! Disconnecting and reconnecting using
new
SSH key...
now ill ssh in and look around
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ 5❓ 1 $ vagrant ssh You appear to be running Vagrant outside of the official
installers.
Note that the installers are what ensure that Vagrant has all
required
dependencies, and Vagrant assumes that these dependencies exist.
By
running outside of the installer environment, Vagrant may not
function
properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com <http://vagrantup.com>.
Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198 [vagrant@test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true
resolv_conf: nameservers: ['8.8.4.4', '8.8.8.8'] searchdomains: - foo.example.com <http://foo.example.com> - bar.example.com <http://bar.example.com> domain: example.com <http://example.com> options: rotate: true timeout: 1
Apparently you are passing in the 'custom_script' some DNS configuration, which may conflict with the DNS configuration that you are passing in the 'dns_servers' and 'dns_search' options. It could be that cloud-init ignores one of them. I'd suggest that you try to remove completely the 'custom_script' parameter.
[vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain
[vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes
[vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i dns [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i local Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017 03:45:01 +0000. Up 1310.61 seconds. Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000. Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1].
Up
15.37 seconds
On the vm host running that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test qemu 12456 1 9 05:03 ? 00:00:19 /usr/libexec/qemu-kvm -name ...snipped [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846.
20fc2db0517e8c06579d7719d8f3fb35.img
. [myoung@ovirt ~]$ sudo mount -o loop,ro c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
/mnt [myoung@ovirt ~]$ find /mnt -type f find: ‘/mnt’: Permission denied [myoung@ovirt ~]$ sudo find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data [myoung@ovirt ~]$ sudo cat /mnt/openstack/content/0000 auto eth0 iface eth0 inet static address 192.168.2.200 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
Cloud-init definitely isn't working right away with resolv.conf, but
definitely passing it correctly to the API, it shows up on the floppy as you described but it's not making its way to /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other settings for sure are (such as address, netmask,)
If i add that information to /etc/sysconfig/network-scripts /ifcfg-eth0 and bounce network it all works:
[vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com
ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com
ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ sudo service network restart Restarting network (via systemctl): [ OK ] [vagrant@test ~]$ ping -c 3 www.google.com <http://www.google.com
PING www.google.com <http://www.google.com> (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=1 ttl=54 time=55.8 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=2 ttl=54 time=43.9 ms ^C --- www.google.com <http://www.google.com> ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms [vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain nameserver 192.168.2.113 nameserver 192.168.2.1
On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand@redhat.com <mailto:jhernand@redhat.com>> wrote:
On 02/17/2017 06:00 AM, Marc Young wrote: > I'm apparently really bad at email, I replied only to Shahar, not the > whole thread. > > Vinzenz your email slipped first, so to answer your question: > > It's the latest Centos 7 with these installed: > > cloud-init-0.7.5-10.el7.centos.1 > kernel-3.10.0-514 <tel:3.10.0-514> <tel:3.10.0-514 <tel:3.10.0-514>>.el7 > ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1. el7.centos > > The setup script I use to create a template is here: > https://github.com/myoung34/vagrant-ovirt4/blob/master/tools /prepare_redhat_for_box.sh <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh> > <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh>> >
In that script you run "chkconfig cloud-init on" *before* installing the cloud-init package. That is irrelevant, as the cloud-init services are enabled by default when the package is installed. But worth changing.
> The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos > The ruby SDK i'm working with is 4.1.2 > > Halfway through I realized that it's actually supported in the API: > > custom_script String > dns_search String > dns_servers String > > > It also shows usage here: > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/ sdk/examples/start_vm_with_cloud_init.rb <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb> > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb>> > > Here's some verification: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search] = > iface_options[:dns_search] unless iface_options[:dns_search].nil? > 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=>nil, > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > But it didn't do anything: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts /ifcfg-eth0 > NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > > > The same is also true using cloud_init: > > ovirt.cloud_init =<<EOF > write_files: > - content: | > wat > path: /tmp/something.txt > permissions: '0644' > network-interfaces: | > auto eth0 > iface eth0 inet static > address 192.168.2.201 > network 192.168.2.0 > netmask 255.255.255.0 > gateway 192.168.2.1 > dns-nameservers 192.168.2.113 192.168.2.1 > EOF >
Is this ^ supposed to work in cloud-init? I didn't find it in the documentation. I thought that the only way to provide network interface configuration is via the 'openstack/content/whatever' file within
I'm the
generated floppy.
> > and inspection: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search]
=
> iface_options[:dns_search] unless
iface_options[:dns_search].nil?
> 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > [1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=> > "write_files:\n - content: |\n wat\n path: > /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n > auto eth0\n iface eth0 inet static\n address
192.168.2.201\n
> network 192.168.2.0\n netmask 255.255.255.0\n gateway > 192.168.2.1\n dns-nameservers 192.168.2.113
192.168.2.1\n",
> :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > And here's my debugging after it comes up: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts
/ifcfg-eth0
> NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > [vagrant@test ~]$ cat /tmp/something.txt > wat > [vagrant@test ~]$ ping -c 3 www.google.com <
> <http://www.google.com/>ping: www.google.com <http://www.google.com> > <http://www.google.com/>: Name or service not known > [vagrant@test ~]$ echo $'nameserver
192.168.2.113\nnameserver
> 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null > [vagrant@test ~]$ ping -c 3 www.google.com <
> <http://www.google.com/>PING www.google.com <http://www.google.com> > <http://www.google.com/> (216.58.217.4) 56(84) bytes of
data.
> 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=1 > ttl=54 time=47.5 ms > 64 bytes from den03s09-in-f4.1e100.net <
http://den03s09-in-f4.1e100.net>
> <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=2 > ttl=54 time=46.9 ms > 64 bytes from den03s09-in-f4.1e100.net <
http://den03s09-in-f4.1e100.net>
> <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=3 > ttl=54 time=44.3 ms > --- www.google.com <http://www.google.com> <http://www.google.com/> ping statistics --- > 3 packets transmitted, 3 received, 0% packet loss, time
2002ms
> rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms >
Not sure if you did that already, but is very useful to check the content of the floppy that the oVirt engine creates and attaches
to the
virtual machine. To do so run the virtual machine, then go to the hypervisor where it is running and locate the corresponding
'qemu-kvm'
process:
# ps -ef | grep qemu-kvm | grep myvm
That will have a very long command line, and should have a
'payload'
option that points to the generated floppy, something like this:
-drive file=/var/run/vdsm/payload/ae3a9cd4-....img
Copy that file somewhere, and mount it:
# mount -o loop,ro that_file.img /mnt
Then explore the content:
# find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data
The 'openstack/content/0000' file should contain your network configuration. Is it what you expected?
The 'openstack/latest/user_data' file should contain the
cloud-init
ocnfiguration generated by oVirt, which your custom script
appended.
Does it look correct?
(Remember to "umount /mnt" when done.)
Its looks like the networkmanager override your configuration. Thanks for reporting. On Mon, Feb 20, 2017 at 10:23 PM, Marc Young <3vilpenguin@gmail.com> wrote:
All good. Turns out the fix was
-echo $'NM_CONTROLLED=no' > /etc/sysconfig/network-scripts/ifcfg-eth0 +echo $'' > /etc/sysconfig/network-scripts/ifcfg-eth0 + +CLOUD_CONFIG=/etc/cloud/cloud.cfg +grep -q ' - resolv-conf' $CLOUD_CONFIG || sed -i -e 's/ - timezone/&\n - resolv-conf/' $CLOUD_CONFIG +chkconfig NetworkManager off
On Mon, Feb 20, 2017 at 1:12 PM, Marc Young <3vilpenguin@gmail.com> wrote:
FWIW I think the problem was cloud.cfg not having the resolv.conf module set up for management. I'm investigating now but it's taking longer than I like due to VPN issues. I'll update when I finally have the resolution
On Mon, Feb 20, 2017 at 10:01 AM, Marc Young <3vilpenguin@gmail.com> wrote:
I started poking around in the cloud-init code. it's running v0.7.5 and the network code (forked) for rhel.py is here: https://github.com/numbe r5/cloud-init/blob/0.7.5/cloudinit/distros/rhel.py#L87
I used journalctl /usr/bin/cloud-init to get debug output and got this:
Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] rhel.py[DEBUG]: Translated ubuntu style network settings auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local into {'eth0': {'dns-search': ['test.local'], 'auto': True, 'dns-nameservers': ['192.168.2.1'], 'netmask': '255.255.255.0', 'bootproto': 'static', 'address': '192.168.2.120', 'gateway': '192.168.2.1'}} Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network-scripts/ifcfg-eth0 (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 17 bytes from /etc/sysconfig/network-scripts/ifcfg-eth0 Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network-scripts/ifcfg-eth0 - wb: [420] 120 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network-scripts/ifcfg-eth0 (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/resolv.conf (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 86 bytes from /etc/resolv.conf Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/resolv.conf - wb: [420] 109 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/resolv.conf (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/resolv.conf (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Reading from /etc/sysconfig/network (quiet=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Read 53 bytes from /etc/sysconfig/network Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Writing to /etc/sysconfig/network - wb: [420] 68 bytes Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False) Feb 19 19:39:03 vagrant cloud-init[647]: [CLOUDINIT] util.py[DEBUG]: Restoring selinux mode for /etc/sysconfig/network (recursive=False)
It definitely feels like something related to cloud-init or my setup . I'll keep looking, perhaps the write to /etc/resolv.conf is failing silently. I can verify that any changes made to /etc/resolv.conf survive a service network restart so i'm not sure if it's not happening at all or what, but i don't think it's being overridden.
On Mon, Feb 20, 2017 at 9:36 AM, Marc Young <3vilpenguin@gmail.com> wrote:
Still no change...
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant up You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com.
Bringing machine 'default' up with 'ovirt4' provider... ==> default: Creating VM with the following settings... ==> default: -- Name: testing ==> default: -- Cluster: Default ==> default: -- Template: vagrant-centos7 ==> default: -- Console Type: vnc ==> default: -- Memory: ==> default: ---- Memory: 2048 MB ==> default: ---- Maximum: 2048 MB ==> default: ---- Guaranteed: 2048 MB ==> default: -- Cpu: ==> default: ---- Cores: 2 ==> default: ---- Sockets: 1 ==> default: ---- Threads: 1 ==> default: -- Cloud-Init: false ==> default: Waiting for VM to become "ready" to start... ==> default: Starting VM.
From: /home/myoung/repos/github/vagrant-ovirt4/lib/vagrant-ovirt4/action/start_vm.rb @ line 79 VagrantPlugins::OVirtProvider::Action::StartVM#call:
74: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil? 75: 76: require 'pry' 77: binding.pry 78: => 79: machine.start( 80: use_cloud_init: true, 81: vm: vm_configuration 82: ) 83: 84: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"testing", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.120", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=>nil, :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's present... default: Key inserted! Disconnecting and reconnecting using new SSH key... myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ $ vagrant ssh You appear to be running Vagrant outside of the official installers. Note that the installers are what ensure that Vagrant has all required dependencies, and Vagrant assumes that these dependencies exist. By running outside of the installer environment, Vagrant may not function properly. To remove this warning, install Vagrant using one of the official packages from vagrantup.com.
Last login: Wed Feb 15 00:23:35 2017 from 192.168.2.32 [vagrant@testing ~]$ cat /etc/resolv.conf # Generated by NetworkManager search localdomain [vagrant@testing ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.120 GATEWAY=192.168.2.1 ONBOOT=yes
If i check out the floppy attached to that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test [sudo] password for myoung: qemu 24311 1 87 01:38 ? 00:00:18 ..... [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/9964609e -8d85-4a5c-9416-8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img . [myoung@ovirt ~]$ mkdir a; sudo mount -o loop,ro 9964609e-8d85-4a5c-9416-8e11b2fd5883.d0d0739973bfb4b6eb67d1f4974c3e0c.img a [myoung@ovirt ~]$ sudo cat $(sudo find a -name 0000 -type f) auto eth0 iface eth0 inet static address 192.168.2.120 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount a
What's the next step? Is there a way to verify on the upstream that dns-search and dns-nameservers actually populate the ifcfg-eth0 file ?
On Sat, Feb 18, 2017 at 6:58 AM, Juan Hernández <jhernand@redhat.com> wrote:
Per the script, i'll change that, probably just wasnt paying attention to where i ran chkconfig
Per cloud-init YAML, not sure. It was in a previous reply so i gave it a shot. But i can't find that usage anywhere either so i gave up on it.
Per troubleshooting:
Spinning up a VM with dns_servers and dns_search:
66: vm_configuration[:initialization][:dns_servers] = iface_options[:dns_servers] unless iface_options[:dns_servers].ni l? 67: vm_configuration[:initialization][:dns_search] = iface_options[:dns_search] unless iface_options[:dns_search].nil ? 68: require 'pry' 69: binding.pry 70: => 71: machine.start( 72: use_cloud_init: true, 73: vm: vm_configuration 74: ) 75: 76: @app.call(env)
[1] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> vm_configuration => {:initialization=> {:host_name=>"test", :nic_configurations=>[{:name=>"eth0", :on_boot=>true, :boot_protocol=>"static", :ip=>{:version=>"v4", :address=>"192.168.2.200", :gateway=>"192.168.2.1", :netmask=>"255.255.255.0"}}], :custom_script=> "manage-resolv-conf: true\n\nresolv_conf:\n nameservers: ['8.8.4.4', '8.8.8.8']\n searchdomains:\n - foo.example.com <http://foo.example.com>\n - bar.example.com <http://bar.example.com>\n domain: example.com <http://example.com>\n options:\n rotate: true\n timeout: 1\n", :dns_servers=>"192.168.2.1", :dns_search=>"test.local"}} [2] pry(#<VagrantPlugins::OVirtProvider::Action::StartVM>)> ==> default: Waiting for VM to get an IP address... ==> default: Machine is booted and ready for use! ==> default: Rsyncing folder: /home/myoung/repos/github/vagrant-ovirt4/ => /vagrant ==> default: Setting hostname... default: default: Vagrant insecure key detected. Vagrant will automatically replace default: this with a newly generated keypair for better security. default: default: Inserting generated public key within guest... default: Removing insecure key from the guest if it's
On 02/17/2017 08:11 PM, Marc Young wrote: present...
default: Key inserted! Disconnecting and reconnecting using
new
SSH key...
now ill ssh in and look around
myoung dev ~ repos github vagrant-ovirt4 feature/gh65 ✎ 5❓ 1 $ vagrant ssh You appear to be running Vagrant outside of the official
installers.
Note that the installers are what ensure that Vagrant has all
required
dependencies, and Vagrant assumes that these dependencies exist.
By
running outside of the installer environment, Vagrant may not
function
properly. To remove this warning, install Vagrant using one of
the
official packages from vagrantup.com <http://vagrantup.com>.
Last login: Thu Feb 16 23:04:00 2017 from 192.168.2.198 [vagrant@test ~]$ sudo cat /var/lib/cloud/instance/user-data.txt #cloud-config output: all: '>> /var/log/cloud-init-output.log' disable_root: 0 runcmd: - 'sed -i ''/^datasource_list: /d'' /etc/cloud/cloud.cfg; echo ''datasource_list: ["NoCloud", "ConfigDrive"]'' >> /etc/cloud/cloud.cfg' ssh_pwauth: true chpasswd: expire: false user: root manage-resolv-conf: true
resolv_conf: nameservers: ['8.8.4.4', '8.8.8.8'] searchdomains: - foo.example.com <http://foo.example.com> - bar.example.com <http://bar.example.com> domain: example.com <http://example.com> options: rotate: true timeout: 1
Apparently you are passing in the 'custom_script' some DNS configuration, which may conflict with the DNS configuration that you are passing in the 'dns_servers' and 'dns_search' options. It could be that cloud-init ignores one of them. I'd suggest that you try to remove completely the 'custom_script' parameter.
[vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain
[vagrant@test ~]$ cat /etc/sysconfig/network-scripts/ifcfg-eth0 NM_CONTROLLED=no NETMASK=255.255.255.0 BOOTPROTO=static DEVICE=eth0 IPADDR=192.168.2.200 GATEWAY=192.168.2.1 ONBOOT=yes
[vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i dns [vagrant@test ~]$ sudo cat /var/log/cloud-init* | grep -i local Feb 14 23:16:45 vagrant cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00
seconds.
Feb 16 23:03:35 test cloud-init: Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 running 'init-local' at Thu, 09 Feb 2017 03:45:01 +0000. Up 1310.61 seconds. Cloud-init v. 0.7.5 running 'init-local' at Wed, 15 Feb 2017 05:16:45 +0000. Up 11.00 seconds. Cloud-init v. 0.7.5 running 'init-local' at Fri, 17 Feb 2017 05:03:35 +0000. Up 5.23 seconds. Cloud-init v. 0.7.5 finished at Fri, 17 Feb 2017 05:03:45 +0000. Datasource DataSourceConfigDrive [local,ver=2][source=/dev/sr1].
Up
15.37 seconds
On the vm host running that VM:
[myoung@ovirt ~]$ sudo ps -ef | grep qemu-kvm | grep test qemu 12456 1 9 05:03 ? 00:00:19 /usr/libexec/qemu-kvm -name ...snipped [myoung@ovirt ~]$ sudo cp /var/run/vdsm/payload/c65751c3-431d-44e4-836c-963b81b1f846.
20fc2db0517e8c06579d7719d8f3fb35.img
. [myoung@ovirt ~]$ sudo mount -o loop,ro c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
/mnt [myoung@ovirt ~]$ find /mnt -type f find: ‘/mnt’: Permission denied [myoung@ovirt ~]$ sudo find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data [myoung@ovirt ~]$ sudo cat /mnt/openstack/content/0000 auto eth0 iface eth0 inet static address 192.168.2.200 netmask 255.255.255.0 gateway 192.168.2.1 dns-nameservers 192.168.2.1 dns-search test.local [myoung@ovirt ~]$ sudo umount c65751c3-431d-44e4-836c-963b81b1f846.20fc2db0517e8c06579d77
19d8f3fb35.img
Cloud-init definitely isn't working right away with resolv.conf, but
I'm
definitely passing it correctly to the API, it shows up on the floppy as you described but it's not making its way to /etc/sysconfig/network-scripts/ifcfg-eth0 even though the other settings for sure are (such as address, netmask,)
If i add that information to /etc/sysconfig/network-scripts /ifcfg-eth0 and bounce network it all works:
[vagrant@test ~]$ ping -c 3 www.google.com < http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ echo $'DNS1=192.168.2.113\nDNS2=192.168.2.1' | sudo tee -a /etc/sysconfig/network-scripts/ifcfg-eth0 >/dev/null [vagrant@test ~]$ ping -c 3 www.google.com < http://www.google.com> ping: www.google.com <http://www.google.com>: Name or service not known [vagrant@test ~]$ sudo service network restart Restarting network (via systemctl): [ OK ] [vagrant@test ~]$ ping -c 3 www.google.com < http://www.google.com> PING www.google.com <http://www.google.com> (216.58.217.4) 56(84) bytes of data. 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=1 ttl=54 time=55.8 ms 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> (216.58.217.4): icmp_seq=2 ttl=54 time=43.9 ms ^C --- www.google.com <http://www.google.com> ping statistics --- 2 packets transmitted, 2 received, 0% packet loss, time 1001ms rtt min/avg/max/mdev = 43.939/49.916/55.894/5.981 ms [vagrant@test ~]$ sudo cat /etc/resolv.conf # Generated by NetworkManager search localdomain nameserver 192.168.2.113 nameserver 192.168.2.1
On Fri, Feb 17, 2017 at 2:21 AM, Juan Hernández <jhernand@redhat.com <mailto:jhernand@redhat.com>> wrote:
On 02/17/2017 06:00 AM, Marc Young wrote: > I'm apparently really bad at email, I replied only to Shahar, not the > whole thread. > > Vinzenz your email slipped first, so to answer your question: > > It's the latest Centos 7 with these installed: > > cloud-init-0.7.5-10.el7.centos.1 > kernel-3.10.0-514 <tel:3.10.0-514> <tel:3.10.0-514 <tel:3.10.0-514>>.el7 > ovirt-guest-agent-common-1.0.13-1.20161220085008.git165fff1. el7.centos > > The setup script I use to create a template is here: > https://github.com/myoung34/vagrant-ovirt4/blob/master/tools /prepare_redhat_for_box.sh <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh> > <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh <https://github.com/myoung34/vagrant-ovirt4/blob/master/too ls/prepare_redhat_for_box.sh>> >
In that script you run "chkconfig cloud-init on" *before* installing the cloud-init package. That is irrelevant, as the cloud-init services are enabled by default when the package is installed. But worth changing.
> The engine-host is oVirt Engine Version: 4.1.0.4-1.el7.centos > The ruby SDK i'm working with is 4.1.2 > > Halfway through I realized that it's actually supported in the API: > > custom_script String > dns_search String > dns_servers String > > > It also shows usage here: > https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master/ sdk/examples/start_vm_with_cloud_init.rb <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb> > <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb <https://github.com/oVirt/ovirt-engine-sdk-ruby/blob/master /sdk/examples/start_vm_with_cloud_init.rb>> > > Here's some verification: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search] = > iface_options[:dns_search] unless iface_options[:dns_search].nil? > 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > > [1] pry(#<VagrantPlugins::OVirtPro vider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=>nil, > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > But it didn't do anything: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts /ifcfg-eth0 > NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > > > The same is also true using cloud_init: > > ovirt.cloud_init =<<EOF > write_files: > - content: | > wat > path: /tmp/something.txt > permissions: '0644' > network-interfaces: | > auto eth0 > iface eth0 inet static > address 192.168.2.201 > network 192.168.2.0 > netmask 255.255.255.0 > gateway 192.168.2.1 > dns-nameservers 192.168.2.113 192.168.2.1 > EOF >
Is this ^ supposed to work in cloud-init? I didn't find it in the documentation. I thought that the only way to provide network interface configuration is via the 'openstack/content/whatever' file within the generated floppy.
> > and inspection: > > 66: vm_configuration[:initialization][:dns_servers] = > iface_options[:dns_servers] unless iface_options[:dns_servers].nil? > 67: vm_configuration[:initialization][:dns_search] = > iface_options[:dns_search] unless iface_options[:dns_search].nil? > 68: require 'pry' > 69: binding.pry > 70: > => 71: machine.start( > 72: use_cloud_init: true, > 73: vm: vm_configuration > 74: ) > 75: > 76: @app.call(env) > [1] pry(#<VagrantPlugins::OVirtPro vider::Action::StartVM>)> > vm_configuration > => {:initialization=> > {:host_name=>"test", > :nic_configurations=>[{:name=>"eth0", :on_boot=>true, > :boot_protocol=>"static", :ip=>{:version=>"v4", > :address=>"192.168.2.200", :gateway=>"192.168.2.1", > :netmask=>"255.255.255.0"}}], > :custom_script=> > "write_files:\n - content: |\n wat\n path: > /tmp/something.txt\n permissions: '0644'\nnetwork-interfaces: |\n > auto eth0\n iface eth0 inet static\n address 192.168.2.201\n > network 192.168.2.0\n netmask 255.255.255.0\n gateway > 192.168.2.1\n dns-nameservers 192.168.2.113 192.168.2.1\n", > :dns_servers=>"192.168.2.1", > :dns_search=>"test.local"}} > > > And here's my debugging after it comes up: > > [vagrant@test ~]$ cat /etc/resolv.conf > # Generated by NetworkManager > search localdomain > [vagrant@test ~]$ cat /etc/sysconfig/network-scripts /ifcfg-eth0 > NM_CONTROLLED=no > NETMASK=255.255.255.0 > BOOTPROTO=static > DEVICE=eth0 > IPADDR=192.168.2.200 > GATEWAY=192.168.2.1 > ONBOOT=yes > [vagrant@test ~]$ cat /tmp/something.txt > wat > [vagrant@test ~]$ ping -c 3 www.google.com < http://www.google.com> > <http://www.google.com/>ping: www.google.com <http://www.google.com> > <http://www.google.com/>: Name or service not known > [vagrant@test ~]$ echo $'nameserver 192.168.2.113\nnameserver > 192.168.2.1' | sudo tee -a /etc/resolv.conf >/dev/null > [vagrant@test ~]$ ping -c 3 www.google.com < http://www.google.com> > <http://www.google.com/>PING www.google.com <http://www.google.com> > <http://www.google.com/> (216.58.217.4) 56(84) bytes of data. > 64 bytes from den03s09-in-f4.1e100.net <http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=1 > ttl=54 time=47.5 ms > 64 bytes from den03s09-in-f4.1e100.net < http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=2 > ttl=54 time=46.9 ms > 64 bytes from den03s09-in-f4.1e100.net < http://den03s09-in-f4.1e100.net> > <http://den03s09-in-f4.1e100.net/ <http://den03s09-in-f4.1e100.net/>> (216.58.217.4): icmp_seq=3 > ttl=54 time=44.3 ms > --- www.google.com <http://www.google.com> <http://www.google.com/> ping statistics --- > 3 packets transmitted, 3 received, 0% packet loss, time 2002ms > rtt min/avg/max/mdev = 44.385/46.307/47.555/1.390 ms >
Not sure if you did that already, but is very useful to check the content of the floppy that the oVirt engine creates and attaches to the virtual machine. To do so run the virtual machine, then go to the hypervisor where it is running and locate the corresponding 'qemu-kvm' process:
# ps -ef | grep qemu-kvm | grep myvm
That will have a very long command line, and should have a 'payload' option that points to the generated floppy, something like this:
-drive file=/var/run/vdsm/payload/ae3a9cd4-....img
Copy that file somewhere, and mount it:
# mount -o loop,ro that_file.img /mnt
Then explore the content:
# find /mnt -type f /mnt/openstack/content/0000 /mnt/openstack/latest/meta_data.json /mnt/openstack/latest/user_data
The 'openstack/content/0000' file should contain your network configuration. Is it what you expected?
The 'openstack/latest/user_data' file should contain the cloud-init ocnfiguration generated by oVirt, which your custom script appended. Does it look correct?
(Remember to "umount /mnt" when done.)
_______________________________________________ Devel mailing list Devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/devel
participants (4)
-
Juan Hernández -
Marc Young -
Shahar Havivi -
Vinzenz Feenstra