Nested KVM: what is the correct setup
Host: CentOS-7_v1611 (or Fedora 25 -my personal fav) VMs: oVirt nodes v4.1.2 To setup nested KVM, I follow these instructions(https://pve.proxmox.com/wiki/Nested_Virtualization) to check if is enabled do ("kvm_intel" for intel cpu, "kvm_amd" for AMD) -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested N (N means it's not, to enable ("kvm-intel" for intel, "kvm-amd" for AMD):) # echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf and reboot or reload the kernel modul modprobe -r kvm_intel modprobe kvm_intel check again -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested Y Question: what other settings need to be enabled in the guest vm? See "test nested virt" at https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM Do we need this setting in the vm?
On 30/05/17 09:42 -0400, ovirt@fateknollogee.com wrote:
Host: CentOS-7_v1611 (or Fedora 25 -my personal fav) VMs: oVirt nodes v4.1.2
To setup nested KVM, I follow these instructions(https://pve.proxmox.com/wiki/Nested_Virtualization)
to check if is enabled do ("kvm_intel" for intel cpu, "kvm_amd" for AMD) -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested N (N means it's not, to enable ("kvm-intel" for intel, "kvm-amd" for AMD):)
# echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf and reboot or reload the kernel modul
modprobe -r kvm_intel modprobe kvm_intel check again
-root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested Y
Question: what other settings need to be enabled in the guest vm?
You need the VDSM hook (on the hypervisor) that sets up vmx feature flag and use host passthrough mode for CPU. https://github.com/oVirt/vdsm/tree/master/vdsm_hooks/nestedvt
See "test nested virt" at https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM Do we need this setting in the vm?
You don't need to setup anything in the VM.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
You need the VDSM hook (on the hypervisor) that sets up vmx feature Yes, I did that.
use host passthrough mode for CPU where is this setting, this is on the VM in Virt-Manager?
On 2017-05-30 09:54, Martin Polednik wrote:
On 30/05/17 09:42 -0400, ovirt@fateknollogee.com wrote:
Host: CentOS-7_v1611 (or Fedora 25 -my personal fav) VMs: oVirt nodes v4.1.2
To setup nested KVM, I follow these instructions(https://pve.proxmox.com/wiki/Nested_Virtualization)
to check if is enabled do ("kvm_intel" for intel cpu, "kvm_amd" for AMD) -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested N (N means it's not, to enable ("kvm-intel" for intel, "kvm-amd" for AMD):)
# echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf and reboot or reload the kernel modul
modprobe -r kvm_intel modprobe kvm_intel check again
-root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested Y
Question: what other settings need to be enabled in the guest vm?
You need the VDSM hook (on the hypervisor) that sets up vmx feature flag and use host passthrough mode for CPU.
https://github.com/oVirt/vdsm/tree/master/vdsm_hooks/nestedvt
See "test nested virt" at https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM Do we need this setting in the vm?
You don't need to setup anything in the VM.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
To install VDSM hook, I followed these instructions: http://community.redhat.com/blog/2013/08/testing-ovirt-3-3-with-nested-kvm/ Is this a proper way to do it? On 2017-05-30 09:54, Martin Polednik wrote:
On 30/05/17 09:42 -0400, ovirt@fateknollogee.com wrote:
Host: CentOS-7_v1611 (or Fedora 25 -my personal fav) VMs: oVirt nodes v4.1.2
To setup nested KVM, I follow these instructions(https://pve.proxmox.com/wiki/Nested_Virtualization)
to check if is enabled do ("kvm_intel" for intel cpu, "kvm_amd" for AMD) -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested N (N means it's not, to enable ("kvm-intel" for intel, "kvm-amd" for AMD):)
# echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf and reboot or reload the kernel modul
modprobe -r kvm_intel modprobe kvm_intel check again
-root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested Y
Question: what other settings need to be enabled in the guest vm?
You need the VDSM hook (on the hypervisor) that sets up vmx feature flag and use host passthrough mode for CPU.
https://github.com/oVirt/vdsm/tree/master/vdsm_hooks/nestedvt
See "test nested virt" at https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM Do we need this setting in the vm?
You don't need to setup anything in the VM.
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
You need in the VMs to have the 'vmx' flag, if you are using libvirt you most likely should use 'host-passthrough' mode, otherwise nested virtualization might not work. Regardless of the emulated CPU type, you need to ensure 'vmx' flag is there, this can also be done explicitly in the libvirt XML. If this is only for testing - I may suggest using oVirt system tests, which will do all these steps for you and setup a working oVirt environment(engine + 2 hosts). But, you will need to run it on a physical host. On Tue, May 30, 2017 at 4:42 PM, <ovirt@fateknollogee.com> wrote:
Host: CentOS-7_v1611 (or Fedora 25 -my personal fav) VMs: oVirt nodes v4.1.2
To setup nested KVM, I follow these instructions(https://pve.proxmox.com/wiki/Nested_Virtualization)
to check if is enabled do ("kvm_intel" for intel cpu, "kvm_amd" for AMD) -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested N (N means it's not, to enable ("kvm-intel" for intel, "kvm-amd" for AMD):)
# echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf and reboot or reload the kernel modul
modprobe -r kvm_intel modprobe kvm_intel check again
-root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested Y
Question: what other settings need to be enabled in the guest vm? See "test nested virt" at https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM Do we need this setting in the vm? _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Yes, this is only for testing I am using the oVirt nodes (engine + 3 hosts) all installed on a physical host (SuperMicro SYS-E200-8D) On 2017-05-30 09:55, Nadav Goldin wrote:
You need in the VMs to have the 'vmx' flag, if you are using libvirt you most likely should use 'host-passthrough' mode, otherwise nested virtualization might not work. Regardless of the emulated CPU type, you need to ensure 'vmx' flag is there, this can also be done explicitly in the libvirt XML.
If this is only for testing - I may suggest using oVirt system tests, which will do all these steps for you and setup a working oVirt environment(engine + 2 hosts). But, you will need to run it on a physical host.
On Tue, May 30, 2017 at 4:42 PM, <ovirt@fateknollogee.com> wrote:
Host: CentOS-7_v1611 (or Fedora 25 -my personal fav) VMs: oVirt nodes v4.1.2
To setup nested KVM, I follow these instructions(https://pve.proxmox.com/wiki/Nested_Virtualization)
to check if is enabled do ("kvm_intel" for intel cpu, "kvm_amd" for AMD) -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested N (N means it's not, to enable ("kvm-intel" for intel, "kvm-amd" for AMD):)
# echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf and reboot or reload the kernel modul
modprobe -r kvm_intel modprobe kvm_intel check again
-root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested Y
Question: what other settings need to be enabled in the guest vm? See "test nested virt" at https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM Do we need this setting in the vm? _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Tue, May 30, 2017 at 3:42 PM, <ovirt@fateknollogee.com> wrote:
Host: CentOS-7_v1611 (or Fedora 25 -my personal fav) VMs: oVirt nodes v4.1.2
To setup nested KVM, I follow these instructions(https://pve.proxm ox.com/wiki/Nested_Virtualization)
to check if is enabled do ("kvm_intel" for intel cpu, "kvm_amd" for AMD) -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested N (N means it's not, to enable ("kvm-intel" for intel, "kvm-amd" for AMD):)
# echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf and reboot or reload the kernel modul
modprobe -r kvm_intel modprobe kvm_intel check again
-root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested Y
Question: what other settings need to be enabled in the guest vm? See "test nested virt" at https://fedoraproject.org/wiki /How_to_enable_nested_virtualization_in_KVM Do we need this setting in the vm? _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
But one of the questions already asked to you also throughout the other thread was: what hypervisor are you using? More precisely, as it is clear it is Qemu-KVM, what is the way you start your VMs that are the candidates to become hypervisors in oVirt? Is it with virt-manager, as described in one of the links you provided, or direct virsh commands or what? If you are using virt-manager, what did you specify in CPUs --> Configuration --> Model ? Did you choose the checkbox to "copy host cpu configuration"? Gianluca
But one of the questions already asked to you also throughout the other thread was: what hypervisor are you using?
CentOS-7 v 1611 Question << What hypervisor should I be using??
More precisely, as it is clear it is Qemu-KVM, what is the way you start your VMs that are the candidates to become hypervisors in oVirt?
Not sure I understand this question?
Is it with virt-manager, as described in one of the links you provided, or direct virsh commands or what?
Virt-Manager
If you are using virt-manager, what did you specify in CPUs --> Configuration --> Model ?
Great question, my answer = none
Did you choose the checkbox to "copy host cpu configuration"?
Great question, my answer = no
On 2017-05-30 09:56, Gianluca Cecchi wrote:
On Tue, May 30, 2017 at 3:42 PM, <ovirt@fateknollogee.com> wrote:
Host: CentOS-7_v1611 (or Fedora 25 -my personal fav) VMs: oVirt nodes v4.1.2
To setup nested KVM, I follow these instructions(https://pve.proxmox.com/wiki/Nested_Virtualization [1])
to check if is enabled do ("kvm_intel" for intel cpu, "kvm_amd" for AMD) -root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested N (N means it's not, to enable ("kvm-intel" for intel, "kvm-amd" for AMD):)
# echo "options kvm-intel nested=Y" > /etc/modprobe.d/kvm-intel.conf and reboot or reload the kernel modul
modprobe -r kvm_intel modprobe kvm_intel check again
-root@proxmox:~# cat /sys/module/kvm_intel/parameters/nested Y
Question: what other settings need to be enabled in the guest vm? See "test nested virt" at
https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM
[2] Do we need this setting in the vm? _______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users [3]
But one of the questions already asked to you also throughout the other thread was: what hypervisor are you using? More precisely, as it is clear it is Qemu-KVM, what is the way you start your VMs that are the candidates to become hypervisors in oVirt? Is it with virt-manager, as described in one of the links you provided, or direct virsh commands or what? If you are using virt-manager, what did you specify in CPUs --> Configuration --> Model ? Did you choose the checkbox to "copy host cpu configuration"?
Gianluca
Links: ------ [1] https://pve.proxmox.com/wiki/Nested_Virtualization [2] https://fedoraproject.org/wiki/How_to_enable_nested_virtualization_in_KVM [3] http://lists.ovirt.org/mailman/listinfo/users
On Tue, May 30, 2017 at 4:08 PM, <ovirt@fateknollogee.com> wrote:
Is it with virt-manager, as described in one of the links you provided, or direct virsh commands or what?
Virt-Manager
If you are using virt-manager, what did you specify in CPUs --> Configuration --> Model ?
Great question, my answer = none
Did you choose the checkbox to "copy host cpu configuration"?
Great question, my answer = no
Well, so you should try to select "copy host cpu configuration" check box and see if you are able to boot the VM and to 1)install the dedicated oVirt Node-NG specialized image http://www.ovirt.org/documentation/install-guide/chap-oVirt_Nodes/ or 2) try to install a plain CentOS 7.3 OS + install hypervisor packages http://www.ovirt.org/documentation/install-guide/chap-Enterprise_Linux_Hosts... enable oVirt repos on this candidate host yum install <http://resources.ovirt.org/pub/yum-repo/ovirt-release41.rpm (btw: this part seems obfuscated in the installation guide....) and add the host to the engine http://www.ovirt.org/documentation/install-guide/chap-Adding_a_Hypervisor/ If you are not able to complete installation of node or OS, probably you have to enable only part of the cpu flags and not all (as you do when you passthrough copy them), and it is possible you to tweak a little to find the better for you... it depends on the real hw and particular cpu where your bare metal host is installed HIH a little clarifying, Gianluca
I just completed #2 right now (yes, I'm starting from scratch) and the host shows: CentOS Linux 7 (Core) Kernel 3.10.0514.el7.x86_64 on an x86_64 I am following these instructions: http://www.ovirt.org/blog/2017/04/up-and-running-with-ovirt-4-1-and-gluster-... Gianluca, thx for the detailed explanation. On 2017-05-30 10:25, Gianluca Cecchi wrote:
On Tue, May 30, 2017 at 4:08 PM, <ovirt@fateknollogee.com> wrote:
Is it with virt-manager, as described in one of the links you provided, or direct virsh commands or what?
Virt-Manager
If you are using virt-manager, what did you specify in CPUs --> Configuration --> Model ?
Great question, my answer = none
Did you choose the checkbox to "copy host cpu configuration"?
Great question, my answer = no
Well, so you should try to select "copy host cpu configuration" check box and see if you are able to boot the VM and to
1)install the dedicated oVirt Node-NG specialized image http://www.ovirt.org/documentation/install-guide/chap-oVirt_Nodes/
or
2) try to install a plain CentOS 7.3 OS + install hypervisor packages http://www.ovirt.org/documentation/install-guide/chap-Enterprise_Linux_Hosts...
enable oVirt repos on this candidate host yum install <http://resources.ovirt.org/pub/yum-repo/ovirt-release41.rpm
(btw: this part seems obfuscated in the installation guide....)
and add the host to the engine http://www.ovirt.org/documentation/install-guide/chap-Adding_a_Hypervisor/
If you are not able to complete installation of node or OS, probably you have to enable only part of the cpu flags and not all (as you do when you passthrough copy them), and it is possible you to tweak a little to find the better for you... it depends on the real hw and particular cpu where your bare metal host is installed
HIH a little clarifying, Gianluca
participants (4)
-
Gianluca Cecchi -
Martin Polednik -
Nadav Goldin -
ovirt＠fateknollogee.com