[ovirt-users] Is it possible to disable qxl video?

Martin Polednik mpolednik at redhat.com
Tue Jul 12 06:13:00 UTC 2016 

On 11/07/16 18:03 +0200, Arman Khalatyan wrote:
>Thanks for clarification. the kvm=off solved the problem.
>NICE!!! I got now inside the VM:
>OpenGL vendor string: NVIDIA Corporation
>OpenGL renderer string: GeForce GT 630/PCIe/SSE2
>OpenGL core profile version string: 4.4.0 NVIDIA 367.27
>OpenGL core profile shading language version string: 4.40 NVIDIA via Cg
>compiler
>OpenGL core profile context flags: (none)
>
>the steps to reproduce:
>1) virsh edit render
>2) change schema of xml: <domain type='kvm' xmlns:qemu='
>http://libvirt.org/schemas/domain/qemu/1.0'>
>3) add following in the end before </domain>:
><qemu:commandline>
>    <qemu:arg value='-cpu'/>
>    <qemu:arg value='host,kvm=off'/>
>  </qemu:commandline>
>4)  start vm, install nvidia driver, check gpu: nvidia smi
>5) install virtualgl, init 3
>6) nvidia-xconfig --enable-all-gpus --separate-x-screens
>--use-display-device=None --virtual=1280x1024
>init 5
>7) vglconnet render01.vm and finally: vglrun glxinfo or whatever!!!
>
>Now back to origin: how to add this tweak in to ovirt?? :)
>

We have hooks just for that! You'll want before_vm_start hook point.
Some idea how the hook could look like:

$ pwd
/usr/libexec/vdsm/hooks/before_vm_start

$ cat 99_maskkvm
#!/usr/bin/python

import hooking


if __name__ == '__main__':
    domxml = hooking.read_domxml()
    domain = domxml.getElementsByTagName('domain')[0]
    cmdline = domxml.createElement('qemu:commandline')
    for arg in ('-cpu', 'host,kvm=off'):
        cmdline_arg = domxml.createElement('qemu:arg')
        cmdline_arg.setAttribute('value', arg)
        cmdline.appendChild(cmdline_arg)

        domain.appendChild(cmdline)
        hooking.write_domxml(domxml)


The hook must be installed on every host that you expect to run
passthrough VMs.

mpolednik

>
>***********************************************************
>
> Dr. Arman Khalatyan  eScience -SuperComputing
> Leibniz-Institut für Astrophysik Potsdam (AIP)
> An der Sternwarte 16, 14482 Potsdam, Germany
>
>***********************************************************
>
>On Mon, Jul 11, 2016 at 5:30 PM, Martin Polednik <mpolednik at redhat.com>
>wrote:
>
>> On 11/07/16 17:16 +0200, Arman Khalatyan wrote:
>>
>>> Yes I have a BusID inside the xorg.conf.
>>> I just tested with virt-manager and indeed it is not a ovirt failure.
>>> It is something to do with qemu-kvm + nvidia.
>>> I found here:https://wiki.debian.org/VGAPassthrough  that nvidia disables
>>> device when it is running on qemu-kvm is it still the case? to overcome
>>> that one need to set:
>>>
>>> -cpu host,kvm=off
>>>
>>> in the command line of qemu.
>>>
>>
>> This applies to GPUs that NVIDIA does not support in virtualized
>> environments (actually your GTX630). Quadro K2200 and higher
>> (incl. grid, tesla) should not require masking the hypervisor. It
>> could be worth to supply dmesg from the host and from the guest so we
>> could see if there is any apparent failure.
>>
>> That being said, I have had failures even with supported card that
>> just stopped at one point. It's worth trying the kvm=off trick.
>>
>> I was trying to convert virsh commands to qemu native, but it fails on
>>> Centos 7 :(
>>> virsh dumpxml > render.xml
>>> virsh  domxml-to-native qemu-argv render.xml
>>> error: internal error: invalid PCI passthrough type 'default'
>>>
>>
>> I usually take the qemu command line from
>> /var/log/libvirt/qemu/${VM_NAME}, tweak it in vim and launch. Works
>> even for oVirt VMs (if you can hack around the storage).
>>
>>
>> ***********************************************************
>>>
>>> Dr. Arman Khalatyan  eScience -SuperComputing
>>> Leibniz-Institut für Astrophysik Potsdam (AIP)
>>> An der Sternwarte 16, 14482 Potsdam, Germany
>>>
>>> ***********************************************************
>>>
>>> On Mon, Jul 11, 2016 at 11:55 AM, Martin Polednik <mpolednik at redhat.com>
>>> wrote:
>>>
>>> On 11/07/16 09:46 +0000, Karli Sjöberg wrote:
>>>>
>>>>
>>>>> Den 11 jul 2016 11:37 fm skrev Arman Khalatyan <arm2arm at gmail.com>:
>>>>>
>>>>>
>>>>>> I just testing several GPUs with pci-passthrough: GTX630, Tesla 2050
>>>>>> and
>>>>>> FX5800.
>>>>>> None of them I got running with opengl/direct rendering inside the
>>>>>> virtual machine.
>>>>>> I was thinking something to do with qxl driver, but after blacklisting
>>>>>> it in the VMs kernel nothing was changed.
>>>>>> The VMs Xorg claiming always no display found even if I configure it
>>>>>> with headless option: nvidia-xconfig -a --use-display-device=None
>>>>>> --virtual=1280x1024
>>>>>> But with nvidia-smi I can see the GPU status, and I can run all cuda
>>>>>> tests where opengl/X11 is not involved.
>>>>>> My test system is CentOS 7 with ovirt 4.
>>>>>> BTW If I clone VM which was not running, to real host on bare metal,
>>>>>> everything is running as expecting.
>>>>>>
>>>>>>
>>>>> Don't remember If this was mentioned but have you tried doing the same
>>>>> thing but with virt-manager on a standalone host, just to rule out oVirt
>>>>> from the equation?
>>>>>
>>>>>
>>>> Also, the xorg must be setup to use the VFIO GPU. There is BusID
>>>> option in device section that must point to the guest address of the
>>>> GPU. I'm not sure if you've done that (and I'm quite sure it's missing
>>>> on the wiki, noted).
>>>>
>>>> [in the guest]
>>>> $ lspci | grep VGA
>>>> 03:00.0 VGA compatible controller: NVIDIA Corporation GM107GL [Quadro
>>>> K2200] (rev a2)
>>>>
>>>> and  using the address, add device section to /etc/X11/xorg.conf
>>>>
>>>> Section "Device"
>>>>    Identifier     "Device0"
>>>>    Driver         "nvidia"
>>>>    VendorName     "NVIDIA Corporation"
>>>>    BusID          "PCI:3:0:0"
>>>> EndSection
>>>>
>>>> That should allow xorg to use the assigned GPU.
>>>>
>>>> mpolednik
>>>>
>>>>
>>>> /K
>>>>
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ***********************************************************
>>>>>>
>>>>>>  Dr. Arman Khalatyan  eScience -SuperComputing
>>>>>>  Leibniz-Institut für Astrophysik Potsdam (AIP)
>>>>>>  An der Sternwarte 16, 14482 Potsdam, Germany
>>>>>>
>>>>>> ***********************************************************
>>>>>>
>>>>>> On Mon, Jul 11, 2016 at 9:06 AM, Martin Polednik <mpolednik at redhat.com
>>>>>> >
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>> On 08/07/16 11:29 +0200, Michal Skrivanek wrote:
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On 06 Jul 2016, at 14:06, Francesco Romani <fromani at redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> From: "Arman Khalatyan" <arm2arm at gmail.com>
>>>>>>>>> To: "users" <users at ovirt.org>
>>>>>>>>> Sent: Monday, July 4, 2016 11:52:24 AM
>>>>>>>>> Subject: [ovirt-users] Is it possible to disable qxl video?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I am doing PCI Passthrough for GPUs.
>>>>>>>>> Is it possible somehow to disable/remove default video qxl?
>>>>>>>>> thanks,
>>>>>>>>> Arman,.
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> The 4.0 Vdsm allows you to run a headless VM, meaning a VM without
>>>>>>>>> graphic device/frontend.
>>>>>>>>> Not sure Engine allows you that however.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> no it doesn’t
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> Even if engine allowed that, it may not be the correct solution.
>>>>>>>
>>>>>>> Running headless with PCI passthrough is considered primary VGA
>>>>>>> passthrough[1]. Using primary VGA passthrough without OVMF could cause
>>>>>>> (will cause) quite a bit of headaches.
>>>>>>>
>>>>>>> If you really want to try that, I assume using before_vm_start hook to
>>>>>>> remove the video device should be enough.
>>>>>>>
>>>>>>> [1] http://wiki.xenproject.org/wiki/Xen_VGA_Passthrough - it's
>>>>>>> possible to read about primary and secondary passthrough between the
>>>>>>> lines
>>>>>>>
>>>>>>> However, if QXL is the problem (why is that so?) then you can switch
>>>>>>> to
>>>>>>>
>>>>>>>> VGA(or CIRRUS if this is still 3.6)
>>>>>>>>
>>>>>>>> I'm afraid you will need to write a Vdsm hook.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> There could be simpler solutions depending on the Engine
>>>>>>>>>
>>>>>>>>> Bests,
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Francesco Romani
>>>>>>>>> RedHat Engineering Virtualization R & D
>>>>>>>>> Phone: 8261328
>>>>>>>>> IRC: fromani
>>>>>>>>> _______________________________________________
>>>>>>>>> Users mailing list
>>>>>>>>> Users at ovirt.org <mailto:Users at ovirt.org>
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users <
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>>> Users mailing list
>>>>>>>> Users at ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>

More information about the Users mailing list