<div dir="ltr"><div><div>Thanks for clarification. the kvm=off solved the problem.<br></div>NICE!!! I got now inside the VM:<br>OpenGL vendor string: NVIDIA Corporation<br>OpenGL renderer string: GeForce GT 630/PCIe/SSE2<br>OpenGL core profile version string: 4.4.0 NVIDIA 367.27<br>OpenGL core profile shading language version string: 4.40 NVIDIA via Cg compiler<br>OpenGL core profile context flags: (none)<br><br></div><div>the steps to reproduce:<br></div><div>1) virsh edit render <br></div><div>2) change schema of xml: <domain type='kvm' xmlns:qemu='<a href="http://libvirt.org/schemas/domain/qemu/1.0">http://libvirt.org/schemas/domain/qemu/1.0</a>'><br></div><div>3) add following in the end before </domain>:<br><qemu:commandline><br> <qemu:arg value='-cpu'/><br> <qemu:arg value='host,kvm=off'/><br> </qemu:commandline><br></div><div>4) start vm, install nvidia driver, check gpu: nvidia smi<br></div><div>5) install virtualgl, init 3 <br>6) nvidia-xconfig --enable-all-gpus --separate-x-screens --use-display-device=None --virtual=1280x1024<br></div><div>init 5<br></div><div>7) vglconnet render01.vm and finally: vglrun glxinfo or whatever!!!<br><br></div><div>Now back to origin: how to add this tweak in to ovirt?? :)<br><br><br><br></div><div><br></div><div><br></div><div><br><br><br></div></div><div class="gmail_extra"><br clear="all"><div><div class="gmail_signature" data-smartmail="gmail_signature">***********************************************************<br><br> Dr. Arman Khalatyan eScience -SuperComputing <br> Leibniz-Institut für Astrophysik Potsdam (AIP) <br> An der Sternwarte 16, 14482 Potsdam, Germany <br><br>***********************************************************</div></div>
<br><div class="gmail_quote">On Mon, Jul 11, 2016 at 5:30 PM, Martin Polednik <span dir="ltr"><<a href="mailto:mpolednik@redhat.com" target="_blank">mpolednik@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 11/07/16 17:16 +0200, Arman Khalatyan wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Yes I have a BusID inside the xorg.conf.<br>
I just tested with virt-manager and indeed it is not a ovirt failure.<br>
It is something to do with qemu-kvm + nvidia.<br>
I found here:<a href="https://wiki.debian.org/VGAPassthrough" rel="noreferrer" target="_blank">https://wiki.debian.org/VGAPassthrough</a> that nvidia disables<br>
device when it is running on qemu-kvm is it still the case? to overcome<br>
that one need to set:<br>
<br>
-cpu host,kvm=off<br>
<br>
in the command line of qemu.<br>
</blockquote>
<br></span>
This applies to GPUs that NVIDIA does not support in virtualized<br>
environments (actually your GTX630). Quadro K2200 and higher<br>
(incl. grid, tesla) should not require masking the hypervisor. It<br>
could be worth to supply dmesg from the host and from the guest so we<br>
could see if there is any apparent failure.<br>
<br>
That being said, I have had failures even with supported card that<br>
just stopped at one point. It's worth trying the kvm=off trick.<span class=""><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I was trying to convert virsh commands to qemu native, but it fails on<br>
Centos 7 :(<br>
virsh dumpxml > render.xml<br>
virsh domxml-to-native qemu-argv render.xml<br>
error: internal error: invalid PCI passthrough type 'default'<br>
</blockquote>
<br></span>
I usually take the qemu command line from<br>
/var/log/libvirt/qemu/${VM_NAME}, tweak it in vim and launch. Works<br>
even for oVirt VMs (if you can hack around the storage).<div class="HOEnZb"><div class="h5"><br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
***********************************************************<br>
<br>
Dr. Arman Khalatyan eScience -SuperComputing<br>
Leibniz-Institut für Astrophysik Potsdam (AIP)<br>
An der Sternwarte 16, 14482 Potsdam, Germany<br>
<br>
***********************************************************<br>
<br>
On Mon, Jul 11, 2016 at 11:55 AM, Martin Polednik <<a href="mailto:mpolednik@redhat.com" target="_blank">mpolednik@redhat.com</a>><br>
wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 11/07/16 09:46 +0000, Karli Sjöberg wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Den 11 jul 2016 11:37 fm skrev Arman Khalatyan <<a href="mailto:arm2arm@gmail.com" target="_blank">arm2arm@gmail.com</a>>:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I just testing several GPUs with pci-passthrough: GTX630, Tesla 2050 and<br>
FX5800.<br>
None of them I got running with opengl/direct rendering inside the<br>
virtual machine.<br>
I was thinking something to do with qxl driver, but after blacklisting<br>
it in the VMs kernel nothing was changed.<br>
The VMs Xorg claiming always no display found even if I configure it<br>
with headless option: nvidia-xconfig -a --use-display-device=None<br>
--virtual=1280x1024<br>
But with nvidia-smi I can see the GPU status, and I can run all cuda<br>
tests where opengl/X11 is not involved.<br>
My test system is CentOS 7 with ovirt 4.<br>
BTW If I clone VM which was not running, to real host on bare metal,<br>
everything is running as expecting.<br>
<br>
</blockquote>
<br>
Don't remember If this was mentioned but have you tried doing the same<br>
thing but with virt-manager on a standalone host, just to rule out oVirt<br>
from the equation?<br>
<br>
</blockquote>
<br>
Also, the xorg must be setup to use the VFIO GPU. There is BusID<br>
option in device section that must point to the guest address of the<br>
GPU. I'm not sure if you've done that (and I'm quite sure it's missing<br>
on the wiki, noted).<br>
<br>
[in the guest]<br>
$ lspci | grep VGA<br>
03:00.0 VGA compatible controller: NVIDIA Corporation GM107GL [Quadro<br>
K2200] (rev a2)<br>
<br>
and using the address, add device section to /etc/X11/xorg.conf<br>
<br>
Section "Device"<br>
Identifier "Device0"<br>
Driver "nvidia"<br>
VendorName "NVIDIA Corporation"<br>
BusID "PCI:3:0:0"<br>
EndSection<br>
<br>
That should allow xorg to use the assigned GPU.<br>
<br>
mpolednik<br>
<br>
<br>
/K<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
<br>
<br>
<br>
<br>
<br>
***********************************************************<br>
<br>
Dr. Arman Khalatyan eScience -SuperComputing<br>
Leibniz-Institut für Astrophysik Potsdam (AIP)<br>
An der Sternwarte 16, 14482 Potsdam, Germany<br>
<br>
***********************************************************<br>
<br>
On Mon, Jul 11, 2016 at 9:06 AM, Martin Polednik <<a href="mailto:mpolednik@redhat.com" target="_blank">mpolednik@redhat.com</a>><br>
wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
On 08/07/16 11:29 +0200, Michal Skrivanek wrote:<br>
<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
On 06 Jul 2016, at 14:06, Francesco Romani <<a href="mailto:fromani@redhat.com" target="_blank">fromani@redhat.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
From: "Arman Khalatyan" <<a href="mailto:arm2arm@gmail.com" target="_blank">arm2arm@gmail.com</a>><br>
To: "users" <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
Sent: Monday, July 4, 2016 11:52:24 AM<br>
Subject: [ovirt-users] Is it possible to disable qxl video?<br>
<br>
<br>
Hi,<br>
I am doing PCI Passthrough for GPUs.<br>
Is it possible somehow to disable/remove default video qxl?<br>
thanks,<br>
Arman,.<br>
Hi,<br>
<br>
The 4.0 Vdsm allows you to run a headless VM, meaning a VM without<br>
graphic device/frontend.<br>
Not sure Engine allows you that however.<br>
<br>
</blockquote>
<br>
<br>
no it doesn’t<br>
<br>
</blockquote>
<br>
<br>
Even if engine allowed that, it may not be the correct solution.<br>
<br>
Running headless with PCI passthrough is considered primary VGA<br>
passthrough[1]. Using primary VGA passthrough without OVMF could cause<br>
(will cause) quite a bit of headaches.<br>
<br>
If you really want to try that, I assume using before_vm_start hook to<br>
remove the video device should be enough.<br>
<br>
[1] <a href="http://wiki.xenproject.org/wiki/Xen_VGA_Passthrough" rel="noreferrer" target="_blank">http://wiki.xenproject.org/wiki/Xen_VGA_Passthrough</a> - it's<br>
possible to read about primary and secondary passthrough between the<br>
lines<br>
<br>
However, if QXL is the problem (why is that so?) then you can switch to<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
VGA(or CIRRUS if this is still 3.6)<br>
<br>
I'm afraid you will need to write a Vdsm hook.<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
There could be simpler solutions depending on the Engine<br>
<br>
Bests,<br>
<br>
--<br>
Francesco Romani<br>
RedHat Engineering Virtualization R & D<br>
Phone: 8261328<br>
IRC: fromani<br>
_______________________________________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a>><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a> <<br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a>><br>
<br>
</blockquote>
<br>
</blockquote>
<br>
_______________________________________________<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
<br>
</blockquote>
<br>
<br>
<br>
</blockquote>
<br>
</blockquote></blockquote></blockquote></blockquote>
</div></div></blockquote></div><br></div>