<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">So, libvirt behaves differently with custom partition? My assumption was that it will perform the same changes. I will have in mind in case oVirt 4 will be released too late.</div><div class="">May be I wrote incorrectly - in oVirt 4.0 RC1 scsi pass-through already working.</div><div class=""><br class=""></div><div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">--</div><div class=""><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="" style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div class="">Dmitry Glushenok</div><div class="">Jet Infosystems</div><div class=""><a href="http://www.jet.msk.su" class="">http://www.jet.msk.su</a></div></div></div></div>+7-495-411-7601 (ext. 1237)</div></div></div></div></div></div>
</div>
<br class=""><div><blockquote type="cite" class=""><div class="">6 июня 2016 г., в 13:35, Martin Polednik <<a href="mailto:mpolednik@redhat.com" class="">mpolednik@redhat.com</a>> написал(а):</div><br class="Apple-interchange-newline"><div class="">On 06/06/16 13:26 +0300, Дмитрий Глушенок wrote:<br class=""><blockquote type="cite" class="">Hello Martin,<br class=""><br class="">Thank you for your time. It is clear how to create partition and assign it to a VM. But libvirt manipulates with contents of devices.list and I didn't find a way to put my devices into devices.list and stop libvirt from removing them.<br class=""></blockquote><br class="">That is, by the way, pretty interesting. I have tried it right now and<br class="">when using custom partition, libvirt doesn't touch my devices.list.<br class=""><br class="">It might make sense to file a libvirt bug.<br class=""><br class=""><blockquote type="cite" class="">Anyway, it looks like that the best solution will be using oVirt 4.0 where scsi pass-through have been fixed.<br class=""></blockquote><br class="">Hoping so!<br class=""><br class=""><blockquote type="cite" class="">--<br class="">Dmitry Glushenok<br class="">Jet Infosystems<br class=""><a href="http://www.jet.msk.su" class="">http://www.jet.msk.su</a> <<a href="http://www.jet.msk.su/" class="">http://www.jet.msk.su/</a>>+7-495-411-7601 (ext. 1237)<br class=""><br class=""><blockquote type="cite" class="">6 июня 2016 г., в 12:33, Martin Polednik <<a href="mailto:mpolednik@redhat.com" class="">mpolednik@redhat.com</a>> написал(а):<br class=""><br class="">On 03/06/16 14:05 +0300, Дмитрий Глушенок wrote:<br class=""><blockquote type="cite" class="">Thank you Martin!<br class=""><br class="">Actually I tried the workaround hook, provided in [2], but then VDSM (oVirt 3.6.6) tries to interpret hostdev in XML as PCI device, which leads to:<br class=""><br class="">::The vm start process failed<br class="">Traceback (most recent call last):<br class="">File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm<br class=""> self._run()<br class="">File "/usr/share/vdsm/virt/vm.py", line 1949, in _run<br class=""> self._domDependentInit()<br class="">File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit<br class=""> self._getUnderlyingVmDevicesInfo()<br class="">File "/usr/share/vdsm/virt/vm.py", line 1738, in _getUnderlyingVmDevicesInfo<br class=""> self._getUnderlyingHostDeviceInfo()<br class="">File "/usr/share/vdsm/virt/vm.py", line 4277, in _getUnderlyingHostDeviceInfo<br class=""> **self._getUnderlyingDeviceAddress(source))<br class="">TypeError: pci_address_to_name() got an unexpected keyword argument 'target'<br class=""><br class="">XML part was:<br class=""><hostdev managed="no" mode="subsystem" rawio="yes" type="scsi"><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><source><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span><adapter name="scsi_host2"/><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span><address bus="0" target="1" unit="0"/><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span></source><br class=""></hostdev><br class=""><hostdev managed="no" mode="subsystem" rawio="yes" type="scsi"><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><source><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span><adapter name="scsi_host2"/><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span><address bus="0" target="2" unit="0"/><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span></source><br class=""></hostdev><br class=""><br class=""><br class="">As of creating custom partition - by default machine.slice has "a *:* rwm" in devices.list. But for every new VM libvirt removes *:* mask and fills the list with actually needed devices (as I understand the process). For example:<br class=""><br class="">c 136:* rw<br class="">c 1:3 rw<br class="">c 1:7 rw<br class="">c 1:5 rw<br class="">c 1:8 rw<br class="">c 1:9 rw<br class="">c 5:2 rw<br class="">c 10:232 rw<br class="">c 253:0 rw<br class="">c 10:228 rw<br class="">c 10:196 rw<br class=""><br class="">What I'm looking for is a way to tell libvirt about my additional devices without breaking oVirt.<br class=""></blockquote><br class="">The solution would be creating your own partition and somehow (e.g.<br class="">VDSM hook) appending<br class=""><br class=""><resource><br class=""> <partition>/machine/custom</partition><br class=""></resource><br class=""><br class="">to the libvirt's <domain> element. I'm not sure how feasible creating<br class="">your own partition is though. I've tried the process as follows:<br class=""><br class="">$ cat ~/create-partition.sh<br class=""># sh ~/create-partition.sh<br class="">for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls perf_event<br class="">do<br class=""> mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition<br class="">done<br class=""><br class="">for i in cpuset.cpus cpuset.mems<br class="">do<br class=""> cat /sys/fs/cgroup/cpuset/machine.slice/$i > /sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i<br class="">done<br class=""><br class="">(creates /machine/custom partition).<br class="">Now, we can create vdsm before_vm_start hook that will set given partition<br class="">for the VM:<br class=""><br class="">$ pwd /usr/libexec/vdsm/hooks/before_vm_start<br class="">$ cat 10_cgroups<br class=""><br class="">#!/usr/bin/python<br class=""><br class="">import hooking<br class=""><br class=""><br class="">def custom_partition(domxml):<br class=""> resource = domxml.createElement('resource')<br class=""> partition = domxml.createElement('partition')<br class=""> partition_text = domxml.createTextNode('/machine/custom')<br class=""> partition.appendChild(partition_text)<br class=""> resource.appendChild(partition)<br class=""><br class=""> return resource<br class=""><br class="">domxml = hooking.read_domxml()<br class="">domain = domxml.getElementsByTagName('domain')[0]<br class="">domain.appendChild(custom_partition(domxml))<br class="">hooking.write_domxml(domxml)<br class=""><br class=""><blockquote type="cite" class="">--<br class="">Dmitry Glushenok<br class="">Jet Infosystems<br class=""><a href="http://www.jet.msk.su" class="">http://www.jet.msk.su</a><br class="">+7-495-411-7601 (ext. 1237)<br class=""><br class=""><blockquote type="cite" class="">3 июня 2016 г., в 12:24, Martin Polednik <mpolednik@redhat.com> написал(а):<br class=""><br class="">On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote:<br class=""><blockquote type="cite" class="">Hello!<br class=""><br class="">Is it possible to tell libvirt to add specific devices to qemu cgroup? By somehow enumerating the devices in XML using a hook for example.<br class="">I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it doesn't work until I remove "devices" from cgroup_controllers in qemu.conf.<br class=""></blockquote><br class="">One way to achieve this is creating a hook to generate the scsi device<br class="">XML instead of modifying qemu cmdline directly. Libvirt assumes<br class="">ownership of all devices created in the XML and therefore adds them to<br class="">the machine cgroup.<br class=""><br class="">Example of the XML taken from [1]:<br class=""><devices><br class=""> <hostdev mode='subsystem' type='scsi' sgio='filtered' rawio='yes'><br class=""> <source><br class=""> <adapter name='scsi_host0'/><br class=""> <address bus='0' target='0' unit='0'/><br class=""> </source><br class=""> <readonly/><br class=""> <address type='drive' controller='0' bus='0' target='0' unit='0'/><br class=""> </hostdev><br class=""></devices><br class=""><br class="">There is slight issue with this approach outlined in [2].<br class=""><br class="">If you want to keep the qemu approach, I think creating a custom<br class="">partition and moving devices there would be the cleanest approach. In<br class="">this case, [3] could help but I'm not entirely sure if that would<br class="">solve the issue.<br class=""><br class="">[1] https://libvirt.org/formatdomain.html<br class="">[2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485<br class="">[3] https://libvirt.org/cgroups.html<br class=""><blockquote type="cite" class="">--<br class="">Dmitry Glushenok<br class="">Jet Infosystems<br class="">http://www.jet.msk.su<br class="">+7-495-411-7601 (ext. 1237)<br class=""><br class="">_______________________________________________<br class="">Users mailing list<br class="">Users@ovirt.org<br class="">http://lists.ovirt.org/mailman/listinfo/users<br class=""></blockquote></blockquote></blockquote></blockquote><br class=""></blockquote></div></blockquote></div><br class=""></body></html>