Re: [ovirt-users] qemu cgroup_controllers

So, libvirt behaves differently with custom partition? My assumption was that it will perform the same changes. I will have in mind in case oVirt 4 will be released too late. May be I wrote incorrectly - in oVirt 4.0 RC1 scsi pass-through already working.

-- Dmitry Glushenok Jet Infosystems http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. 1237) > 6 июня 2016 г., в 13:35, Martin Polednik <mpolednik(a)redhat.com&gt; написал(а): > > On 06/06/16 13:26 +0300, Дмитрий Глушенок wrote: >> Hello Martin, >> >> Thank you for your time. It is clear how to create partition and assign it to a VM. But libvirt manipulates with contents of devices.list and I didn't find a way to put my devices into devices.list and stop libvirt from removing them. > > That is, by the way, pretty interesting. I have tried it right now and > when using custom partition, libvirt doesn't touch my devices.list. > > It might make sense to file a libvirt bug. > >> Anyway, it looks like that the best solution will be using oVirt 4.0 where scsi pass-through have been fixed. > > Hoping so! > >> -- >> Dmitry Glushenok >> Jet Infosystems >> http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. 1237) >> >>> 6 июня 2016 г., в 12:33, Martin Polednik <mpolednik(a)redhat.com&gt; написал(а): >>> >>> On 03/06/16 14:05 +0300, Дмитрий Глушенок wrote: >>>> Thank you Martin! >>>> >>>> Actually I tried the workaround hook, provided in [2], but then VDSM (oVirt 3.6.6) tries to interpret hostdev in XML as PCI device, which leads to: >>>> >>>> ::The vm start process failed >>>> Traceback (most recent call last): >>>> File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm >>>> self._run() >>>> File "/usr/share/vdsm/virt/vm.py", line 1949, in _run >>>> self._domDependentInit() >>>> File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit >>>> self._getUnderlyingVmDevicesInfo() >>>> File "/usr/share/vdsm/virt/vm.py", line 1738, in _getUnderlyingVmDevicesInfo >>>> self._getUnderlyingHostDeviceInfo() >>>> File "/usr/share/vdsm/virt/vm.py", line 4277, in _getUnderlyingHostDeviceInfo >>>> **self._getUnderlyingDeviceAddress(source)) >>>> TypeError: pci_address_to_name() got an unexpected keyword argument 'target' >>>> >>>> XML part was: >>>> <hostdev managed="no" mode="subsystem" rawio="yes" type="scsi"> >>>> <source> >>>> <adapter name="scsi_host2"/> >>>> <address bus="0" target="1" unit="0"/> >>>> </source> >>>> </hostdev> >>>> <hostdev managed="no" mode="subsystem" rawio="yes" type="scsi"> >>>> <source> >>>> <adapter name="scsi_host2"/> >>>> <address bus="0" target="2" unit="0"/> >>>> </source> >>>> </hostdev> >>>> >>>> >>>> As of creating custom partition - by default machine.slice has "a *:* rwm" in devices.list. But for every new VM libvirt removes *:* mask and fills the list with actually needed devices (as I understand the process). For example: >>>> >>>> c 136:* rw >>>> c 1:3 rw >>>> c 1:7 rw >>>> c 1:5 rw >>>> c 1:8 rw >>>> c 1:9 rw >>>> c 5:2 rw >>>> c 10:232 rw >>>> c 253:0 rw >>>> c 10:228 rw >>>> c 10:196 rw >>>> >>>> What I'm looking for is a way to tell libvirt about my additional devices without breaking oVirt. >>> >>> The solution would be creating your own partition and somehow (e.g. >>> VDSM hook) appending >>> >>> <resource> >>> <partition>/machine/custom</partition> >>> </resource> >>> >>> to the libvirt's <domain> element. I'm not sure how feasible creating >>> your own partition is though. I've tried the process as follows: >>> >>> $ cat ~/create-partition.sh >>> # sh ~/create-partition.sh >>> for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls perf_event >>> do >>> mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition >>> done >>> >>> for i in cpuset.cpus cpuset.mems >>> do >>> cat /sys/fs/cgroup/cpuset/machine.slice/$i > /sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i >>> done >>> >>> (creates /machine/custom partition). >>> Now, we can create vdsm before_vm_start hook that will set given partition >>> for the VM: >>> >>> $ pwd /usr/libexec/vdsm/hooks/before_vm_start >>> $ cat 10_cgroups >>> >>> #!/usr/bin/python >>> >>> import hooking >>> >>> >>> def custom_partition(domxml): >>> resource = domxml.createElement('resource') >>> partition = domxml.createElement('partition') >>> partition_text = domxml.createTextNode('/machine/custom') >>> partition.appendChild(partition_text) >>> resource.appendChild(partition) >>> >>> return resource >>> >>> domxml = hooking.read_domxml() >>> domain = domxml.getElementsByTagName('domain')[0] >>> domain.appendChild(custom_partition(domxml)) >>> hooking.write_domxml(domxml) >>> >>>> -- >>>> Dmitry Glushenok >>>> Jet Infosystems >>>> http://www.jet.msk.su >>>> +7-495-411-7601 (ext. 1237) >>>> >>>>> 3 июня 2016 г., в 12:24, Martin Polednik <mpolednik(a)redhat.com&gt; написал(а): >>>>> >>>>> On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote: >>>>>> Hello! >>>>>> >>>>>> Is it possible to tell libvirt to add specific devices to qemu cgroup? By somehow enumerating the devices in XML using a hook for example. >>>>>> I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it doesn't work until I remove "devices" from cgroup_controllers in qemu.conf. >>>>> >>>>> One way to achieve this is creating a hook to generate the scsi device >>>>> XML instead of modifying qemu cmdline directly. Libvirt assumes >>>>> ownership of all devices created in the XML and therefore adds them to >>>>> the machine cgroup. >>>>> >>>>> Example of the XML taken from [1]: >>>>> <devices> >>>>> <hostdev mode='subsystem' type='scsi' sgio='filtered' rawio='yes'> >>>>> <source> >>>>> <adapter name='scsi_host0'/> >>>>> <address bus='0' target='0' unit='0'/> >>>>> </source> >>>>> <readonly/> >>>>> <address type='drive' controller='0' bus='0' target='0' unit='0'/> >>>>> </hostdev> >>>>> </devices> >>>>> >>>>> There is slight issue with this approach outlined in [2]. >>>>> >>>>> If you want to keep the qemu approach, I think creating a custom >>>>> partition and moving devices there would be the cleanest approach. In >>>>> this case, [3] could help but I'm not entirely sure if that would >>>>> solve the issue. >>>>> >>>>> [1] https://libvirt.org/formatdomain.html >>>>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485 >>>>> [3] https://libvirt.org/cgroups.html >>>>>> -- >>>>>> Dmitry Glushenok >>>>>> Jet Infosystems >>>>>> http://www.jet.msk.su >>>>>> +7-495-411-7601 (ext. 1237) >>>>>> >>>>>> _______________________________________________ >>>>>> Users mailing list >>>>>> Users(a)ovirt.org >>>>>> http://lists.ovirt.org/mailman/listinfo/users >>