[ovirt-users] qemu cgroup_controllers
Дмитрий Глушенок
glush at jet.msk.su
Mon Jun 6 13:15:31 UTC 2016
So, libvirt behaves differently with custom partition? My assumption was that it will perform the same changes. I will have in mind in case oVirt 4 will be released too late.
May be I wrote incorrectly - in oVirt 4.0 RC1 scsi pass-through already working.
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. 1237)
> 6 июня 2016 г., в 13:35, Martin Polednik <mpolednik at redhat.com> написал(а):
>
> On 06/06/16 13:26 +0300, Дмитрий Глушенок wrote:
>> Hello Martin,
>>
>> Thank you for your time. It is clear how to create partition and assign it to a VM. But libvirt manipulates with contents of devices.list and I didn't find a way to put my devices into devices.list and stop libvirt from removing them.
>
> That is, by the way, pretty interesting. I have tried it right now and
> when using custom partition, libvirt doesn't touch my devices.list.
>
> It might make sense to file a libvirt bug.
>
>> Anyway, it looks like that the best solution will be using oVirt 4.0 where scsi pass-through have been fixed.
>
> Hoping so!
>
>> --
>> Dmitry Glushenok
>> Jet Infosystems
>> http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. 1237)
>>
>>> 6 июня 2016 г., в 12:33, Martin Polednik <mpolednik at redhat.com> написал(а):
>>>
>>> On 03/06/16 14:05 +0300, Дмитрий Глушенок wrote:
>>>> Thank you Martin!
>>>>
>>>> Actually I tried the workaround hook, provided in [2], but then VDSM (oVirt 3.6.6) tries to interpret hostdev in XML as PCI device, which leads to:
>>>>
>>>> ::The vm start process failed
>>>> Traceback (most recent call last):
>>>> File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm
>>>> self._run()
>>>> File "/usr/share/vdsm/virt/vm.py", line 1949, in _run
>>>> self._domDependentInit()
>>>> File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit
>>>> self._getUnderlyingVmDevicesInfo()
>>>> File "/usr/share/vdsm/virt/vm.py", line 1738, in _getUnderlyingVmDevicesInfo
>>>> self._getUnderlyingHostDeviceInfo()
>>>> File "/usr/share/vdsm/virt/vm.py", line 4277, in _getUnderlyingHostDeviceInfo
>>>> **self._getUnderlyingDeviceAddress(source))
>>>> TypeError: pci_address_to_name() got an unexpected keyword argument 'target'
>>>>
>>>> XML part was:
>>>> <hostdev managed="no" mode="subsystem" rawio="yes" type="scsi">
>>>> <source>
>>>> <adapter name="scsi_host2"/>
>>>> <address bus="0" target="1" unit="0"/>
>>>> </source>
>>>> </hostdev>
>>>> <hostdev managed="no" mode="subsystem" rawio="yes" type="scsi">
>>>> <source>
>>>> <adapter name="scsi_host2"/>
>>>> <address bus="0" target="2" unit="0"/>
>>>> </source>
>>>> </hostdev>
>>>>
>>>>
>>>> As of creating custom partition - by default machine.slice has "a *:* rwm" in devices.list. But for every new VM libvirt removes *:* mask and fills the list with actually needed devices (as I understand the process). For example:
>>>>
>>>> c 136:* rw
>>>> c 1:3 rw
>>>> c 1:7 rw
>>>> c 1:5 rw
>>>> c 1:8 rw
>>>> c 1:9 rw
>>>> c 5:2 rw
>>>> c 10:232 rw
>>>> c 253:0 rw
>>>> c 10:228 rw
>>>> c 10:196 rw
>>>>
>>>> What I'm looking for is a way to tell libvirt about my additional devices without breaking oVirt.
>>>
>>> The solution would be creating your own partition and somehow (e.g.
>>> VDSM hook) appending
>>>
>>> <resource>
>>> <partition>/machine/custom</partition>
>>> </resource>
>>>
>>> to the libvirt's <domain> element. I'm not sure how feasible creating
>>> your own partition is though. I've tried the process as follows:
>>>
>>> $ cat ~/create-partition.sh
>>> # sh ~/create-partition.sh
>>> for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls perf_event
>>> do
>>> mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition
>>> done
>>>
>>> for i in cpuset.cpus cpuset.mems
>>> do
>>> cat /sys/fs/cgroup/cpuset/machine.slice/$i > /sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i
>>> done
>>>
>>> (creates /machine/custom partition).
>>> Now, we can create vdsm before_vm_start hook that will set given partition
>>> for the VM:
>>>
>>> $ pwd /usr/libexec/vdsm/hooks/before_vm_start
>>> $ cat 10_cgroups
>>>
>>> #!/usr/bin/python
>>>
>>> import hooking
>>>
>>>
>>> def custom_partition(domxml):
>>> resource = domxml.createElement('resource')
>>> partition = domxml.createElement('partition')
>>> partition_text = domxml.createTextNode('/machine/custom')
>>> partition.appendChild(partition_text)
>>> resource.appendChild(partition)
>>>
>>> return resource
>>>
>>> domxml = hooking.read_domxml()
>>> domain = domxml.getElementsByTagName('domain')[0]
>>> domain.appendChild(custom_partition(domxml))
>>> hooking.write_domxml(domxml)
>>>
>>>> --
>>>> Dmitry Glushenok
>>>> Jet Infosystems
>>>> http://www.jet.msk.su
>>>> +7-495-411-7601 (ext. 1237)
>>>>
>>>>> 3 июня 2016 г., в 12:24, Martin Polednik <mpolednik at redhat.com> написал(а):
>>>>>
>>>>> On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote:
>>>>>> Hello!
>>>>>>
>>>>>> Is it possible to tell libvirt to add specific devices to qemu cgroup? By somehow enumerating the devices in XML using a hook for example.
>>>>>> I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it doesn't work until I remove "devices" from cgroup_controllers in qemu.conf.
>>>>>
>>>>> One way to achieve this is creating a hook to generate the scsi device
>>>>> XML instead of modifying qemu cmdline directly. Libvirt assumes
>>>>> ownership of all devices created in the XML and therefore adds them to
>>>>> the machine cgroup.
>>>>>
>>>>> Example of the XML taken from [1]:
>>>>> <devices>
>>>>> <hostdev mode='subsystem' type='scsi' sgio='filtered' rawio='yes'>
>>>>> <source>
>>>>> <adapter name='scsi_host0'/>
>>>>> <address bus='0' target='0' unit='0'/>
>>>>> </source>
>>>>> <readonly/>
>>>>> <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>>>>> </hostdev>
>>>>> </devices>
>>>>>
>>>>> There is slight issue with this approach outlined in [2].
>>>>>
>>>>> If you want to keep the qemu approach, I think creating a custom
>>>>> partition and moving devices there would be the cleanest approach. In
>>>>> this case, [3] could help but I'm not entirely sure if that would
>>>>> solve the issue.
>>>>>
>>>>> [1] https://libvirt.org/formatdomain.html
>>>>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485
>>>>> [3] https://libvirt.org/cgroups.html
>>>>>> --
>>>>>> Dmitry Glushenok
>>>>>> Jet Infosystems
>>>>>> http://www.jet.msk.su
>>>>>> +7-495-411-7601 (ext. 1237)
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users at ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160606/5f9d6f46/attachment-0001.html>
More information about the Users
mailing list