[ovirt-users] qemu cgroup_controllers

Дмитрий Глушенок glush at jet.msk.su
Mon Jun 6 06:26:25 EDT 2016 

Hello Martin,

Thank you for your time. It is clear how to create partition and assign it to a VM. But libvirt manipulates with contents of devices.list and I didn't find a way to put my devices into devices.list and stop libvirt from removing them.

Anyway, it looks like that the best solution will be using oVirt 4.0 where scsi pass-through have been fixed.

--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. 1237)

> 6 июня 2016 г., в 12:33, Martin Polednik <mpolednik at redhat.com> написал(а):
> 
> On 03/06/16 14:05 +0300, Дмитрий Глушенок wrote:
>> Thank you Martin!
>> 
>> Actually I tried the workaround hook, provided in [2], but then VDSM (oVirt 3.6.6) tries to interpret hostdev in XML as PCI device, which leads to:
>> 
>> ::The vm start process failed
>> Traceback (most recent call last):
>> File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm
>>   self._run()
>> File "/usr/share/vdsm/virt/vm.py", line 1949, in _run
>>   self._domDependentInit()
>> File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit
>>   self._getUnderlyingVmDevicesInfo()
>> File "/usr/share/vdsm/virt/vm.py", line 1738, in _getUnderlyingVmDevicesInfo
>>   self._getUnderlyingHostDeviceInfo()
>> File "/usr/share/vdsm/virt/vm.py", line 4277, in _getUnderlyingHostDeviceInfo
>>   **self._getUnderlyingDeviceAddress(source))
>> TypeError: pci_address_to_name() got an unexpected keyword argument 'target'
>> 
>> XML part was:
>> <hostdev managed="no" mode="subsystem" rawio="yes" type="scsi">
>> 	<source>
>> 		<adapter name="scsi_host2"/>
>> 		<address bus="0" target="1" unit="0"/>
>> 	</source>
>> </hostdev>
>> <hostdev managed="no" mode="subsystem" rawio="yes" type="scsi">
>> 	<source>
>> 		<adapter name="scsi_host2"/>
>> 		<address bus="0" target="2" unit="0"/>
>> 	</source>
>> </hostdev>
>> 
>> 
>> As of creating custom partition - by default machine.slice has "a *:* rwm" in devices.list. But for every new VM libvirt removes *:* mask and fills the list with actually needed devices (as I understand the process). For example:
>> 
>> c 136:* rw
>> c 1:3 rw
>> c 1:7 rw
>> c 1:5 rw
>> c 1:8 rw
>> c 1:9 rw
>> c 5:2 rw
>> c 10:232 rw
>> c 253:0 rw
>> c 10:228 rw
>> c 10:196 rw
>> 
>> What I'm looking for is a way to tell libvirt about my additional devices without breaking oVirt.
> 
> The solution would be creating your own partition and somehow (e.g.
> VDSM hook) appending
> 
> <resource>
>   <partition>/machine/custom</partition>
> </resource>
> 
> to the libvirt's <domain> element. I'm not sure how feasible creating
> your own partition is though. I've tried the process as follows:
> 
> $ cat ~/create-partition.sh
> # sh ~/create-partition.sh
> for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls perf_event
> do
>   mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition
> done
> 
> for i in cpuset.cpus cpuset.mems
> do
>   cat /sys/fs/cgroup/cpuset/machine.slice/$i > /sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i
> done
> 
> (creates /machine/custom partition).
> Now, we can create vdsm before_vm_start hook that will set given partition
> for the VM:
> 
> $ pwd /usr/libexec/vdsm/hooks/before_vm_start
> $ cat 10_cgroups
> 
> #!/usr/bin/python
> 
> import hooking
> 
> 
> def custom_partition(domxml):
>   resource = domxml.createElement('resource')
>   partition = domxml.createElement('partition')
>   partition_text = domxml.createTextNode('/machine/custom')
>   partition.appendChild(partition_text)
>   resource.appendChild(partition)
> 
>   return resource
> 
> domxml = hooking.read_domxml()
> domain = domxml.getElementsByTagName('domain')[0]
> domain.appendChild(custom_partition(domxml))
> hooking.write_domxml(domxml)
> 
>> --
>> Dmitry Glushenok
>> Jet Infosystems
>> http://www.jet.msk.su
>> +7-495-411-7601 (ext. 1237)
>> 
>>> 3 июня 2016 г., в 12:24, Martin Polednik <mpolednik at redhat.com> написал(а):
>>> 
>>> On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote:
>>>> Hello!
>>>> 
>>>> Is it possible to tell libvirt to add specific devices to qemu cgroup? By somehow enumerating the devices in XML using a hook for example.
>>>> I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it doesn't work until I remove "devices" from cgroup_controllers in qemu.conf.
>>> 
>>> One way to achieve this is creating a hook to generate the scsi device
>>> XML instead of modifying qemu cmdline directly. Libvirt assumes
>>> ownership of all devices created in the XML and therefore adds them to
>>> the machine cgroup.
>>> 
>>> Example of the XML taken from [1]:
>>> <devices>
>>>  <hostdev mode='subsystem' type='scsi' sgio='filtered' rawio='yes'>
>>>      <source>
>>>          <adapter name='scsi_host0'/>
>>>          <address bus='0' target='0' unit='0'/>
>>>      </source>
>>>      <readonly/>
>>>      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
>>>  </hostdev>
>>> </devices>
>>> 
>>> There is slight issue with this approach outlined in [2].
>>> 
>>> If you want to keep the qemu approach, I think creating a custom
>>> partition and moving devices there would be the cleanest approach. In
>>> this case, [3] could help but I'm not entirely sure if that would
>>> solve the issue.
>>> 
>>> [1] https://libvirt.org/formatdomain.html
>>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485
>>> [3] https://libvirt.org/cgroups.html
>>>> --
>>>> Dmitry Glushenok
>>>> Jet Infosystems
>>>> http://www.jet.msk.su
>>>> +7-495-411-7601 (ext. 1237)
>>>> 
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160606/6e666c39/attachment-0001.html>

More information about the Users mailing list