11:48 a.m.
Hello!
Is it possible to tell libvirt to add specific devices to qemu cgroup? By somehow
enumerating the devices in XML using a hook for example.
I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it
doesn't work until I remove "devices" from cgroup_controllers in qemu.conf.
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su
+7-495-411-7601 (ext. 1237)
12:24 p.m.
On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote:
Hello!
Is it possible to tell libvirt to add specific devices to qemu cgroup? By somehow
enumerating the devices in XML using a hook for example.
I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it
doesn't work until I remove "devices" from cgroup_controllers in qemu.conf.
One way to achieve this is creating a hook to generate the scsi device
XML instead of modifying qemu cmdline directly. Libvirt assumes
ownership of all devices created in the XML and therefore adds them to
the machine cgroup.
Example of the XML taken from [1]:
<devices>
<hostdev mode='subsystem' type='scsi' sgio='filtered'
rawio='yes'>
<source>
<adapter name='scsi_host0'/>
<address bus='0' target='0' unit='0'/>
</source>
<readonly/>
<address type='drive' controller='0' bus='0'
target='0' unit='0'/>
</hostdev>
</devices>
There is slight issue with this approach outlined in [2].
If you want to keep the qemu approach, I think creating a custom
partition and moving devices there would be the cleanest approach. In
this case, [3] could help but I'm not entirely sure if that would
solve the issue.
[1] https://libvirt.org/formatdomain.html
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485
[3] https://libvirt.org/cgroups.html
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su
+7-495-411-7601 (ext. 1237)
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2:05 p.m.
Thank you Martin!
Actually I tried the workaround hook, provided in [2], but then VDSM (oVirt 3.6.6) tries
to interpret hostdev in XML as PCI device, which leads to:
::The vm start process failed
Traceback (most recent call last):
File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm
self._run()
File "/usr/share/vdsm/virt/vm.py", line 1949, in _run
self._domDependentInit()
File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit
self._getUnderlyingVmDevicesInfo()
File "/usr/share/vdsm/virt/vm.py", line 1738, in _getUnderlyingVmDevicesInfo
self._getUnderlyingHostDeviceInfo()
File "/usr/share/vdsm/virt/vm.py", line 4277, in _getUnderlyingHostDeviceInfo
**self._getUnderlyingDeviceAddress(source))
TypeError: pci_address_to_name() got an unexpected keyword argument 'target'
XML part was:
<hostdev managed="no" mode="subsystem" rawio="yes"
type="scsi">
<source>
<adapter name="scsi_host2"/>
<address bus="0" target="1" unit="0"/>
</source>
</hostdev>
<hostdev managed="no" mode="subsystem" rawio="yes"
type="scsi">
<source>
<adapter name="scsi_host2"/>
<address bus="0" target="2" unit="0"/>
</source>
</hostdev>
As of creating custom partition - by default machine.slice has "a *:* rwm" in
devices.list. But for every new VM libvirt removes *:* mask and fills the list with
actually needed devices (as I understand the process). For example:
c 136:* rw
c 1:3 rw
c 1:7 rw
c 1:5 rw
c 1:8 rw
c 1:9 rw
c 5:2 rw
c 10:232 rw
c 253:0 rw
c 10:228 rw
c 10:196 rw
What I'm looking for is a way to tell libvirt about my additional devices without
breaking oVirt.
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su
+7-495-411-7601 (ext. 1237)
3 июня 2016 г., в 12:24, Martin Polednik <mpolednik(a)redhat.com>
написал(а):
On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote:
> Hello!
>
> Is it possible to tell libvirt to add specific devices to qemu cgroup? By somehow
enumerating the devices in XML using a hook for example.
> I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it
doesn't work until I remove "devices" from cgroup_controllers in qemu.conf.
One way to achieve this is creating a hook to generate the scsi device
XML instead of modifying qemu cmdline directly. Libvirt assumes
ownership of all devices created in the XML and therefore adds them to
the machine cgroup.
Example of the XML taken from [1]:
<devices>
<hostdev mode='subsystem' type='scsi' sgio='filtered'
rawio='yes'>
<source>
<adapter name='scsi_host0'/>
<address bus='0' target='0' unit='0'/>
</source>
<readonly/>
<address type='drive' controller='0' bus='0'
target='0' unit='0'/>
</hostdev>
</devices>
There is slight issue with this approach outlined in [2].
If you want to keep the qemu approach, I think creating a custom
partition and moving devices there would be the cleanest approach. In
this case, [3] could help but I'm not entirely sure if that would
solve the issue.
[1] https://libvirt.org/formatdomain.html
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485
[3] https://libvirt.org/cgroups.html
> --
> Dmitry Glushenok
> Jet Infosystems
> http://www.jet.msk.su
> +7-495-411-7601 (ext. 1237)
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
12:33 p.m.
On 03/06/16 14:05 +0300, Дмитрий Глушенок wrote:
Thank you Martin!
Actually I tried the workaround hook, provided in [2], but then VDSM (oVirt 3.6.6) tries
to interpret hostdev in XML as PCI device, which leads to:
::The vm start process failed
Traceback (most recent call last):
File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm
self._run()
File "/usr/share/vdsm/virt/vm.py", line 1949, in _run
self._domDependentInit()
File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit
self._getUnderlyingVmDevicesInfo()
File "/usr/share/vdsm/virt/vm.py", line 1738, in _getUnderlyingVmDevicesInfo
self._getUnderlyingHostDeviceInfo()
File "/usr/share/vdsm/virt/vm.py", line 4277, in _getUnderlyingHostDeviceInfo
**self._getUnderlyingDeviceAddress(source))
TypeError: pci_address_to_name() got an unexpected keyword argument 'target'
XML part was:
<hostdev managed="no" mode="subsystem" rawio="yes"
type="scsi">
<source>
<adapter name="scsi_host2"/>
<address bus="0" target="1" unit="0"/>
</source>
</hostdev>
<hostdev managed="no" mode="subsystem" rawio="yes"
type="scsi">
<source>
<adapter name="scsi_host2"/>
<address bus="0" target="2" unit="0"/>
</source>
</hostdev>
As of creating custom partition - by default machine.slice has "a *:* rwm" in
devices.list. But for every new VM libvirt removes *:* mask and fills the list with
actually needed devices (as I understand the process). For example:
c 136:* rw
c 1:3 rw
c 1:7 rw
c 1:5 rw
c 1:8 rw
c 1:9 rw
c 5:2 rw
c 10:232 rw
c 253:0 rw
c 10:228 rw
c 10:196 rw
What I'm looking for is a way to tell libvirt about my additional devices without
breaking oVirt.
The solution would be creating your own partition and somehow (e.g.
VDSM hook) appending
<resource>
<partition>/machine/custom</partition>
</resource>
to the libvirt's <domain> element. I'm not sure how feasible creating
your own partition is though. I've tried the process as follows:
$ cat ~/create-partition.sh
# sh ~/create-partition.sh
for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls perf_event
do
mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition
done
for i in cpuset.cpus cpuset.mems
do
cat /sys/fs/cgroup/cpuset/machine.slice/$i >
/sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i
done
(creates /machine/custom partition).
Now, we can create vdsm before_vm_start hook that will set given partition
for the VM:
$ pwd /usr/libexec/vdsm/hooks/before_vm_start
$ cat 10_cgroups
#!/usr/bin/python
import hooking
def custom_partition(domxml):
resource = domxml.createElement('resource')
partition = domxml.createElement('partition')
partition_text = domxml.createTextNode('/machine/custom')
partition.appendChild(partition_text)
resource.appendChild(partition)
return resource
domxml = hooking.read_domxml()
domain = domxml.getElementsByTagName('domain')[0]
domain.appendChild(custom_partition(domxml))
hooking.write_domxml(domxml)
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su
+7-495-411-7601 (ext. 1237)
> 3 июня 2016 г., в 12:24, Martin Polednik <mpolednik(a)redhat.com> написал(а):
>
> On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote:
>> Hello!
>>
>> Is it possible to tell libvirt to add specific devices to qemu cgroup? By somehow
enumerating the devices in XML using a hook for example.
>> I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it
doesn't work until I remove "devices" from cgroup_controllers in qemu.conf.
>
> One way to achieve this is creating a hook to generate the scsi device
> XML instead of modifying qemu cmdline directly. Libvirt assumes
> ownership of all devices created in the XML and therefore adds them to
> the machine cgroup.
>
> Example of the XML taken from [1]:
> <devices>
> <hostdev mode='subsystem' type='scsi' sgio='filtered'
rawio='yes'>
> <source>
> <adapter name='scsi_host0'/>
> <address bus='0' target='0' unit='0'/>
> </source>
> <readonly/>
> <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
> </hostdev>
> </devices>
>
> There is slight issue with this approach outlined in [2].
>
> If you want to keep the qemu approach, I think creating a custom
> partition and moving devices there would be the cleanest approach. In
> this case, [3] could help but I'm not entirely sure if that would
> solve the issue.
>
> [1] https://libvirt.org/formatdomain.html
> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485
> [3] https://libvirt.org/cgroups.html
>> --
>> Dmitry Glushenok
>> Jet Infosystems
>> http://www.jet.msk.su
>> +7-495-411-7601 (ext. 1237)
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
1:26 p.m.
--Apple-Mail=_64C09473-EDEA-4FA9-8643-DC7E66D3674C
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
Hello Martin,
Thank you for your time. It is clear how to create partition and assign =
it to a VM. But libvirt manipulates with contents of devices.list and I =
didn't find a way to put my devices into devices.list and stop libvirt =
from removing them.
Anyway, it looks like that the best solution will be using oVirt 4.0 =
where scsi pass-through have been fixed.
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. =
1237)
6 =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 12:33, Martin
Polednik =
<mpolednik(a)redhat.com>
=D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0):=
=20
On 03/06/16 14:05 +0300, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =
=D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=BD=D0=BE=D0=BA wrote:
> Thank you Martin!
>=20
> Actually I tried the workaround hook, provided in [2], but then VDSM =
(oVirt
3.6.6) tries to interpret hostdev in XML as PCI device, which =
leads to:
>=20
> ::The vm start process failed
> Traceback (most recent call last):
> File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm
> self._run()
> File "/usr/share/vdsm/virt/vm.py", line 1949, in _run
> self._domDependentInit()
> File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit
> self._getUnderlyingVmDevicesInfo()
> File "/usr/share/vdsm/virt/vm.py", line 1738, in =
_getUnderlyingVmDevicesInfo
> self._getUnderlyingHostDeviceInfo()
> File "/usr/share/vdsm/virt/vm.py", line 4277, in =
_getUnderlyingHostDeviceInfo
> **self._getUnderlyingDeviceAddress(source))
> TypeError: pci_address_to_name() got an unexpected keyword argument =
'target'
>=20
> XML part was:
> <hostdev managed=3D"no" mode=3D"subsystem"
rawio=3D"yes" type=3D"scsi">=
> <source>
> <adapter name=3D"scsi_host2"/>
> <address bus=3D"0" target=3D"1" unit=3D"0"/>
> </source>
> </hostdev>
> <hostdev managed=3D"no" mode=3D"subsystem"
rawio=3D"yes" type=3D"scsi">=
> <source>
> <adapter name=3D"scsi_host2"/>
> <address bus=3D"0" target=3D"2" unit=3D"0"/>
> </source>
> </hostdev>
>=20
>=20
> As of creating custom partition - by default machine.slice has "a *:* =
rwm" in devices.list. But for every new VM libvirt removes *:* mask and =
fills the list with actually needed devices (as I understand the =
process). For example:
>=20
> c 136:* rw
> c 1:3 rw
> c 1:7 rw
> c 1:5 rw
> c 1:8 rw
> c 1:9 rw
> c 5:2 rw
> c 10:232 rw
> c 253:0 rw
> c 10:228 rw
> c 10:196 rw
>=20
> What I'm looking for is a way to tell libvirt about my additional =
devices
without breaking oVirt.
=20
The solution would be creating your own partition and somehow (e.g.
VDSM hook) appending
=20
<resource>
<partition>/machine/custom</partition>
</resource>
=20
to the libvirt's <domain> element. I'm not sure how feasible creating
your own partition is though. I've tried the process as follows:
=20
$ cat ~/create-partition.sh
# sh ~/create-partition.sh
for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls =
perf_event
do
mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition
done
=20
for i in cpuset.cpus cpuset.mems
do
cat /sys/fs/cgroup/cpuset/machine.slice/$i > =
/sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i
done
=20
(creates /machine/custom partition).
Now, we can create vdsm before_vm_start hook that will set given =
partition
for the VM:
=20
$ pwd /usr/libexec/vdsm/hooks/before_vm_start
$ cat 10_cgroups
=20
#!/usr/bin/python
=20
import hooking
=20
=20
def custom_partition(domxml):
resource =3D domxml.createElement('resource')
partition =3D domxml.createElement('partition')
partition_text =3D domxml.createTextNode('/machine/custom')
partition.appendChild(partition_text)
resource.appendChild(partition)
=20
return resource
=20
domxml =3D hooking.read_domxml()
domain =3D domxml.getElementsByTagName('domain')[0]
domain.appendChild(custom_partition(domxml))
hooking.write_domxml(domxml)
=20
> --
> Dmitry Glushenok
> Jet Infosystems
> http://www.jet.msk.su
> +7-495-411-7601 (ext. 1237)
>=20
>> 3 =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 12:24, Martin =
Polednik
<mpolednik(a)redhat.com> =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB=
(=D0=B0):
>>=20
>> On 03/06/16 11:48 +0300, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =
=D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=BD=D0=BE=D0=BA wrote:
>>> Hello!
>>>=20
>>> Is it possible to tell libvirt to add specific devices to qemu =
cgroup? By somehow enumerating the devices in XML using a hook for =
example.
>>> I'm passing scsi-generic disks (/dev/sgX) to VM using
qemucmdline =
hook and it doesn't work until I remove "devices" from
=
cgroup_controllers in qemu.conf.
>>=20
>> One way to achieve this is creating a hook to generate the scsi =
device
>> XML instead of modifying qemu cmdline directly. Libvirt
assumes
>> ownership of all devices created in the XML and therefore adds them =
to
>> the machine cgroup.
>>=20
>> Example of the XML taken from [1]:
>> <devices>
>> <hostdev mode=3D'subsystem' type=3D'scsi'
sgio=3D'filtered' =
rawio=3D'yes'>
>> <source>
>> <adapter name=3D'scsi_host0'/>
>> <address bus=3D'0' target=3D'0'
unit=3D'0'/>
>> </source>
>> <readonly/>
>> <address type=3D'drive' controller=3D'0'
bus=3D'0' target=3D'0' =
unit=3D'0'/>
>> </hostdev>
>> </devices>
>>=20
>> There is slight issue with this approach outlined in [2].
>>=20
>> If you want to keep the qemu approach, I think creating a custom
>> partition and moving devices there would be the cleanest approach. =
In
>> this case, [3] could help but I'm not entirely sure if
that would
>> solve the issue.
>>=20
>> [1] https://libvirt.org/formatdomain.html
>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=3D1325485
>> [3] https://libvirt.org/cgroups.html
>>> --
>>> Dmitry Glushenok
>>> Jet Infosystems
>>> http://www.jet.msk.su
>>> +7-495-411-7601 (ext. 1237)
>>>=20
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
--Apple-Mail=_64C09473-EDEA-4FA9-8643-DC7E66D3674C
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D"">Hello Martin,</div><div
class=3D""><br =
class=3D""></div><div class=3D"">Thank you for your
time. It is clear =
how to create partition and assign it to a VM. But libvirt manipulates =
with contents of devices.list and I didn't find a way to put my devices =
into devices.list and stop libvirt from removing them.</div><div =
class=3D""><br class=3D""></div><div
class=3D"">Anyway, it looks like =
that the best solution will be using oVirt 4.0 where scsi pass-through =
have been fixed.</div><br class=3D""><div
class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D"">--</div><div
class=3D""><div class=3D"" =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;"><div class=3D"" =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;"><div class=3D"" =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;"><div class=3D"">Dmitry =
Glushenok</div><div class=3D"">Jet Infosystems</div><div
class=3D""><a =
href=3D"http://www.jet.msk.su" =
class=3D"">http://www.jet.msk.su</a></div></div></div></div>+7-495-411-760=
1 (ext. 1237)</div></div></div></div></div></div>
</div>
<br class=3D""><div><blockquote type=3D"cite"
class=3D""><div class=3D"">6=
=D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 12:33, Martin Polednik =
<<a href=3D"mailto:mpolednik@redhat.com" =
class=3D"">mpolednik(a)redhat.com</a>&gt;
=D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=
=D0=BB(=D0=B0):</div><br class=3D"Apple-interchange-newline"><div
=
class=3D""><span style=3D"font-family: Menlo-Regular; font-size:
13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;" class=3D"">On 03/06/16 14:05 +0300,
=
=D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=
=BD=D0=BE=D0=BA wrote:</span><br style=3D"font-family: Menlo-Regular; =
font-size: 13px; font-style: normal; font-variant: normal; font-weight: =
normal; letter-spacing: normal; line-height: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><blockquote type=3D"cite"
style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D"">Thank you Martin!<br
=
class=3D""><br class=3D"">Actually I tried the workaround
hook, provided =
in [2], but then VDSM (oVirt 3.6.6) tries to interpret hostdev in XML as =
PCI device, which leads to:<br class=3D""><br
class=3D"">::The vm start =
process failed<br class=3D"">Traceback (most recent call last):<br =
class=3D"">File "/usr/share/vdsm/virt/vm.py", line 703, in =
_startUnderlyingVm<br class=3D""> self._run()<br =
class=3D"">File "/usr/share/vdsm/virt/vm.py", line 1949, in
_run<br =
class=3D""> self._domDependentInit()<br
class=3D"">File =
"/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit<br =
class=3D""> self._getUnderlyingVmDevicesInfo()<br =
class=3D"">File "/usr/share/vdsm/virt/vm.py", line 1738, in =
_getUnderlyingVmDevicesInfo<br =
class=3D""> self._getUnderlyingHostDeviceInfo()<br =
class=3D"">File "/usr/share/vdsm/virt/vm.py", line 4277, in =
_getUnderlyingHostDeviceInfo<br =
class=3D""> **self._getUnderlyingDeviceAddress(source))<br
=
class=3D"">TypeError: pci_address_to_name() got an unexpected keyword =
argument 'target'<br class=3D""><br class=3D"">XML
part was:<br =
class=3D""><hostdev managed=3D"no"
mode=3D"subsystem" rawio=3D"yes" =
type=3D"scsi"><br class=3D""><span
class=3D"Apple-tab-span" =
style=3D"white-space: pre;"> </span><source><br
class=3D""><span =
class=3D"Apple-tab-span" style=3D"white-space:
pre;"> </span><span =
class=3D"Apple-tab-span" style=3D"white-space: pre;"> =
</span><adapter name=3D"scsi_host2"/><br
class=3D""><span =
class=3D"Apple-tab-span" style=3D"white-space:
pre;"> </span><span =
class=3D"Apple-tab-span" style=3D"white-space: pre;"> =
</span><address bus=3D"0" target=3D"1"
unit=3D"0"/><br =
class=3D""><span class=3D"Apple-tab-span"
style=3D"white-space: pre;"> =
</span></source><br
class=3D""></hostdev><br =
class=3D""><hostdev managed=3D"no"
mode=3D"subsystem" rawio=3D"yes" =
type=3D"scsi"><br class=3D""><span
class=3D"Apple-tab-span" =
style=3D"white-space: pre;"> </span><source><br
class=3D""><span =
class=3D"Apple-tab-span" style=3D"white-space:
pre;"> </span><span =
class=3D"Apple-tab-span" style=3D"white-space: pre;"> =
</span><adapter name=3D"scsi_host2"/><br
class=3D""><span =
class=3D"Apple-tab-span" style=3D"white-space:
pre;"> </span><span =
class=3D"Apple-tab-span" style=3D"white-space: pre;"> =
</span><address bus=3D"0" target=3D"2"
unit=3D"0"/><br =
class=3D""><span class=3D"Apple-tab-span"
style=3D"white-space: pre;"> =
</span></source><br
class=3D""></hostdev><br class=3D""><br =
class=3D""><br class=3D"">As of creating custom partition -
by default =
machine.slice has "a *:* rwm" in devices.list. But for every new VM =
libvirt removes *:* mask and fills the list with actually needed devices =
(as I understand the process). For example:<br class=3D""><br
class=3D"">c=
136:* rw<br class=3D"">c 1:3 rw<br class=3D"">c 1:7
rw<br class=3D"">c =
1:5 rw<br class=3D"">c 1:8 rw<br class=3D"">c 1:9 rw<br
class=3D"">c 5:2 =
rw<br class=3D"">c 10:232 rw<br class=3D"">c 253:0
rw<br class=3D"">c =
10:228 rw<br class=3D"">c 10:196 rw<br class=3D""><br
class=3D"">What =
I'm looking for is a way to tell libvirt about my additional devices =
without breaking oVirt.<br class=3D""></blockquote><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">The solution would be creating
=
your own partition and somehow (e.g.</span><br style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><span =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">VDSM hook)
appending</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;"
class=3D""><resource></span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" =
class=3D""> <partition>/machine/custom</partition><=
/span><br style=3D"font-family: Menlo-Regular; font-size: 13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><span style=3D"font-family: Menlo-Regular; font-size:
13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;"
class=3D""></resource></span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">to the libvirt's
<domain> =
element. I'm not sure how feasible creating</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">your own partition is though. =
I've tried the process as follows:</span><br style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><br
style=3D"font-family:=
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><span =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">$ cat =
~/create-partition.sh</span><br style=3D"font-family: Menlo-Regular; =
font-size: 13px; font-style: normal; font-variant: normal; font-weight: =
normal; letter-spacing: normal; line-height: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-family: Menlo-Regular;
font-size: =
13px; font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;" class=3D""># sh =
~/create-partition.sh</span><br style=3D"font-family: Menlo-Regular; =
font-size: 13px; font-style: normal; font-variant: normal; font-weight: =
normal; letter-spacing: normal; line-height: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-family: Menlo-Regular;
font-size: =
13px; font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;" class=3D"">for i in blkio cpu,cpuacct
=
cpuset devices freezer memory net_cls perf_event</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">do</span><br
style=3D"font-family:=
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><span =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D""> mkdir =
/sys/fs/cgroup/$i/machine.slice/custom.partition</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">done</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">for i in cpuset.cpus =
cpuset.mems</span><br style=3D"font-family: Menlo-Regular; font-size: =
13px; font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><span style=3D"font-family: Menlo-Regular; font-size:
13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;" class=3D"">do</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D""> cat =
/sys/fs/cgroup/cpuset/machine.slice/$i > =
/sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">done</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">(creates /machine/custom =
partition).</span><br style=3D"font-family: Menlo-Regular; font-size: =
13px; font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><span style=3D"font-family: Menlo-Regular; font-size:
13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;" class=3D"">Now, we can create vdsm =
before_vm_start hook that will set given partition</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">for the VM:</span><br
=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">$ pwd =
/usr/libexec/vdsm/hooks/before_vm_start</span><br style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><span =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">$ cat
10_cgroups</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;"
class=3D"">#!/usr/bin/python</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">import
hooking</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D"">def =
custom_partition(domxml):</span><br style=3D"font-family: Menlo-Regular; =
font-size: 13px; font-style: normal; font-variant: normal; font-weight: =
normal; letter-spacing: normal; line-height: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-family: Menlo-Regular;
font-size: =
13px; font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;"
class=3D""> resource =3D =
domxml.createElement('resource')</span><br style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><span =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D""> partition
=3D =
domxml.createElement('partition')</span><br style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><span =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;"
class=3D""> partition_text =3D =
domxml.createTextNode('/machine/custom')</span><br
style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><span =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" =
class=3D""> partition.appendChild(partition_text)</span><br
=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" =
class=3D""> resource.appendChild(partition)</span><br
=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" class=3D""> return =
resource</span><br style=3D"font-family: Menlo-Regular; font-size: 13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><br style=3D"font-family: Menlo-Regular; font-size: 13px;
=
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" =
class=3D""><span style=3D"font-family: Menlo-Regular; font-size:
13px; =
font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;" class=3D"">domxml =3D =
hooking.read_domxml()</span><br style=3D"font-family: Menlo-Regular; =
font-size: 13px; font-style: normal; font-variant: normal; font-weight: =
normal; letter-spacing: normal; line-height: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: =
0px;" class=3D""><span style=3D"font-family: Menlo-Regular;
font-size: =
13px; font-style: normal; font-variant: normal; font-weight: normal; =
letter-spacing: normal; line-height: normal; orphans: auto; text-align: =
start; text-indent: 0px; text-transform: none; white-space: normal; =
widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: =
none; display: inline !important;" class=3D"">domain =3D =
domxml.getElementsByTagName('domain')[0]</span><br
style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><span =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" =
class=3D"">domain.appendChild(custom_partition(domxml))</span><br =
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D""><span=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; =
display: inline !important;" =
class=3D"">hooking.write_domxml(domxml)</span><br
style=3D"font-family: =
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><br
style=3D"font-family:=
Menlo-Regular; font-size: 13px; font-style: normal; font-variant: =
normal; font-weight: normal; letter-spacing: normal; line-height: =
normal; orphans: auto; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; widows: auto; word-spacing: =
0px; -webkit-text-stroke-width: 0px;" class=3D""><blockquote
type=3D"cite"=
style=3D"font-family: Menlo-Regular; font-size: 13px; font-style: =
normal; font-variant: normal; font-weight: normal; letter-spacing: =
normal; line-height: normal; orphans: auto; text-align: start; =
text-indent: 0px; text-transform: none; white-space: normal; widows: =
auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"
class=3D"">--<br=
class=3D"">Dmitry Glushenok<br class=3D"">Jet
Infosystems<br =
class=3D""><a href=3D"http://www.jet.msk.su" =
class=3D"">http://www.jet.msk.su</a><br
class=3D"">+7-495-411-7601 (ext. =
1237)<br class=3D""><br class=3D""><blockquote
type=3D"cite" class=3D"">3 =
=D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 12:24, Martin Polednik =
&lt;mpolednik(a)redhat.com&gt; =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=
=B0):<br class=3D""><br class=3D"">On 03/06/16 11:48
+0300, =
=D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=
=BD=D0=BE=D0=BA wrote:<br class=3D""><blockquote
type=3D"cite" =
class=3D"">Hello!<br class=3D""><br
class=3D"">Is it possible to tell =
libvirt to add specific devices to qemu cgroup? By somehow enumerating =
the devices in XML using a hook for example.<br class=3D"">I'm passing
=
scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it =
doesn't work until I remove "devices" from cgroup_controllers in =
qemu.conf.<br class=3D""></blockquote><br
class=3D"">One way to achieve =
this is creating a hook to generate the scsi device<br class=3D"">XML =
instead of modifying qemu cmdline directly. Libvirt assumes<br =
class=3D"">ownership of all devices created in the XML and therefore =
adds them to<br class=3D"">the machine cgroup.<br
class=3D""><br =
class=3D"">Example of the XML taken from [1]:<br =
class=3D""><devices><br
class=3D""> <hostdev =
mode=3D'subsystem' type=3D'scsi' sgio=3D'filtered'
rawio=3D'yes'><br =
class=3D""> <source><br
=
class=3D""> <adapt=
er name=3D'scsi_host0'/><br =
class=3D""> <addre=
ss bus=3D'0' target=3D'0' unit=3D'0'/><br =
class=3D""> </source><br
=
class=3D""> <readonly/><br
=
class=3D""> <address
type=3D'drive' =
controller=3D'0' bus=3D'0' target=3D'0'
unit=3D'0'/><br =
class=3D""> </hostdev><br
class=3D""></devices><br =
class=3D""><br class=3D"">There is slight issue with this
approach =
outlined in [2].<br class=3D""><br class=3D"">If you want
to keep the =
qemu approach, I think creating a custom<br class=3D"">partition and =
moving devices there would be the cleanest approach. In<br
class=3D"">this=
case, [3] could help but I'm not entirely sure if that would<br =
class=3D"">solve the issue.<br class=3D""><br
class=3D"">[1] =
https://libvirt.org/formatdomain.html<br class=3D"">[2] =
https://bugzilla.redhat.com/show_bug.cgi?id=3D1325485<br class=3D"">[3] =
https://libvirt.org/cgroups.html<br class=3D""><blockquote
type=3D"cite" =
class=3D"">--<br class=3D"">Dmitry Glushenok<br
class=3D"">Jet =
Infosystems<br class=3D"">http://www.jet.msk.su<br =
class=3D"">+7-495-411-7601 (ext. 1237)<br class=3D""><br
=
class=3D"">_______________________________________________<br =
class=3D"">Users mailing list<br
class=3D"">Users(a)ovirt.org<br =
class=3D"">http://lists.ovirt.org/mailman/listinfo/users<...
ckquote></blockquote></div></blockquote></div><br =
class=3D""></body></html>=
--Apple-Mail=_64C09473-EDEA-4FA9-8643-DC7E66D3674C--
1:35 p.m.
On 06/06/16 13:26 +0300, Дмитрий Глушенок wrote:
Hello Martin,
Thank you for your time. It is clear how to create partition and assign it to a VM. But
libvirt manipulates with contents of devices.list and I didn't find a way to put my
devices into devices.list and stop libvirt from removing them.
That is, by the way, pretty interesting. I have tried it right now and
when using custom partition, libvirt doesn't touch my devices.list.
It might make sense to file a libvirt bug.
Anyway, it looks like that the best solution will be using oVirt 4.0
where scsi pass-through have been fixed.
Hoping so!
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. 1237)
> 6 июня 2016 г., в 12:33, Martin Polednik <mpolednik(a)redhat.com> написал(а):
>
> On 03/06/16 14:05 +0300, Дмитрий Глушенок wrote:
>> Thank you Martin!
>>
>> Actually I tried the workaround hook, provided in [2], but then VDSM (oVirt
3.6.6) tries to interpret hostdev in XML as PCI device, which leads to:
>>
>> ::The vm start process failed
>> Traceback (most recent call last):
>> File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm
>> self._run()
>> File "/usr/share/vdsm/virt/vm.py", line 1949, in _run
>> self._domDependentInit()
>> File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit
>> self._getUnderlyingVmDevicesInfo()
>> File "/usr/share/vdsm/virt/vm.py", line 1738, in
_getUnderlyingVmDevicesInfo
>> self._getUnderlyingHostDeviceInfo()
>> File "/usr/share/vdsm/virt/vm.py", line 4277, in
_getUnderlyingHostDeviceInfo
>> **self._getUnderlyingDeviceAddress(source))
>> TypeError: pci_address_to_name() got an unexpected keyword argument
'target'
>>
>> XML part was:
>> <hostdev managed="no" mode="subsystem"
rawio="yes" type="scsi">
>> <source>
>> <adapter name="scsi_host2"/>
>> <address bus="0" target="1" unit="0"/>
>> </source>
>> </hostdev>
>> <hostdev managed="no" mode="subsystem"
rawio="yes" type="scsi">
>> <source>
>> <adapter name="scsi_host2"/>
>> <address bus="0" target="2" unit="0"/>
>> </source>
>> </hostdev>
>>
>>
>> As of creating custom partition - by default machine.slice has "a *:*
rwm" in devices.list. But for every new VM libvirt removes *:* mask and fills the
list with actually needed devices (as I understand the process). For example:
>>
>> c 136:* rw
>> c 1:3 rw
>> c 1:7 rw
>> c 1:5 rw
>> c 1:8 rw
>> c 1:9 rw
>> c 5:2 rw
>> c 10:232 rw
>> c 253:0 rw
>> c 10:228 rw
>> c 10:196 rw
>>
>> What I'm looking for is a way to tell libvirt about my additional devices
without breaking oVirt.
>
> The solution would be creating your own partition and somehow (e.g.
> VDSM hook) appending
>
> <resource>
> <partition>/machine/custom</partition>
> </resource>
>
> to the libvirt's <domain> element. I'm not sure how feasible creating
> your own partition is though. I've tried the process as follows:
>
> $ cat ~/create-partition.sh
> # sh ~/create-partition.sh
> for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls perf_event
> do
> mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition
> done
>
> for i in cpuset.cpus cpuset.mems
> do
> cat /sys/fs/cgroup/cpuset/machine.slice/$i >
/sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i
> done
>
> (creates /machine/custom partition).
> Now, we can create vdsm before_vm_start hook that will set given partition
> for the VM:
>
> $ pwd /usr/libexec/vdsm/hooks/before_vm_start
> $ cat 10_cgroups
>
> #!/usr/bin/python
>
> import hooking
>
>
> def custom_partition(domxml):
> resource = domxml.createElement('resource')
> partition = domxml.createElement('partition')
> partition_text = domxml.createTextNode('/machine/custom')
> partition.appendChild(partition_text)
> resource.appendChild(partition)
>
> return resource
>
> domxml = hooking.read_domxml()
> domain = domxml.getElementsByTagName('domain')[0]
> domain.appendChild(custom_partition(domxml))
> hooking.write_domxml(domxml)
>
>> --
>> Dmitry Glushenok
>> Jet Infosystems
>> http://www.jet.msk.su
>> +7-495-411-7601 (ext. 1237)
>>
>>> 3 июня 2016 г., в 12:24, Martin Polednik <mpolednik(a)redhat.com>
написал(а):
>>>
>>> On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote:
>>>> Hello!
>>>>
>>>> Is it possible to tell libvirt to add specific devices to qemu cgroup? By
somehow enumerating the devices in XML using a hook for example.
>>>> I'm passing scsi-generic disks (/dev/sgX) to VM using qemucmdline
hook and it doesn't work until I remove "devices" from cgroup_controllers in
qemu.conf.
>>>
>>> One way to achieve this is creating a hook to generate the scsi device
>>> XML instead of modifying qemu cmdline directly. Libvirt assumes
>>> ownership of all devices created in the XML and therefore adds them to
>>> the machine cgroup.
>>>
>>> Example of the XML taken from [1]:
>>> <devices>
>>> <hostdev mode='subsystem' type='scsi'
sgio='filtered' rawio='yes'>
>>> <source>
>>> <adapter name='scsi_host0'/>
>>> <address bus='0' target='0'
unit='0'/>
>>> </source>
>>> <readonly/>
>>> <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
>>> </hostdev>
>>> </devices>
>>>
>>> There is slight issue with this approach outlined in [2].
>>>
>>> If you want to keep the qemu approach, I think creating a custom
>>> partition and moving devices there would be the cleanest approach. In
>>> this case, [3] could help but I'm not entirely sure if that would
>>> solve the issue.
>>>
>>> [1] https://libvirt.org/formatdomain.html
>>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485
>>> [3] https://libvirt.org/cgroups.html
>>>> --
>>>> Dmitry Glushenok
>>>> Jet Infosystems
>>>> http://www.jet.msk.su
>>>> +7-495-411-7601 (ext. 1237)
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
4:15 p.m.
--Apple-Mail=_F5070364-047E-4F4A-BEE4-93C0242B9F73
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
charset=utf-8
So, libvirt behaves differently with custom partition? My assumption was =
that it will perform the same changes. I will have in mind in case oVirt =
4 will be released too late.
May be I wrote incorrectly - in oVirt 4.0 RC1 scsi pass-through already =
working.
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. =
1237)
6 =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 13:35, Martin
Polednik =
<mpolednik(a)redhat.com>
=D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=B0):=
=20
On 06/06/16 13:26 +0300, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =
=D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=BD=D0=BE=D0=BA wrote:
> Hello Martin,
>=20
> Thank you for your time. It is clear how to create partition and =
assign it to
a VM. But libvirt manipulates with contents of devices.list =
and I didn't find a way to put my devices into devices.list and stop =
libvirt from removing them.
=20
That is, by the way, pretty interesting. I have tried it right now and
when using custom partition, libvirt doesn't touch my devices.list.
=20
It might make sense to file a libvirt bug.
=20
> Anyway, it looks like that the best solution will be using oVirt 4.0 =
where
scsi pass-through have been fixed.
=20
Hoping so!
=20
> --
> Dmitry Glushenok
> Jet Infosystems
> http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. =
1237)
>=20
>> 6 =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 12:33, Martin =
Polednik
<mpolednik(a)redhat.com> =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB=
(=D0=B0):
>>=20
>> On 03/06/16 14:05 +0300, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =
=D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=BD=D0=BE=D0=BA wrote:
>>> Thank you Martin!
>>>=20
>>> Actually I tried the workaround hook, provided in [2], but then =
VDSM
(oVirt 3.6.6) tries to interpret hostdev in XML as PCI device, =
which leads to:
>>>=20
>>> ::The vm start process failed
>>> Traceback (most recent call last):
>>> File "/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm
>>> self._run()
>>> File "/usr/share/vdsm/virt/vm.py", line 1949, in _run
>>> self._domDependentInit()
>>> File "/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit
>>> self._getUnderlyingVmDevicesInfo()
>>> File "/usr/share/vdsm/virt/vm.py", line 1738, in =
_getUnderlyingVmDevicesInfo
>>> self._getUnderlyingHostDeviceInfo()
>>> File "/usr/share/vdsm/virt/vm.py", line 4277, in =
_getUnderlyingHostDeviceInfo
>>> **self._getUnderlyingDeviceAddress(source))
>>> TypeError: pci_address_to_name() got an unexpected keyword argument =
'target'
>>>=20
>>> XML part was:
>>> <hostdev managed=3D"no" mode=3D"subsystem"
rawio=3D"yes" =
type=3D"scsi">
>>> <source>
>>> <adapter name=3D"scsi_host2"/>
>>> <address bus=3D"0" target=3D"1"
unit=3D"0"/>
>>> </source>
>>> </hostdev>
>>> <hostdev managed=3D"no" mode=3D"subsystem"
rawio=3D"yes" =
type=3D"scsi">
>>> <source>
>>> <adapter name=3D"scsi_host2"/>
>>> <address bus=3D"0" target=3D"2"
unit=3D"0"/>
>>> </source>
>>> </hostdev>
>>>=20
>>>=20
>>> As of creating custom partition - by default machine.slice has "a =
*:* rwm" in devices.list. But for every new VM libvirt removes *:* mask =
and fills the list with actually needed devices (as I understand the =
process). For example:
>>>=20
>>> c 136:* rw
>>> c 1:3 rw
>>> c 1:7 rw
>>> c 1:5 rw
>>> c 1:8 rw
>>> c 1:9 rw
>>> c 5:2 rw
>>> c 10:232 rw
>>> c 253:0 rw
>>> c 10:228 rw
>>> c 10:196 rw
>>>=20
>>> What I'm looking for is a way to tell libvirt about my additional =
devices without breaking oVirt.
>>=20
>> The solution would be creating your own partition and somehow (e.g.
>> VDSM hook) appending
>>=20
>> <resource>
>> <partition>/machine/custom</partition>
>> </resource>
>>=20
>> to the libvirt's <domain> element. I'm not sure how feasible =
creating
>> your own partition is though. I've tried the process as
follows:
>>=20
>> $ cat ~/create-partition.sh
>> # sh ~/create-partition.sh
>> for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls =
perf_event
>> do
>> mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition
>> done
>>=20
>> for i in cpuset.cpus cpuset.mems
>> do
>> cat /sys/fs/cgroup/cpuset/machine.slice/$i > =
/sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i
>> done
>>=20
>> (creates /machine/custom partition).
>> Now, we can create vdsm before_vm_start hook that will set given =
partition
>> for the VM:
>>=20
>> $ pwd /usr/libexec/vdsm/hooks/before_vm_start
>> $ cat 10_cgroups
>>=20
>> #!/usr/bin/python
>>=20
>> import hooking
>>=20
>>=20
>> def custom_partition(domxml):
>> resource =3D domxml.createElement('resource')
>> partition =3D domxml.createElement('partition')
>> partition_text =3D domxml.createTextNode('/machine/custom')
>> partition.appendChild(partition_text)
>> resource.appendChild(partition)
>>=20
>> return resource
>>=20
>> domxml =3D hooking.read_domxml()
>> domain =3D domxml.getElementsByTagName('domain')[0]
>> domain.appendChild(custom_partition(domxml))
>> hooking.write_domxml(domxml)
>>=20
>>> --
>>> Dmitry Glushenok
>>> Jet Infosystems
>>> http://www.jet.msk.su
>>> +7-495-411-7601 (ext. 1237)
>>>=20
>>>> 3 =D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 12:24, Martin =
Polednik <mpolednik(a)redhat.com> =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB=
(=D0=B0):
>>>>=20
>>>> On 03/06/16 11:48 +0300, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =
=D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=BD=D0=BE=D0=BA wrote:
>>>>> Hello!
>>>>>=20
>>>>> Is it possible to tell libvirt to add specific devices to qemu =
cgroup? By somehow enumerating the devices in XML using a hook for =
example.
>>>>> I'm passing scsi-generic disks (/dev/sgX) to
VM using qemucmdline =
hook and it doesn't work until I remove
"devices" from =
cgroup_controllers in qemu.conf.
>>>>=20
>>>> One way to achieve this is creating a hook to generate the scsi =
device
>>>> XML instead of modifying qemu cmdline directly.
Libvirt assumes
>>>> ownership of all devices created in the XML and therefore adds =
them to
>>>> the machine cgroup.
>>>>=20
>>>> Example of the XML taken from [1]:
>>>> <devices>
>>>> <hostdev mode=3D'subsystem' type=3D'scsi'
sgio=3D'filtered' =
rawio=3D'yes'>
>>>> <source>
>>>> <adapter name=3D'scsi_host0'/>
>>>> <address bus=3D'0' target=3D'0'
unit=3D'0'/>
>>>> </source>
>>>> <readonly/>
>>>> <address type=3D'drive' controller=3D'0'
bus=3D'0' target=3D'0' =
unit=3D'0'/>
>>>> </hostdev>
>>>> </devices>
>>>>=20
>>>> There is slight issue with this approach outlined in [2].
>>>>=20
>>>> If you want to keep the qemu approach, I think creating a custom
>>>> partition and moving devices there would be the cleanest approach. =
In
>>>> this case, [3] could help but I'm not entirely
sure if that would
>>>> solve the issue.
>>>>=20
>>>> [1] https://libvirt.org/formatdomain.html
>>>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=3D1325485
>>>> [3] https://libvirt.org/cgroups.html
>>>>> --
>>>>> Dmitry Glushenok
>>>>> Jet Infosystems
>>>>> http://www.jet.msk.su
>>>>> +7-495-411-7601 (ext. 1237)
>>>>>=20
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>=20
--Apple-Mail=_F5070364-047E-4F4A-BEE4-93C0242B9F73
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
charset=utf-8
<html><head><meta http-equiv=3D"Content-Type"
content=3D"text/html =
charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D"">So, libvirt behaves differently
with custom =
partition? My assumption was that it will perform the same changes. I =
will have in mind in case oVirt 4 will be released too late.</div><div =
class=3D"">May be I wrote incorrectly - in oVirt 4.0 RC1 scsi =
pass-through already working.</div><div class=3D""><br =
class=3D""></div><div class=3D"">
<div style=3D"color: rgb(0, 0, 0); letter-spacing: normal; orphans: =
auto; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div style=3D"color: rgb(0, 0, 0); letter-spacing: normal;
=
orphans: auto; text-align: start; text-indent: 0px; text-transform: =
none; white-space: normal; widows: auto; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" =
class=3D""><div class=3D"">--</div><div
class=3D""><div class=3D"" =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;"><div class=3D"" =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;"><div class=3D"" =
style=3D"word-wrap: break-word; -webkit-nbsp-mode: space; =
-webkit-line-break: after-white-space;"><div class=3D"">Dmitry =
Glushenok</div><div class=3D"">Jet Infosystems</div><div
class=3D""><a =
href=3D"http://www.jet.msk.su" =
class=3D"">http://www.jet.msk.su</a></div></div></div></div>+7-495-411-760=
1 (ext. 1237)</div></div></div></div></div></div>
</div>
<br class=3D""><div><blockquote type=3D"cite"
class=3D""><div class=3D"">6=
=D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 13:35, Martin Polednik =
<<a href=3D"mailto:mpolednik@redhat.com" =
class=3D"">mpolednik(a)redhat.com</a>&gt;
=D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=
=D0=BB(=D0=B0):</div><br class=3D"Apple-interchange-newline"><div
=
class=3D"">On 06/06/16 13:26 +0300, =D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=
=B9 =D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=BD=D0=BE=D0=BA wrote:<br =
class=3D""><blockquote type=3D"cite"
class=3D"">Hello Martin,<br =
class=3D""><br class=3D"">Thank you for your time. It is
clear how to =
create partition and assign it to a VM. But libvirt manipulates with =
contents of devices.list and I didn't find a way to put my devices into =
devices.list and stop libvirt from removing them.<br =
class=3D""></blockquote><br class=3D"">That is, by the
way, pretty =
interesting. I have tried it right now and<br class=3D"">when using =
custom partition, libvirt doesn't touch my devices.list.<br
class=3D""><br=
class=3D"">It might make sense to file a libvirt bug.<br
class=3D""><br =
class=3D""><blockquote type=3D"cite"
class=3D"">Anyway, it looks like =
that the best solution will be using oVirt 4.0 where scsi pass-through =
have been fixed.<br class=3D""></blockquote><br
class=3D"">Hoping so!<br =
class=3D""><br class=3D""><blockquote
type=3D"cite" class=3D"">--<br =
class=3D"">Dmitry Glushenok<br class=3D"">Jet
Infosystems<br class=3D""><a=
href=3D"http://www.jet.msk.su"
class=3D"">http://www.jet.msk.su</a> =
<<a href=3D"http://www.jet.msk.su/" =
class=3D"">http://www.jet.msk.su/</a>>+7-495-411-7601 (ext.
1237)<br =
class=3D""><br class=3D""><blockquote
type=3D"cite" class=3D"">6 =
=D0=B8=D1=8E=D0=BD=D1=8F 2016 =D0=B3., =D0=B2 12:33, Martin Polednik =
<<a href=3D"mailto:mpolednik@redhat.com" =
class=3D"">mpolednik(a)redhat.com</a>&gt;
=D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=
=D0=BB(=D0=B0):<br class=3D""><br class=3D"">On 03/06/16
14:05 +0300, =
=D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=
=BD=D0=BE=D0=BA wrote:<br class=3D""><blockquote
type=3D"cite" =
class=3D"">Thank you Martin!<br class=3D""><br
class=3D"">Actually I =
tried the workaround hook, provided in [2], but then VDSM (oVirt 3.6.6) =
tries to interpret hostdev in XML as PCI device, which leads to:<br =
class=3D""><br class=3D"">::The vm start process failed<br
=
class=3D"">Traceback (most recent call last):<br
class=3D"">File =
"/usr/share/vdsm/virt/vm.py", line 703, in _startUnderlyingVm<br =
class=3D""> self._run()<br class=3D"">File =
"/usr/share/vdsm/virt/vm.py", line 1949, in _run<br class=3D"">
=
self._domDependentInit()<br class=3D"">File =
"/usr/share/vdsm/virt/vm.py", line 1797, in _domDependentInit<br =
class=3D""> self._getUnderlyingVmDevicesInfo()<br
class=3D"">File =
"/usr/share/vdsm/virt/vm.py", line 1738, in =
_getUnderlyingVmDevicesInfo<br class=3D""> =
self._getUnderlyingHostDeviceInfo()<br class=3D"">File =
"/usr/share/vdsm/virt/vm.py", line 4277, in =
_getUnderlyingHostDeviceInfo<br class=3D""> =
**self._getUnderlyingDeviceAddress(source))<br
class=3D"">TypeError:=
pci_address_to_name() got an unexpected keyword argument 'target'<br =
class=3D""><br class=3D"">XML part was:<br
class=3D""><hostdev =
managed=3D"no" mode=3D"subsystem" rawio=3D"yes"
type=3D"scsi"><br =
class=3D""><span class=3D"Apple-tab-span"
style=3D"white-space:pre"> =
</span><source><br class=3D""><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span><adapter =
name=3D"scsi_host2"/><br class=3D""><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span><address bus=3D"0"
target=3D"1" =
unit=3D"0"/><br class=3D""><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span></source><br =
class=3D""></hostdev><br
class=3D""><hostdev managed=3D"no" =
mode=3D"subsystem" rawio=3D"yes" type=3D"scsi"><br
class=3D""><span =
class=3D"Apple-tab-span" style=3D"white-space:pre"> =
</span><source><br class=3D""><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span><adapter =
name=3D"scsi_host2"/><br class=3D""><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span><address bus=3D"0"
target=3D"2" =
unit=3D"0"/><br class=3D""><span
class=3D"Apple-tab-span" =
style=3D"white-space:pre"> </span></source><br =
class=3D""></hostdev><br class=3D""><br
class=3D""><br class=3D"">As=
of creating custom partition - by default machine.slice has "a *:* rwm" =
in devices.list. But for every new VM libvirt removes *:* mask and fills =
the list with actually needed devices (as I understand the process). For =
example:<br class=3D""><br class=3D"">c 136:* rw<br
class=3D"">c 1:3 =
rw<br class=3D"">c 1:7 rw<br class=3D"">c 1:5 rw<br
class=3D"">c 1:8 =
rw<br class=3D"">c 1:9 rw<br class=3D"">c 5:2 rw<br
class=3D"">c 10:232 =
rw<br class=3D"">c 253:0 rw<br class=3D"">c 10:228
rw<br class=3D"">c =
10:196 rw<br class=3D""><br class=3D"">What I'm
looking for is a way to =
tell libvirt about my additional devices without breaking oVirt.<br =
class=3D""></blockquote><br class=3D"">The solution
would be creating =
your own partition and somehow (e.g.<br class=3D"">VDSM hook) =
appending<br class=3D""><br
class=3D""><resource><br class=3D""> =
<partition>/machine/custom</partition><br =
class=3D""></resource><br class=3D""><br
class=3D"">to the =
libvirt's <domain> element. I'm not sure how feasible creating<br
=
class=3D"">your own partition is though. I've tried the process as =
follows:<br class=3D""><br class=3D"">$ cat
~/create-partition.sh<br =
class=3D""># sh ~/create-partition.sh<br class=3D"">for i in
blkio =
cpu,cpuacct cpuset devices freezer memory net_cls perf_event<br =
class=3D"">do<br class=3D""> mkdir =
/sys/fs/cgroup/$i/machine.slice/custom.partition<br class=3D"">done<br
=
class=3D""><br class=3D"">for i in cpuset.cpus
cpuset.mems<br =
class=3D"">do<br class=3D""> cat =
/sys/fs/cgroup/cpuset/machine.slice/$i > =
/sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i<br =
class=3D"">done<br class=3D""><br
class=3D"">(creates /machine/custom =
partition).<br class=3D"">Now, we can create vdsm before_vm_start hook =
that will set given partition<br class=3D"">for the VM:<br
class=3D""><br =
class=3D"">$ pwd /usr/libexec/vdsm/hooks/before_vm_start<br
class=3D"">$ =
cat 10_cgroups<br class=3D""><br
class=3D"">#!/usr/bin/python<br =
class=3D""><br class=3D"">import hooking<br
class=3D""><br class=3D""><br =
class=3D"">def custom_partition(domxml):<br class=3D"">
resource =3D=
domxml.createElement('resource')<br class=3D"">
partition =3D =
domxml.createElement('partition')<br class=3D"">
partition_text =3D =
domxml.createTextNode('/machine/custom')<br class=3D""> =
partition.appendChild(partition_text)<br class=3D""> =
resource.appendChild(partition)<br class=3D""><br
class=3D""> =
return resource<br class=3D""><br
class=3D"">domxml =3D =
hooking.read_domxml()<br class=3D"">domain =3D =
domxml.getElementsByTagName('domain')[0]<br =
class=3D"">domain.appendChild(custom_partition(domxml))<br =
class=3D"">hooking.write_domxml(domxml)<br class=3D""><br
=
class=3D""><blockquote type=3D"cite"
class=3D"">--<br class=3D"">Dmitry =
Glushenok<br class=3D"">Jet Infosystems<br
class=3D""><a =
href=3D"http://www.jet.msk.su"
class=3D"">http://www.jet.msk.su</a><br =
class=3D"">+7-495-411-7601 (ext. 1237)<br class=3D""><br
=
class=3D""><blockquote type=3D"cite" class=3D"">3
=D0=B8=D1=8E=D0=BD=D1=8F=
2016 =D0=B3., =D0=B2 12:24, Martin Polednik =
&lt;mpolednik(a)redhat.com&gt; =D0=BD=D0=B0=D0=BF=D0=B8=D1=81=D0=B0=D0=BB(=D0=
=B0):<br class=3D""><br class=3D"">On 03/06/16 11:48
+0300, =
=D0=94=D0=BC=D0=B8=D1=82=D1=80=D0=B8=D0=B9 =D0=93=D0=BB=D1=83=D1=88=D0=B5=D0=
=BD=D0=BE=D0=BA wrote:<br class=3D""><blockquote
type=3D"cite" =
class=3D"">Hello!<br class=3D""><br
class=3D"">Is it possible to tell =
libvirt to add specific devices to qemu cgroup? By somehow enumerating =
the devices in XML using a hook for example.<br class=3D"">I'm passing
=
scsi-generic disks (/dev/sgX) to VM using qemucmdline hook and it =
doesn't work until I remove "devices" from cgroup_controllers in =
qemu.conf.<br class=3D""></blockquote><br
class=3D"">One way to achieve =
this is creating a hook to generate the scsi device<br class=3D"">XML =
instead of modifying qemu cmdline directly. Libvirt assumes<br =
class=3D"">ownership of all devices created in the XML and therefore =
adds them to<br class=3D"">the machine cgroup.<br
class=3D""><br =
class=3D"">Example of the XML taken from [1]:<br =
class=3D""><devices><br class=3D"">
<hostdev mode=3D'subsystem' =
type=3D'scsi' sgio=3D'filtered' rawio=3D'yes'><br
class=3D""> =
<source><br
class=3D""> =
<adapter
=
name=3D'scsi_host0'/><br class=3D""> =
<address
bus=3D'0' =
target=3D'0' unit=3D'0'/><br class=3D""> =
</source><br
class=3D""> =
<readonly/><br
class=3D""> =
<address type=3D'drive'
controller=3D'0' =
bus=3D'0' target=3D'0' unit=3D'0'/><br
class=3D""> =
</hostdev><br class=3D""></devices><br
class=3D""><br =
class=3D"">There is slight issue with this approach outlined in [2].<br =
class=3D""><br class=3D"">If you want to keep the qemu
approach, I think =
creating a custom<br class=3D"">partition and moving devices there would
=
be the cleanest approach. In<br class=3D"">this case, [3] could help but
=
I'm not entirely sure if that would<br class=3D"">solve the
issue.<br =
class=3D""><br class=3D"">[1]
https://libvirt.org/formatdomain.html<br =
class=3D"">[2] https://bugzilla.redhat.com/show_bug.cgi?id=3D1325485<br =
class=3D"">[3] https://libvirt.org/cgroups.html<br
class=3D""><blockquote =
type=3D"cite" class=3D"">--<br class=3D"">Dmitry
Glushenok<br =
class=3D"">Jet Infosystems<br
class=3D"">http://www.jet.msk.su<br =
class=3D"">+7-495-411-7601 (ext. 1237)<br class=3D""><br
=
class=3D"">_______________________________________________<br =
class=3D"">Users mailing list<br
class=3D"">Users(a)ovirt.org<br =
class=3D"">http://lists.ovirt.org/mailman/listinfo/users<br =
class=3D""></blockquote></blockquote></blockquote></blockquote><br
=
class=3D""></blockquote></div></blockquote></div><br
=
class=3D""></body></html>=
--Apple-Mail=_F5070364-047E-4F4A-BEE4-93C0242B9F73--
4:39 p.m.
On 06/06/16 16:15 +0300, Дмитрий Глушенок wrote:
So, libvirt behaves differently with custom partition? My assumption
was that it will perform the same changes. I will have in mind in case oVirt 4 will be
released too late.
May be I wrote incorrectly - in oVirt 4.0 RC1 scsi pass-through already working.
It should be working.
--
Dmitry Glushenok
Jet Infosystems
http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. 1237)
> 6 июня 2016 г., в 13:35, Martin Polednik <mpolednik(a)redhat.com> написал(а):
>
> On 06/06/16 13:26 +0300, Дмитрий Глушенок wrote:
>> Hello Martin,
>>
>> Thank you for your time. It is clear how to create partition and assign it to a
VM. But libvirt manipulates with contents of devices.list and I didn't find a way to
put my devices into devices.list and stop libvirt from removing them.
>
> That is, by the way, pretty interesting. I have tried it right now and
> when using custom partition, libvirt doesn't touch my devices.list.
>
> It might make sense to file a libvirt bug.
>
>> Anyway, it looks like that the best solution will be using oVirt 4.0 where scsi
pass-through have been fixed.
>
> Hoping so!
>
>> --
>> Dmitry Glushenok
>> Jet Infosystems
>> http://www.jet.msk.su <http://www.jet.msk.su/>+7-495-411-7601 (ext. 1237)
>>
>>> 6 июня 2016 г., в 12:33, Martin Polednik <mpolednik(a)redhat.com>
написал(а):
>>>
>>> On 03/06/16 14:05 +0300, Дмитрий Глушенок wrote:
>>>> Thank you Martin!
>>>>
>>>> Actually I tried the workaround hook, provided in [2], but then VDSM
(oVirt 3.6.6) tries to interpret hostdev in XML as PCI device, which leads to:
>>>>
>>>> ::The vm start process failed
>>>> Traceback (most recent call last):
>>>> File "/usr/share/vdsm/virt/vm.py", line 703, in
_startUnderlyingVm
>>>> self._run()
>>>> File "/usr/share/vdsm/virt/vm.py", line 1949, in _run
>>>> self._domDependentInit()
>>>> File "/usr/share/vdsm/virt/vm.py", line 1797, in
_domDependentInit
>>>> self._getUnderlyingVmDevicesInfo()
>>>> File "/usr/share/vdsm/virt/vm.py", line 1738, in
_getUnderlyingVmDevicesInfo
>>>> self._getUnderlyingHostDeviceInfo()
>>>> File "/usr/share/vdsm/virt/vm.py", line 4277, in
_getUnderlyingHostDeviceInfo
>>>> **self._getUnderlyingDeviceAddress(source))
>>>> TypeError: pci_address_to_name() got an unexpected keyword argument
'target'
>>>>
>>>> XML part was:
>>>> <hostdev managed="no" mode="subsystem"
rawio="yes" type="scsi">
>>>> <source>
>>>> <adapter name="scsi_host2"/>
>>>> <address bus="0" target="1"
unit="0"/>
>>>> </source>
>>>> </hostdev>
>>>> <hostdev managed="no" mode="subsystem"
rawio="yes" type="scsi">
>>>> <source>
>>>> <adapter name="scsi_host2"/>
>>>> <address bus="0" target="2"
unit="0"/>
>>>> </source>
>>>> </hostdev>
>>>>
>>>>
>>>> As of creating custom partition - by default machine.slice has "a
*:* rwm" in devices.list. But for every new VM libvirt removes *:* mask and fills the
list with actually needed devices (as I understand the process). For example:
>>>>
>>>> c 136:* rw
>>>> c 1:3 rw
>>>> c 1:7 rw
>>>> c 1:5 rw
>>>> c 1:8 rw
>>>> c 1:9 rw
>>>> c 5:2 rw
>>>> c 10:232 rw
>>>> c 253:0 rw
>>>> c 10:228 rw
>>>> c 10:196 rw
>>>>
>>>> What I'm looking for is a way to tell libvirt about my additional
devices without breaking oVirt.
>>>
>>> The solution would be creating your own partition and somehow (e.g.
>>> VDSM hook) appending
>>>
>>> <resource>
>>> <partition>/machine/custom</partition>
>>> </resource>
>>>
>>> to the libvirt's <domain> element. I'm not sure how feasible
creating
>>> your own partition is though. I've tried the process as follows:
>>>
>>> $ cat ~/create-partition.sh
>>> # sh ~/create-partition.sh
>>> for i in blkio cpu,cpuacct cpuset devices freezer memory net_cls perf_event
>>> do
>>> mkdir /sys/fs/cgroup/$i/machine.slice/custom.partition
>>> done
>>>
>>> for i in cpuset.cpus cpuset.mems
>>> do
>>> cat /sys/fs/cgroup/cpuset/machine.slice/$i >
/sys/fs/cgroup/cpuset/machine.slice/custom.partition/$i
>>> done
>>>
>>> (creates /machine/custom partition).
>>> Now, we can create vdsm before_vm_start hook that will set given partition
>>> for the VM:
>>>
>>> $ pwd /usr/libexec/vdsm/hooks/before_vm_start
>>> $ cat 10_cgroups
>>>
>>> #!/usr/bin/python
>>>
>>> import hooking
>>>
>>>
>>> def custom_partition(domxml):
>>> resource = domxml.createElement('resource')
>>> partition = domxml.createElement('partition')
>>> partition_text = domxml.createTextNode('/machine/custom')
>>> partition.appendChild(partition_text)
>>> resource.appendChild(partition)
>>>
>>> return resource
>>>
>>> domxml = hooking.read_domxml()
>>> domain = domxml.getElementsByTagName('domain')[0]
>>> domain.appendChild(custom_partition(domxml))
>>> hooking.write_domxml(domxml)
>>>
>>>> --
>>>> Dmitry Glushenok
>>>> Jet Infosystems
>>>> http://www.jet.msk.su
>>>> +7-495-411-7601 (ext. 1237)
>>>>
>>>>> 3 июня 2016 г., в 12:24, Martin Polednik <mpolednik(a)redhat.com>
написал(а):
>>>>>
>>>>> On 03/06/16 11:48 +0300, Дмитрий Глушенок wrote:
>>>>>> Hello!
>>>>>>
>>>>>> Is it possible to tell libvirt to add specific devices to qemu
cgroup? By somehow enumerating the devices in XML using a hook for example.
>>>>>> I'm passing scsi-generic disks (/dev/sgX) to VM using
qemucmdline hook and it doesn't work until I remove "devices" from
cgroup_controllers in qemu.conf.
>>>>>
>>>>> One way to achieve this is creating a hook to generate the scsi
device
>>>>> XML instead of modifying qemu cmdline directly. Libvirt assumes
>>>>> ownership of all devices created in the XML and therefore adds them
to
>>>>> the machine cgroup.
>>>>>
>>>>> Example of the XML taken from [1]:
>>>>> <devices>
>>>>> <hostdev mode='subsystem' type='scsi'
sgio='filtered' rawio='yes'>
>>>>> <source>
>>>>> <adapter name='scsi_host0'/>
>>>>> <address bus='0' target='0'
unit='0'/>
>>>>> </source>
>>>>> <readonly/>
>>>>> <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>>>>> </hostdev>
>>>>> </devices>
>>>>>
>>>>> There is slight issue with this approach outlined in [2].
>>>>>
>>>>> If you want to keep the qemu approach, I think creating a custom
>>>>> partition and moving devices there would be the cleanest approach.
In
>>>>> this case, [3] could help but I'm not entirely sure if that
would
>>>>> solve the issue.
>>>>>
>>>>> [1] https://libvirt.org/formatdomain.html
>>>>> [2] https://bugzilla.redhat.com/show_bug.cgi?id=1325485
>>>>> [3] https://libvirt.org/cgroups.html
>>>>>> --
>>>>>> Dmitry Glushenok
>>>>>> Jet Infosystems
>>>>>> http://www.jet.msk.su
>>>>>> +7-495-411-7601 (ext. 1237)
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>
3091
days inactive
3094
days old
7 comments
2 participants
participants (2)
-
Martin Polednik -
Дмитрий Глушенок