[ovirt-devel] [VDSM] health: Introduce Vdsm health monitoring

Nir Soffer nsoffer at redhat.com
Thu Jan 14 18:00:50 UTC 2016


Here are example logs with running and stopping one vm:

# tail -f /var/log/vdsm/vdsm.log | grep health
Thread-13::DEBUG::2016-01-14
19:42:04,625::health::84::health::(_check) Checking health
Thread-13::DEBUG::2016-01-14
19:42:04,651::health::86::health::(_check) Collected 25 objects
Thread-13::WARNING::2016-01-14
19:42:04,656::health::100::health::(_check) Found 6085 uncollectible
objects, 0 suspects implementing __del__: []

Vm was started here...

Thread-13::DEBUG::2016-01-14
19:43:04,657::health::84::health::(_check) Checking health
Thread-13::DEBUG::2016-01-14
19:43:04,688::health::86::health::(_check) Collected 1680 objects
Thread-13::WARNING::2016-01-14
19:43:04,697::health::100::health::(_check) Found 11041 uncollectible
objects, 0 suspects implementing __del__: []
Thread-13::DEBUG::2016-01-14
19:44:04,698::health::84::health::(_check) Checking health
Thread-13::DEBUG::2016-01-14
19:44:04,729::health::86::health::(_check) Collected 57 objects
Thread-13::WARNING::2016-01-14
19:44:04,737::health::100::health::(_check) Found 11096 uncollectible
objects, 0 suspects implementing __del__: []
Thread-13::DEBUG::2016-01-14
19:45:04,738::health::84::health::(_check) Checking health
Thread-13::DEBUG::2016-01-14
19:45:04,769::health::86::health::(_check) Collected 69 objects
Thread-13::WARNING::2016-01-14
19:45:04,778::health::100::health::(_check) Found 11163 uncollectible
objects, 0 suspects implementing __del__: []

Vm was shutdown here...

Thread-13::DEBUG::2016-01-14
19:46:04,779::health::84::health::(_check) Checking health
Thread-13::DEBUG::2016-01-14
19:46:04,809::health::86::health::(_check) Collected 2272 objects
Thread-13::WARNING::2016-01-14
19:46:04,819::health::100::health::(_check) Found 13464 uncollectible
objects, 28 suspects implementing __del__: ['<pthreading.Lock object
at 0x7f1d8016f908>', '<pthreading.Lock object at 0x7f1d801804d0>',
'<pthreading.Lock object at 0x7f1cf8010f38>', '<pthreading.Lock object
at 0x7f1d8016f7a0>', '<pthreading.Lock object at 0x7f1d20003680>',
'<pthreading.Lock object at 0x7f1da8112998>', '<pthreading.Lock object
at 0x7f1d8016fa28>', '<pthreading.Lock object at 0x7f1da8112830>',
'<pthread.Cond object at 0x7f1da811cf50>', '<pthreading.Lock object at
0x7f1d8016f200>', '<pthread.Cond object at 0x7f1da811ca50>',
'<pthreading.Lock object at 0x7f1cf8010fc8>', '<pthread.Cond object at
0x7f1da8115c50>', '<pthreading.Lock object at 0x7f1da8112878>',
'<pthread.Cond object at 0x7f1da811c550>', '<pthreading.Lock object at
0x7f1d8016f9e0>', '<pthread.Cond object at 0x7f1da811ca90>',
'<pthreading.Lock object at 0x7f1d8016f680>', '<pthread.Cond object at
0x7f1da811cc50>', '<libvirt.virDomain object at 0x1ba8210>',
'<pthreading.Lock object at 0x7f1d8016f758>', '<pthread.Cond object at
0x7f1da811cb50>', '<pthreading.Lock object at 0x7f1d8016fb90>',
'<pthread.Cond object at 0x7f1da811cc10>', '<pthread.Cond object at
0x233ed50>', '<pthreading.Lock object at 0x7f1ce0227c20>',
'<pthreading.Lock object at 0x7f1ce0227ab8>', '<pthread.Cond object at
0x233ed10>']

Need to test with should be fixed by https://gerrit.ovirt.org/51630,
which should
eliminate '<libvirt.virDomain object at 0x1ba8210>', and maybe also
the pthreading
objects.

Nir

On Thu, Jan 14, 2016 at 7:38 PM, Nir Soffer <nsoffer at redhat.com> wrote:
> Hi all,
>
> Continuing the leak investigation Francesco and Milan are working on,
> I posted this patch, adding leak health monitoring to vdsm [1]. This patch
> currently monitor collectible objects, and we may add other interesting
> stuff later if needed.
>
> To enable the monitor, you must set:
>
> [devel]
> health_monitor_enable = true
>
> In vdsm.conf, and restart vdsm.
>
> Here is an example logs - this was logged 60 seconds after starting vdsm,
> when running as spm:
>
> Thread-13::DEBUG::2016-01-14
> 18:53:43,239::health::84::health::(_check) Checking health
> Thread-13::DEBUG::2016-01-14
> 18:53:43,272::health::86::health::(_check) Collected 460 objects
> Thread-13::WARNING::2016-01-14
> 18:53:43,277::health::100::health::(_check) Found 5873 uncollectible
> objects, 10 suspects implementing __del__: ['<pthreading.Lock object
> at 0x7f012c115320>', '<pthreading.Lock object at 0x7f012c115440>',
> '<pthreading.Lock object at 0x7f012c1153b0>', '<pthreading.Lock object
> at 0x7f012c1154d0>', '<pthread.Cond object at 0x7f012c111350>',
> '<pthread.Cond object at 0x7f012c1112d0>', '<pthread.Cond object at
> 0x7f012c111250>', '<pthread.Cond object at 0x7f012c111850>',
> '<pthread.Cond object at 0x7f012c1117d0>', '<pthread.Cond object at
> 0x7f012c111750>']
>
> We found 460 uncollectible objects, 10 of them are objects implementing __del__.
> Having such object in a reference cycle make the entire cycle uncollectible.
>
> The pthreading objects in this log grow as vdsm continue to run, even without
> any interaction with engine. The pthreading objects leak is caused by a cycle
> in storage.misc.RWLock, fixed in patch [2].
>
> After solving the pthreading leak, we still have many uncollectible
> objects in the logs,
> listing them, it seems that the suspect is hiding inside a collection
> (dict, list, tuple)
> and does not appear in the gc.garbage list. I will address this in later patch,
> searching in the entire object graph instead of the top level of gc.garbage.
>
> See end of this mail for list of these objects. I printed them like this:
>
> Enable manhole debugging shell:
>
> [devel]
> manhole_enable = true
>
> Connect to vdsm using manhole:
>
> nc -U /run/vdsm/vdsmd.manhole
>
> Print garbage:
>
>>>> import pprint
>>>> import gc
>>>> with open('/tmp/garbage.txt', 'w') as f:
>            pprint.pprint(gc.garbage, stream=f, indent=4)
>
> These objects looks related to libvirt - Francesco, can you take a look?
>
> [1] https://gerrit.ovirt.org/51708
> [2] https://gerrit.ovirt.org/51868
>
> Nir
>
> --- garbage after patch [2] ---
>
> [   (<type '_ctypes.Array'>,),
>     {   '__dict__': <attribute '__dict__' of 'c_char_Array_60' objects>,
>         '__doc__': None,
>         '__module__': 'vdsm.netlink',
>         '__weakref__': <attribute '__weakref__' of 'c_char_Array_60' objects>,
>         '_length_': 60,
>         '_type_': <class 'ctypes.c_char'>,
>         'raw': <attribute 'raw' of 'c_char_Array_60' objects>,
>         'value': <attribute 'value' of 'c_char_Array_60' objects>},
>     <class 'vdsm.netlink.c_char_Array_60'>,
>     <attribute '__dict__' of 'c_char_Array_60' objects>,
>     <attribute '__weakref__' of 'c_char_Array_60' objects>,
>     (   <class 'vdsm.netlink.c_char_Array_60'>,
>         <type '_ctypes.Array'>,
>         <type '_ctypes._CData'>,
>         <type 'object'>),
>     <attribute 'raw' of 'c_char_Array_60' objects>,
>     <attribute 'value' of 'c_char_Array_60' objects>,
>     <cell at 0x23662f0: bool object at 0x7f1e086c8c40>,
>     <cell at 0x2366328: NoneType object at 0x7f1e086dcf00>,
>     <cell at 0x2366360: function object at 0x233f488>,
>     (   <cell at 0x23662f0: bool object at 0x7f1e086c8c40>,
>         <cell at 0x2366328: NoneType object at 0x7f1e086dcf00>,
>         <cell at 0x2366360: function object at 0x233f488>),
>     <function wrapMethod at 0x233f488>,
>     [   [   <Recursion on list with id=34033464>,
>             <Recursion on list with id=34033464>,
>             'class'],
>         [   <Recursion on list with id=34033464>,
>             <Recursion on list with id=34033464>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=33979064>,
>             <Recursion on list with id=33979064>,
>             None],
>         [   <Recursion on list with id=33979064>,
>             <Recursion on list with id=33979064>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=35535256>,
>             <Recursion on list with id=35535256>,
>             'class'],
>         [   <Recursion on list with id=35535256>,
>             <Recursion on list with id=35535256>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=35535832>,
>             <Recursion on list with id=35535832>,
>             None],
>         [   <Recursion on list with id=35535832>,
>             <Recursion on list with id=35535832>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=35556312>,
>             <Recursion on list with id=35556312>,
>             'class'],
>         [   <Recursion on list with id=35556312>,
>             <Recursion on list with id=35556312>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=35556600>,
>             <Recursion on list with id=35556600>,
>             None],
>         [   <Recursion on list with id=35556600>,
>             <Recursion on list with id=35556600>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=35556960>,
>             <Recursion on list with id=35556960>,
>             'class'],
>         [   <Recursion on list with id=35556960>,
>             <Recursion on list with id=35556960>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=35561912>,
>             <Recursion on list with id=35561912>,
>             None],
>         [   <Recursion on list with id=35561912>,
>             <Recursion on list with id=35561912>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=35583904>,
>             <Recursion on list with id=35583904>,
>             'class'],
>         [   <Recursion on list with id=35583904>,
>             <Recursion on list with id=35583904>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=35584912>,
>             <Recursion on list with id=35584912>,
>             None],
>         [   <Recursion on list with id=35584912>,
>             <Recursion on list with id=35584912>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=36340368>,
>             <Recursion on list with id=36340368>,
>             'class'],
>         [   <Recursion on list with id=36340368>,
>             <Recursion on list with id=36340368>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=36340872>,
>             <Recursion on list with id=36340872>,
>             None],
>         [   <Recursion on list with id=36340872>,
>             <Recursion on list with id=36340872>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=36364872>,
>             <Recursion on list with id=36364872>,
>             'class'],
>         [   <Recursion on list with id=36364872>,
>             <Recursion on list with id=36364872>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=36365520>,
>             <Recursion on list with id=36365520>,
>             None],
>         [   <Recursion on list with id=36365520>,
>             <Recursion on list with id=36365520>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=36457216>,
>             <Recursion on list with id=36457216>,
>             'class'],
>         [   <Recursion on list with id=36457216>,
>             <Recursion on list with id=36457216>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=36458008>,
>             <Recursion on list with id=36458008>,
>             None],
>         [   <Recursion on list with id=36458008>,
>             <Recursion on list with id=36458008>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=36460520>,
>             <Recursion on list with id=36460520>,
>             'class'],
>         [   <Recursion on list with id=36460520>,
>             <Recursion on list with id=36460520>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=36461168>,
>             <Recursion on list with id=36461168>,
>             None],
>         [   <Recursion on list with id=36461168>,
>             <Recursion on list with id=36461168>,
>             None],
>         'class'],
>     [   [   <Recursion on list with id=36562344>,
>             <Recursion on list with id=36562344>,
>             'class'],
>         [   <Recursion on list with id=36562344>,
>             <Recursion on list with id=36562344>,
>             'class'],
>         None],
>     [   [   <Recursion on list with id=36563280>,
>             <Recursion on list with id=36563280>,
>             None],
>         [   <Recursion on list with id=36563280>,
>             <Recursion on list with id=36563280>,
>             None],
>         'class'],
>     <Element 'memory' at 0x2364710>,
>     {   '_children': [],
>         'attrib': {   'unit': 'KiB'},
>         'tag': 'memory',
>         'tail': '\n          ',
>         'text': '4095476'},
>     [],
>     <Element 'pages' at 0x2364750>,
>     {   '_children': [],
>         'attrib': {   'size': '4', 'unit': 'KiB'},
>         'tag': 'pages',
>         'tail': '\n          ',
>         'text': '1023869'},
>     [],
>     <Element 'pages' at 0x2364790>,
>     {   '_children': [],
>         'attrib': {   'size': '2048', 'unit': 'KiB'},
>         'tag': 'pages',
>         'tail': '\n          ',
>         'text': '0'},
>     [],
>     <Element 'distances' at 0x23647d0>,
>     {   '_children': [<Element 'sibling' at 0x2364810>],
>         'attrib': {   },
>         'tag': 'distances',
>         'tail': '\n          ',
>         'text': '\n            '},
>     [<Element 'sibling' at 0x2364810>],
>     <Element 'sibling' at 0x2364810>,
>     {   '_children': [],
>         'attrib': {   'id': '0', 'value': '10'},
>         'tag': 'sibling',
>         'tail': '\n          '},
>     [],
>     <Element 'cpus' at 0x2364850>,
>     {   '_children': [   <Element 'cpu' at 0x2364890>,
>                          <Element 'cpu' at 0x23648d0>,
>                          <Element 'cpu' at 0x2364910>,
>                          <Element 'cpu' at 0x2364950>,
>                          <Element 'cpu' at 0x2364990>,
>                          <Element 'cpu' at 0x23649d0>,
>                          <Element 'cpu' at 0x2364a10>,
>                          <Element 'cpu' at 0x2364a50>],
>         'attrib': {   'num': '8'},
>         'tag': 'cpus',
>         'tail': '\n        ',
>         'text': '\n            '},
>     [   <Element 'cpu' at 0x2364890>,
>         <Element 'cpu' at 0x23648d0>,
>         <Element 'cpu' at 0x2364910>,
>         <Element 'cpu' at 0x2364950>,
>         <Element 'cpu' at 0x2364990>,
>         <Element 'cpu' at 0x23649d0>,
>         <Element 'cpu' at 0x2364a10>,
>         <Element 'cpu' at 0x2364a50>],
>     <Element 'cpu' at 0x2364890>,
>     {   '_children': [],
>         'attrib': {   'core_id': '0',
>                       'id': '0',
>                       'siblings': '0',
>                       'socket_id': '0'},
>         'tag': 'cpu',
>         'tail': '\n            '},
>     [],
>     <Element 'cpu' at 0x23648d0>,
>     {   '_children': [],
>         'attrib': {   'core_id': '0',
>                       'id': '1',
>                       'siblings': '1',
>                       'socket_id': '1'},
>         'tag': 'cpu',
>         'tail': '\n            '},
>     [],
>     <Element 'cpu' at 0x2364910>,
>     {   '_children': [],
>         'attrib': {   'core_id': '0',
>                       'id': '2',
>                       'siblings': '2',
>                       'socket_id': '2'},
>         'tag': 'cpu',
>         'tail': '\n            '},
>     [],
>     <Element 'cpu' at 0x2364950>,
>     {   '_children': [],
>         'attrib': {   'core_id': '0',
>                       'id': '3',
>                       'siblings': '3',
>                       'socket_id': '3'},
>         'tag': 'cpu',
>         'tail': '\n            '},
>     [],
>     <Element 'cpu' at 0x2364990>,
>     {   '_children': [],
>         'attrib': {   'core_id': '0',
>                       'id': '4',
>                       'siblings': '4',
>                       'socket_id': '4'},
>         'tag': 'cpu',
>         'tail': '\n            '},
>     [],
>     <Element 'cpu' at 0x23649d0>,
>     {   '_children': [],
>         'attrib': {   'core_id': '0',
>                       'id': '5',
>                       'siblings': '5',
>                       'socket_id': '5'},
>         'tag': 'cpu',
>         'tail': '\n            '},
>     [],
>     <Element 'cpu' at 0x2364a10>,
>     {   '_children': [],
>         'attrib': {   'core_id': '0',
>                       'id': '6',
>                       'siblings': '6',
>                       'socket_id': '6'},
>         'tag': 'cpu',
>         'tail': '\n            '},
>     [],
>     <Element 'cpu' at 0x2364a50>,
>     {   '_children': [],
>         'attrib': {   'core_id': '0',
>                       'id': '7',
>                       'siblings': '7',
>                       'socket_id': '7'},
>         'tag': 'cpu',
>         'tail': '\n          '},
>     [],
>     <Element 'secmodel' at 0x2364a90>,
>     {   '_children': [   <Element 'model' at 0x2364ad0>,
>                          <Element 'doi' at 0x2364b10>,
>                          <Element 'baselabel' at 0x2364b50>,
>                          <Element 'baselabel' at 0x2364b90>],
>         'attrib': {   },
>         'tag': 'secmodel',
>         'tail': '\n    ',
>         'text': '\n      '},
>     [   <Element 'model' at 0x2364ad0>,
>         <Element 'doi' at 0x2364b10>,
>         <Element 'baselabel' at 0x2364b50>,
>         <Element 'baselabel' at 0x2364b90>],
>     <Element 'model' at 0x2364ad0>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'model',
>         'tail': '\n      ',
>         'text': 'selinux'},
>     [],
>     <Element 'doi' at 0x2364b10>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'doi',
>         'tail': '\n      ',
>         'text': '0'},
>     [],
>     <Element 'baselabel' at 0x2364b50>,
>     {   '_children': [],
>         'attrib': {   'type': 'kvm'},
>         'tag': 'baselabel',
>         'tail': '\n      ',
>         'text': 'system_u:system_r:svirt_t:s0'},
>     [],
>     <Element 'baselabel' at 0x2364b90>,
>     {   '_children': [],
>         'attrib': {   'type': 'qemu'},
>         'tag': 'baselabel',
>         'tail': '\n    ',
>         'text': 'system_u:system_r:svirt_tcg_t:s0'},
>     [],
>     <Element 'secmodel' at 0x2364bd0>,
>     {   '_children': [   <Element 'model' at 0x2364c10>,
>                          <Element 'doi' at 0x2364c50>,
>                          <Element 'baselabel' at 0x2364c90>,
>                          <Element 'baselabel' at 0x2364cd0>],
>         'attrib': {   },
>         'tag': 'secmodel',
>         'tail': '\n  ',
>         'text': '\n      '},
>     [   <Element 'model' at 0x2364c10>,
>         <Element 'doi' at 0x2364c50>,
>         <Element 'baselabel' at 0x2364c90>,
>         <Element 'baselabel' at 0x2364cd0>],
>     <Element 'model' at 0x2364c10>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'model',
>         'tail': '\n      ',
>         'text': 'dac'},
>     [],
>     <Element 'doi' at 0x2364c50>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'doi',
>         'tail': '\n      ',
>         'text': '0'},
>     [],
>     <Element 'baselabel' at 0x2364c90>,
>     {   '_children': [],
>         'attrib': {   'type': 'kvm'},
>         'tag': 'baselabel',
>         'tail': '\n      ',
>         'text': '+107:+107'},
>     [],
>     <Element 'baselabel' at 0x2364cd0>,
>     {   '_children': [],
>         'attrib': {   'type': 'qemu'},
>         'tag': 'baselabel',
>         'tail': '\n    ',
>         'text': '+107:+107'},
>     [],
>     <Element 'guest' at 0x2364d10>,
>     {   '_children': [   <Element 'os_type' at 0x2364d50>,
>                          <Element 'arch' at 0x2364d90>,
>                          <Element 'features' at 0x2369390>],
>         'attrib': {   },
>         'tag': 'guest',
>         'tail': '\n\n  ',
>         'text': '\n    '},
>     [   <Element 'os_type' at 0x2364d50>,
>         <Element 'arch' at 0x2364d90>,
>         <Element 'features' at 0x2369390>],
>     <Element 'os_type' at 0x2364d50>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'os_type',
>         'tail': '\n    ',
>         'text': 'hvm'},
>     [],
>     <Element 'arch' at 0x2364d90>,
>     {   '_children': [   <Element 'wordsize' at 0x2364dd0>,
>                          <Element 'emulator' at 0x2364e10>,
>                          <Element 'machine' at 0x2364e90>,
>                          <Element 'machine' at 0x2364ed0>,
>                          <Element 'machine' at 0x2364f10>,
>                          <Element 'machine' at 0x2364f50>,
>                          <Element 'machine' at 0x2364f90>,
>                          <Element 'machine' at 0x2364fd0>,
>                          <Element 'machine' at 0x2369050>,
>                          <Element 'machine' at 0x2369090>,
>                          <Element 'machine' at 0x23690d0>,
>                          <Element 'machine' at 0x2369110>,
>                          <Element 'machine' at 0x2369150>,
>                          <Element 'machine' at 0x2369190>,
>                          <Element 'machine' at 0x23691d0>,
>                          <Element 'machine' at 0x2369210>,
>                          <Element 'machine' at 0x2369250>,
>                          <Element 'domain' at 0x2369290>,
>                          <Element 'domain' at 0x23692d0>],
>         'attrib': {   'name': 'i686'},
>         'tag': 'arch',
>         'tail': '\n    ',
>         'text': '\n      '},
>     [   <Element 'wordsize' at 0x2364dd0>,
>         <Element 'emulator' at 0x2364e10>,
>         <Element 'machine' at 0x2364e90>,
>         <Element 'machine' at 0x2364ed0>,
>         <Element 'machine' at 0x2364f10>,
>         <Element 'machine' at 0x2364f50>,
>         <Element 'machine' at 0x2364f90>,
>         <Element 'machine' at 0x2364fd0>,
>         <Element 'machine' at 0x2369050>,
>         <Element 'machine' at 0x2369090>,
>         <Element 'machine' at 0x23690d0>,
>         <Element 'machine' at 0x2369110>,
>         <Element 'machine' at 0x2369150>,
>         <Element 'machine' at 0x2369190>,
>         <Element 'machine' at 0x23691d0>,
>         <Element 'machine' at 0x2369210>,
>         <Element 'machine' at 0x2369250>,
>         <Element 'domain' at 0x2369290>,
>         <Element 'domain' at 0x23692d0>],
>     <Element 'wordsize' at 0x2364dd0>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'wordsize',
>         'tail': '\n      ',
>         'text': '32'},
>     [],
>     <Element 'emulator' at 0x2364e10>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'emulator',
>         'tail': '\n      ',
>         'text': '/usr/libexec/qemu-kvm'},
>     [],
>     <Element 'machine' at 0x2364e90>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-i440fx-rhel7.2.0'},
>     [],
>     <Element 'machine' at 0x2364ed0>,
>     {   '_children': [],
>         'attrib': {   'canonical': 'pc-i440fx-rhel7.2.0', 'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc'},
>     [],
>     <Element 'machine' at 0x2364f10>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-i440fx-rhel7.0.0'},
>     [],
>     <Element 'machine' at 0x2364f50>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-q35-rhel7.1.0'},
>     [],
>     <Element 'machine' at 0x2364f90>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.3.0'},
>     [],
>     <Element 'machine' at 0x2364fd0>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-q35-rhel7.2.0'},
>     [],
>     <Element 'machine' at 0x2369050>,
>     {   '_children': [],
>         'attrib': {   'canonical': 'pc-q35-rhel7.2.0', 'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'q35'},
>     [],
>     <Element 'machine' at 0x2369090>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.4.0'},
>     [],
>     <Element 'machine' at 0x23690d0>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.0.0'},
>     [],
>     <Element 'machine' at 0x2369110>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-i440fx-rhel7.1.0'},
>     [],
>     <Element 'machine' at 0x2369150>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.5.0'},
>     [],
>     <Element 'machine' at 0x2369190>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.6.0'},
>     [],
>     <Element 'machine' at 0x23691d0>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.1.0'},
>     [],
>     <Element 'machine' at 0x2369210>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-q35-rhel7.0.0'},
>     [],
>     <Element 'machine' at 0x2369250>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.2.0'},
>     [],
>     <Element 'domain' at 0x2369290>,
>     {   '_children': [],
>         'attrib': {   'type': 'qemu'},
>         'tag': 'domain',
>         'tail': '\n      '},
>     [],
>     <Element 'domain' at 0x23692d0>,
>     {   '_children': [<Element 'emulator' at 0x2369310>],
>         'attrib': {   'type': 'kvm'},
>         'tag': 'domain',
>         'tail': '\n    ',
>         'text': '\n        '},
>     [<Element 'emulator' at 0x2369310>],
>     <Element 'emulator' at 0x2369310>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'emulator',
>         'tail': '\n      ',
>         'text': '/usr/libexec/qemu-kvm'},
>     [],
>     <Element 'features' at 0x2369390>,
>     {   '_children': [   <Element 'cpuselection' at 0x23693d0>,
>                          <Element 'deviceboot' at 0x2369410>,
>                          <Element 'disksnapshot' at 0x2369450>,
>                          <Element 'acpi' at 0x2369490>,
>                          <Element 'apic' at 0x23694d0>,
>                          <Element 'pae' at 0x2369510>,
>                          <Element 'nonpae' at 0x2369550>],
>         'attrib': {   },
>         'tag': 'features',
>         'tail': '\n  ',
>         'text': '\n      '},
>     [   <Element 'cpuselection' at 0x23693d0>,
>         <Element 'deviceboot' at 0x2369410>,
>         <Element 'disksnapshot' at 0x2369450>,
>         <Element 'acpi' at 0x2369490>,
>         <Element 'apic' at 0x23694d0>,
>         <Element 'pae' at 0x2369510>,
>         <Element 'nonpae' at 0x2369550>],
>     <Element 'cpuselection' at 0x23693d0>,
>     {   '_children': [], 'attrib': {   }, 'tag': 'cpuselection',
> 'tail': '\n      '},
>     [],
>     <Element 'deviceboot' at 0x2369410>,
>     {   '_children': [], 'attrib': {   }, 'tag': 'deviceboot', 'tail':
> '\n      '},
>     [],
>     <Element 'disksnapshot' at 0x2369450>,
>     {   '_children': [],
>         'attrib': {   'default': 'on', 'toggle': 'no'},
>         'tag': 'disksnapshot',
>         'tail': '\n      '},
>     [],
>     <Element 'acpi' at 0x2369490>,
>     {   '_children': [],
>         'attrib': {   'default': 'on', 'toggle': 'yes'},
>         'tag': 'acpi',
>         'tail': '\n      '},
>     [],
>     <Element 'apic' at 0x23694d0>,
>     {   '_children': [],
>         'attrib': {   'default': 'on', 'toggle': 'no'},
>         'tag': 'apic',
>         'tail': '\n      '},
>     [],
>     <Element 'pae' at 0x2369510>,
>     {   '_children': [], 'attrib': {   }, 'tag': 'pae', 'tail': '\n      '},
>     [],
>     <Element 'nonpae' at 0x2369550>,
>     {   '_children': [], 'attrib': {   }, 'tag': 'nonpae', 'tail': '\n    '},
>     [],
>     <Element 'guest' at 0x2369590>,
>     {   '_children': [   <Element 'os_type' at 0x23695d0>,
>                          <Element 'arch' at 0x2369610>,
>                          <Element 'features' at 0x2369bd0>],
>         'attrib': {   },
>         'tag': 'guest',
>         'tail': '\n\n',
>         'text': '\n    '},
>     [   <Element 'os_type' at 0x23695d0>,
>         <Element 'arch' at 0x2369610>,
>         <Element 'features' at 0x2369bd0>],
>     <Element 'os_type' at 0x23695d0>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'os_type',
>         'tail': '\n    ',
>         'text': 'hvm'},
>     [],
>     <Element 'arch' at 0x2369610>,
>     {   '_children': [   <Element 'wordsize' at 0x2369650>,
>                          <Element 'emulator' at 0x2369690>,
>                          <Element 'machine' at 0x2369710>,
>                          <Element 'machine' at 0x2369750>,
>                          <Element 'machine' at 0x2369790>,
>                          <Element 'machine' at 0x23697d0>,
>                          <Element 'machine' at 0x2369810>,
>                          <Element 'machine' at 0x2369850>,
>                          <Element 'machine' at 0x2369890>,
>                          <Element 'machine' at 0x23698d0>,
>                          <Element 'machine' at 0x2369910>,
>                          <Element 'machine' at 0x2369950>,
>                          <Element 'machine' at 0x2369990>,
>                          <Element 'machine' at 0x23699d0>,
>                          <Element 'machine' at 0x2369a10>,
>                          <Element 'machine' at 0x2369a50>,
>                          <Element 'machine' at 0x2369a90>,
>                          <Element 'domain' at 0x2369ad0>,
>                          <Element 'domain' at 0x2369b10>],
>         'attrib': {   'name': 'x86_64'},
>         'tag': 'arch',
>         'tail': '\n    ',
>         'text': '\n      '},
>     [   <Element 'wordsize' at 0x2369650>,
>         <Element 'emulator' at 0x2369690>,
>         <Element 'machine' at 0x2369710>,
>         <Element 'machine' at 0x2369750>,
>         <Element 'machine' at 0x2369790>,
>         <Element 'machine' at 0x23697d0>,
>         <Element 'machine' at 0x2369810>,
>         <Element 'machine' at 0x2369850>,
>         <Element 'machine' at 0x2369890>,
>         <Element 'machine' at 0x23698d0>,
>         <Element 'machine' at 0x2369910>,
>         <Element 'machine' at 0x2369950>,
>         <Element 'machine' at 0x2369990>,
>         <Element 'machine' at 0x23699d0>,
>         <Element 'machine' at 0x2369a10>,
>         <Element 'machine' at 0x2369a50>,
>         <Element 'machine' at 0x2369a90>,
>         <Element 'domain' at 0x2369ad0>,
>         <Element 'domain' at 0x2369b10>],
>     <Element 'wordsize' at 0x2369650>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'wordsize',
>         'tail': '\n      ',
>         'text': '64'},
>     [],
>     <Element 'emulator' at 0x2369690>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'emulator',
>         'tail': '\n      ',
>         'text': '/usr/libexec/qemu-kvm'},
>     [],
>     <Element 'machine' at 0x2369710>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-i440fx-rhel7.2.0'},
>     [],
>     <Element 'machine' at 0x2369750>,
>     {   '_children': [],
>         'attrib': {   'canonical': 'pc-i440fx-rhel7.2.0', 'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc'},
>     [],
>     <Element 'machine' at 0x2369790>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-i440fx-rhel7.0.0'},
>     [],
>     <Element 'machine' at 0x23697d0>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-q35-rhel7.1.0'},
>     [],
>     <Element 'machine' at 0x2369810>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.3.0'},
>     [],
>     <Element 'machine' at 0x2369850>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-q35-rhel7.2.0'},
>     [],
>     <Element 'machine' at 0x2369890>,
>     {   '_children': [],
>         'attrib': {   'canonical': 'pc-q35-rhel7.2.0', 'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'q35'},
>     [],
>     <Element 'machine' at 0x23698d0>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.4.0'},
>     [],
>     <Element 'machine' at 0x2369910>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.0.0'},
>     [],
>     <Element 'machine' at 0x2369950>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-i440fx-rhel7.1.0'},
>     [],
>     <Element 'machine' at 0x2369990>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.5.0'},
>     [],
>     <Element 'machine' at 0x23699d0>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.6.0'},
>     [],
>     <Element 'machine' at 0x2369a10>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.1.0'},
>     [],
>     <Element 'machine' at 0x2369a50>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'pc-q35-rhel7.0.0'},
>     [],
>     <Element 'machine' at 0x2369a90>,
>     {   '_children': [],
>         'attrib': {   'maxCpus': '240'},
>         'tag': 'machine',
>         'tail': '\n      ',
>         'text': 'rhel6.2.0'},
>     [],
>     <Element 'domain' at 0x2369ad0>,
>     {   '_children': [],
>         'attrib': {   'type': 'qemu'},
>         'tag': 'domain',
>         'tail': '\n      '},
>     [],
>     <Element 'domain' at 0x2369b10>,
>     {   '_children': [<Element 'emulator' at 0x2369b50>],
>         'attrib': {   'type': 'kvm'},
>         'tag': 'domain',
>         'tail': '\n    ',
>         'text': '\n        '},
>     [<Element 'emulator' at 0x2369b50>],
>     <Element 'emulator' at 0x2369b50>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'emulator',
>         'tail': '\n      ',
>         'text': '/usr/libexec/qemu-kvm'},
>     [],
>     <Element 'features' at 0x2369bd0>,
>     {   '_children': [   <Element 'cpuselection' at 0x2369c10>,
>                          <Element 'deviceboot' at 0x2369c50>,
>                          <Element 'disksnapshot' at 0x2369c90>,
>                          <Element 'acpi' at 0x2369cd0>,
>                          <Element 'apic' at 0x2369d10>],
>         'attrib': {   },
>         'tag': 'features',
>         'tail': '\n  ',
>         'text': '\n      '},
>     [   <Element 'cpuselection' at 0x2369c10>,
>         <Element 'deviceboot' at 0x2369c50>,
>         <Element 'disksnapshot' at 0x2369c90>,
>         <Element 'acpi' at 0x2369cd0>,
>         <Element 'apic' at 0x2369d10>],
>     <Element 'cpuselection' at 0x2369c10>,
>     {   '_children': [], 'attrib': {   }, 'tag': 'cpuselection',
> 'tail': '\n      '},
>     [],
>     <Element 'deviceboot' at 0x2369c50>,
>     {   '_children': [], 'attrib': {   }, 'tag': 'deviceboot', 'tail':
> '\n      '},
>     [],
>     <Element 'disksnapshot' at 0x2369c90>,
>     {   '_children': [],
>         'attrib': {   'default': 'on', 'toggle': 'no'},
>         'tag': 'disksnapshot',
>         'tail': '\n      '},
>     [],
>     <Element 'acpi' at 0x2369cd0>,
>     {   '_children': [],
>         'attrib': {   'default': 'on', 'toggle': 'yes'},
>         'tag': 'acpi',
>         'tail': '\n      '},
>     [],
>     <Element 'apic' at 0x2369d10>,
>     {   '_children': [],
>         'attrib': {   'default': 'on', 'toggle': 'no'},
>         'tag': 'apic',
>         'tail': '\n    '},
>     [],
>     [],
>     (<type '_ctypes.Array'>,),
>     {   '__dict__': <attribute '__dict__' of 'c_char_Array_60' objects>,
>         '__doc__': None,
>         '__module__': 'vdsm.netlink',
>         '__weakref__': <attribute '__weakref__' of 'c_char_Array_60' objects>,
>         '_length_': 60,
>         '_type_': <class 'ctypes.c_char'>,
>         'raw': <attribute 'raw' of 'c_char_Array_60' objects>,
>         'value': <attribute 'value' of 'c_char_Array_60' objects>},
>     <class 'vdsm.netlink.c_char_Array_60'>,
>     <attribute '__dict__' of 'c_char_Array_60' objects>,
>     <attribute '__weakref__' of 'c_char_Array_60' objects>,
>     (   <class 'vdsm.netlink.c_char_Array_60'>,
>         <type '_ctypes.Array'>,
>         <type '_ctypes._CData'>,
>         <type 'object'>),
>     <attribute 'raw' of 'c_char_Array_60' objects>,
>     <attribute 'value' of 'c_char_Array_60' objects>,
>     <xml.etree.ElementTree.XMLParser object at 0x2357f50>,
>     {   '_doctype': None,
>         '_error': <class 'xml.parsers.expat.ExpatError'>,
>         '_names': {   u'acpi': 'acpi',
>                       u'apic': 'apic',
>                       u'arch': 'arch',
>                       u'baselabel': 'baselabel',
>                       u'canonical': 'canonical',
>                       u'capabilities': 'capabilities',
>                       u'cell': 'cell',
>                       u'cells': 'cells',
>                       u'core_id': 'core_id',
>                       u'cores': 'cores',
>                       u'cpu': 'cpu',
>                       u'cpus': 'cpus',
>                       u'cpuselection': 'cpuselection',
>                       u'default': 'default',
>                       u'deviceboot': 'deviceboot',
>                       u'disksnapshot': 'disksnapshot',
>                       u'distances': 'distances',
>                       u'doi': 'doi',
>                       u'domain': 'domain',
>                       u'emulator': 'emulator',
>                       u'feature': 'feature',
>                       u'features': 'features',
>                       u'guest': 'guest',
>                       u'host': 'host',
>                       u'id': 'id',
>                       u'live': 'live',
>                       u'machine': 'machine',
>                       u'maxCpus': 'maxCpus',
>                       u'memory': 'memory',
>                       u'migration_features': 'migration_features',
>                       u'model': 'model',
>                       u'name': 'name',
>                       u'nonpae': 'nonpae',
>                       u'num': 'num',
>                       u'os_type': 'os_type',
>                       u'pae': 'pae',
>                       u'pages': 'pages',
>                       u'power_management': 'power_management',
>                       u'secmodel': 'secmodel',
>                       u'sibling': 'sibling',
>                       u'siblings': 'siblings',
>                       u'size': 'size',
>                       u'socket_id': 'socket_id',
>                       u'sockets': 'sockets',
>                       u'suspend_disk': 'suspend_disk',
>                       u'suspend_hybrid': 'suspend_hybrid',
>                       u'suspend_mem': 'suspend_mem',
>                       u'threads': 'threads',
>                       u'toggle': 'toggle',
>                       u'topology': 'topology',
>                       u'type': 'type',
>                       u'unit': 'unit',
>                       u'uri_transport': 'uri_transport',
>                       u'uri_transports': 'uri_transports',
>                       u'uuid': 'uuid',
>                       u'value': 'value',
>                       u'vendor': 'vendor',
>                       u'wordsize': 'wordsize'},
>         '_target': <xml.etree.ElementTree.TreeBuilder object at 0x2357f10>,
>         'entity': {   },
>         'parser': <pyexpat.xmlparser object at 0x22e5360>,
>         'version': 'Expat 2.1.0'},
>     <pyexpat.xmlparser object at 0x22e5360>,
>     <xml.etree.ElementTree.TreeBuilder object at 0x2357f10>,
>     <bound method XMLParser._start_list of
> <xml.etree.ElementTree.XMLParser object at 0x2357f50>>,
>     <bound method XMLParser._end of <xml.etree.ElementTree.XMLParser
> object at 0x2357f50>>,
>     <bound method XMLParser._pi of <xml.etree.ElementTree.XMLParser
> object at 0x2357f50>>,
>     <bound method XMLParser._data of <xml.etree.ElementTree.XMLParser
> object at 0x2357f50>>,
>     <bound method XMLParser._comment of
> <xml.etree.ElementTree.XMLParser object at 0x2357f50>>,
>     <bound method XMLParser._default of
> <xml.etree.ElementTree.XMLParser object at 0x2357f50>>,
>     {   '_data': [],
>         '_elem': [],
>         '_factory': <class 'xml.etree.ElementTree.Element'>,
>         '_last': <Element 'capabilities' at 0x2357f90>,
>         '_tail': 1},
>     [],
>     <Element 'cell' at 0x23646d0>,
>     <Element 'cells' at 0x2364690>,
>     <Element 'topology' at 0x2364650>,
>     <Element 'host' at 0x2357fd0>,
>     <Element 'capabilities' at 0x2357f90>,
>     {   '_children': [   <Element 'memory' at 0x2364710>,
>                          <Element 'pages' at 0x2364750>,
>                          <Element 'pages' at 0x2364790>,
>                          <Element 'distances' at 0x23647d0>,
>                          <Element 'cpus' at 0x2364850>],
>         'attrib': {   'id': '0'},
>         'tag': 'cell',
>         'tail': '\n      ',
>         'text': '\n          '},
>     {   '_children': [<Element 'cell' at 0x23646d0>],
>         'attrib': {   'num': '1'},
>         'tag': 'cells',
>         'tail': '\n    ',
>         'text': '\n        '},
>     {   '_children': [<Element 'cells' at 0x2364690>],
>         'attrib': {   },
>         'tag': 'topology',
>         'tail': '\n    ',
>         'text': '\n      '},
>     {   '_children': [   <Element 'uuid' at 0x2364050>,
>                          <Element 'cpu' at 0x2364090>,
>                          <Element 'power_management' at 0x2364390>,
>                          <Element 'migration_features' at 0x23644d0>,
>                          <Element 'topology' at 0x2364650>,
>                          <Element 'secmodel' at 0x2364a90>,
>                          <Element 'secmodel' at 0x2364bd0>],
>         'attrib': {   },
>         'tag': 'host',
>         'tail': '\n\n  ',
>         'text': '\n    '},
>     {   '_children': [   <Element 'host' at 0x2357fd0>,
>                          <Element 'guest' at 0x2364d10>,
>                          <Element 'guest' at 0x2369590>],
>         'attrib': {   },
>         'tag': 'capabilities',
>         'text': '\n\n  '},
>     [   <Element 'memory' at 0x2364710>,
>         <Element 'pages' at 0x2364750>,
>         <Element 'pages' at 0x2364790>,
>         <Element 'distances' at 0x23647d0>,
>         <Element 'cpus' at 0x2364850>],
>     [<Element 'cell' at 0x23646d0>],
>     [<Element 'cells' at 0x2364690>],
>     [   <Element 'uuid' at 0x2364050>,
>         <Element 'cpu' at 0x2364090>,
>         <Element 'power_management' at 0x2364390>,
>         <Element 'migration_features' at 0x23644d0>,
>         <Element 'topology' at 0x2364650>,
>         <Element 'secmodel' at 0x2364a90>,
>         <Element 'secmodel' at 0x2364bd0>],
>     [   <Element 'host' at 0x2357fd0>,
>         <Element 'guest' at 0x2364d10>,
>         <Element 'guest' at 0x2369590>],
>     <Element 'migration_features' at 0x23644d0>,
>     <Element 'power_management' at 0x2364390>,
>     <Element 'cpu' at 0x2364090>,
>     <Element 'uuid' at 0x2364050>,
>     {   '_children': [   <Element 'live' at 0x2364510>,
>                          <Element 'uri_transports' at 0x2364590>],
>         'attrib': {   },
>         'tag': 'migration_features',
>         'tail': '\n    ',
>         'text': '\n      '},
>     {   '_children': [   <Element 'suspend_mem' at 0x23643d0>,
>                          <Element 'suspend_disk' at 0x2364410>,
>                          <Element 'suspend_hybrid' at 0x2364490>],
>         'attrib': {   },
>         'tag': 'power_management',
>         'tail': '\n    ',
>         'text': '\n      '},
>     {   '_children': [   <Element 'arch' at 0x23640d0>,
>                          <Element 'model' at 0x2364110>,
>                          <Element 'vendor' at 0x2364150>,
>                          <Element 'topology' at 0x2364190>,
>                          <Element 'feature' at 0x23641d0>,
>                          <Element 'feature' at 0x2364210>,
>                          <Element 'feature' at 0x2364250>,
>                          <Element 'feature' at 0x2364290>,
>                          <Element 'feature' at 0x23642d0>,
>                          <Element 'pages' at 0x2364310>,
>                          <Element 'pages' at 0x2364350>],
>         'attrib': {   },
>         'tag': 'cpu',
>         'tail': '\n    ',
>         'text': '\n      '},
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'uuid',
>         'tail': '\n    ',
>         'text': 'a920d963-c18f-4fda-aa5a-deee9bbcde9d'},
>     [<Element 'live' at 0x2364510>, <Element 'uri_transports' at 0x2364590>],
>     [   <Element 'suspend_mem' at 0x23643d0>,
>         <Element 'suspend_disk' at 0x2364410>,
>         <Element 'suspend_hybrid' at 0x2364490>],
>     [   <Element 'arch' at 0x23640d0>,
>         <Element 'model' at 0x2364110>,
>         <Element 'vendor' at 0x2364150>,
>         <Element 'topology' at 0x2364190>,
>         <Element 'feature' at 0x23641d0>,
>         <Element 'feature' at 0x2364210>,
>         <Element 'feature' at 0x2364250>,
>         <Element 'feature' at 0x2364290>,
>         <Element 'feature' at 0x23642d0>,
>         <Element 'pages' at 0x2364310>,
>         <Element 'pages' at 0x2364350>],
>     [],
>     <Element 'uri_transports' at 0x2364590>,
>     <Element 'live' at 0x2364510>,
>     <Element 'suspend_hybrid' at 0x2364490>,
>     <Element 'suspend_disk' at 0x2364410>,
>     <Element 'suspend_mem' at 0x23643d0>,
>     <Element 'pages' at 0x2364350>,
>     <Element 'pages' at 0x2364310>,
>     <Element 'feature' at 0x23642d0>,
>     <Element 'feature' at 0x2364290>,
>     <Element 'feature' at 0x2364250>,
>     <Element 'feature' at 0x2364210>,
>     <Element 'feature' at 0x23641d0>,
>     <Element 'topology' at 0x2364190>,
>     <Element 'vendor' at 0x2364150>,
>     <Element 'model' at 0x2364110>,
>     <Element 'arch' at 0x23640d0>,
>     {   '_children': [   <Element 'uri_transport' at 0x23645d0>,
>                          <Element 'uri_transport' at 0x2364610>],
>         'attrib': {   },
>         'tag': 'uri_transports',
>         'tail': '\n    ',
>         'text': '\n        '},
>     {   '_children': [], 'attrib': {   }, 'tag': 'live', 'tail': '\n      '},
>     {   '_children': [], 'attrib': {   }, 'tag': 'suspend_hybrid',
> 'tail': '\n    '},
>     {   '_children': [], 'attrib': {   }, 'tag': 'suspend_disk',
> 'tail': '\n      '},
>     {   '_children': [], 'attrib': {   }, 'tag': 'suspend_mem',
> 'tail': '\n      '},
>     {   '_children': [],
>         'attrib': {   'size': '2048', 'unit': 'KiB'},
>         'tag': 'pages',
>         'tail': '\n    '},
>     {   '_children': [],
>         'attrib': {   'size': '4', 'unit': 'KiB'},
>         'tag': 'pages',
>         'tail': '\n      '},
>     {   '_children': [],
>         'attrib': {   'name': 'ss'},
>         'tag': 'feature',
>         'tail': '\n      '},
>     {   '_children': [],
>         'attrib': {   'name': 'vmx'},
>         'tag': 'feature',
>         'tail': '\n      '},
>     {   '_children': [],
>         'attrib': {   'name': 'pcid'},
>         'tag': 'feature',
>         'tail': '\n      '},
>     {   '_children': [],
>         'attrib': {   'name': 'osxsave'},
>         'tag': 'feature',
>         'tail': '\n      '},
>     {   '_children': [],
>         'attrib': {   'name': 'hypervisor'},
>         'tag': 'feature',
>         'tail': '\n      '},
>     {   '_children': [],
>         'attrib': {   'cores': '1', 'sockets': '8', 'threads': '1'},
>         'tag': 'topology',
>         'tail': '\n      '},
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'vendor',
>         'tail': '\n      ',
>         'text': 'Intel'},
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'model',
>         'tail': '\n      ',
>         'text': 'IvyBridge'},
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'arch',
>         'tail': '\n      ',
>         'text': 'x86_64'},
>     [   <Element 'uri_transport' at 0x23645d0>,
>         <Element 'uri_transport' at 0x2364610>],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     [],
>     <Element 'uri_transport' at 0x2364610>,
>     <Element 'uri_transport' at 0x23645d0>,
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'uri_transport',
>         'tail': '\n      ',
>         'text': 'rdma'},
>     {   '_children': [],
>         'attrib': {   },
>         'tag': 'uri_transport',
>         'tail': '\n        ',
>         'text': 'tcp'},
>     [],
>     [],
>     (<type '_ctypes.Array'>,),
>     {   '__dict__': <attribute '__dict__' of 'c_char_Array_60' objects>,
>         '__doc__': None,
>         '__module__': 'vdsm.netlink',
>         '__weakref__': <attribute '__weakref__' of 'c_char_Array_60' objects>,
>         '_length_': 60,
>         '_type_': <class 'ctypes.c_char'>,
>         'raw': <attribute 'raw' of 'c_char_Array_60' objects>,
>         'value': <attribute 'value' of 'c_char_Array_60' objects>},
>     <class 'vdsm.netlink.c_char_Array_60'>,
>     <attribute '__dict__' of 'c_char_Array_60' objects>,
>     <attribute '__weakref__' of 'c_char_Array_60' objects>,
>     (   <class 'vdsm.netlink.c_char_Array_60'>,
>         <type '_ctypes.Array'>,
>         <type '_ctypes._CData'>,
>         <type 'object'>),
>     <attribute 'raw' of 'c_char_Array_60' objects>,
>     <attribute 'value' of 'c_char_Array_60' objects>]



More information about the Devel mailing list