<div dir="ltr">Thanks for your time on this Dan!<div><br></div><div>The output from the hostdev looks like it may be unparseable, so I'm hoping this is the issue (and that it can be easily remedied).</div><div><br></div><div>I've also create a log of the other items you asked for, available at: <a href="https://www.dropbox.com/s/qh0yw1ptpivpatm/typescript?dl=0">https://www.dropbox.com/s/qh0yw1ptpivpatm/typescript?dl=0</a></div><div><br></div><div><div>[root@ovirt01 vdsm]# vdsm-tool restore-nets</div><div><device></div><div> <name>computer</name></div><div> <capability type='system'></div><div> <product>���������������������������������</product></div><div> <hardware></div><div> <vendor>���������������������������������</vendor></div><div> <version>���������������������������������</version></div><div> <serial>���������������������������������</serial></div><div> <uuid>d6a3e3c1-c5cb-42e9-a54c-ff8d0df91722</uuid></div><div> </hardware></div><div> <firmware></div><div> <vendor>Intel Corp.</vendor></div><div> <version>FYBYT10H.86A.0052.2015.0923.1845</version></div><div> <release_date>09/23/2015</release_date></div><div> </firmware></div><div> </capability></div><div></device></div><div><br></div><div>Traceback (most recent call last):</div><div> File "/usr/share/vdsm/vdsm-restore-net-config", line 429, in <module></div><div> restore(args)</div><div> File "/usr/share/vdsm/vdsm-restore-net-config", line 387, in restore</div><div> _restore_sriov_numvfs()</div><div> File "/usr/share/vdsm/vdsm-restore-net-config", line 82, in _restore_sriov_numvfs</div><div> sriov_devices = _get_sriov_devices()</div><div> File "/usr/share/vdsm/vdsm-restore-net-config", line 54, in _get_sriov_devices</div><div> devices = hostdev.list_by_caps()</div><div> File "/usr/share/vdsm/hostdev.py", line 177, in list_by_caps</div><div> libvirt_devices = _get_devices_from_libvirt()</div><div> File "/usr/share/vdsm/hostdev.py", line 162, in _get_devices_from_libvirt</div><div> for device in libvirtconnection.get().listAllDevices(0))</div><div> File "/usr/share/vdsm/hostdev.py", line 162, in <genexpr></div><div> for device in libvirtconnection.get().listAllDevices(0))</div><div> File "/usr/share/vdsm/hostdev.py", line 109, in _parse_device_params</div><div> devXML = etree.fromstring(device_xml)</div><div> File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1300, in XML</div><div> parser.feed(text)</div><div> File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed</div><div> self._raiseerror(v)</div><div> File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror</div><div> raise err</div><div>xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 4, column 13</div><div>Traceback (most recent call last):</div><div> File "/bin/vdsm-tool", line 219, in main</div><div> return tool_command[cmd]["command"](*args)</div><div> File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 41, in restore_command</div><div> exec_restore(cmd)</div><div> File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line 54, in exec_restore</div><div> raise EnvironmentError('Failed to restore the persisted networks')</div><div>EnvironmentError: Failed to restore the persisted networks</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 7, 2016 at 10:37 AM, Dan Kenigsberg <span dir="ltr"><<a href="mailto:danken@redhat.com" target="_blank">danken@redhat.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Mon, Mar 07, 2016 at 03:16:06PM +0200, Yedidyah Bar David wrote:<br>
> On Mon, Mar 7, 2016 at 3:01 PM, Jonathan Sherman <<a href="mailto:haviland@gmail.com">haviland@gmail.com</a>> wrote:<br>
> > Didi -<br>
> ><br>
> > The vdsmd service error is what led me to the vdsm-tool restore-nets<br>
> > concern. See the chain of errors below (along with the other details you<br>
> > requested.)<br>
><br>
> Thanks.<br>
><br>
> Changing subject and adding Dan.<br>
<br>
> ><br>
</span><div><div class="h5">> > [root@ovirt01 log]# vdsm-tool restore-nets<br>
> > Traceback (most recent call last):<br>
> > File "/usr/share/vdsm/vdsm-restore-net-config", line 429, in <module><br>
> > restore(args)<br>
> > File "/usr/share/vdsm/vdsm-restore-net-config", line 387, in restore<br>
> > _restore_sriov_numvfs()<br>
> > File "/usr/share/vdsm/vdsm-restore-net-config", line 82, in<br>
> > _restore_sriov_numvfs<br>
> > sriov_devices = _get_sriov_devices()<br>
> > File "/usr/share/vdsm/vdsm-restore-net-config", line 54, in<br>
> > _get_sriov_devices<br>
> > devices = hostdev.list_by_caps()<br>
> > File "/usr/share/vdsm/hostdev.py", line 175, in list_by_caps<br>
> > libvirt_devices = _get_devices_from_libvirt()<br>
> > File "/usr/share/vdsm/hostdev.py", line 160, in _get_devices_from_libvirt<br>
> > for device in libvirtconnection.get().listAllDevices(0))<br>
> > File "/usr/share/vdsm/hostdev.py", line 160, in <genexpr><br>
> > for device in libvirtconnection.get().listAllDevices(0))<br>
> > File "/usr/share/vdsm/hostdev.py", line 107, in _parse_device_params<br>
> > devXML = etree.fromstring(device_xml)<br>
> > File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1300, in XML<br>
> > parser.feed(text)<br>
> > File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed<br>
> > self._raiseerror(v)<br>
> > File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in<br>
> > _raiseerror<br>
> > raise err<br>
> > xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 4,<br>
> > column 13<br>
> > Traceback (most recent call last):<br>
> > File "/bin/vdsm-tool", line 219, in main<br>
> > return tool_command[cmd]["command"](*args)<br>
> > File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line<br>
> > 41, in restore_command<br>
> > exec_restore(cmd)<br>
> > File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line<br>
> > 54, in exec_restore<br>
> > raise EnvironmentError('Failed to restore the persisted networks')<br>
> > EnvironmentError: Failed to restore the persisted networks<br>
<br>
</div></div>Jonathan, thanks for this disturbing report. For some reason, Vdsm fails<br>
to parse libvirt's output.<br>
<br>
Could you share your libvirt version as well the output of<br>
for d in `virsh -r nodedev-list`; do echo ====$d====; virsh -r nodedev-dumpxml $d; done<br>
<br>
if all seems like valid xml, can you add the following print() and run<br>
`restore-net` again?<br>
<br>
diff --git a/lib/vdsm/hostdev.py b/lib/vdsm/hostdev.py<br>
index 6ba3189..9d22e9e 100644<br>
--- a/lib/vdsm/hostdev.py<br>
+++ b/lib/vdsm/hostdev.py<br>
@@ -113,6 +113,7 @@ def _parse_device_params(device_xml):<br>
Process device_xml and return dict of found known parameters,<br>
also doing sysfs lookups for sr-iov related information<br>
"""<br>
+ print(device_xml)<br>
address_parser = {'pci': _parse_pci_address,<br>
'scsi': _parse_scsi_address,<br>
'usb_device': _parse_usb_address}<br>
<br>
Regards,<br>
Dan.<br>
</blockquote></div><br></div>