The SMBIOS settings were indeed indeed the issue that was blocking me.  I investigated how to configure the SMBIOS settings and now restore-nets works, and I'm getting past where I was failing on the hosted-engine --deploy.

FYI, I had to download the "Intel Integrator Toolkit" (which is now EOL) and create a custom BIOS to add those settings in for my system, which is an Intel NUC DN2820FYKH.

I am happy to back this out and test any changes if you'd like, but this has gotten me to where I can continue oVirt testing for now.  I'll likely be reinstalling a few times along the way to polish my documentation, so let me know if you want me to revert my BIOS and test anything.

Thanks all!
-js

On Mon, Mar 7, 2016 at 11:22 AM, Martin Polednik <mpolednik@redhat.com> wrote:
On 07/03/16 11:09 -0500, Jonathan Sherman wrote:
Thanks for your time on this Dan!

The output from the hostdev looks like it may be unparseable, so I'm hoping
this is the issue (and that it can be easily remedied).

I've also create a log of the other items you asked for, available at:
https://www.dropbox.com/s/qh0yw1ptpivpatm/typescript?dl=0

[root@ovirt01 vdsm]# vdsm-tool restore-nets
<device>
 <name>computer</name>
 <capability type='system'>
   <product>���������������������������������</product>
   <hardware>
     <vendor>���������������������������������</vendor>
     <version>���������������������������������</version>
     <serial>���������������������������������</serial>
     <uuid>d6a3e3c1-c5cb-42e9-a54c-ff8d0df91722</uuid>
   </hardware>
   <firmware>
     <vendor>Intel Corp.</vendor>
     <version>FYBYT10H.86A.0052.2015.0923.1845</version>
     <release_date>09/23/2015</release_date>
   </firmware>
 </capability>
</device>

That seems to be the issue. Even bigger issue is that we can
not skip this device easily, as it is the root of device tree and must
be present in database.

I can think of logging the exception but letting the call go through
and create a hook to fake a (minimal) device tree. Dan, what do you think?

mpolednik


Traceback (most recent call last):
 File "/usr/share/vdsm/vdsm-restore-net-config", line 429, in <module>
   restore(args)
 File "/usr/share/vdsm/vdsm-restore-net-config", line 387, in restore
   _restore_sriov_numvfs()
 File "/usr/share/vdsm/vdsm-restore-net-config", line 82, in
_restore_sriov_numvfs
   sriov_devices = _get_sriov_devices()
 File "/usr/share/vdsm/vdsm-restore-net-config", line 54, in
_get_sriov_devices
   devices = hostdev.list_by_caps()
 File "/usr/share/vdsm/hostdev.py", line 177, in list_by_caps
   libvirt_devices = _get_devices_from_libvirt()
 File "/usr/share/vdsm/hostdev.py", line 162, in _get_devices_from_libvirt
   for device in libvirtconnection.get().listAllDevices(0))
 File "/usr/share/vdsm/hostdev.py", line 162, in <genexpr>
   for device in libvirtconnection.get().listAllDevices(0))
 File "/usr/share/vdsm/hostdev.py", line 109, in _parse_device_params
   devXML = etree.fromstring(device_xml)
 File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1300, in XML
   parser.feed(text)
 File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
   self._raiseerror(v)
 File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in
_raiseerror
   raise err
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 4,
column 13
Traceback (most recent call last):
 File "/bin/vdsm-tool", line 219, in main
   return tool_command[cmd]["command"](*args)
 File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line
41, in restore_command
   exec_restore(cmd)
 File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py", line
54, in exec_restore
   raise EnvironmentError('Failed to restore the persisted networks')
EnvironmentError: Failed to restore the persisted networks

On Mon, Mar 7, 2016 at 10:37 AM, Dan Kenigsberg <danken@redhat.com> wrote:

On Mon, Mar 07, 2016 at 03:16:06PM +0200, Yedidyah Bar David wrote:
> On Mon, Mar 7, 2016 at 3:01 PM, Jonathan Sherman <haviland@gmail.com>
wrote:
> > Didi -
> >
> > The vdsmd service error is what led me to the vdsm-tool restore-nets
> > concern.  See the chain of errors below (along with the other details
you
> > requested.)
>
> Thanks.
>
> Changing subject and adding Dan.

> >
> > [root@ovirt01 log]# vdsm-tool restore-nets
> > Traceback (most recent call last):
> >   File "/usr/share/vdsm/vdsm-restore-net-config", line 429, in <module>
> >     restore(args)
> >   File "/usr/share/vdsm/vdsm-restore-net-config", line 387, in restore
> >     _restore_sriov_numvfs()
> >   File "/usr/share/vdsm/vdsm-restore-net-config", line 82, in
> > _restore_sriov_numvfs
> >     sriov_devices = _get_sriov_devices()
> >   File "/usr/share/vdsm/vdsm-restore-net-config", line 54, in
> > _get_sriov_devices
> >     devices = hostdev.list_by_caps()
> >   File "/usr/share/vdsm/hostdev.py", line 175, in list_by_caps
> >     libvirt_devices = _get_devices_from_libvirt()
> >   File "/usr/share/vdsm/hostdev.py", line 160, in
_get_devices_from_libvirt
> >     for device in libvirtconnection.get().listAllDevices(0))
> >   File "/usr/share/vdsm/hostdev.py", line 160, in <genexpr>
> >     for device in libvirtconnection.get().listAllDevices(0))
> >   File "/usr/share/vdsm/hostdev.py", line 107, in _parse_device_params
> >     devXML = etree.fromstring(device_xml)
> >   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1300, in
XML
> >     parser.feed(text)
> >   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in
feed
> >     self._raiseerror(v)
> >   File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in
> > _raiseerror
> >     raise err
> > xml.etree.ElementTree.ParseError: not well-formed (invalid token):
line 4,
> > column 13
> > Traceback (most recent call last):
> >   File "/bin/vdsm-tool", line 219, in main
> >     return tool_command[cmd]["command"](*args)
> >   File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py",
line
> > 41, in restore_command
> >     exec_restore(cmd)
> >   File "/usr/lib/python2.7/site-packages/vdsm/tool/restore_nets.py",
line
> > 54, in exec_restore
> >     raise EnvironmentError('Failed to restore the persisted networks')
> > EnvironmentError: Failed to restore the persisted networks

Jonathan, thanks for this disturbing report. For some reason, Vdsm fails
to parse libvirt's output.

Could you share your libvirt version as well the output of
    for d in `virsh -r nodedev-list`; do echo ====$d====; virsh -r
nodedev-dumpxml $d; done

if all seems like valid xml, can you add the following print() and run
`restore-net` again?

diff --git a/lib/vdsm/hostdev.py b/lib/vdsm/hostdev.py
index 6ba3189..9d22e9e 100644
--- a/lib/vdsm/hostdev.py
+++ b/lib/vdsm/hostdev.py
@@ -113,6 +113,7 @@ def _parse_device_params(device_xml):
     Process device_xml and return dict of found known parameters,
     also doing sysfs lookups for sr-iov related information
     """
+    print(device_xml)
     address_parser = {'pci': _parse_pci_address,
                       'scsi': _parse_scsi_address,
                       'usb_device': _parse_usb_address}

Regards,
Dan.