<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Courier New";
panose-1:2 7 3 9 2 2 5 2 4 4;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"MS Mincho";
panose-1:2 2 6 9 4 2 5 8 3 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
span.m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space
{mso-style-name:m_-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:175385002;
mso-list-type:hybrid;
mso-list-template-ids:-2033157306 1746011708 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style>
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">There were no other defined networks. I had tried doing the ‘sync network’ but in every case it resulted in the host losing connectivity, which it regained after a reboot. There’s nothing
exotic in the networking, just a single 10G interface and ovirtmgmt. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">My hunch is that initially selecting ‘ovs’ vs. ‘legacy’ switch type in my newly created cluster introduced something in the networking stack that couldn’t be recovered from. At least, not
without more knowledge of OVS. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">(Just a suggestion: I question the word choice of ‘legacy’ for the default switch type. “Legacy” usually denotes something very outdated and soon to be deprecated, but standard Linux bridging
(while limited) is perfectly effective in most cases and isn’t going away anytime soon. Whereas, OVS in oVirt is still pretty bleeding edge. And most people using oVirt (or vSphere, etc.) consider live migration an essential feature. It might be helpful to
put something in the release notes about this. As in, “unless you have specific reason to and know what you’re doing, don’t select OVS switch.”)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">So I went ahead did as follows:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Removed the oVirt Node 4 hosts from engine via (from web interface); wiped their disks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Problem not necessary but due to past experiences, removed:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">/etc/pki/ovirt-engine/requests/<oVirtNode4Hostnames>*<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">/etc/pki/ovirt-engine/certs/<oVirtNode4Hostnames>*<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">(Kept two oVirt Node 3.6 hosts (identical hardware and networking) running with VMs)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Removed the new oVirt 4 compatible cluster<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Re-installed oVirt Node 4 hosts (two) using ISO<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Configured basic host configuration within the Anaconda setup; rebooted and ran ‘yum update’. Did not customize anything; did not manually add ‘ovirtmgmt’ bridge.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Re-added hosts to oVirt engine via root password authentication, this time adding them to the existing (default) cluster which is in 3.6 compatibility mode<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">oVirt engine created the ‘ovirtmgmt’ bridge. Hosts activated fine and were able to run VMs.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">I did note that the DNS server that was in my original device ifcfg file (static IPv4 configiuration – setup in the Anaconda installer) was not transferred over to the ovirtmgmt bridge.
Consequently, I got errors about those two new hosts not being able to access the NFS based ISO and Export storage domains. I had to manually add “nameserver my.name.server.IP” to /etc/resolv.conf on those hosts. The original (default) /etc/resolv.conf file
looked like this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"># Generated by NetworkManager<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">search lab.ingram.io<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"># No nameservers found; try putting DNS servers into your<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"># ifcfg files in /etc/sysconfig/network-scripts like so:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">#<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"># DNS1=xxx.xxx.xxx.xxx<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"># DNS2=xxx.xxx.xxx.xxx<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"># DOMAIN=lab.foo.com bar.foo.com<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">I can start up a VM on one of the oVirt Node 3.6 hosts, live migrate it among 3.6 hosts, live migrate it to (and among) 4.x hosts, and even back to a 3.6 host.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Since then I’ve migrated everything to the 4.x hosts, removed the 3.6 hosts, upgraded the cluster compatibility mode (taking care not to upgrade switch from “legacy”), and upgraded the
data center to 4.0 mode. Everything’s working great. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">When I wipe and rebuild the 3.6 hosts, I’m going to try building one as a traditional/generic Centos 7 install rather than a “node” ISO install and compare.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Thanks for pointing me in the right direction!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><br>
Best,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri">Daniel<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Calibri"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-family:Calibri;color:black">From: </span>
</b><span style="font-family:Calibri;color:black">Ivan Bulatovic <combuster@gmail.com><br>
<b>Date: </b>Monday, November 7, 2016 at 11:49 PM<br>
<b>To: </b>"Beckman, Daniel" <Daniel.Beckman@ingramcontent.com><br>
<b>Cc: </b>Michal Skrivanek <michal.skrivanek@redhat.com>, users <users@ovirt.org><br>
<b>Subject: </b>Re: [ovirt-users] oVirt 4.04 Can't Migrate VMs - Cryptic "Migration Failed" Error<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Host configuration looks correct, but it shows that ovirtmgmt network is out of sync on the node, so first step should be to select the "Sync Network" in networking tab. I would try to delete all other defined networks for the cluster (if
there are any). If that doesn't work out, another round of engine and vdsm logs after a failed migration could help to find the culprit for this issue. If there are no other networks defined on the cluster level (other than ovirtmgmt) and vdsm still reports
that it can't find the ovs bridged interface and thus failing the lm, there could be a bug lurking somewhere.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Mon, Nov 7, 2016 at 4:29 PM, Beckman, Daniel <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank">Daniel.Beckman@ingramcontent.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">When I created the new cluster and selected 4.0 for Compatibility Mode, I may have initially selected OVS (vs. Legacy) for Switch
Type. But since then I had changed it back to Legacy. So could there be something left over from OVS? We definitely want to retain live migration.
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Here’s my networking configuration files:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">[root@labvmhostt05 network-scripts]# ls</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">ifcfg-em1 ifcfg-ovirtmgmt ifdown-eth ifdown-ovs ifdown-Team ifup-bnep ifup-isdn ifup-ppp ifup-tunnel</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">ifcfg-em2 ifcfg-p5p1 ifdown-ib ifdown-post ifdown-TeamPort ifup-eth ifup-ovs ifup-routes ifup-wireless</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">ifcfg-em3 ifcfg-p5p2 ifdown-ippp ifdown-ppp ifdown-tunnel ifup-ib ifup-plip ifup-sit init.ipv6-global</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">ifcfg-em4 ifdown ifdown-ipv6 ifdown-routes ifup ifup-ippp ifup-plusb ifup-Team network-functions</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">ifcfg-lo ifdown-bnep ifdown-isdn ifdown-sit ifup-aliases ifup-ipv6 ifup-post ifup-TeamPort network-functions-ipv6</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">[root@labvmhostt05 network-scripts]#
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">So “ifcfg-p5p1” is the single 10G Ethernet interface being used:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">DEVICE=p5p1</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">HWADDR=a0:36:9f:ba:00:3c</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">BRIDGE=ovirtmgmt</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">ONBOOT=yes</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">MTU=1500</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">NM_CONTROLLED=no</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">IPV6INIT=no</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">PEERDNS=no</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">And here is the bridge “ovirtmgmt”:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">DEVICE=ovirtmgmt</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">TYPE=Bridge</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">DELAY=0</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">STP=off</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">ONBOOT=yes</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">IPADDR=10.50.3.97</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">NETMASK=255.255.255.0</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">GATEWAY=10.50.3.1</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">BOOTPROTO=none</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">MTU=1500</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">DEFROUTE=yes</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">NM_CONTROLLED=no</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">IPV6INIT=no</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">HOTPLUG=no</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">PEERDNS=no</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">The other interfaces are unconfigured (just the default per Anaconda setup), set to ONBOOT=no, and not physically connected to
anything. I attached a screenshot of networking within the admin interface. No other interfaces are enabled for anything.
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Is there some remnant of OVS I need to remove?
</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thanks,</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Daniel</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-family:Calibri;color:black">From:
</span></b><span style="font-family:Calibri;color:black">Ivan Bulatovic <<a href="mailto:combuster@gmail.com" target="_blank">combuster@gmail.com</a>><br>
<b>Date: </b>Saturday, November 5, 2016 at 3:40 PM<br>
<b>To: </b>"Beckman, Daniel" <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank">Daniel.Beckman@ingramcontent.com</a>><br>
<b>Cc: </b>Michal Skrivanek <<a href="mailto:michal.skrivanek@redhat.com" target="_blank">michal.skrivanek@redhat.com</a>>, users <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
<b>Subject: </b>Re: [ovirt-users] oVirt 4.04 Can't Migrate VMs - Cryptic "Migration Failed" Error</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">After digging some more, culprit is that live migration is not yet supported for Open vSwitch network configuration. There is an open bug report to track the progress:<br>
<br>
<a href="https://bugzilla.redhat.com/show_bug.cgi?id=1362495" target="_blank">https://bugzilla.redhat.com/show_bug.cgi?id=1362495</a><o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I haven't had any experience with ovs networking on oVirt, but it should be fairly straightforward to switch from ovs until this is resolved (if lm is a requirement).<o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Sat, Nov 5, 2016 at 8:50 PM, Ivan Bulatovic <<a href="mailto:combuster@gmail.com" target="_blank">combuster@gmail.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt">libvirtError: Cannot get interface MTU on 'vdsmbr_YMv4HqCU': No such device<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">This is the error vdsm reports after a failed migration. You have two bridges defined, ovirtmgmt is up, vdmsbr_YMv4HqCU is down and it looks like a remnant of openvswitch configuration.
Can you remove or disable every other interface from Networking tab on affected hosts, and just leave ovirtmgmt bridge and underlying p5p1 interface 10Gbps interface configured? After that, make sure that ovirtmgmt network is tagged as a Live Migration network
as shown here:<br>
<a href="https://www.ovirt.org/develop/release-management/features/network/migration-network/" target="_blank">https://www.ovirt.org/develop/release-management/features/network/migration-network/</a><br>
<br>
Regarding communication issues I've mentioned, your nodes were unresponsive and I've seen from the engine logs several attempts of fencing the nodes just after the live migration failed to complete.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Fri, Nov 4, 2016 at 8:31 PM, Beckman, Daniel <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank">Daniel.Beckman@ingramcontent.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">I’m attaching vdsm.log from the three hyervisors running oVirt Node 4vdsd, captured just after several failed VM migrations. I
had stopped iptables on the hosts, and even stopped firewalld on the engine. Here is a snippet from one of the vdsm.log files:</span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/5::DEBUG::2016-11-04 14:09:49,699::API::591::vds::(migrationCreate) Migration create</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-834::DEBUG::2016-11-04 14:09:49,702::vm::692::virt.vm::(_startUnderlyingVm) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Acquiring
incoming migration semaphore.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/5::DEBUG::2016-11-04 14:09:49,703::vm::4385::virt.vm::(waitForMigrationDestinationPrepare) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::migration
destination: waiting for VM creation</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/5::DEBUG::2016-11-04 14:09:49,704::vm::4390::virt.vm::(waitForMigrationDestinationPrepare) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::migration
destination: waiting 48s for path preparation</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/5::ERROR::2016-11-04 14:09:49,745::vm::4365::virt.vm::(_updateDevicesDomxmlCache) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Alias
not found for device type redir during migration at destination host</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/5::ERROR::2016-11-04 14:09:49,745::vm::4365::virt.vm::(_updateDevicesDomxmlCache) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Alias
not found for device type redir during migration at destination host</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/5::ERROR::2016-11-04 14:09:49,745::vm::4365::virt.vm::(_updateDevicesDomxmlCache) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Alias
not found for device type redir during migration at destination host</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/5::ERROR::2016-11-04 14:09:49,745::vm::4365::virt.vm::(_updateDevicesDomxmlCache) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Alias
not found for device type redir during migration at destination host</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">and…</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-834::ERROR::2016-11-04 14:09:49,855::vm::759::virt.vm::(_startUnderlyingVm) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Failed
to start a migration destination vm</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-834::INFO::2016-11-04 14:09:49,857::vm::1308::virt.vm::(setDownStatus) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Changed
state to Down: VM failed to migrate (code=8)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-834::DEBUG::2016-11-04 14:09:49,857::__init__::208::jsonrpc.Notification::(emit) Sending event {"params": {"ee985ac2-1c53-40a1-a764-6e14799d28ad":
{"status": "Down", "timeOffset": "0", "exitReason": 8, "exitMessage": "VM failed to migrate", "exitCode": 1}, "notify_time": 4299218670}, "jsonrpc": "2.0", "method": "|virt|VM_status|ee985ac2-1c53-40a1-a764-6e14799d28ad"}</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-834::DEBUG::2016-11-04 14:09:49,858::vm::769::virt.vm::(_startUnderlyingVm) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Releasing
incoming migration semaphore</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/4::DEBUG::2016-11-04 14:09:51,607::__init__::554::jsonrpc.JsonRpcServer::(_handle_request) Return 'VM.getMigrationStatus'
in bridge with {'status': {'message': 'Fatal error during migration', 'code': 12}, 'progress': 0}</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">On another host:</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/6::DEBUG::2016-11-04 14:09:48,378::__init__::555::jsonrpc.JsonRpcServer::(_handle_request) Return 'VM.migrate'
in bridge with True</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/6::INFO::2016-11-04 14:09:48,378::__init__::513::jsonrpc.JsonRpcServer::(_serveRequest) RPC call VM.migrate succeeded
in 0.01 seconds</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::DEBUG::2016-11-04 14:09:49,649::migration::188::virt.vm::(_setupVdsConnection) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Initiating
connection with destination</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::DEBUG::2016-11-04 14:09:49,651::migration::200::virt.vm::(_setupVdsConnection) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Destination
server is: <a href="http://labvmhostt05.lab.ingram.io:54321" target="_blank">labvmhostt05.lab.ingram.io:54321</a></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::DEBUG::2016-11-04 14:09:49,653::migration::246::virt.vm::(_prepareGuest) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Migration
started</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::DEBUG::2016-11-04 14:09:49,662::guestagent::502::virt.vm::(send_lifecycle_event) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::send_lifecycle_event
before_migration called</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::DEBUG::2016-11-04 14:09:49,662::migration::353::virt.vm::(run) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::migration
semaphore acquired after 1 seconds</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::INFO::2016-11-04 14:09:49,754::migration::407::virt.vm::(_startUnderlyingMigration) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Creation
of destination VM took: 0 seconds</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::INFO::2016-11-04 14:09:49,754::migration::429::virt.vm::(_startUnderlyingMigration) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::starting
migration to qemu+tls://<a href="http://labvmhostt05.lab.ingram.io/system" target="_blank">labvmhostt05.lab.ingram.io/system</a> with miguri tcp://<a href="http://10.50.3.97" target="_blank">10.50.3.97</a></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::DEBUG::2016-11-04 14:09:49,754::migration::494::virt.vm::(_perform_with_conv_schedule) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::performing
migration with conv schedule</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-940::DEBUG::2016-11-04 14:09:49,754::migration::620::virt.vm::(run) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::starting
migration monitor thread</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-940::DEBUG::2016-11-04 14:09:49,755::migration::739::virt.vm::(_execute_action_with_params) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Setting
downtime to 100</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::DEBUG::2016-11-04 14:09:49,834::migration::715::virt.vm::(stop) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::stopping
migration monitor thread</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::ERROR::2016-11-04 14:09:49,834::migration::252::virt.vm::(_recover) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Cannot
get interface MTU on 'vdsmbr_YMv4HqCU': No such device</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-940::DEBUG::2016-11-04 14:09:49,835::migration::629::virt.vm::(run) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::stopped
migration monitor thread</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::DEBUG::2016-11-04 14:09:49,858::guestagent::502::virt.vm::(send_lifecycle_event) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::send_lifecycle_event
after_migration called</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thread-939::ERROR::2016-11-04 14:09:49,872::migration::381::virt.vm::(run) vmId=`ee985ac2-1c53-40a1-a764-6e14799d28ad`::Failed
to migrate</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> File "/usr/share/vdsm/virt/migration.py", line 363, in run</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> File "/usr/share/vdsm/virt/migration.py", line 436, in _startUnderlyingMigration</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> File "/usr/share/vdsm/virt/migration.py", line 496, in _perform_with_conv_schedule</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> self._perform_migration(duri, muri)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> File "/usr/share/vdsm/virt/migration.py", line 476, in _perform_migration</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> self._vm._dom.migrateToURI3(duri, params, flags)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1836, in migrateToURI3</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">jsonrpc.Executor/0::DEBUG::2016-11-04 14:09:59,301::__init__::555::jsonrpc.JsonRpcServer::(_handle_request) Return 'VM.getMigrationStatus'
in bridge with {'status': {'message': 'Fatal error during migration', 'code': 12}, 'progress': 0}</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Thanks,</span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri">Daniel</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-family:Calibri;color:black">From:
</span></b><span style="font-family:Calibri;color:black">Michal Skrivanek <<a href="mailto:michal.skrivanek@redhat.com" target="_blank">michal.skrivanek@redhat.com</a>><br>
<b>Date: </b>Friday, November 4, 2016 at 1:32 PM<br>
<b>To: </b>"Beckman, Daniel" <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank">Daniel.Beckman@ingramcontent.com</a>><br>
<b>Cc: </b>Ivan Bulatovic <<a href="mailto:combuster@gmail.com" target="_blank">combuster@gmail.com</a>>, users <<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>><br>
<b>Subject: </b>Re: [ovirt-users] oVirt 4.04 Can't Migrate VMs - Cryptic "Migration Failed" Error</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On 4 Nov 2016, at 19:18, Beckman, Daniel <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank">Daniel.Beckman@ingramcontent.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Well, I manually created the ovirtmgmt bridge and the hosts came back up fine. I can still run any VM on any oVirt Node 4 host, but I still can’t migrate VMs among them. Not sure what else to check.<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></span><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">can you please post vdsm.log including that migration failure?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">there was/is a known firewall service issue, on hosts. did you try to disable firewalld and enable iptables (or manually clear the rules and open it completely) on both source and
destination hosts?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">michal<o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"><o:p> </o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<b><span style="font-family:Calibri">From:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></span></b><span style="font-family:Calibri"><<a href="mailto:users-bounces@ovirt.org" target="_blank"><span style="color:purple">users-bounces@ovirt.org</span></a>>
on behalf of "Beckman, Daniel" <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank"><span style="color:purple">Daniel.Beckman@ingramcontent.com</span></a>><br>
<b>Date:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Friday, November 4, 2016 at 11:57 AM<br>
<b>To:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Ivan Bulatovic <<a href="mailto:combuster@gmail.com" target="_blank"><span style="color:purple">combuster@gmail.com</span></a>>, users <<a href="mailto:users@ovirt.org" target="_blank"><span style="color:purple">users@ovirt.org</span></a>><br>
<b>Subject:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Re: [ovirt-users] oVirt 4.04 Can't Migrate VMs - Cryptic "Migration Failed" Error</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">I haven’t fixed the problem but I’ve got a hunch.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">In oVirt Node 3.6, when you setup network interfaces (in the TUI) it creates a bridge, ovirtmgmt. In oVirt Node (“Next”) 4, there is no such procedure creating that bridge. The “ovirtmgmt” network shows up
in the web admin interface for an oVirt Node 4 host, but it shows as not being synced, and when you look on the host itself there is no interface file (ifcfg-ovirtmgmt). If I try to “sync” the network from the web admin portal, this succeeds in breaking communication
with the host altogether and I have to go power cycle it.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">I guess it’s back to manually editing ifcfg files for me. Brings back memories of RHEV 2.2…</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<b><span style="font-family:Calibri">From:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></span></b><span style="font-family:Calibri"><<a href="mailto:users-bounces@ovirt.org" target="_blank"><span style="color:purple">users-bounces@ovirt.org</span></a>>
on behalf of "Beckman, Daniel" <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank"><span style="color:purple">Daniel.Beckman@ingramcontent.com</span></a>><br>
<b>Date:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Friday, November 4, 2016 at 10:21 AM<br>
<b>To:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Ivan Bulatovic <<a href="mailto:combuster@gmail.com" target="_blank"><span style="color:purple">combuster@gmail.com</span></a>>, users <<a href="mailto:users@ovirt.org" target="_blank"><span style="color:purple">users@ovirt.org</span></a>><br>
<b>Subject:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Re: [ovirt-users] oVirt 4.04 Can't Migrate VMs - Cryptic "Migration Failed" Error</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Hi Ivan,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">What do you mean by communication issues? I was able to add them to the engine and everything looks good. There are no firewalls in between the hosts and the engine, and name resolution is working. These are
the same physical hosts used with no problems on oVirt Node 3.6, with the same networking. Is there something in particular I should look for?</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">I tried creating a new VM on that new 4.x cluster and got the same results; can’t migrate. I tried temporarily stopping iptables on the hosts and firewalld on the engine, with no impact. </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Here’s what vdsm service shows on one of the oVirt Node 4 hosts:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">[root@labvmhostt05 ~]# systemctl status vdsmd.service</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">● vdsmd.service - Virtual Desktop Server Manager</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor preset: enabled)</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> Active: active (running) since Thu 2016-11-03 12:44:11 CDT; 21h ago</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> Process: 4163 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh --pre-start (code=exited, status=0/SUCCESS)</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Main PID: 4247 (vdsm)</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> CGroup: /system.slice/vdsmd.service</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> <span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></span><span style="font-size:11.0pt;font-family:"MS Mincho"">├</span><span style="font-size:11.0pt;font-family:Calibri">─4247
/usr/bin/python /usr/share/vdsm/vdsm</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> <span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></span><span style="font-size:11.0pt;font-family:"MS Mincho"">├</span><span style="font-size:11.0pt;font-family:Calibri">─4428
/usr/libexec/ioprocess --read-pipe-fd 73 --write-pipe-fd 72 --max-threads 10 --max-queued-requests 10</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> <span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></span><span style="font-size:11.0pt;font-family:"MS Mincho"">├</span><span style="font-size:11.0pt;font-family:Calibri">─4461
/usr/libexec/ioprocess --read-pipe-fd 93 --write-pipe-fd 92 --max-threads 10 --max-queued-requests 10</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><span style="font-size:11.0pt;font-family:"MS Mincho"">├</span><span style="font-size:11.0pt;font-family:Calibri">─4469 /usr/libexec/ioprocess --read-pipe-fd 100 --write-pipe-fd 98 --max-threads
10 --max-queued-requests 10</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> └─4477 /usr/libexec/ioprocess --read-pipe-fd 109 --write-pipe-fd 108 --max-threads 10 --max-queued-requests 10</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 03 12:44:28<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm MOM WARNING MOM not available, KSM stats will be missing.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 03 12:44:28<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm root ERROR Report host stats failed</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> Traceback (most recent call last):</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> File "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 113, in report_stats...</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 03 12:44:28<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm MOM WARNING MOM not available, Policy could not be set.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 03 12:45:42<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm vds.dispatcher ERROR SSL error during reading data: unexpected eof</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 03 12:47:33<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm root WARNING File: /var/lib/libvirt/qemu/channels/cddd2019-5baf-4321-aa8b-130b2889d1b0.com.redh...y removed</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 03 12:47:33<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm root WARNING File: /var/lib/libvirt/qemu/channels/cddd2019-5baf-4321-aa8b-130b2889d1b0.org.qemu...y removed</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 03 12:51:30<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm root WARNING File: /var/lib/libvirt/qemu/channels/68d40cdf-344b-4750-9ddd-9cc5eaef18ec.com.redh...y removed</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 03 12:51:30<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm root WARNING File: /var/lib/libvirt/qemu/channels/68d40cdf-344b-4750-9ddd-9cc5eaef18ec.org.qemu...y removed</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 04 09:44:43<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm root WARNING File: /var/lib/libvirt/qemu/channels/ee985ac2-1c53-40a1-a764-6e14799d28ad.com.redh...y removed</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Nov 04 09:44:43<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span><a href="http://labvmhostt05.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt05.lab.ingram.io</span></a><span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span>vdsm[4247]:
vdsm root WARNING File: /var/lib/libvirt/qemu/channels/ee985ac2-1c53-40a1-a764-6e14799d28ad.org.qemu...y removed</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">I attached the vdsm.log from one of the oVirt Node 4 hosts.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Thanks,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri">Daniel</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt;font-family:Calibri"> </span><o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<b><span style="font-family:Calibri">From:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></span></b><span style="font-family:Calibri">Ivan Bulatovic <<a href="mailto:combuster@gmail.com" target="_blank"><span style="color:purple">combuster@gmail.com</span></a>><br>
<b>Date:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Thursday, November 3, 2016 at 6:34 PM<br>
<b>To:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>"Beckman, Daniel" <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank"><span style="color:purple">Daniel.Beckman@ingramcontent.com</span></a>><br>
<b>Subject:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Re: [ovirt-users] oVirt 4.04 Can't Migrate VMs - Cryptic "Migration Failed" Error</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<o:p></o:p></p>
</div>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt;background:white">
Hi Daniel,<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt;background:white">
from the engine log it seems that your nodes have communication issues with the engine. VDSM logs from the source and destination nodes after the failed migration (or after host fencing) should hold more information that can help you figure out why is this
happening. systemctl status vdsmd.service would be interesting also. I would also try to create a new VM on the cluster that contains Node 4 hosts and retry the migration procedure (if everything checks out regarding engine-node network communication).<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt;background:white">
Regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
Ivan<o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<o:p></o:p></p>
</div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
On Thu, Nov 3, 2016 at 7:52 PM, Beckman, Daniel <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank"><span style="color:purple">Daniel.Beckman@ingramcontent.com</span></a>> wrote:<o:p></o:p></p>
</div>
</div>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">I’ve attached the engine.log in compressed form.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">For whatever it’s worth, I tried moving back two of the oVirt Node 4 hosts to the original (default) cluster that is running in 3.6 compatibility mode. I get the same behavior within that cluster – the hosts running oVirt Node
4 can’t migrate VMs but can start and run them. The hosts running oVirt Node 3 within the same cluster can migrate VMs with no problems.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<b>From:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>"Beckman, Daniel" <<a href="mailto:Daniel.Beckman@ingramcontent.com" target="_blank"><span style="color:purple">Daniel.Beckman@ingramcontent.com</span></a>><br>
<b>Date:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>Thursday, November 3, 2016 at 12:40 PM<br>
<b>To:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>users <<a href="mailto:users@ovirt.org" target="_blank"><span style="color:purple">users@ovirt.org</span></a>><br>
<b>Subject:<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></b>oVirt 4.04 Can't Migrate VMs - Cryptic "Migration Failed" Error<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">I recently upgraded from oVirt 3.6 to 4.04. After upgrading the engine, I rebuilt a couple of the hosts with oVirt Node 4.0 and added them to a new cluster for testing in 4.0 compatibility mode.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">I can run any number of VMs on any of the hosts in this new cluster, by starting them up on a particular host (via ‘run once…”). But I cannot migrate a running VM to another host within the cluster. My networking setup is very
simple – just a single 10G</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">Interface with the default ‘ovirtmgmt’ and everything else running on it. The storage is NFS based. These are the same hosts that were able to migrate under 3.6 just fine. Most hosts are Dell PowerEdge R730 models; 1 is a HP Proliant
DL380 G8.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">My existing cluster of hosts running oVirt Node 3.6 continue to function OK in 3.6 compatibility mode. They can migrate VMs (within that cluster) just fine.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">When trying to migrate a VM, I get this terse error with no information: “Migration failed.”</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">Engine version:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">oVirt Engine Version: 4.0.4.4-1.el7.centos</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">oVirt Node version:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">OS Version:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">RHEL - 7 - 2.1511.el7.centos.2.10</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">OS Description:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">oVirt Node 4.0.3</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">Any suggestions? I was going to attach my engine.log but it’s too large. Here is part of it:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:49:59,655 INFO [org.ovirt.engine.core.bll.storage.pool.ConnectHostToStoragePoolServersCommand] (DefaultQuartzScheduler7) [4f789867] Host 'labvmhostt01' storage connection was succeeded<span class="m-8496170754045529610m5675251263651723542m-4809531396642446850apple-converted-space"> </span></span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:49:59,668 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-33) [] START, ConnectStoragePoolVDSCommand(HostName = labvmhostt01, ConnectStoragePoolVDSCommandParameters:{runAsync='true',
hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', vdsId='c8fe257b-6506-4ef7-8629-2b8d003ae164', storagePoolId='00000001-0001-0001-0001-000000000366', masterVersion='6'}), log id: 4ca9f290</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:00,977 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ConnectStoragePoolVDSCommand] (org.ovirt.thread.pool-8-thread-33) [] FINISH, ConnectStoragePoolVDSCommand, log id: 4ca9f290</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:02,246 INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (DefaultQuartzScheduler7) [4f789867] Correlation ID: null, Call Stack: null, Custom Event ID: -1, Message: Executing power management
status on Host labvmhostt01 using Proxy Host labvmhostt05 and Fence Agent ipmilan:<a href="http://labvmhostt01-mgmt.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt01-mgmt.lab.ingram.io</span></a>.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:02,247 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (DefaultQuartzScheduler7) [4f789867] START, FenceVdsVDSCommand(HostName = labvmhostt05, FenceVdsVDSCommandParameters:{runAsync='true',
hostId='d1a4198b-5889-43b7-85ee-c1f21d1f3db9', targetVdsId='c8fe257b-6506-4ef7-8629-2b8d003ae164', action='STATUS', agent='FenceAgent:{id='6a4bf631-ab8a-434b-839c-3588af862b68', hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', order='1', type='ipmilan', ip='<a href="http://labvmhostt01-mgmt.lab.ingram.io/" target="_blank"><span style="color:purple">labvmhostt01-mgmt.lab.ingram.io</span></a>',
port='null', user='root', password='***', encryptOptions='false', options='privlvl=OPERATOR</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">delay=10</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">lanplus=1'}', policy='null'}), log id: 179d490b</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:03,439 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.FenceVdsVDSCommand] (DefaultQuartzScheduler7) [4f789867] FINISH, FenceVdsVDSCommand, return: FenceOperationResult:{status='SUCCESS', powerStatus='ON', message=''},
log id: 179d490b</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:03,468 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler7) [4f789867] START, SetMOMPolicyParametersVDSCommand(HostName = labvmhostt01, MomPolicyVDSParameters:{runAsync='true',
hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164'}), log id: 5e9d2daa</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:04,424 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.SetMOMPolicyParametersVDSCommand] (DefaultQuartzScheduler7) [4f789867] FINISH, SetMOMPolicyParametersVDSCommand, log id: 5e9d2daa</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:04,461 INFO [org.ovirt.engine.core.bll.hostdev.RefreshHostDevicesCommand] (DefaultQuartzScheduler7) [2bab1a0c] Running command: RefreshHostDevicesCommand internal: true. Entities affected : ID: c8fe257b-6506-4ef7-8629-2b8d003ae164
Type: VDSAction group MANIPULATE_HOST with role type ADMIN</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:04,463 INFO [org.ovirt.engine.core.vdsbroker.HostDevListByCapsVDSCommand] (DefaultQuartzScheduler7) [2bab1a0c] START, HostDevListByCapsVDSCommand(HostName = labvmhostt01, VdsIdAndVdsVDSCommandParametersBase:{runAsync='true',
hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', vds='Host[labvmhostt01,c8fe257b-6506-4ef7-8629-2b8d003ae164]'}), log id: 578b59c0</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;background:white">
<span style="font-size:11.0pt">2016-11-03 11:50:04,646 INFO [org.ovirt.engine.core.vdsbroker.HostDevListByCapsVDSCommand] (DefaultQuartzScheduler7) [2bab1a0c] FINISH, HostDevListByCapsVDSCommand, return: [HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_7f_1f_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit', productId='0x6f88', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_17_1', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Thermal Control', productId='0x6fd1', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null',
driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_1f_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon
D Power Control Unit', productId='0x6f8a', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_ff_17_3', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 1 Error', productId='0x6fd3', vendorName='Intel Corporation', vendorId='0x8086',
parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_17_4', parentDeviceName='computer',
capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface', productId='0x6fb8', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null',
driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_17_5', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D
DDRIO Channel 2/3 Interface', productId='0x6fb9', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_ff_17_6', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface', productId='0x6fba', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_17_7', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Interface', productId='0x6fbb', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null',
assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_17_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller
1 - Channel 0 Error', productId='0x6fd2', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='scsi_target10_0_0', parentDeviceName='scsi_host10', capability='scsi_target', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null',
driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_16_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D
Target Address/Thermal/RAS', productId='0x6f79', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='usb_3_1_1_0', parentDeviceName='usb_3_1', capability='usb', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='hub',
assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_00_1d_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='C610/X99 series chipset USB Enhanced Host Controller #1',
productId='0x8d26', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='ehci-pci', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_7f_16_6', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Broadcast', productId='0x6f6e', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_16_7', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast', productId='0x6f6f', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='net_vdsmbr_sBo2AHc7_6a_74_14_37_7c_c5', parentDeviceName='computer', capability='net', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null',
parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='vdsmbr_sBo2AHc7', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='usb_1_2_1_0', parentDeviceName='usb_1_2',
capability='usb', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='usbhid', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_08_00_0', parentDeviceName='pci_0000_07_00_0', capability='pci', iommuGroup='null', productName='null', productId='0x001d', vendorName='Renesas Technology Corp.', vendorId='0x1912', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='pcieport', assignable='false', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_13_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon
E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target Address/Thermal/RAS', productId='0x6f71', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null',
assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_80_03_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root
Port 3', productId='0x6f0a', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='pcieport', assignable='false', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_80_03_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 3', productId='0x6f08', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='pcieport', assignable='false', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_0f_6', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffe', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='block_sda_361866da06a431e001f5aaac9308f23e7', parentDeviceName='scsi_0_2_0_0', capability='storage', iommuGroup='null', productName='PERC H730 Mini', productId='null', vendorName='DELL',
vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='net_p5p2_a0_36_9f_bc_be_52', parentDeviceName='pci_0000_04_00_1',
capability='net', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='p5p2', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_09_00_0', parentDeviceName='pci_0000_08_00_0', capability='pci', iommuGroup='null', productName='null', productId='0x001a', vendorName='Renesas Technology Corp.', vendorId='0x1912', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='false', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_00_11_4', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='C610/X99
series chipset sSATA Controller [AHCI mode]', productId='0x8d62', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='ahci', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='usb_3_0_1_0', parentDeviceName='usb_usb3', capability='usb', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null',
driver='hub', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_80_01_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D
PCI Express Root Port 1', productId='0x6f02', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='pcieport', assignable='false', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_00_11_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='C610/X99 series chipset SPSR', productId='0x8d7c', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_14_6', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface', productId='0x6fbe', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_14_7', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface', productId='0x6fbf',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_14_4',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface', productId='0x6fbc', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_14_5', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Interface', productId='0x6fbd', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_14_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Error', productId='0x6fb2',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_14_3',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 1 Error', productId='0x6fb3', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_14_0', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel 0 Thermal Control', productId='0x6fb0', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null',
driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_14_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon
D Memory Controller 0 - Channel 1 Thermal Control', productId='0x6fb1', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='usb_1_2', parentDeviceName='usb_usb1', capability='usb_device', iommuGroup='null', productName='USB Keyboard & Mouse', productId='0x0101', vendorName='Motorola', vendorId='0x1061', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='usb', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_0f_1', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ff9', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_17_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 1 - Channel 0 Thermal
Control', productId='0x6fd0', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_0a_00_0', parentDeviceName='pci_0000_09_00_0', capability='pci', iommuGroup='null', productName='G200eR2', productId='0x0534', vendorName='Matrox Electronics Systems Ltd.', vendorId='0x102b', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='mgag200', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_0f_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7
v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ff8', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_13_6', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 0/1 Broadcast', productId='0x6fae',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_00_01_0',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D PCI Express Root Port 1', productId='0x6f02', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='pcieport', assignable='false', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_13_7', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon
E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast', productId='0x6faf', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_13_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Target Address/Thermal/RAS',
productId='0x6fa8', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_7f_10_7', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Ubox', productId='0x6f1f', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_16_0', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS', productId='0x6f68', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null',
assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_10_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R2PCIe Agent',
productId='0x6f1d', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_7f_10_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R2PCIe Agent', productId='0x6f34', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_10_6', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Ubox', productId='0x6f7d', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_00_1f_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='C610/X99 series chipset LPC Controller', productId='0x8d44', vendorName='Intel Corporation',
vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='lpc_ich', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_13_2', parentDeviceName='computer',
capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Memory Controller 0 - Channel Target Address Decoder', productId='0x6faa', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_10_5', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Ubox', productId='0x6f1e', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='usb_1_12', parentDeviceName='usb_usb1', capability='usb_device', iommuGroup='null', productName='Hub', productId='0xa001', vendorName='Dell Computer Corp.', vendorId='0x413c', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null',
driver='usb', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_13_3', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D
Memory Controller 0 - Channel Target Address Decoder', productId='0x6fab', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_16_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Channel Target Address Decoder', productId='0x6f6a',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_12_4',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 1', productId='0x6f60', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_12_5', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Home Agent 1', productId='0x6f38', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_7f_12_6', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 1 Debug', productId='0x6f78', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_16_3', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Channel Target Address Decoder', productId='0x6f6b', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null',
assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_12_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 0',
productId='0x6fa0', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='sbridge_edac', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_7f_12_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 0', productId='0x6f30', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='scsi_10_0_0_0', parentDeviceName='scsi_target10_0_0', capability='scsi',
iommuGroup='null', productName='DVD-ROM DTA0N', productId='null', vendorName='HL-DT-ST', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='sr', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='net_p5p1_a0_36_9f_bc_be_50', parentDeviceName='pci_0000_04_00_0', capability='net', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='p5p1',
driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_00_1c_7', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='C610/X99 series chipset PCI Express Root
Port #8', productId='0x8d1e', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='pcieport', assignable='false', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_ff_0b_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1', productId='0x6f37', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0b_3', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link Debug', productId='0x6f76', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0b_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1', productId='0x6f81',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0b_1',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R3 QPI Link 0/1', productId='0x6f36', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_16_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon
E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS', productId='0x6f79', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_16_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Target Address/Thermal/RAS', productId='0x6f68',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_16_3',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Channel Target Address Decoder', productId='0x6f6b', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_16_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Channel Target Address Decoder', productId='0x6f6a', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_16_7', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Global Broadcast', productId='0x6f6f', vendorName='Intel
Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_16_6',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D DDRIO Channel 2/3 Broadcast', productId='0x6f6e', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_80_05_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D IIO Hot Plug', productId='0x6f29', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_80_05_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Map/VTd_Misc/System Management', productId='0x6f28', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='computer', parentDeviceName='computer', capability='system', iommuGroup='null',
productName='PowerEdge R730', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_80_05_4', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D I/O APIC', productId='0x6f2c', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='block_sr0_HL_DT_ST_DVD_ROM_DTA0N_KMHG5CA3953', parentDeviceName='scsi_10_0_0_0',
capability='storage', iommuGroup='null', productName='DVD-ROM DTA0N', productId='null', vendorName='HL-DT-ST', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_10_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D R2PCIe Agent', productId='0x6f34', vendorName='Intel
Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_09_3',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D QPI Link 1', productId='0x6f93', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_10_7', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Ubox', productId='0x6f1f', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_ff_10_6', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Ubox', productId='0x6f7d', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_10_5', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Ubox', productId='0x6f1e', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0f_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffa', vendorName='Intel
Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0f_3',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffb', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0f_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ff8', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_ff_0f_1', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ff9', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0f_6', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffe', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_00_1a_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='C610/X99 series chipset USB Enhanced Host Controller #2', productId='0x8d2d',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='ehci-pci', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0f_5',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffd', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='net_em4_24_6e_96_20_82_8d', parentDeviceName='pci_0000_06_00_1', capability='net', iommuGroup='null', productName='null',
productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='em4', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0d_4',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fec', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0d_5', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fed', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_7f_0f_5', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffd', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_0f_4', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffc', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_0f_3', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffb',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_0f_2',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6ffa', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0d_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fea', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_ff_0d_3', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6feb', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_00_1c_0', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='C610/X99 series chipset PCI Express Root Port #1', productId='0x8d10', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='pcieport', assignable='false',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_00_1c_4', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='C610/X99 series chipset PCI Express Root Port #5', productId='0x8d18', vendorName='Intel
Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='pcieport', assignable='false', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='net_ovirtmgmt_e6_1a_ad_9d_44_62',
parentDeviceName='computer', capability='net', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='ovirtmgmt', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0c_7', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fe7',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0c_6',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fe6', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0c_5', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fe5', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_ff_0c_4', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fe4', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0c_3', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fe3', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0c_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fe2',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0c_1',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fe1', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_0c_0', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon
E5 v4/Xeon E3 v4/Xeon D Caching Agent', productId='0x6fe0', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='usb_4_0_1_0', parentDeviceName='usb_usb4', capability='usb', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null',
driver='hub', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='usb_4_1_1_0', parentDeviceName='usb_4_1', capability='usb', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null',
parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='hub', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='usb_1_0_1_0', parentDeviceName='usb_usb1', capability='usb',
iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='hub', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='net_ovs_system_da_c3_0f_4f_d4_f0', parentDeviceName='computer', capability='net', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='ovs-system',
driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='scsi_0_2_0_0', parentDeviceName='scsi_target0_2_0', capability='scsi', iommuGroup='null', productName='PERC H730 Mini', productId='null',
vendorName='DELL', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='sd', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='scsi_host3',
parentDeviceName='pci_0000_00_11_4', capability='scsi_host', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null',
assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='usb_usb2', parentDeviceName='pci_0000_00_14_0', capability='usb_device', iommuGroup='null', productName='3.0 root hub', productId='0x0003', vendorName='Linux
Foundation', vendorId='0x1d6b', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='usb', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='scsi_host2', parentDeviceName='pci_0000_00_11_4',
capability='scsi_host', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_ff_12_5', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 1', productId='0x6f38', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_12_4', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 1', productId='0x6f60', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true',
vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_12_6', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 1 Debug', productId='0x6f78',
vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_12_1',
parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 0', productId='0x6f30', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null',
networkInterfaceName='null', driver='bdx_uncore', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_12_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon
E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 0 Debug', productId='0x6f70', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'},
HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_ff_12_2', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Home Agent 0 Debug', productId='0x6f70', vendorName='Intel
Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_04_00_0',
parentDeviceName='pci_0000_00_03_2', capability='pci', iommuGroup='null', productName='Ethernet Controller 10-Gigabit X540-AT2', productId='0x1528', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='63',
networkInterfaceName='null', driver='ixgbe', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_04_00_1', parentDeviceName='pci_0000_00_03_2', capability='pci', iommuGroup='null', productName='Ethernet
Controller 10-Gigabit X540-AT2', productId='0x1528', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='63', networkInterfaceName='null', driver='ixgbe', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='net_em1_24_6e_96_20_82_88', parentDeviceName='pci_0000_01_00_0', capability='net', iommuGroup='null', productName='null', productId='null', vendorName='null', vendorId='null', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='em1',
driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_1e_4', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D
Power Control Unit', productId='0x6f9c', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null', totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164',
deviceName='pci_0000_7f_1e_3', parentDeviceName='computer', capability='pci', iommuGroup='null', productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Control Unit', productId='0x6fc0', vendorName='Intel Corporation', vendorId='0x8086', parentPhysicalFunction='null',
totalVirtualFunctions='null', networkInterfaceName='null', driver='null', assignable='true', vmId='null'}, HostDevice:{hostId='c8fe257b-6506-4ef7-8629-2b8d003ae164', deviceName='pci_0000_7f_1e_2', parentDeviceName='computer', capability='pci', iommuGroup='null',
productName='Xeon E7 v4/Xeon E5 v4/Xeon E3 v4/Xeon D Power Cont</span><o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">...<br>
<br>
[Message clipped] <o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</blockquote>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>