<div dir="ltr"><span style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Hi Charles,</span><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><br class=""></div><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">I think the problem I am having is due to the setup failing and not something in vdsm configs as I have never gotten this server to start up properly and the BRIDGE ethernet interface + ovirt routes are not setup.</div><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><br class=""></div><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">I put the logs here: <a href="https://www.dropbox.com/sh/5ugyykqh1lgru9l/AACXxRYWr3tgd0WbBVFW5twHa?dl=0" class="">https://www.dropbox.com/sh/5ugyykqh1lgru9l/AACXxRYWr3tgd0WbBVFW5twHa?dl=0</a></div><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><br class=""></div><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">hosted-engine--deploy-logs.zip<span class="" style="white-space:pre">        </span># Logs from when I tried to deploy and it failed</div><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">vdsm.tar.gz<span class="" style="white-space:pre">                                        </span># /var/log/vdsm</div><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><br class=""></div><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px">Output from running vdsm from the command line:</div><blockquote class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px;margin:0px 0px 0px 40px;border:none;padding:0px"><div class="">[root@cultivar2 log]# su -s /bin/bash vdsm</div><div class="">[vdsm@cultivar2 log]$ python /usr/share/vdsm/vdsm</div><div class="">(PID: 6521) I am the actual vdsm 4.17.26-1.el7 <a href="http://cultivar2.grove.silverorange.com/" class="">cultivar2.grove.silverorange.com</a> (3.10.0-327.el7.x86_64)</div><div class="">VDSM will run with cpu affinity: frozenset([1])</div><div class="">/usr/bin/taskset --all-tasks --pid --cpu-list 1 6521 (cwd None)</div><div class="">SUCCESS: &lt;err&gt; = &#39;&#39;; &lt;rc&gt; = 0</div><div class="">Starting scheduler vdsm.Scheduler</div><div class="">started</div><div class="">Run and protect: registerDomainStateChangeCallback(callbackFunc=&lt;functools.partial object at 0x381b158&gt;)</div><div class="">Run and protect: registerDomainStateChangeCallback, Return response: None</div><div class="">Trying to connect to Super Vdsm</div><div class="">Preparing MOM interface</div><div class="">Using named unix socket /var/run/vdsm/mom-vdsm.sock</div><div class="">Unregistering all secrests</div><div class="">trying to connect libvirt</div><div class="">recovery: started</div><div class="">Setting channels&#39; timeout to 30 seconds.</div><div class="">Starting VM channels listener thread.</div><div class="">Listening at <a href="http://0.0.0.0:54321">0.0.0.0:54321</a></div><div class="">Adding detector &lt;rpc.bindingxmlrpc.XmlDetector instance at 0x3b4ecb0&gt;</div><div class="">recovery: completed in 0s</div><div class="">Adding detector &lt;yajsonrpc.stompreactor.StompDetector instance at 0x382e5a8&gt;</div><div class="">Starting executor</div><div class="">Starting worker jsonrpc.Executor/0</div><div class="">Worker started</div><div class="">Starting worker jsonrpc.Executor/1</div><div class="">Worker started</div><div class="">Starting worker jsonrpc.Executor/2</div><div class="">Worker started</div><div class="">Starting worker jsonrpc.Executor/3</div><div class="">Worker started</div><div class="">Starting worker jsonrpc.Executor/4</div><div class="">Worker started</div><div class="">Starting worker jsonrpc.Executor/5</div><div class="">Worker started</div><div class="">Starting worker jsonrpc.Executor/6</div><div class="">Worker started</div><div class="">Starting worker jsonrpc.Executor/7</div><div class="">Worker started</div><div class="">XMLRPC server running</div><div class="">Starting executor</div><div class="">Starting worker periodic/0</div><div class="">Worker started</div><div class="">Starting worker periodic/1</div><div class="">Worker started</div><div class="">Starting worker periodic/2</div><div class="">Worker started</div><div class="">Starting worker periodic/3</div><div class="">Worker started</div><div class="">trying to connect libvirt</div><div class="">Panic: Connect to supervdsm service failed: [Errno 2] No such file or directory</div><div class="">Traceback (most recent call last):</div><div class="">  File &quot;/usr/share/vdsm/supervdsm.py&quot;, line 78, in _connect</div><div class="">    utils.retry(self._manager.connect, Exception, timeout=60, tries=3)</div><div class="">  File &quot;/usr/lib/python2.7/site-packages/vdsm/utils.py&quot;, line 959, in retry</div><div class="">    return func()</div><div class="">  File &quot;/usr/lib64/python2.7/multiprocessing/managers.py&quot;, line 500, in connect</div><div class="">    conn = Client(self._address, authkey=self._authkey)</div><div class="">  File &quot;/usr/lib64/python2.7/multiprocessing/connection.py&quot;, line 173, in Client</div><div class="">    c = SocketClient(address)</div><div class="">  File &quot;/usr/lib64/python2.7/multiprocessing/connection.py&quot;, line 308, in SocketClient</div><div class="">    s.connect(address)</div><div class="">  File &quot;/usr/lib64/python2.7/socket.py&quot;, line 224, in meth</div><div class="">    return getattr(self._sock,name)(*args)</div><div class="">error: [Errno 2] No such file or directory</div><div class="">Killed</div></blockquote><div class="" style="color:rgb(0,0,0);font-family:Helvetica;font-size:12px"><div class=""><br class=""></div><div class="">Thanks for the help. It&#39;s really appreciated.</div><div class=""><div id="signature" class=""><br class="">Cheers,<br class="">Gervais</div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, May 13, 2016 at 12:55 AM, Charles Tassell <span dir="ltr">&lt;<a href="mailto:ctassell@gmail.com" target="_blank">ctassell@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
  
    
  
  <div text="#000000" bgcolor="#FFFFFF">
    <div>Hi Gervais,<br>
      <br>
        Hmm, can you tar up the logfiles (/var/log/vdsm/* on the host
      you are installing on) and put them somewhere to look at?  Also, I
      found that starting VDSM from the command line is useful as it
      sometimes spits out error messages that don&#39;t show up in the
      logs.  I think the command I used was:<br>
      su -s /bin/bash vdsm<br>
      python /usr/share/vdsm/vdsm<br>
      <br>
      My problem was that I customized the logging settings in
      /etc/vdsm/*conf to try and tone down the debugging stuff and had a
      syntax error.<div><div class="h5"><br>
      <br>
      On 16-05-12 10:24 PM, Gervais de Montbrun wrote:<br>
    </div></div></div><div><div class="h5">
    <blockquote type="cite">
      <div dir="ltr">
        <div dir="auto" style="word-wrap:break-word">Hi Charles,<br>
          <br>
          Thanks for the suggestion.<br>
          <br>
          I cleaned up again using the bash script from the
          recoving-from-failed-install link below, then reinstalled (yum
          install ovirt-hosted-engine-setup).<br>
          <br>
          I enabled NetworkManager and firewalld as you suggested. The
          install stops very early on with an error:<br>
          <span style="white-space:pre-wrap">        </span>[ ERROR ] Failed to
          execute stage &#39;Programs detection&#39;: hosted-engine cannot be
          deployed while NetworkManager is running, please stop and
          disable it before proceeding <br>
          <br>
          I disabled and stopped NetworkManager and tried again. Same
          result. :(<br>
          <br>
          Any more guesses?<br>
          <br>
          Cheers,<br>
          Gervais<br>
          <br>
          <br>
          <br>
          <blockquote type="cite">On May 12, 2016, at 9:08 PM, Charles
            Tassell &lt;<a href="mailto:ctassell@gmail.com" target="_blank">ctassell@gmail.com</a>&gt;
            wrote:<br>
            <br>
            Hey Gervais,<br>
            <br>
            Try enabling NetworkManager and firewalld before doing the
            hosted-engine --deploy.  I have run into problems with oVirt
            trying to perform tasks on hosts where firewalld is
            disabled, so maybe you are running into a similar problem. 
            Also, I think the setup script will disable NetworkManager
            if it needs to.  I know I didn&#39;t manually disable it on any
            of the boxes I installed on.<br>
            <br>
            On 16-05-12 04:49 PM, <a href="mailto:users-request@ovirt.org" target="_blank">users-request@ovirt.org</a>
            wrote:<br>
            <blockquote type="cite">Message: 1<br>
              Date: Thu, 12 May 2016 14:22:12 -0300<br>
              From: Gervais de Montbrun &lt;<a href="mailto:gervais@demontbrun.com" target="_blank">gervais@demontbrun.com</a>&gt;<br>
              To: Wee Sritippho &lt;<a href="mailto:wee.s@forest.go.th" target="_blank">wee.s@forest.go.th</a>&gt;<br>
              Cc: users &lt;<a href="mailto:users@ovirt.org" target="_blank">users@ovirt.org</a>&gt;<br>
              Subject: Re: [ovirt-users] Adding another host to my
              cluster<br>
              Message-ID: &lt;<a href="mailto:28B7FC74-5C52-4F60-B9F3-39A36621A7CA@demontbrun.com" target="_blank">28B7FC74-5C52-4F60-B9F3-39A36621A7CA@demontbrun.com</a>&gt;<br>
              Content-Type: text/plain; charset=&quot;utf-8&quot;<br>
              <br>
              Hi Wee<br>
              (and others)<br>
              <br>
              Thanks for the reply. I tried what you suggested, but I am
              in the exact same state. :-(<br>
              <br>
              I don&#39;t want to completely remove my hosted engine setup
              as it is working on the two other hosts in my cluster. I
              did not run the rm -rf stes listed here (<a href="https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install" target="_blank"></a><a href="https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install" target="_blank">https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install</a>
              &lt;<a href="https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install" target="_blank">https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install</a>&gt;)
              that would wipe my hosted_engine nfs mount. If you know
              that this is 100% necessary, please let me know.<br>
              <br>
              I did:<br>
              hosted-engine --clean-metadata --force-cleanup --host-id=3<br>
              run the bash script to remove all of the ovirt packages
              and config files<br>
              reinstalled ovirt-hosted-engine-setup<br>
              ran &quot;hosted-engine --deploy&quot;<br>
              <br>
              I&#39;m back exactly where I started. Is there a way to run
              just the network configuration part of the deploy?<br>
              <br>
              Since the last attempt, I did upgrade my hosted engine and
              my cluster is now running oVirt 3.6.5.<br>
              <br>
              Cheers,<br>
              Gervais<br>
              <br>
              <br>
              <br>
              <blockquote type="cite">On May 12, 2016, at 11:50 AM, Wee
                Sritippho &lt;<a href="mailto:wee.s@forest.go.th" target="_blank">wee.s@forest.go.th</a>&gt;
                wrote:<br>
                <br>
                Hi,<br>
                <br>
                I used to have a similar problem where one of my host
                can&#39;t be deployed due to the absence of ovirtmgmt
                bridge. Simone said it&#39;s a bug ( <a href="https://bugzilla.redhat.com/1323465" target="_blank"></a><a href="https://bugzilla.redhat.com/1323465" target="_blank">https://bugzilla.redhat.com/1323465</a>
                &lt;<a href="https://bugzilla.redhat.com/1323465" target="_blank">https://bugzilla.redhat.com/1323465</a>&gt;
                ) which would be fixed in 3.6.6.<br>
                <br>
                This is what I&#39;ve done to solve it:<br>
                <br>
                1. In the web UI, set the failed host to maintenance.<br>
                2. Remove it.<br>
                3. In that host, run a script from <a href="https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install" target="_blank"></a><a href="https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install" target="_blank">https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install</a>
                &lt;<a href="https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install" target="_blank">https://www.ovirt.org/documentation/how-to/hosted-engine/#recoving-from-failed-install</a>&gt;<br>
                4. Install ovirt-hosted-engine-setup again.<br>
                5. Redeploy again.<br>
                <br>
                Hope that helps<br>
                <br>
                On 11 ??????? 2016 22 ?????? 48 ???? 58 ??????
                GMT+07:00, Gervais de Montbrun &lt;<a href="mailto:gervais@demontbrun.com" target="_blank"></a><a href="mailto:gervais@demontbrun.com" target="_blank">gervais@demontbrun.com</a>&gt;
                wrote:<br>
                Hi Folks,<br>
                <br>
                I hate to reply to my own message, but I&#39;m really hoping
                someone can help me with my issue<br>
                <a href="http://lists.ovirt.org/pipermail/users/2016-May/039690.html" target="_blank">http://lists.ovirt.org/pipermail/users/2016-May/039690.html</a>
                &lt;<a href="http://lists.ovirt.org/pipermail/users/2016-May/039690.html" target="_blank">http://lists.ovirt.org/pipermail/users/2016-May/039690.html</a>&gt;<br>
                <br>
                Does anyone have a suggestion for me? If there is any
                more information that I can provide that would help you
                to help me, please advise.<br>
                <br>
                Cheers,<br>
                Gervais<br>
                <br>
                <br>
                <br>
                <blockquote type="cite">On May 9, 2016, at 1:42 PM,
                  Gervais de Montbrun &lt;<a href="mailto:gervais@demontbrun.com" target="_blank">gervais@demontbrun.com</a>
                  &lt;mailto:<a href="mailto:gervais@demontbrun.com" target="_blank">gervais@demontbrun.com</a>&gt;&gt;
                  wrote:<br>
                  <br>
                  Hi All,<br>
                  <br>
                  I&#39;m trying to add a third host into my oVirt cluster.
                  I have hosted engine setup on the first two. It&#39;s
                  failing to finish the hosted-engine --deploy on this
                  third host. I wiped the server and did a CentOS 7
                  minimum install and ran it again to have a clean
                  machine.<br>
                  <br>
                  My setup:<br>
                  CentOS 7 clean install<br>
                  yum install -y <a href="http://resources.ovirt.org/pub/yum-repo/ovirt-release36.rpm" target="_blank">http://resources.ovirt.org/pub/yum-repo/ovirt-release36.rpm</a>
                  &lt;<a href="http://resources.ovirt.org/pub/yum-repo/ovirt-release36.rpm" target="_blank">http://resources.ovirt.org/pub/yum-repo/ovirt-release36.rpm</a>&gt;<br>
                  yum install -y ovirt-hosted-engine-setup<br>
                  yum upgrade -y &amp;&amp; reboot<br>
                  systemctl disable NetworkManager ; systemctl stop
                  NetworkManager ; systemctl disable firewalld ;
                  systemctl stop firewalld<br>
                  hosted-engine --deploy<br>
                  <br>
                  hosted-engine --deploy always throws an error:<br>
                  [ ERROR ] The VDSM host was found in a failed state.
                  Please check engine and bootstrap installation logs.<br>
                  [ ERROR ] Unable to add Cultivar2 to the manager<br>
                  and then echo&#39;s<br>
                  [ INFO  ] Waiting for VDSM hardware info<br>
                  ...<br>
                  [ ERROR ] Failed to execute stage &#39;Closing up&#39;: VDSM
                  did not start within 120 seconds<br>
                  [ INFO  ] Stage: Clean up<br>
                  [ INFO  ] Generating answer file
                  &#39;/var/lib/ovirt-hosted-engine-setup/answers/answers-20160509131103.conf&#39;<br>
                  [ INFO  ] Stage: Pre-termination<br>
                  [ INFO  ] Stage: Termination<br>
                  [ ERROR ] Hosted Engine deployment failed: this system
                  is not reliable, please check the issue, fix and
                  redeploy<br>
                      Log file is located at
/var/log/ovirt-hosted-engine-setup/ovirt-hosted-engine-setup-20160509130658-qb8ev0.log<br>
                  <br>
                  Full output of hosted-engine --deploy included in the
                  attached zip file.<br>
                  I&#39;ve also included vdsm.log (There is more than one
                  tries worth of tries in there).<br>
                  You&#39;ll also find the
                  ovirt-hosted-engine-setup-20160509130658-qb8ev0.log
                  listed above.<br>
                  <br>
                  This is my &quot;test&quot; setup. Cultivar0 is my first host
                  and my nfs server for storage. I have two hosts in the
                  setup already and everything is working fine. The host
                  does show up in the oVirt admin, but shows &quot;Installed
                  Failed&quot;<br>
                  &lt;PastedGraphic-1.png&gt;<br>
                  <br>
                  Trying to reinstall from within the interface just
                  fails again.<br>
                  <br>
                  The ovirt bridge interface is not configured and there
                  are no config files in /etc/sysconfi/network-scripts
                  related to ovirt.<br>
                  <br>
                  OS:<br>
                  [root@cultivar2 ovirt-hosted-engine-setup]# cat
                  /etc/redhat-release<br>
                  CentOS Linux release 7.2.1511 (Core)<br>
                  <br>
                  [root@cultivar2 ovirt-hosted-engine-setup]# uname -a<br>
                  Linux <a href="http://cultivar2.grove.silverorange.com" target="_blank">cultivar2.grove.silverorange.com</a>
                  &lt;<a href="http://cultivar2.grove.silverorange.com/" target="_blank">http://cultivar2.grove.silverorange.com/</a>&gt;
                  3.10.0-327.13.1.el7.x86_64 #1 SMP Thu Mar 31 16:04:38
                  UTC 2016 x86_64 x86_64 x86_64 GNU/Linux<br>
                  <br>
                  Versions:<br>
                  [root@cultivar2 ovirt-hosted-engine-setup]# rpm -qa |
                  grep -i ovirt<br>
                  libgovirt-0.3.3-1.el7_2.1.x86_64<br>
                  ovirt-hosted-engine-setup-1.3.5.0-1.1.el7.noarch<br>
                  ovirt-host-deploy-1.4.1-1.el7.centos.noarch<br>
                  ovirt-vmconsole-1.0.0-1.el7.centos.noarch<br>
                  ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch<br>
                  ovirt-release36-007-1.noarch<br>
                  ovirt-engine-sdk-python-3.6.5.0-1.el7.centos.noarch<br>
                  ovirt-setup-lib-1.0.1-1.el7.centos.noarch<br>
                  ovirt-hosted-engine-ha-1.3.5.3-1.1.el7.noarch<br>
                  [root@cultivar2 ovirt-hosted-engine-setup]#<br>
                  [root@cultivar2 ovirt-hosted-engine-setup]#<br>
                  [root@cultivar2 ovirt-hosted-engine-setup]#<br>
                  [root@cultivar2 ovirt-hosted-engine-setup]# rpm -qa |
                  grep -i virt<br>
                  libvirt-daemon-driver-secret-1.2.17-13.el7_2.4.x86_64<br>
                  virt-viewer-2.0-6.el7.x86_64<br>
                  libgovirt-0.3.3-1.el7_2.1.x86_64<br>
                  libvirt-daemon-kvm-1.2.17-13.el7_2.4.x86_64<br>
                  ovirt-hosted-engine-setup-1.3.5.0-1.1.el7.noarch<br>
                  fence-virt-0.3.2-2.el7.x86_64<br>
                  virt-what-1.13-6.el7.x86_64<br>
                  libvirt-python-1.2.17-2.el7.x86_64<br>
                  libvirt-daemon-1.2.17-13.el7_2.4.x86_64<br>
libvirt-daemon-config-nwfilter-1.2.17-13.el7_2.4.x86_64<br>
                  libvirt-lock-sanlock-1.2.17-13.el7_2.4.x86_64<br>
                  libvirt-daemon-driver-nodedev-1.2.17-13.el7_2.4.x86_64<br>
                  libvirt-daemon-driver-network-1.2.17-13.el7_2.4.x86_64<br>
                  libvirt-daemon-driver-storage-1.2.17-13.el7_2.4.x86_64<br>
                  ovirt-host-deploy-1.4.1-1.el7.centos.noarch<br>
                  virt-v2v-1.28.1-1.55.el7.centos.2.x86_64<br>
                  ovirt-vmconsole-1.0.0-1.el7.centos.noarch<br>
                  ovirt-vmconsole-host-1.0.0-1.el7.centos.noarch<br>
                  libvirt-client-1.2.17-13.el7_2.4.x86_64<br>
libvirt-daemon-driver-nwfilter-1.2.17-13.el7_2.4.x86_64<br>
                  ovirt-release36-007-1.noarch<br>
libvirt-daemon-driver-interface-1.2.17-13.el7_2.4.x86_64<br>
                  libvirt-daemon-driver-qemu-1.2.17-13.el7_2.4.x86_64<br>
                  ovirt-engine-sdk-python-3.6.5.0-1.el7.centos.noarch<br>
                  ovirt-setup-lib-1.0.1-1.el7.centos.noarch<br>
                  ovirt-hosted-engine-ha-1.3.5.3-1.1.el7.noarch<br>
                  <br>
                  I also have a series of stuck tasks that I can&#39;t clear
                  related to the host that can&#39;t be added... This is a
                  secondary issue and I don&#39;t want to get off track, but
                  they look like this:<br>
                  &lt;PastedGraphic-2.png&gt;<br>
                  <br>
                  I&#39;d appreciate any help that can be offered.<br>
                  <br>
                  Cheers,<br>
                  Gervais<br>
                  <br>
                  <br>
                  Gervais de Montbrun<br>
                  Systems Administrator  / silverorange Inc.<br>
                  <br>
                  Phone                                   <span style="white-space:pre-wrap">        </span><a href="tel:%2B1%20902%20367%204532%20ext.%20104" value="+19023674532" target="_blank">+1 902 367 4532
                    ext. 104</a> &lt;tel:<a href="tel:%2B1%20902%20367%204532%20ext.%20104" value="+19023674532" target="_blank">+1 902 367 4532
                    ext. 104</a>&gt;<br>
                  Mobile                                   <span style="white-space:pre-wrap">        </span><a href="tel:%2B1%20902%20978%200009" value="+19029780009" target="_blank">+1 902 978 0009</a>
                  &lt;tel:<a href="tel:%2B1%20902%20978%200009" value="+19029780009" target="_blank">+1 902 978 0009</a>&gt;<br>
                  <br>
                  &lt;hosted-engine--deploy-logs.zip&gt;<br>
                </blockquote>
                <br>
                <br>
                Users mailing list<br>
                <a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
                <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a>
                &lt;<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a>&gt;<br>
                <br>
                -- <br>
                Wee<br>
              </blockquote>
              <br>
            </blockquote>
            <br>
            _______________________________________________<br>
            Users mailing list<br>
            <a href="mailto:Users@ovirt.org" target="_blank">Users@ovirt.org</a><br>
            <a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/mailman/listinfo/users</a><br>
          </blockquote>
          <br>
        </div>
      </div>
    </blockquote>
    <br>
  </div></div></div>

</blockquote></div><br></div>