<html>
  <head>
    <meta content="text/html; charset=windows-1252"
      http-equiv="Content-Type">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <br>
    <br>
    <div class="moz-cite-prefix">On 02/17/2016 04:57 PM, Luiz Claudio
      Prazeres Goncalves wrote:<br>
    </div>
    <blockquote
cite="mid:CABUQ0hBjOXpMWN00EQsVAHVgLU4cznA2iYjPXRoh=HcXB1mbMg@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div><br>
        </div>
        <div>Hi, I realised that gluster 3.7.8 was released for GA. So I
          updated manually using "yum -y install glusterfs*" gluster
          get's updated normally, but  unfortunately things stopped to
          work completely. ovirt_hosted_engine_ha was not able to
          connect to my gluster storage domain (as you can see below).
          I'm not using ovirt + gluster in hyperconverged way. I have 3
          external gluster hosts. <br>
        </div>
      </div>
    </blockquote>
    <br>
    <br>
    What does the mount log on the hypervisor say? Could you attach
    that? <br>
    <br>
    <blockquote
cite="mid:CABUQ0hBjOXpMWN00EQsVAHVgLU4cznA2iYjPXRoh=HcXB1mbMg@mail.gmail.com"
      type="cite">
      <div dir="ltr">
        <div><br>
        </div>
        <div>As a workaround I've executed "<span class="s1"><b>yum</b></span><span
            class="s2"> downgrade glusterfs*" and after doing this ovirt
            started to work again...but as a collateral effect now I
            can't migrate my vm's anymore.  On the vdsm logs I can see
            the following errors when trying to manually migrate a any
            VM.... Anyone knows how to fix it? </span></div>
        <div><span class="s2"><br>
          </span></div>
        <div><span class="s2">
            <p class="p1"><span class="s1">Thread-66055::DEBUG::2016-02-17
                11:05:57,497::migration::453::virt.vm::(stop)
                vmId=`05681896-a76a-4ae1-879e-8fe5d28634e1`::stopping
                migration downtime thread</span></p>
            <p class="p1"><span class="s1">Thread-66055::ERROR::2016-02-17
                11:05:57,497::migration::208::virt.vm::(_recover)
                vmId=`05681896-a76a-4ae1-879e-8fe5d28634e1`::unable to
                connect to server at '<a moz-do-not-send="true"
                  href="http://kvm2.brightsquid.com:49152">kvm2.brightsquid.com:49152</a>':
                No route to host</span></p>
            <p class="p1"><span class="s1">Thread-66055::DEBUG::2016-02-17
                11:05:57,497::stompreactor::389::jsonrpc.AsyncoreClient::(send)
                Sending response</span></p>
            <p class="p1"><span class="s1">Thread-66056::DEBUG::2016-02-17
                11:05:57,498::migration::450::virt.vm::(run)
                vmId=`05681896-a76a-4ae1-879e-8fe5d28634e1`::migration
                downtime thread exiting</span></p>
            <p class="p1"><span class="s1">Thread-66055::DEBUG::2016-02-17
                11:05:57,540::__init__::206::jsonrpc.Notification::(emit)
                Sending event {"params": {"notify_time": 4327804740,
                "05681896-a76a-4ae1-879e-8fe5d28634e1": {"status":
                "Migration Source"}}, "jsonrpc": "2.0", "method":
                "|virt|VM_status|05681896-a76a-4ae1-879e-8fe5d28634e1"}</span></p>
            <p class="p1"><span class="s1">Thread-66055::ERROR::2016-02-17
                11:05:57,541::migration::310::virt.vm::(run)
                vmId=`05681896-a76a-4ae1-879e-8fe5d28634e1`::</span><span
                class="s2">Failed</span><span class="s1"> to migrate</span></p>
            <p class="p1"><span class="s1">Traceback (most recent call
                last):</span></p>
            <p class="p1"><span class="s1">  File
                "/usr/share/vdsm/virt/migration.py", line 294, in run</span></p>
            <p class="p1"><span class="s1">   
                self._startUnderlyingMigration(time.time())</span></p>
            <p class="p1"><span class="s1">  File
                "/usr/share/vdsm/virt/migration.py", line 364, in
                _startUnderlyingMigration</span></p>
            <p class="p1"><span class="s1">   
                self._perform_migration(duri, muri)</span></p>
            <p class="p1"><span class="s1">  File
                "/usr/share/vdsm/virt/migration.py", line 403, in
                _perform_migration</span></p>
            <p class="p1"><span class="s1">   
                self._vm._dom.migrateToURI3(duri, params, flags)</span></p>
            <p class="p1"><span class="s1">  File
                "/usr/share/vdsm/virt/virdomain.py", line 68, in f</span></p>
            <p class="p1"><span class="s1">    ret = attr(*args,
                **kwargs)</span></p>
            <p class="p1"><span class="s1">  File
                "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py",
                line 124, in wrapper</span></p>
            <p class="p1"><span class="s1">    ret = f(*args, **kwargs)</span></p>
            <p class="p1"><span class="s1">  File
                "/usr/lib64/python2.7/site-packages/libvirt.py", line
                1836, in migrateToURI3</span></p>
            <p class="p1"><span class="s1">    if ret == -1: raise
                libvirtError ('virDomainMigrateToURI3() failed',
                dom=self)</span></p>
            <p class="p1"><span class="s1">libvirtError: unable to
                connect to server at '<a moz-do-not-send="true"
                  href="http://kvm2.brightsquid.com:49152">kvm2.brightsquid.com:49152</a>':
                No route to host</span></p>
            <p class="p1"><span class="s1">Thread-68::DEBUG::2016-02-17
                11:05:57,652::fileSD::173::Storage.Misc.excCmd::(getReadDelay)
                /usr/bin/taskset --cpu-list 0-39 /usr/bin/dd
                if=/rhev/data-center/mnt/gluster2.brightsquid.com:_home_export_iso/61827b7b-e255-44f5-a791-482a144be29f/dom_md/metadata
                iflag=direct of=/dev/null bs=4096 count=1 (cwd None)</span></p>
            <p class="p1"><span class="s1">Thread-68::DEBUG::2016-02-17
                11:05:57,662::fileSD::173::Storage.Misc.excCmd::(getReadDelay)
                SUCCESS: &lt;err&gt; = '0+1 records in\n0+1 records
                out\n341 bytes (341 B) copied, 0.000713967 s, 478
                kB/s\n'; &lt;rc&gt; = 0</span></p>
            <p class="p1"><span class="s1"><br>
              </span></p>
          </span></div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div>Issue to connect to the gluster storage.</div>
        <div>*****************************************************************************</div>
        <div>Feb 17 01:25:40 kvm2 ovirt-ha-agent:
          <a class="moz-txt-link-freetext" href="INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting">INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting</a>
          storage server</div>
        <div>Feb 17 01:25:40 kvm2 ovirt-ha-agent:
          <a class="moz-txt-link-freetext" href="INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting">INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting</a>
          storage server</div>
        <div>Feb 17 01:25:40 kvm2 journal: ovirt-ha-agent
          ovirt_hosted_engine_ha.agent.agent.Agent ERROR Error:
          'Connection to storage server failed' - trying to restart
          agent</div>
        <div>Feb 17 01:25:40 kvm2 ovirt-ha-agent:
          ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Error:
          'Connection to storage server failed' - trying to restart
          agent</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker:
          <a class="moz-txt-link-freetext" href="INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection">INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection</a>
          established</div>
        <div>Feb 17 01:25:41 kvm2 journal: ovirt-ha-broker
          ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR
          Error handling request, data: 'set-storage-domain
          FilesystemBackend dom_type=glusterfs
          sd_uuid=7d376952-312b-4539-b809-a8fa740f7883'#012Traceback
          (most recent call last):#012  File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py",
          line 166, in handle#012    data)#012  File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py",
          line 299, in _dispatch#012    .set_storage_domain(client,
          sd_type, **options)#012  File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py",
          line 66, in set_storage_domain#012  
           self._backends[client].connect()#012  File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py",
          line 456, in connect#012    self._dom_type)#012  File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py",
          line 108, in get_domain_path#012    " in {1}".format(sd_uuid,
          parent))#012BackendFailureException: path to storage domain
          7d376952-312b-4539-b809-a8fa740f7883 not found in
          /rhev/data-center/mnt/glusterSD</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker:
          ERROR:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Error
          handling request, data: 'set-storage-domain FilesystemBackend
          dom_type=glusterfs
          sd_uuid=7d376952-312b-4539-b809-a8fa740f7883'</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: Traceback (most
          recent call last):</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py",
          line 166, in handle</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: data)</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py",
          line 299, in _dispatch</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker:
          .set_storage_domain(client, sd_type, **options)</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py",
          line 66, in set_storage_domain</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker:
          self._backends[client].connect()</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py",
          line 456, in connect</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: self._dom_type)</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py",
          line 108, in get_domain_path</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker: " in
          {1}".format(sd_uuid, parent))</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker:
          BackendFailureException: path to storage domain
          7d376952-312b-4539-b809-a8fa740f7883 not found in
          /rhev/data-center/mnt/glusterSD</div>
        <div>Feb 17 01:25:41 kvm2 journal: vdsm vds ERROR failed to
          retrieve Hosted Engine HA info#012Traceback (most recent call
          last):#012  File "/usr/share/vdsm/API.py", line 1842, in
          _getHaInfo#012    stats = instance.get_all_stats()#012  File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
          line 103, in get_all_stats#012  
           self._configure_broker_conn(broker)#012  File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py",
          line 180, in _configure_broker_conn#012  
           dom_type=dom_type)#012  File
          "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py",
          line 176, in set_storage_domain#012    .format(sd_type,
          options, e))#012RequestError: Failed to set storage domain
          FilesystemBackend, options {'dom_type': 'glusterfs',
          'sd_uuid': '7d376952-312b-4539-b809-a8fa740f7883'}: Request
          failed: &lt;class
'ovirt_hosted_engine_ha.lib.storage_backends.BackendFailureException'&gt;</div>
        <div>Feb 17 01:25:41 kvm2 ovirt-ha-broker:
          <a class="moz-txt-link-freetext" href="INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection">INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection</a>
          closed</div>
        <div>^C</div>
        <div><br>
        </div>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>