[ovirt-users] oVirt 3.6.2 issue when updating to gluster 3.7.8

Sahina Bose sabose at redhat.com
Wed Feb 17 13:13:14 UTC 2016 


On 02/17/2016 04:57 PM, Luiz Claudio Prazeres Goncalves wrote:
>
> Hi, I realised that gluster 3.7.8 was released for GA. So I updated 
> manually using "yum -y install glusterfs*" gluster get's updated 
> normally, but  unfortunately things stopped to work completely. 
> ovirt_hosted_engine_ha was not able to connect to my gluster storage 
> domain (as you can see below). I'm not using ovirt + gluster in 
> hyperconverged way. I have 3 external gluster hosts.


What does the mount log on the hypervisor say? Could you attach that?

>
> As a workaround I've executed "*yum*downgrade glusterfs*" and after 
> doing this ovirt started to work again...but as a collateral effect 
> now I can't migrate my vm's anymore.  On the vdsm logs I can see the 
> following errors when trying to manually migrate a any VM.... Anyone 
> knows how to fix it?
>
> Thread-66055::DEBUG::2016-02-17 
> 11:05:57,497::migration::453::virt.vm::(stop) 
> vmId=`05681896-a76a-4ae1-879e-8fe5d28634e1`::stopping migration 
> downtime thread
>
> Thread-66055::ERROR::2016-02-17 
> 11:05:57,497::migration::208::virt.vm::(_recover) 
> vmId=`05681896-a76a-4ae1-879e-8fe5d28634e1`::unable to connect to 
> server at 'kvm2.brightsquid.com:49152 
> <http://kvm2.brightsquid.com:49152>': No route to host
>
> Thread-66055::DEBUG::2016-02-17 
> 11:05:57,497::stompreactor::389::jsonrpc.AsyncoreClient::(send) 
> Sending response
>
> Thread-66056::DEBUG::2016-02-17 
> 11:05:57,498::migration::450::virt.vm::(run) 
> vmId=`05681896-a76a-4ae1-879e-8fe5d28634e1`::migration downtime thread 
> exiting
>
> Thread-66055::DEBUG::2016-02-17 
> 11:05:57,540::__init__::206::jsonrpc.Notification::(emit) Sending 
> event {"params": {"notify_time": 4327804740, 
> "05681896-a76a-4ae1-879e-8fe5d28634e1": {"status": "Migration 
> Source"}}, "jsonrpc": "2.0", "method": 
> "|virt|VM_status|05681896-a76a-4ae1-879e-8fe5d28634e1"}
>
> Thread-66055::ERROR::2016-02-17 
> 11:05:57,541::migration::310::virt.vm::(run) 
> vmId=`05681896-a76a-4ae1-879e-8fe5d28634e1`::Failedto migrate
>
> Traceback (most recent call last):
>
>   File "/usr/share/vdsm/virt/migration.py", line 294, in run
>
> self._startUnderlyingMigration(time.time())
>
>   File "/usr/share/vdsm/virt/migration.py", line 364, in 
> _startUnderlyingMigration
>
> self._perform_migration(duri, muri)
>
>   File "/usr/share/vdsm/virt/migration.py", line 403, in 
> _perform_migration
>
> self._vm._dom.migrateToURI3(duri, params, flags)
>
>   File "/usr/share/vdsm/virt/virdomain.py", line 68, in f
>
>     ret = attr(*args, **kwargs)
>
>   File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", 
> line 124, in wrapper
>
>     ret = f(*args, **kwargs)
>
>   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 1836, in 
> migrateToURI3
>
>     if ret == -1: raise libvirtError ('virDomainMigrateToURI3() 
> failed', dom=self)
>
> libvirtError: unable to connect to server at 
> 'kvm2.brightsquid.com:49152 <http://kvm2.brightsquid.com:49152>': No 
> route to host
>
> Thread-68::DEBUG::2016-02-17 
> 11:05:57,652::fileSD::173::Storage.Misc.excCmd::(getReadDelay) 
> /usr/bin/taskset --cpu-list 0-39 /usr/bin/dd 
> if=/rhev/data-center/mnt/gluster2.brightsquid.com:_home_export_iso/61827b7b-e255-44f5-a791-482a144be29f/dom_md/metadata 
> iflag=direct of=/dev/null bs=4096 count=1 (cwd None)
>
> Thread-68::DEBUG::2016-02-17 
> 11:05:57,662::fileSD::173::Storage.Misc.excCmd::(getReadDelay) 
> SUCCESS: <err> = '0+1 records in\n0+1 records out\n341 bytes (341 B) 
> copied, 0.000713967 s, 478 kB/s\n'; <rc> = 0
>
>
>
>
>
>
>
>
>
>
> Issue to connect to the gluster storage.
> *****************************************************************************
> Feb 17 01:25:40 kvm2 ovirt-ha-agent: 
> INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting storage 
> server
> Feb 17 01:25:40 kvm2 ovirt-ha-agent: 
> INFO:ovirt_hosted_engine_ha.lib.storage_server.StorageServer:Connecting storage 
> server
> Feb 17 01:25:40 kvm2 journal: ovirt-ha-agent 
> ovirt_hosted_engine_ha.agent.agent.Agent ERROR Error: 'Connection to 
> storage server failed' - trying to restart agent
> Feb 17 01:25:40 kvm2 ovirt-ha-agent: 
> ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Error: 'Connection to 
> storage server failed' - trying to restart agent
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: 
> INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection 
> established
> Feb 17 01:25:41 kvm2 journal: ovirt-ha-broker 
> ovirt_hosted_engine_ha.broker.listener.ConnectionHandler ERROR Error 
> handling request, data: 'set-storage-domain FilesystemBackend 
> dom_type=glusterfs 
> sd_uuid=7d376952-312b-4539-b809-a8fa740f7883'#012Traceback (most 
> recent call last):#012  File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", 
> line 166, in handle#012    data)#012  File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", 
> line 299, in _dispatch#012    .set_storage_domain(client, sd_type, 
> **options)#012  File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", 
> line 66, in set_storage_domain#012 
>  self._backends[client].connect()#012  File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", 
> line 456, in connect#012    self._dom_type)#012  File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", 
> line 108, in get_domain_path#012    " in {1}".format(sd_uuid, 
> parent))#012BackendFailureException: path to storage domain 
> 7d376952-312b-4539-b809-a8fa740f7883 not found in 
> /rhev/data-center/mnt/glusterSD
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: 
> ERROR:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Error 
> handling request, data: 'set-storage-domain FilesystemBackend 
> dom_type=glusterfs sd_uuid=7d376952-312b-4539-b809-a8fa740f7883'
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: Traceback (most recent call last):
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", 
> line 166, in handle
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: data)
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/listener.py", 
> line 299, in _dispatch
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: .set_storage_domain(client, 
> sd_type, **options)
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/broker/storage_broker.py", 
> line 66, in set_storage_domain
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: self._backends[client].connect()
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", 
> line 456, in connect
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: self._dom_type)
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/storage_backends.py", 
> line 108, in get_domain_path
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: " in {1}".format(sd_uuid, parent))
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: BackendFailureException: path to 
> storage domain 7d376952-312b-4539-b809-a8fa740f7883 not found in 
> /rhev/data-center/mnt/glusterSD
> Feb 17 01:25:41 kvm2 journal: vdsm vds ERROR failed to retrieve Hosted 
> Engine HA info#012Traceback (most recent call last):#012  File 
> "/usr/share/vdsm/API.py", line 1842, in _getHaInfo#012    stats = 
> instance.get_all_stats()#012  File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", 
> line 103, in get_all_stats#012 
>  self._configure_broker_conn(broker)#012  File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", 
> line 180, in _configure_broker_conn#012  dom_type=dom_type)#012  File 
> "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", 
> line 176, in set_storage_domain#012    .format(sd_type, options, 
> e))#012RequestError: Failed to set storage domain FilesystemBackend, 
> options {'dom_type': 'glusterfs', 'sd_uuid': 
> '7d376952-312b-4539-b809-a8fa740f7883'}: Request failed: <class 
> 'ovirt_hosted_engine_ha.lib.storage_backends.BackendFailureException'>
> Feb 17 01:25:41 kvm2 ovirt-ha-broker: 
> INFO:ovirt_hosted_engine_ha.broker.listener.ConnectionHandler:Connection 
> closed
> ^C
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20160217/4d46fcb0/attachment-0001.html>

More information about the Users mailing list