Hello!

- 1 oVirt Engine (not hosted), 4 nodes, 3 in "Default" datacenter, 1 in a local-storage type datacenter;

- Recently upgraded from 3.4.2 to 3.4.3;

"Default"Datacenter has, along "mandatory"ISO and Export domain, two storage domains "storage_gluster2" aprox 550Gb  and "storage_gluster3" aprox 130 GB.

As you can guess :-), the three nodes serve also as gluster bricks, each with 2 bricks in the above mentioned storage domains, both in replicated mode.

Master domain is "storage_gluster3".

After create new VM, trying to add new disk on "storage_gluster2 ", which is NOT master domain raise (ovirt-engine.log)



-- Message: VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = [Errno 2] No such file or directory: '/rhev/data-center/5849b030-626e-47cb-ad90-3ce782d831b3/b694642b-4603-4dc6-ba36-c0d82ca83fa2/images/5de04fd7-a8a5-483a-be53-cffb17db3d94', code = 100,
-- Exception: VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = [Errno 2] No such file or directory: '/rhev/data-center/5849b030-626e-47cb-ad90-3ce782d831b3/b694642b-4603-4dc6-ba36-c0d82ca83fa2/images/5de04fd7-a8a5-483a-be53-cffb17db3d94', code = 100

On SPM:


vdsClient -s 0 getConnectedStoragePoolsList
5849b030-626e-47cb-ad90-3ce782d831b3


vdsClient -s 0 getStoragePoolInfo 5849b030-626e-47cb-ad90-3ce782d831b3
        name = Default
        isoprefix = /rhev/data-center/mnt/10.125.1.193:_mnt_storage__filer1_export__nfs2_iso/bb09aac2-05d5-45c5-b573-ebff9292ad6d/images/11111111-1111-1111-1111-111111111111
        pool_status = connected
        lver = 23
        spm_id = 1
        master_uuid = aa401118-fc30-4399-8f01-f1751f89fff4
        version = 3
        domains = b694642b-4603-4dc6-ba36-c0d82ca83fa2:Active,bb09aac2-05d5-45c5-b573-ebff9292ad6d:Active,aa401118-fc30-4399-8f01-f1751f89fff4:Active,62f795ed-486c-443d-93e9-2c16595fe546:Active
        type = GLUSTERFS
        master_ver = 1136
        b694642b-4603-4dc6-ba36-c0d82ca83fa2 = {'status': 'Active', 'diskfree': '324000022528', 'isoprefix': '', 'alerts': [], 'disktotal': '590231371776', 'version': 3}
        bb09aac2-05d5-45c5-b573-ebff9292ad6d = {'status': 'Active', 'diskfree': '1904214016', 'isoprefix': '/rhev/data-center/mnt/10.125.1.193:_mnt_storage__filer1_export__nfs2_iso/bb09aac2-05d5-45c5-b573-ebff9292ad6d/images/11111111-1111-1111-1111-111111111111', 'alerts': [], 'disktotal': '8049917952', 'version': 0}
        aa401118-fc30-4399-8f01-f1751f89fff4 = {'status': 'Active', 'diskfree': '111680159744', 'isoprefix': '', 'alerts': [], 'disktotal': '144146563072', 'version': 3}
        62f795ed-486c-443d-93e9-2c16595fe546 = {'status': 'Active', 'diskfree': '131499819008', 'isoprefix': '', 'alerts': [], 'disktotal': '234763583488', 'version': 0}

mount 

.....

127.0.0.1:gluster_data3 on /rhev/data-center/mnt/glusterSD/127.0.0.1:gluster__data3 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
127.0.0.1:gluster_data2 on /rhev/data-center/mnt/glusterSD/127.0.0.1:gluster__data2 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
........

ls -la /rhev/data-center/mnt/
total 20
drwxr-xr-x. 5 vdsm kvm 4096 Sep  2 16:46 .
drwxr-xr-x. 5 vdsm kvm 4096 Sep  2 16:46 ..
drwxrwsrwx+ 3   96  96 4096 Feb  7  2014 10.125.1.193:_mnt_storage__filer1_export__nfs1_export
drwxrwsrwx+ 3   96  96 4096 May  8 17:16 10.125.1.193:_mnt_storage__filer1_export__nfs2_iso
drwxr-xr-x. 5 vdsm kvm 4096 Sep  2 16:45 glusterSD

but no:

'/rhev/data-center/5849b030-626e-47cb-ad90-3ce782d831b3/b694642b-4603-4dc6-ba36-c0d82ca83fa2/images/5de04fd7-a8a5-483a-be53-cffb17db3d94',

as per error from engine log posted above.

Do you have any opinions/ideeas/advice?

I''ll try to restart the whole thing ( engine plus 3 nodes) but , since it is in production it is a "'little bit" dificult :-)