Active Storage Domains as Problematic
Hello, I ,ve setup a test lab with 3 nodes installed with centos 7 I configured manualy gluster fs. Glusterfs is up and running [root@kvm380 ~]# gluster peer status Number of Peers: 2 Hostname: kvm320.durchhalten.intern Uuid: dac066db-55f7-4770-900d-4830c740ffbf State: Peer in Cluster (Connected) Hostname: kvm360.durchhalten.intern Uuid: 4291be40-f77f-4f41-98f6-dc48fd993842 State: Peer in Cluster (Connected) [root@kvm380 ~]# gluster volume info Volume Name: data Type: Replicate Volume ID: 3586de82-e504-4c62-972b-448abead13d3 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: kvm380.durchhalten.intern:/gluster/data Brick2: kvm360.durchhalten.intern:/gluster/data Brick3: kvm320.durchhalten.intern:/gluster/data Options Reconfigured: storage.owner-uid: 36 storage.owner-gid: 36 features.shard: on performance.low-prio-threads: 32 performance.strict-o-direct: on network.ping-timeout: 30 user.cifs: off network.remote-dio: off performance.quick-read: off performance.read-ahead: off performance.io-cache: off cluster.eager-lock: enable transport.address-family: inet nfs.disable: on performance.client-io-threads: off Volume Name: engine Type: Replicate Volume ID: dcfbd322-5dd0-4bfe-a775-99ecc79e1416 Status: Started Snapshot Count: 0 Number of Bricks: 1 x 3 = 3 Transport-type: tcp Bricks: Brick1: kvm380.durchhalten.intern:/gluster/engine Brick2: kvm360.durchhalten.intern:/gluster/engine Brick3: kvm320.durchhalten.intern:/gluster/engine Options Reconfigured: storage.owner-uid: 36 storage.owner-gid: 36 features.shard: on performance.low-prio-threads: 32 performance.strict-o-direct: on network.remote-dio: off network.ping-timeout: 30 user.cifs: off performance.quick-read: off performance.read-ahead: off performance.io-cache: off cluster.eager-lock: enable transport.address-family: inet nfs.disable: on performance.client-io-threads: off After that I deployed a selfhosted engine And add the two other hosts, at the beginning it looks good, but without changing anything I got following error by two hosts ! 20.12.2018 11:35:05 Failed to connect Host kvm320.durchhalten.intern to Storage Pool Default ! 20.12.2018 11:35:05 Host kvm320.durchhalten.intern cannot access the Storage Domain(s) hosted_storage attached to the Data Center Default. Setting Host state to Non-Operational. X 20.12.2018 11:35:05 Host kvm320.durchhalten.intern reports about one of the Active Storage Domains as Problematic. ! 20.12.2018 11:35:05 Kdump integration is enabled for host kvm320.durchhalten.intern, but kdump is not configured properly on host. ! 20.12.2018 11:35:04 Failed to connect Host kvm360.durchhalten.intern to Storage Pool Default ! 20.12.2018 11:35:04 Host kvm360.durchhalten.intern cannot access the Storage Domain(s) hosted_storage attached to the Data Center Default. Setting Host state to Non-Operational. X 20.12.2018 11:35:04 Host kvm360.durchhalten.intern reports about one of the Active Storage Domains as Problematic. Before glusterfs I had a setup with nfs on 4. Server Where is the problem? thx
here is what i found in the logs of the hosts 2018-12-20 12:34:04,824+0100 INFO (periodic/0) [vdsm.api] START repoStats(domains=()) from=internal, task_id=09235382-a5b5-48da-853d-f94cae092684 (api:46) 2018-12-20 12:34:04,825+0100 INFO (periodic/0) [vdsm.api] FINISH repoStats return={u'20651d3d-08d7-482a-ae4e-7cd0e33cc907': {'code': 399, 'actual': True, 'version': -1, 'acquired': False, 'delay': '0', 'lastCheck': '6.1', 'valid': False}, u'ae9e4cbd-3946-481d-b01a-e8a38bf00efb': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.0013974', 'lastCheck': '1.1', 'valid': True}} from=internal, task_id=09235382-a5b5-48da-853d-f94cae092684 (api:52) 2018-12-20 12:34:04,826+0100 INFO (periodic/0) [vdsm.api] START multipath_health() from=internal, task_id=8f6166cb-aa41-4f46-823d-d38e4e85f02a (api:46) 2018-12-20 12:34:04,826+0100 INFO (periodic/0) [vdsm.api] FINISH multipath_health return={} from=internal, task_id=8f6166cb-aa41-4f46-823d-d38e4e85f02a (api:52) 2018-12-20 12:34:04,832+0100 INFO (jsonrpc/4) [vdsm.api] START prepareImage(sdUUID=u'20651d3d-08d7-482a-ae4e-7cd0e33cc907', spUUID=u'00000000-0000-0000-0000-000000000000', imgUUID=u'c1ebc7ad-dfb6-4cc1-8e24-40f0be3f4afe', leafUUID=u'e7e76dd8-d166-46a0-9761-fa6391aa047b', allowIllegal=False) from=::1,55348, task_id=996206fc-65eb-4056-b3b9-2ac0e1780c2c (api:46) 2018-12-20 12:34:04,836+0100 ERROR (periodic/0) [root] failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? (api:196) 2018-12-20 12:34:04,847+0100 ERROR (jsonrpc/4) [storage.fileSD] Underlying file system doesn't supportdirect IO (fileSD:108) 2018-12-20 12:34:04,847+0100 INFO (jsonrpc/4) [vdsm.api] FINISH prepareImage error=Storage Domain target is unsupported: () from=::1,55348, task_id=996206fc-65eb-4056-b3b9-2ac0e1780c2c (api:50) 2018-12-20 12:34:04,847+0100 ERROR (jsonrpc/4) [storage.TaskManager.Task] (Task='996206fc-65eb-4056-b3b9-2ac0e1780c2c') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in prepareImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3173, in prepareImage dom = sdCache.produce(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110, in produce domain.getRealDomain() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51, in getRealDomain return self._cache._realProduce(self._sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134, in _realProduce domain = self._findDomain(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 151, in _findDomain return findMethod(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/glusterSD.py", line 55, in findDomain return GlusterStorageDomain(GlusterStorageDomain.findDomainPath(sdUUID)) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 400, in __init__ validateFileSystemFeatures(manifest.sdUUID, manifest.mountpoint) File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 110, in validateFileSystemFeatures raise se.StorageDomainTargetUnsupported() StorageDomainTargetUnsupported: Storage Domain target is unsupported: () 2018-12-20 12:34:04,847+0100 INFO (jsonrpc/4) [storage.TaskManager.Task] (Task='996206fc-65eb-4056-b3b9-2ac0e1780c2c') aborting: Task is aborted: 'Storage Domain target is unsupported: ()' - code 399 (task:1181) 2018-12-20 12:34:04,848+0100 ERROR (jsonrpc/4) [storage.Dispatcher] FINISH prepareImage error=Storage Domain target is unsupported: () (dispatcher:82) 2018-12-20 12:34:04,848+0100 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call Image.prepare failed (error 399) in 0.02 seconds (__init__:573) ^ especially this part 2018-12-20 12:34:04,847+0100 ERROR (jsonrpc/4) [storage.fileSD] Underlying file system doesn't supportdirect IO (fileSD:108) 2018-12-20 12:34:04,847+0100 INFO (jsonrpc/4) [vdsm.api] FINISH prepareImage error=Storage Domain target is unsupported: () from=::1,55348, task_id=996206fc-65eb-4056-b3b9-2ac0e1780c2c (api:50) and i am not sure why he is asking this 2018-12-20 12:34:04,836+0100 ERROR (periodic/0) [root] failed to retrieve Hosted Engine HA score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished? (api:196)
yes i think this too, but as you see at the top
[root@kvm380 ~]# gluster volume info ... performance.strict-o-direct: on ... it was already set
i did a one cluster setup with ovirt and I uses this result Volume Name: engine Type: Distribute Volume ID: a40e848b-a8f1-4990-9d32-133b46db6f1d Status: Started Snapshot Count: 0 Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: kvm360.durchhalten.intern:/gluster_bricks/engine/engine Options Reconfigured: cluster.eager-lock: enable performance.io-cache: off performance.read-ahead: off performance.quick-read: off user.cifs: off network.ping-timeout: 30 network.remote-dio: off performance.strict-o-direct: on performance.low-prio-threads: 32 features.shard: on storage.owner-gid: 36 storage.owner-uid: 36 transport.address-family: inet nfs.disable: on could there be an other reason?
On 2018-12-20 07:14, Stefan Wolf wrote:
yes i think this too, but as you see at the top
[root@kvm380 ~]# gluster volume info ... performance.strict-o-direct: on ... it was already set
i did a one cluster setup with ovirt and I uses this result
Volume Name: engine Type: Distribute Volume ID: a40e848b-a8f1-4990-9d32-133b46db6f1d Status: Started Snapshot Count: 0 Number of Bricks: 1 Transport-type: tcp Bricks: Brick1: kvm360.durchhalten.intern:/gluster_bricks/engine/engine Options Reconfigured: cluster.eager-lock: enable performance.io-cache: off performance.read-ahead: off performance.quick-read: off user.cifs: off network.ping-timeout: 30 network.remote-dio: off performance.strict-o-direct: on performance.low-prio-threads: 32 features.shard: on storage.owner-gid: 36 storage.owner-uid: 36 transport.address-family: inet nfs.disable: on
could there be an other reason?
are you mounting via the gluster GUI? I'm not sure how it handles mounting of manual gluster volumes, but the direct-io-mode=enable mount option comes to mind. I assume direct-io is also enabled on the other volume? It needs to be on all of them.
i 've mounted it during the hosted-engine --deploy process I selected glusterfs and entered server:/engine I dont enter any mount options yes it is enabled for both. I dont got errors for the second one, but may it doesn't check after the first fail
On 2018-12-20 07:53, Stefan Wolf wrote:
i 've mounted it during the hosted-engine --deploy process I selected glusterfs and entered server:/engine I dont enter any mount options yes it is enabled for both. I dont got errors for the second one, but may it doesn't check after the first fail _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/SYX6BQBW2MMV4Y...
try server:/engine -o direct-io-mode=enable
participants (2)
-
Alex McWhirter -
Stefan Wolf