1:40 p.m.
Hello,
I ,ve setup a test lab with 3 nodes installed with centos 7
I configured manualy gluster fs. Glusterfs is up and running
[root@kvm380 ~]# gluster peer status
Number of Peers: 2
Hostname: kvm320.durchhalten.intern
Uuid: dac066db-55f7-4770-900d-4830c740ffbf
State: Peer in Cluster (Connected)
Hostname: kvm360.durchhalten.intern
Uuid: 4291be40-f77f-4f41-98f6-dc48fd993842
State: Peer in Cluster (Connected)
[root@kvm380 ~]# gluster volume info
Volume Name: data
Type: Replicate
Volume ID: 3586de82-e504-4c62-972b-448abead13d3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: kvm380.durchhalten.intern:/gluster/data
Brick2: kvm360.durchhalten.intern:/gluster/data
Brick3: kvm320.durchhalten.intern:/gluster/data
Options Reconfigured:
storage.owner-uid: 36
storage.owner-gid: 36
features.shard: on
performance.low-prio-threads: 32
performance.strict-o-direct: on
network.ping-timeout: 30
user.cifs: off
network.remote-dio: off
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
cluster.eager-lock: enable
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
Volume Name: engine
Type: Replicate
Volume ID: dcfbd322-5dd0-4bfe-a775-99ecc79e1416
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: kvm380.durchhalten.intern:/gluster/engine
Brick2: kvm360.durchhalten.intern:/gluster/engine
Brick3: kvm320.durchhalten.intern:/gluster/engine
Options Reconfigured:
storage.owner-uid: 36
storage.owner-gid: 36
features.shard: on
performance.low-prio-threads: 32
performance.strict-o-direct: on
network.remote-dio: off
network.ping-timeout: 30
user.cifs: off
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
cluster.eager-lock: enable
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
After that I deployed a selfhosted engine
And add the two other hosts, at the beginning it looks good, but without
changing anything I got following error by two hosts
!
20.12.2018 11:35:05
Failed to connect Host kvm320.durchhalten.intern to Storage Pool Default
!
20.12.2018 11:35:05
Host kvm320.durchhalten.intern cannot access the Storage Domain(s)
hosted_storage attached to the Data Center Default. Setting Host state to
Non-Operational.
X
20.12.2018 11:35:05
Host kvm320.durchhalten.intern reports about one of the Active Storage
Domains as Problematic.
!
20.12.2018 11:35:05
Kdump integration is enabled for host kvm320.durchhalten.intern, but kdump
is not configured properly on host.
!
20.12.2018 11:35:04
Failed to connect Host kvm360.durchhalten.intern to Storage Pool Default
!
20.12.2018 11:35:04
Host kvm360.durchhalten.intern cannot access the Storage Domain(s)
hosted_storage attached to the Data Center Default. Setting Host state to
Non-Operational.
X
20.12.2018 11:35:04
Host kvm360.durchhalten.intern reports about one of the Active Storage
Domains as Problematic.
Before glusterfs I had a setup with nfs on 4. Server
Where is the problem?
thx
2:38 p.m.
here is what i found in the logs of the hosts
2018-12-20 12:34:04,824+0100 INFO (periodic/0) [vdsm.api] START repoStats(domains=())
from=internal, task_id=09235382-a5b5-48da-853d-f94cae092684 (api:46)
2018-12-20 12:34:04,825+0100 INFO (periodic/0) [vdsm.api] FINISH repoStats
return={u'20651d3d-08d7-482a-ae4e-7cd0e33cc907': {'code': 399,
'actual': True, 'version': -1, 'acquired': False, 'delay':
'0', 'lastCheck': '6.1', 'valid': False},
u'ae9e4cbd-3946-481d-b01a-e8a38bf00efb': {'code': 0, 'actual':
True, 'version': 4, 'acquired': True, 'delay':
'0.0013974', 'lastCheck': '1.1', 'valid': True}}
from=internal, task_id=09235382-a5b5-48da-853d-f94cae092684 (api:52)
2018-12-20 12:34:04,826+0100 INFO (periodic/0) [vdsm.api] START multipath_health()
from=internal, task_id=8f6166cb-aa41-4f46-823d-d38e4e85f02a (api:46)
2018-12-20 12:34:04,826+0100 INFO (periodic/0) [vdsm.api] FINISH multipath_health
return={} from=internal, task_id=8f6166cb-aa41-4f46-823d-d38e4e85f02a (api:52)
2018-12-20 12:34:04,832+0100 INFO (jsonrpc/4) [vdsm.api] START
prepareImage(sdUUID=u'20651d3d-08d7-482a-ae4e-7cd0e33cc907',
spUUID=u'00000000-0000-0000-0000-000000000000',
imgUUID=u'c1ebc7ad-dfb6-4cc1-8e24-40f0be3f4afe',
leafUUID=u'e7e76dd8-d166-46a0-9761-fa6391aa047b', allowIllegal=False)
from=::1,55348, task_id=996206fc-65eb-4056-b3b9-2ac0e1780c2c (api:46)
2018-12-20 12:34:04,836+0100 ERROR (periodic/0) [root] failed to retrieve Hosted Engine HA
score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished?
(api:196)
2018-12-20 12:34:04,847+0100 ERROR (jsonrpc/4) [storage.fileSD] Underlying file system
doesn't supportdirect IO (fileSD:108)
2018-12-20 12:34:04,847+0100 INFO (jsonrpc/4) [vdsm.api] FINISH prepareImage
error=Storage Domain target is unsupported: () from=::1,55348,
task_id=996206fc-65eb-4056-b3b9-2ac0e1780c2c (api:50)
2018-12-20 12:34:04,847+0100 ERROR (jsonrpc/4) [storage.TaskManager.Task]
(Task='996206fc-65eb-4056-b3b9-2ac0e1780c2c') Unexpected error (task:875)
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in
_run
return fn(*args, **kargs)
File "<string>", line 2, in prepareImage
File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in
method
ret = func(*args, **kwargs)
File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3173, in
prepareImage
dom = sdCache.produce(sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 110, in
produce
domain.getRealDomain()
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 51, in
getRealDomain
return self._cache._realProduce(self._sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 134, in
_realProduce
domain = self._findDomain(sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/sdc.py", line 151, in
_findDomain
return findMethod(sdUUID)
File "/usr/lib/python2.7/site-packages/vdsm/storage/glusterSD.py", line 55, in
findDomain
return GlusterStorageDomain(GlusterStorageDomain.findDomainPath(sdUUID))
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 400, in
__init__
validateFileSystemFeatures(manifest.sdUUID, manifest.mountpoint)
File "/usr/lib/python2.7/site-packages/vdsm/storage/fileSD.py", line 110, in
validateFileSystemFeatures
raise se.StorageDomainTargetUnsupported()
StorageDomainTargetUnsupported: Storage Domain target is unsupported: ()
2018-12-20 12:34:04,847+0100 INFO (jsonrpc/4) [storage.TaskManager.Task]
(Task='996206fc-65eb-4056-b3b9-2ac0e1780c2c') aborting: Task is aborted:
'Storage Domain target is unsupported: ()' - code 399 (task:1181)
2018-12-20 12:34:04,848+0100 ERROR (jsonrpc/4) [storage.Dispatcher] FINISH prepareImage
error=Storage Domain target is unsupported: () (dispatcher:82)
2018-12-20 12:34:04,848+0100 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call
Image.prepare failed (error 399) in 0.02 seconds (__init__:573)
^
especially this part
2018-12-20 12:34:04,847+0100 ERROR (jsonrpc/4) [storage.fileSD] Underlying file system
doesn't supportdirect IO (fileSD:108)
2018-12-20 12:34:04,847+0100 INFO (jsonrpc/4) [vdsm.api] FINISH prepareImage
error=Storage Domain target is unsupported: () from=::1,55348,
task_id=996206fc-65eb-4056-b3b9-2ac0e1780c2c (api:50)
and i am not sure why he is asking this
2018-12-20 12:34:04,836+0100 ERROR (periodic/0) [root] failed to retrieve Hosted Engine HA
score '[Errno 2] No such file or directory'Is the Hosted Engine setup finished?
(api:196)
3:14 p.m.
yes i think this too, but as you see at the top
[root@kvm380 ~]# gluster volume info
...
performance.strict-o-direct: on
...
it was already set
i did a one cluster setup with ovirt and I uses this result
Volume Name: engine
Type: Distribute
Volume ID: a40e848b-a8f1-4990-9d32-133b46db6f1d
Status: Started
Snapshot Count: 0
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: kvm360.durchhalten.intern:/gluster_bricks/engine/engine
Options Reconfigured:
cluster.eager-lock: enable
performance.io-cache: off
performance.read-ahead: off
performance.quick-read: off
user.cifs: off
network.ping-timeout: 30
network.remote-dio: off
performance.strict-o-direct: on
performance.low-prio-threads: 32
features.shard: on
storage.owner-gid: 36
storage.owner-uid: 36
transport.address-family: inet
nfs.disable: on
could there be an other reason?
3:42 p.m.
On 2018-12-20 07:14, Stefan Wolf wrote:
yes i think this too, but as you see at the top
> [root@kvm380 ~]# gluster volume info
> ...
> performance.strict-o-direct: on
...
it was already set
i did a one cluster setup with ovirt and I uses this result
Volume Name: engine
Type: Distribute
Volume ID: a40e848b-a8f1-4990-9d32-133b46db6f1d
Status: Started
Snapshot Count: 0
Number of Bricks: 1
Transport-type: tcp
Bricks:
Brick1: kvm360.durchhalten.intern:/gluster_bricks/engine/engine
Options Reconfigured:
cluster.eager-lock: enable
performance.io-cache: off
performance.read-ahead: off
performance.quick-read: off
user.cifs: off
network.ping-timeout: 30
network.remote-dio: off
performance.strict-o-direct: on
performance.low-prio-threads: 32
features.shard: on
storage.owner-gid: 36
storage.owner-uid: 36
transport.address-family: inet
nfs.disable: on
could there be an other reason?
are you mounting via the gluster GUI? I'm not sure how it handles
mounting of manual gluster volumes, but the direct-io-mode=enable mount
option comes to mind. I assume direct-io is also enabled on the other
volume? It needs to be on all of them.
3:53 p.m.
i 've mounted it during the hosted-engine --deploy process
I selected glusterfs
and entered server:/engine
I dont enter any mount options
yes it is enabled for both. I dont got errors for the second one, but may it doesn't
check after the first fail
4:29 p.m.
On 2018-12-20 07:53, Stefan Wolf wrote:
i 've mounted it during the hosted-engine --deploy process
I selected glusterfs
and entered server:/engine
I dont enter any mount options
yes it is enabled for both. I dont got errors for the second one, but
may it doesn't check after the first fail
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/SYX6BQBW2MM...
try server:/engine -o direct-io-mode=enable
2164
days inactive
2164
days old
6 comments
2 participants
participants (2)
-
Alex McWhirter -
Stefan Wolf