2:55 a.m.
When I was able to load CentosOS as a host OS, Was able to use RDMA, but it seems like
the 4.3x branch of nodeNG is missing RDMA support? I enabled rdma and started the
service, but gluster refuses to recognize that RDMA is available and always reports RDMA
port as 0 and when I try to make a new drive with tcp,rdma transport options, it always
fails.
1:04 p.m.
Il giorno ven 26 apr 2019 alle ore 01:56 <michael(a)wanderingmad.com> ha
scritto:
When I was able to load CentosOS as a host OS, Was able to use RDMA,
but
it seems like the 4.3x branch of nodeNG is missing RDMA support? I enabled
rdma and started the service, but gluster refuses to recognize that RDMA is
available and always reports RDMA port as 0 and when I try to make a new
drive with tcp,rdma transport options, it always fails.
Sahina, any hint?
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/site/privacy-policy/
oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/E73V472TTQV...
--
SANDRO BONAZZOLA
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo(a)redhat.com
<https://red.ht/sig>
3:49 p.m.
On Tue, Apr 30, 2019 at 3:35 PM Sandro Bonazzola <sbonazzo(a)redhat.com>
wrote:
Il giorno ven 26 apr 2019 alle ore 01:56 <michael(a)wanderingmad.com> ha
scritto:
> When I was able to load CentosOS as a host OS, Was able to use RDMA, but
> it seems like the 4.3x branch of nodeNG is missing RDMA support? I enabled
> rdma and started the service, but gluster refuses to recognize that RDMA is
> available and always reports RDMA port as 0 and when I try to make a new
> drive with tcp,rdma transport options, it always fails.
>
Sahina, any hint?
Adding Rafi. Any errors in glusterd logs?
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/E73V472TTQV...
>
--
SANDRO BONAZZOLA
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo(a)redhat.com
<https://red.ht/sig>
7:58 p.m.
this is what I see in the logs when I try to add RDMA:
[2019-05-06 16:54:50.305297] I [MSGID: 106521]
[glusterd-op-sm.c:2953:glusterd_op_set_volume] 0-management: changing transport-type for
volume storage_ssd to tcp,rdma
[2019-05-06 16:54:50.309122] W [MSGID: 101095] [xlator.c:180:xlator_volopt_dynload]
0-xlator: /usr/lib64/glusterfs/5.3/xlator/nfs/server.so: cannot open shared object file:
No such file or directory
[2019-05-06 16:54:50.321422] E [MSGID: 106068]
[glusterd-volgen.c:1025:volgen_write_volfile] 0-management: failed to create volfile
[2019-05-06 16:54:50.321463] E [glusterd-volgen.c:6556:glusterd_create_volfiles]
0-management: Could not generate gfproxy client volfiles
[2019-05-06 16:54:50.321476] E [MSGID: 106068]
[glusterd-op-sm.c:3062:glusterd_op_set_volume] 0-management: Unable to create volfile for
'volume set'
[2019-05-06 16:54:50.321488] E [MSGID: 106122] [glusterd-syncop.c:1434:gd_commit_op_phase]
0-management: Commit of operation 'Volume Set' failed on localhost
[2019-05-06 16:54:50.323610] E [MSGID: 101191]
[event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
rmda status:
root@Prometheus glusterfs]# systemctl status rdma
● rdma.service - Initialize the iWARP/InfiniBand/RDMA stack in the kernel
Loaded: loaded (/usr/lib/systemd/system/rdma.service; enabled; vendor preset:
disabled)
Active: active (exited) since Sun 2019-04-21 23:53:42 EDT; 2 weeks 0 days ago
Docs: file:/etc/rdma/rdma.conf
Main PID: 4968 (code=exited, status=0/SUCCESS)
Tasks: 0
CGroup: /system.slice/rdma.service
volume status:
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick 10.100.50.12:/gluster_bricks/storage_
nvme2/storage_nvme2 49153 0 Y 26888
Brick 10.100.50.14:/gluster_bricks/storage_
nvme2/storage_nvme2 49153 0 Y 3827
Brick 10.100.50.16:/gluster_bricks/storage_
nvme2/storage_nvme2 49153 0 Y 24238
Self-heal Daemon on localhost N/A N/A Y 29452
Self-heal Daemon on 10.100.50.16 N/A N/A Y 11058
Self-heal Daemon on 10.100.50.14 N/A N/A Y 8573
6:14 p.m.
Rafi, can you take a look?
On Mon, May 6, 2019 at 10:29 PM <michael(a)wanderingmad.com> wrote:
>
> this is what I see in the logs when I try to add RDMA:
>
> [2019-05-06 16:54:50.305297] I [MSGID: 106521]
[glusterd-op-sm.c:2953:glusterd_op_set_volume] 0-management: changing transport-type for
volume storage_ssd to tcp,rdma
> [2019-05-06 16:54:50.309122] W [MSGID: 101095] [xlator.c:180:xlator_volopt_dynload]
0-xlator: /usr/lib64/glusterfs/5.3/xlator/nfs/server.so: cannot open shared object file:
No such file or directory
> [2019-05-06 16:54:50.321422] E [MSGID: 106068]
[glusterd-volgen.c:1025:volgen_write_volfile] 0-management: failed to create volfile
> [2019-05-06 16:54:50.321463] E [glusterd-volgen.c:6556:glusterd_create_volfiles]
0-management: Could not generate gfproxy client volfiles
> [2019-05-06 16:54:50.321476] E [MSGID: 106068]
[glusterd-op-sm.c:3062:glusterd_op_set_volume] 0-management: Unable to create volfile for
'volume set'
> [2019-05-06 16:54:50.321488] E [MSGID: 106122]
[glusterd-syncop.c:1434:gd_commit_op_phase] 0-management: Commit of operation 'Volume
Set' failed on localhost
> [2019-05-06 16:54:50.323610] E [MSGID: 101191]
[event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
>
> rmda status:
> root@Prometheus glusterfs]# systemctl status rdma
> ● rdma.service - Initialize the iWARP/InfiniBand/RDMA stack in the kernel
> Loaded: loaded (/usr/lib/systemd/system/rdma.service; enabled; vendor preset:
disabled)
> Active: active (exited) since Sun 2019-04-21 23:53:42 EDT; 2 weeks 0 days ago
> Docs: file:/etc/rdma/rdma.conf
> Main PID: 4968 (code=exited, status=0/SUCCESS)
> Tasks: 0
> CGroup: /system.slice/rdma.service
>
> volume status:
>
> Gluster process TCP Port RDMA Port Online Pid
> ------------------------------------------------------------------------------
> Brick 10.100.50.12:/gluster_bricks/storage_
> nvme2/storage_nvme2 49153 0 Y 26888
> Brick 10.100.50.14:/gluster_bricks/storage_
> nvme2/storage_nvme2 49153 0 Y 3827
> Brick 10.100.50.16:/gluster_bricks/storage_
> nvme2/storage_nvme2 49153 0 Y 24238
> Self-heal Daemon on localhost N/A N/A Y 29452
> Self-heal Daemon on 10.100.50.16 N/A N/A Y 11058
> Self-heal Daemon on 10.100.50.14 N/A N/A Y 8573
>
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/4JZJXH3OFH3...
10:25 a.m.
On 5/7/19 8:44 PM, Sahina Bose wrote:
Rafi, can you take a look?
On Mon, May 6, 2019 at 10:29 PM <michael(a)wanderingmad.com> wrote:
> this is what I see in the logs when I try to add RDMA:
Are you trying to change the transport type of the volume. If so, Have
you followed the document [1] ?
[1] :
https://docs.gluster.org/en/latest/Administrator%20Guide/RDMA%20Transport/
>>
>> [2019-05-06 16:54:50.305297] I [MSGID: 106521]
[glusterd-op-sm.c:2953:glusterd_op_set_volume] 0-management: changing transport-type for
volume storage_ssd to tcp,rdma
>> [2019-05-06 16:54:50.309122] W [MSGID: 101095]
[xlator.c:180:xlator_volopt_dynload] 0-xlator:
/usr/lib64/glusterfs/5.3/xlator/nfs/server.so: cannot open shared object file: No such
file or directory
>> [2019-05-06 16:54:50.321422] E [MSGID: 106068]
[glusterd-volgen.c:1025:volgen_write_volfile] 0-management: failed to create volfile
>> [2019-05-06 16:54:50.321463] E [glusterd-volgen.c:6556:glusterd_create_volfiles]
0-management: Could not generate gfproxy client volfiles
>> [2019-05-06 16:54:50.321476] E [MSGID: 106068]
[glusterd-op-sm.c:3062:glusterd_op_set_volume] 0-management: Unable to create volfile for
'volume set'
>> [2019-05-06 16:54:50.321488] E [MSGID: 106122]
[glusterd-syncop.c:1434:gd_commit_op_phase] 0-management: Commit of operation 'Volume
Set' failed on localhost
>> [2019-05-06 16:54:50.323610] E [MSGID: 101191]
[event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
>>
>> rmda status:
>> root@Prometheus glusterfs]# systemctl status rdma
>> ● rdma.service - Initialize the iWARP/InfiniBand/RDMA stack in the kernel
>> Loaded: loaded (/usr/lib/systemd/system/rdma.service; enabled; vendor preset:
disabled)
>> Active: active (exited) since Sun 2019-04-21 23:53:42 EDT; 2 weeks 0 days
ago
>> Docs: file:/etc/rdma/rdma.conf
>> Main PID: 4968 (code=exited, status=0/SUCCESS)
>> Tasks: 0
>> CGroup: /system.slice/rdma.service
>>
>> volume status:
>>
>> Gluster process TCP Port RDMA Port Online Pid
>> ------------------------------------------------------------------------------
>> Brick 10.100.50.12:/gluster_bricks/storage_
>> nvme2/storage_nvme2 49153 0 Y 26888
>> Brick 10.100.50.14:/gluster_bricks/storage_
>> nvme2/storage_nvme2 49153 0 Y 3827
>> Brick 10.100.50.16:/gluster_bricks/storage_
>> nvme2/storage_nvme2 49153 0 Y 24238
>> Self-heal Daemon on localhost N/A N/A Y 29452
>> Self-heal Daemon on 10.100.50.16 N/A N/A Y 11058
>> Self-heal Daemon on 10.100.50.14 N/A N/A Y 8573
>>
>> _______________________________________________
>> Users mailing list -- users(a)ovirt.org
>> To unsubscribe send an email to users-leave(a)ovirt.org
>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
>> oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
>> List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/4JZJXH3OFH3...
12:13 a.m.
Tried that, get this error:
[root@Icarus ~]# gluster volume stop storage_ssd
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: storage_ssd: success
[root@Icarus ~]# gluster volume set storage_ssd config.transport tcp,rdma
volume set: failed: Commit failed on localhost. Please check the log file for more
details.
and here is the glusterd.log
[2019-05-09 21:10:45.945855] W [MSGID: 101095] [xlator.c:456:xlator_dynload] 0-xlator:
/usr/lib64/glusterfs/5.3/xlator/nfs/server.so: cannot open shared object file: No such
file or directory
[2019-05-09 21:10:46.041341] I [MSGID: 106521]
[glusterd-op-sm.c:2953:glusterd_op_set_volume] 0-management: changing transport-type for
volume storage_ssd to tcp,rdma
[2019-05-09 21:10:46.041965] W [MSGID: 101095] [xlator.c:180:xlator_volopt_dynload]
0-xlator: /usr/lib64/glusterfs/5.3/xlator/nfs/server.so: cannot open shared object file:
No such file or di
rectory
[2019-05-09 21:10:46.059356] E [MSGID: 106068]
[glusterd-volgen.c:1025:volgen_write_volfile] 0-management: failed to create volfile
[2019-05-09 21:10:46.059394] E [glusterd-volgen.c:6556:glusterd_create_volfiles]
0-management: Could not generate gfproxy client volfiles
[2019-05-09 21:10:46.059406] E [MSGID: 106068]
[glusterd-op-sm.c:3062:glusterd_op_set_volume] 0-management: Unable to create volfile for
'volume set'
[2019-05-09 21:10:46.059420] E [MSGID: 106122] [glusterd-syncop.c:1434:gd_commit_op_phase]
0-management: Commit of operation 'Volume Set' failed on localhost
[2019-05-09 21:10:46.062304] E [MSGID: 101191]
[event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler
[root@prometheus ~]#
1832
days inactive
1846
days old
6 comments
4 participants
participants (4)
-
michael＠wanderingmad.com -
RAFI KC -
Sahina Bose -
Sandro Bonazzola