From the host-deploy logs - the error seems to be "RuntimeError: Failed to start service 'glusterd'"

Can you try to start the glusterd service (systemctl start glusterd) on the failed nodes? Please attach /var/log/glusterfs/etc-glusterfs-glusterd.vol.log to analyze the failure.

On Wed, Nov 2, 2016 at 4:31 AM, Thing <thing.thing@gmail.com> wrote:
DNS responds fine,

==========
[root@ovirt1 host-deploy]# host 192.168.1.31
31.1.168.192.in-addr.arpa domain name pointer glusterp1.ods.graywitch.co.nz.
[root@ovirt1 host-deploy]# host 192.168.1.32
32.1.168.192.in-addr.arpa domain name pointer glusterp2.ods.graywitch.co.nz.
[root@ovirt1 host-deploy]# host 192.168.1.33
33.1.168.192.in-addr.arpa domain name pointer glusterp3.ods.graywitch.co.nz.
[root@ovirt1 host-deploy]# host 192.168.1.34
34.1.168.192.in-addr.arpa domain name pointer ovirt1.ods.graywitch.co.nz.
[root@ovirt1 host-deploy]# host glusterp1.ods.graywitch.co.nz
glusterp1.ods.graywitch.co.nz has address 192.168.1.31
[root@ovirt1 host-deploy]# host glusterp2.ods.graywitch.co.nz
glusterp2.ods.graywitch.co.nz has address 192.168.1.32
[root@ovirt1 host-deploy]# host glusterp3.ods.graywitch.co.nz
glusterp3.ods.graywitch.co.nz has address 192.168.1.33
[root@ovirt1 host-deploy]# more /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.31    glusterp1.ods.graywitch.co.nz       glusterp1
192.168.1.32    glusterp2.ods.graywitch.co.nz       glusterp2
192.168.1.33    glusterp3.ods.graywitch.co.nz       glusterp3
192.168.1.34    ovirt1.ods.graywitch.co.nz    ovirt1
[root@ovirt1 host-deploy]# host glusterp2.ods.graywitch.co.nz glusterp1.ods.graywitch.co.nz
Using domain server:
Name: glusterp1.ods.graywitch.co.nz
Address: 192.168.1.31#53
Aliases:

glusterp2.ods.graywitch.co.nz has address 192.168.1.32
[root@ovirt1 host-deploy]# host glusterp1.ods.graywitch.co.nz glusterp1.ods.graywitch.co.nz
Using domain server:
Name: glusterp1.ods.graywitch.co.nz
Address: 192.168.1.31#53
Aliases:

glusterp1.ods.graywitch.co.nz has address 192.168.1.31
[root@ovirt1 host-deploy]# host glusterp3.ods.graywitch.co.nz glusterp1.ods.graywitch.co.nz
Using domain server:
Name: glusterp1.ods.graywitch.co.nz
Address: 192.168.1.31#53
Aliases:

glusterp3.ods.graywitch.co.nz has address 192.168.1.33
[root@ovirt1 host-deploy]# host glusterp2.ods.graywitch.co.nz glusterp2.ods.graywitch.co.nz
Using domain server:
Name: glusterp2.ods.graywitch.co.nz
Address: 192.168.1.32#53
Aliases:

glusterp2.ods.graywitch.co.nz has address 192.168.1.32
[root@ovirt1 host-deploy]# host glusterp1.ods.graywitch.co.nz glusterp2.ods.graywitch.co.nz
Using domain server:
Name: glusterp2.ods.graywitch.co.nz
Address: 192.168.1.32#53
Aliases:

glusterp1.ods.graywitch.co.nz has address 192.168.1.31
[root@ovirt1 host-deploy]# host glusterp3.ods.graywitch.co.nz glusterp2.ods.graywitch.co.nz
Using domain server:
Name: glusterp2.ods.graywitch.co.nz
Address: 192.168.1.32#53
Aliases:

glusterp3.ods.graywitch.co.nz has address 192.168.1.33
[root@ovirt1 host-deploy]# host ovirt1.ods.graywitch.co.nz glusterp2.ods.graywitch.co.nz
Using domain server:
Name: glusterp2.ods.graywitch.co.nz
Address: 192.168.1.32#53
Aliases:

ovirt1.ods.graywitch.co.nz has address 192.168.1.34
[root@ovirt1 host-deploy]#
==========

ssh keys work fine from the command line,

==========
[root@ovirt1 host-deploy]# uname -a
Linux ovirt1.ods.graywitch.co.nz 3.10.0-327.36.2.el7.x86_64 #1 SMP Mon Oct 10 23:08:37 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
[root@ovirt1 host-deploy]# ssh glusterp1
Last login: Wed Nov  2 10:46:58 2016 from ovirt1.ods.graywitch.co.nz
[root@glusterp1 ~]# ^C
[root@glusterp1 ~]# logout
Connection to glusterp1 closed.
[root@ovirt1 host-deploy]# ssh glusterp2
Last login: Wed Nov  2 10:14:09 2016
[root@glusterp2 ~]# logout
Connection to glusterp2 closed.
[root@ovirt1 host-deploy]# ssh glusterp3
Last login: Wed Nov  2 10:14:19 2016
[root@glusterp3 ~]# logout
Connection to glusterp3 closed.
[root@ovirt1 host-deploy]#
==========

iptables is disabled,

========
[root@glusterp2 log]# iptables -L -n
Chain INPUT (policy ACCEPT)
target     prot opt source               destination        

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination        

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination        
[root@glusterp2 log]#
=========







On 2 November 2016 at 11:24, Thing <thing.thing@gmail.com> wrote:
I have 3 gluster nodes, 1 and 2 repeatedly fail to come in to ovirt but node 3 worked first time.

It appears to be saying ssh-keys / root password is failing, but I can ssh in fine from teh command line so this message makes no sense.

attached is a host deploy log






_______________________________________________
Users mailing list
Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users