8:57 a.m.
Hello all,
I am hoping someone can help me with a oVirt installation that has just gotten the better
of me after weeks of trying.
After setting up ssh-keys and making sure each host is known to the primary host
(sr-svr04), I go though Cockpit and "Configure Gluster storage and oVirt hosted
engine", enter all of the details with <host>.san.lennoxconsulting.com.au for
the storage network FQDN and <host>.core.lennoxconsulting.com.au for the public
interfaces. Connectivity on each of the VLAN's test out as basically working
(everything is pingable and ssh connections work) and the hosts are generally usable on
the network.
But the install ultimately dies with the following ansible error:
----------- gluster-deployment.log ---------
:
:
TASK [gluster.infra/roles/backend_setup : Create volume groups] ****************
task path: /etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/vg_create.yml:63
failed: [sr-svr04.san.lennoxconsulting.com.au] (item={'key':
'gluster_vg_sdb', 'value': [{'vgname': 'gluster_vg_sdb',
'pvname': '/dev/sdb'}]}) => {"ansible_loop_var":
"item", "changed": true, "cmd": ["vgcreate",
"--dataalignment", "1536K", "-s", "1536K",
"gluster_vg_sdb", "/dev/sdb"], "delta":
"0:00:00.058528", "end": "2022-07-16 16:18:37.018563",
"item": {"key": "gluster_vg_sdb", "value":
[{"pvname": "/dev/sdb", "vgname":
"gluster_vg_sdb"}]}, "msg": "non-zero return code",
"rc": 5, "start": "2022-07-16 16:18:36.960035",
"stderr": " A volume group called gluster_vg_sdb already exists.",
"stderr_lines": [" A volume group called gluster_vg_sdb already
exists."], "stdout": "", "stdout_lines": []}
failed: [sr-svr05.san.lennoxconsulting.com.au] (item={'key':
'gluster_vg_sdb', 'value': [{'vgname': 'gluster_vg_sdb',
'pvname': '/dev/sdb'}]}) => {"ansible_loop_var":
"item", "changed": true, "cmd": ["vgcreate",
"--dataalignment", "1536K", "-s", "1536K",
"gluster_vg_sdb", "/dev/sdb"], "delta":
"0:00:00.057186", "end": "2022-07-16 16:18:37.784063",
"item": {"key": "gluster_vg_sdb", "value":
[{"pvname": "/dev/sdb", "vgname":
"gluster_vg_sdb"}]}, "msg": "non-zero return code",
"rc": 5, "start": "2022-07-16 16:18:37.726877",
"stderr": " A volume group called gluster_vg_sdb already exists.",
"stderr_lines": [" A volume group called gluster_vg_sdb already
exists."], "stdout": "", "stdout_lines": []}
failed: [sr-svr06.san.lennoxconsulting.com.au] (item={'key':
'gluster_vg_sdb', 'value': [{'vgname': 'gluster_vg_sdb',
'pvname': '/dev/sdb'}]}) => {"ansible_loop_var":
"item", "changed": true, "cmd": ["vgcreate",
"--dataalignment", "1536K", "-s", "1536K",
"gluster_vg_sdb", "/dev/sdb"], "delta":
"0:00:00.062212", "end": "2022-07-16 16:18:37.250371",
"item": {"key": "gluster_vg_sdb", "value":
[{"pvname": "/dev/sdb", "vgname":
"gluster_vg_sdb"}]}, "msg": "non-zero return code",
"rc": 5, "start": "2022-07-16 16:18:37.188159",
"stderr": " A volume group called gluster_vg_sdb already exists.",
"stderr_lines": [" A volume group called gluster_vg_sdb already
exists."], "stdout": "", "stdout_lines": []}
NO MORE HOSTS LEFT *************************************************************
NO MORE HOSTS LEFT *************************************************************
PLAY RECAP *********************************************************************
sr-svr04.san.lennoxconsulting.com.au : ok=32 changed=13 unreachable=0 failed=1
skipped=27 rescued=0 ignored=1
sr-svr05.san.lennoxconsulting.com.au : ok=31 changed=12 unreachable=0 failed=1
skipped=27 rescued=0 ignored=1
sr-svr06.san.lennoxconsulting.com.au : ok=31 changed=12 unreachable=0 failed=1
skipped=27 rescued=0 ignored=1
----------- gluster-deployment.log ---------
A "gluster v status" gives me no volumes present and that is where I am stuck!
Any ideas of what I start trying next?
I have tried this with oVirt Node 4.5.1 el8 and el9 images as well as 4.5 el8 images so it
has got to be somewhere in my infrastructure configuration but I am out of ideas.
My hardware configuration is 3 x HP DL360's with oVirt Node 4.5.1 el8 installed on
2x146gb RAID1 array and a gluster 6x900gb RAID5 array.
Network configuration is:
root@sr-svr04 ~]# ip addr show up
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0
state UP group default qlen 1000
link/ether 1c:98:ec:29:41:68 brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0
state UP group default qlen 1000
link/ether 1c:98:ec:29:41:68 brd ff:ff:ff:ff:ff:ff permaddr 1c:98:ec:29:41:69
4: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default
qlen 1000
link/ether 1c:98:ec:29:41:6a brd ff:ff:ff:ff:ff:ff
5: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group
default qlen 1000
link/ether 1c:98:ec:29:41:6b brd ff:ff:ff:ff:ff:ff
6: eno3.3@eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether 1c:98:ec:29:41:6a brd ff:ff:ff:ff:ff:ff
inet 192.168.3.11/24 brd 192.168.3.255 scope global noprefixroute eno3.3
valid_lft forever preferred_lft forever
inet6 fe80::ec:d7be:760e:8eda/64 scope link noprefixroute
valid_lft forever preferred_lft forever
7: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether 1c:98:ec:29:41:68 brd ff:ff:ff:ff:ff:ff
8: bond0.4@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether 1c:98:ec:29:41:68 brd ff:ff:ff:ff:ff:ff
inet 192.168.4.11/24 brd 192.168.4.255 scope global noprefixroute bond0.4
valid_lft forever preferred_lft forever
inet6 fe80::f503:a54:8421:ea8b/64 scope link noprefixroute
valid_lft forever preferred_lft forever
9: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
group default qlen 1000
link/ether 52:54:00:5b:fd:ac brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
The public network is core.lennoxconsulting.com.au which is 192.168.4.0/24 and the storage
network is san.lennoxconsulting.com.au which is 192.168.3.0/24
Any help to move forward please is appreciated.
- Dave.
10:21 a.m.
Can you cat this file
/etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/vg_create.yml
It seems that the VG creation is not idempotent .As a workaround, delete the VG
'gluster_vg_sdb' on all Gluster nodes:
vgremove gluster_vg_sdb
Best Regards,Strahil Nikolov
Hello all,
I am hoping someone can help me with a oVirt installation that has just gotten the better
of me after weeks of trying.
After setting up ssh-keys and making sure each host is known to the primary host
(sr-svr04), I go though Cockpit and "Configure Gluster storage and oVirt hosted
engine", enter all of the details with <host>.san.lennoxconsulting.com.au for
the storage network FQDN and <host>.core.lennoxconsulting.com.au for the public
interfaces. Connectivity on each of the VLAN's test out as basically working
(everything is pingable and ssh connections work) and the hosts are generally usable on
the network.
But the install ultimately dies with the following ansible error:
----------- gluster-deployment.log ---------
:
:
TASK [gluster.infra/roles/backend_setup : Create volume groups] ****************
task path: /etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/vg_create.yml:63
failed: [sr-svr04.san.lennoxconsulting.com.au] (item={'key':
'gluster_vg_sdb', 'value': [{'vgname': 'gluster_vg_sdb',
'pvname': '/dev/sdb'}]}) => {"ansible_loop_var":
"item", "changed": true, "cmd": ["vgcreate",
"--dataalignment", "1536K", "-s", "1536K",
"gluster_vg_sdb", "/dev/sdb"], "delta":
"0:00:00.058528", "end": "2022-07-16 16:18:37.018563",
"item": {"key": "gluster_vg_sdb", "value":
[{"pvname": "/dev/sdb", "vgname":
"gluster_vg_sdb"}]}, "msg": "non-zero return code",
"rc": 5, "start": "2022-07-16 16:18:36.960035",
"stderr": " A volume group called gluster_vg_sdb already exists.",
"stderr_lines": [" A volume group called gluster_vg_sdb already
exists."], "stdout": "", "stdout_lines": []}
failed: [sr-svr05.san.lennoxconsulting.com.au] (item={'key':
'gluster_vg_sdb', 'value': [{'vgname': 'gluster_vg_sdb',
'pvname': '/dev/sdb'}]}) => {"ansible_loop_var":
"item", "changed": true, "cmd": ["vgcreate",
"--dataalignment", "1536K", "-s", "1536K",
"gluster_vg_sdb", "/dev/sdb"], "delta":
"0:00:00.057186", "end": "2022-07-16 16:18:37.784063",
"item": {"key": "gluster_vg_sdb", "value":
[{"pvname": "/dev/sdb", "vgname":
"gluster_vg_sdb"}]}, "msg": "non-zero return code",
"rc": 5, "start": "2022-07-16 16:18:37.726877",
"stderr": " A volume group called gluster_vg_sdb already exists.",
"stderr_lines": [" A volume group called gluster_vg_sdb already
exists."], "stdout": "", "stdout_lines": []}
failed: [sr-svr06.san.lennoxconsulting.com.au] (item={'key':
'gluster_vg_sdb', 'value': [{'vgname': 'gluster_vg_sdb',
'pvname': '/dev/sdb'}]}) => {"ansible_loop_var":
"item", "changed": true, "cmd": ["vgcreate",
"--dataalignment", "1536K", "-s", "1536K",
"gluster_vg_sdb", "/dev/sdb"], "delta":
"0:00:00.062212", "end": "2022-07-16 16:18:37.250371",
"item": {"key": "gluster_vg_sdb", "value":
[{"pvname": "/dev/sdb", "vgname":
"gluster_vg_sdb"}]}, "msg": "non-zero return code",
"rc": 5, "start": "2022-07-16 16:18:37.188159",
"stderr": " A volume group called gluster_vg_sdb already exists.",
"stderr_lines": [" A volume group called gluster_vg_sdb already
exists."], "stdout": "", "stdout_lines": []}
NO MORE HOSTS LEFT *************************************************************
NO MORE HOSTS LEFT *************************************************************
PLAY RECAP *********************************************************************
sr-svr04.san.lennoxconsulting.com.au : ok=32 changed=13 unreachable=0 failed=1
skipped=27 rescued=0 ignored=1
sr-svr05.san.lennoxconsulting.com.au : ok=31 changed=12 unreachable=0 failed=1
skipped=27 rescued=0 ignored=1
sr-svr06.san.lennoxconsulting.com.au : ok=31 changed=12 unreachable=0 failed=1
skipped=27 rescued=0 ignored=1
----------- gluster-deployment.log ---------
A "gluster v status" gives me no volumes present and that is where I am stuck!
Any ideas of what I start trying next?
I have tried this with oVirt Node 4.5.1 el8 and el9 images as well as 4.5 el8 images so it
has got to be somewhere in my infrastructure configuration but I am out of ideas.
My hardware configuration is 3 x HP DL360's with oVirt Node 4.5.1 el8 installed on
2x146gb RAID1 array and a gluster 6x900gb RAID5 array.
Network configuration is:
root@sr-svr04 ~]# ip addr show up
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default
qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0
state UP group default qlen 1000
link/ether 1c:98:ec:29:41:68 brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0
state UP group default qlen 1000
link/ether 1c:98:ec:29:41:68 brd ff:ff:ff:ff:ff:ff permaddr 1c:98:ec:29:41:69
4: eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default
qlen 1000
link/ether 1c:98:ec:29:41:6a brd ff:ff:ff:ff:ff:ff
5: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group
default qlen 1000
link/ether 1c:98:ec:29:41:6b brd ff:ff:ff:ff:ff:ff
6: eno3.3@eno3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether 1c:98:ec:29:41:6a brd ff:ff:ff:ff:ff:ff
inet 192.168.3.11/24 brd 192.168.3.255 scope global noprefixroute eno3.3
valid_lft forever preferred_lft forever
inet6 fe80::ec:d7be:760e:8eda/64 scope link noprefixroute
valid_lft forever preferred_lft forever
7: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether 1c:98:ec:29:41:68 brd ff:ff:ff:ff:ff:ff
8: bond0.4@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
group default qlen 1000
link/ether 1c:98:ec:29:41:68 brd ff:ff:ff:ff:ff:ff
inet 192.168.4.11/24 brd 192.168.4.255 scope global noprefixroute bond0.4
valid_lft forever preferred_lft forever
inet6 fe80::f503:a54:8421:ea8b/64 scope link noprefixroute
valid_lft forever preferred_lft forever
9: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
group default qlen 1000
link/ether 52:54:00:5b:fd:ac brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
The public network is core.lennoxconsulting.com.au which is 192.168.4.0/24 and the storage
network is san.lennoxconsulting.com.au which is 192.168.3.0/24
Any help to move forward please is appreciated.
- Dave.
_______________________________________________
Users mailing list -- users(a)ovirt.org
To unsubscribe send an email to users-leave(a)ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/
List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZCJASQR3EV4...
1:07 a.m.
Can you cat this file
/etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/vg_create.yml
It seems that the VG creation is not idempotent .As a workaround, delete the VG
'gluster_vg_sdb' on all Gluster nodes:
vgremove gluster_vg_sdb
Best Regards,Strahil Nikolov
Strahil,
Thank you for your response, I have added the file for you, but with my continued efforts
over the past day
I have finally managed to get oVirt to install but not without issues. Over a period of 2
weeks I have kicked off the install process over 20 times. Occasionally it would return
the error posted here, but most of the time the install process would hang on this step.
I posted the error as that was the only log/error I could get, when it hung no log files
or any footprint was created it just sat there until I either reboot the host or cancelled
the install.
The problem I have found is that nodectl doesn't return while the installer is in
progress, so I assume that when the installer tries to ssh to the localhost it never gets
to a shell because nodectl is waiting indefinitely for something. So I removed
nodectl-motd.sh and nodectl-run-banner.sh from /etc/profile.d and now the Gluster install
wizards works perfectly.
Next, the Cockpit wizard refused to identify any of my network devices, but the command
line installer was fine, so I have now got a self hosted engine running on one node via:
hosted-engine --deploy
However my next issue now is that when I try to login to the Administration Portal with
the admin user, it is giving me "Invalid username or password". I can log into
the Monitoring Portal just fine, but the Administration and VM Portal doesn't like the
admin credentials. So now to work out why authentication isn't working.
- Dave.
----- /etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/vg_create.yml -----
---
# We have to set the dataalignment for physical volumes, and physicalextentsize
# for volume groups. For JBODs we use a constant alignment value of 256K
# however, for RAID we calculate it by multiplying the RAID stripe unit size
# with the number of data disks. Hence in case of RAID stripe_unit_size and data
# disks are mandatory parameters.
- name: Check if valid disktype is provided
fail:
msg: "Unknown disktype. Allowed disktypes: JBOD, RAID6, RAID10, RAID5."
when: gluster_infra_disktype not in [ 'JBOD', 'RAID6', 'RAID10',
'RAID5' ]
# Set data alignment for JBODs, by default it is 256K. This set_fact is not
# needed if we can always assume 256K for JBOD, however we provide this extra
# variable to override it.
- name: Set PV data alignment for JBOD
set_fact:
pv_dataalign: "{{ gluster_infra_dalign | default('256K') }}"
when: gluster_infra_disktype == 'JBOD'
# Set data alignment for RAID
# We need KiB: ensure to keep the trailing `K' in the pv_dataalign calculation.
- name: Set PV data alignment for RAID
set_fact:
pv_dataalign: >
{{ gluster_infra_diskcount|int *
gluster_infra_stripe_unit_size|int }}K
when: >
gluster_infra_disktype == 'RAID6' or
gluster_infra_disktype == 'RAID10' or
gluster_infra_disktype == 'RAID5'
- name: Set VG physical extent size for RAID
set_fact:
vg_pesize: >
{{ gluster_infra_diskcount|int *
gluster_infra_stripe_unit_size|int }}K
when: >
gluster_infra_disktype == 'RAID6' or
gluster_infra_disktype == 'RAID10' or
gluster_infra_disktype == 'RAID5'
- include_tasks: get_vg_groupings.yml
vars:
volume_groups: "{{ gluster_infra_volume_groups }}"
when: gluster_infra_volume_groups is defined and gluster_infra_volume_groups is not none
and gluster_infra_volume_groups|length >0
- name: Record for missing devices for phase 2
set_fact:
gluster_phase2_has_missing_devices: true
loop: "{{ vg_device_exists.results }}"
when: item.stdout_lines is defined and "0" in item.stdout_lines
- name: Print the gateway for each host when defined
ansible.builtin.debug:
msg: vg names {{ gluster_volumes_by_groupname }}
# Tasks to create a volume group
# The devices in `pvs' can be a regular device or a VDO device
# Please take note; only the first item per volume group will define the actual
configuraton!
#TODO: fix pesize // {{ ((item.value | first).vg_pesize || vg_pesize) | default(4) }}
- name: Create volume groups
register: gluster_changed_vgs
command: vgcreate --dataalignment {{ item.value.pv_dataalign | default(pv_dataalign) }}
-s {{ vg_pesize | default(4) }} {{ (item.value | first).vgname }} {{ item.value |
ovirt.ovirt.json_query('[].pvname') | unique | join(',') }}
# lvg:
# state: present
# vg: "{{ (item.value | first).vgname }}"
# pvs: "{{ item.value | json_query('[].pvname') | unique |
join(',') }}"
# pv_options: "--dataalignment {{ item.value.pv_dataalign | default(pv_dataalign)
}}"
# pesize is 4m by default for JBODs
# pesize: "{{ vg_pesize | default(4) }}"
loop: "{{gluster_volumes_by_groupname | default({}) | dict2items}}"
when: gluster_volumes_by_groupname is defined and item.value|length>0
- name: update LVM fact's
setup:
filter: 'ansible_lvm'
when: gluster_changed_vgs.changed
654
days inactive
655
days old
2 comments
2 participants
participants (2)
-
david.lennox＠frontlinedigital.com.au -
Strahil Nikolov