- Users - oVirt List Archives

4.3.3 single node hyperconverged wizard failing because var/log is too small?
by Edward Berger 10 May '19

10 May '19

I'm trying to bring up a single node hyperconverged with the current node-ng ISO installation, but it ends with this failure message. TASK [gluster.features/roles/gluster_hci : Check if /var/log has enough disk space] *** fatal: [br014.bridges.psc.edu] FAILED! => {"changed": true, "cmd": "df -m /var/log | awk '/[0-9]%/ {print $4}'", "delta": "0:00:00.008513", "end": "2019-05-09 20:09:27.914400", "failed_when_result": true, "rc": 0, "start": "2019-05-09 20:09:27.905887", "stderr": "", "stderr_lines": [], "stdout": "7470", "stdout_lines": ["7470"]} I have what the installer created by default for /var/log, so I don't know why its complaining. [root@br014 ~]# df -kh Filesystem Size Used Avail Use% Mounted on /dev/mapper/onn_br014-ovirt--node--ng--4.3.3.1--0.20190417.0+1 3.5T 2.1G 3.3T 1% / devtmpfs 63G 0 63G 0% /dev tmpfs 63G 4.0K 63G 1% /dev/shm tmpfs 63G 18M 63G 1% /run tmpfs 63G 0 63G 0% /sys/fs/cgroup /dev/mapper/onn_br014-home 976M 2.6M 907M 1% /home /dev/mapper/onn_br014-tmp 976M 2.8M 906M 1% /tmp /dev/mapper/onn_br014-var 15G 42M 14G 1% /var /dev/sda2 976M 173M 737M 19% /boot /dev/mapper/onn_br014-var_log 7.8G 41M 7.3G 1% /var/log /dev/mapper/onn_br014-var_log_audit 2.0G 7.6M 1.8G 1% /var/log/audit /dev/mapper/onn_br014-var_crash 9.8G 37M 9.2G 1% /var/crash /dev/sda1 200M 12M 189M 6% /boot/efi tmpfs 13G 0 13G 0% /run/user/1000 tmpfs 13G 0 13G 0% /run/user/0 /dev/mapper/gluster_vg_sdb-gluster_lv_engine 3.7T 33M 3.7T 1% /gluster_bricks/engine /dev/mapper/gluster_vg_sdc-gluster_lv_data 3.7T 34M 3.7T 1% /gluster_bricks/data /dev/mapper/gluster_vg_sdd-gluster_lv_vmstore 3.7T 34M 3.7T 1% /gluster_bricks/vmstore The machine had 4 4TB disks, so sda is the installation for oVirt node-ng, the other 3 disks for the gluster volumes. root@br014 ~]# pvs PV VG Fmt Attr PSize PFree /dev/sda3 onn_br014 lvm2 a-- <3.64t 100.00g /dev/sdb gluster_vg_sdb lvm2 a-- <3.64t <26.02g /dev/sdc gluster_vg_sdc lvm2 a-- <3.64t 0 /dev/sdd gluster_vg_sdd lvm2 a-- <3.64t 0 [root@br014 ~]# vgs VG #PV #LV #SN Attr VSize VFree gluster_vg_sdb 1 1 0 wz--n- <3.64t <26.02g gluster_vg_sdc 1 2 0 wz--n- <3.64t 0 gluster_vg_sdd 1 2 0 wz--n- <3.64t 0 onn_br014 1 11 0 wz--n- <3.64t 100.00g [root@br014 ~]# lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert gluster_lv_engine gluster_vg_sdb -wi-ao---- 3.61t gluster_lv_data gluster_vg_sdc Vwi-aot--- 3.61t gluster_thinpool_gluster_vg_sdc 0.05 gluster_thinpool_gluster_vg_sdc gluster_vg_sdc twi-aot--- <3.61t 0.05 0.13 gluster_lv_vmstore gluster_vg_sdd Vwi-aot--- 3.61t gluster_thinpool_gluster_vg_sdd 0.05 gluster_thinpool_gluster_vg_sdd gluster_vg_sdd twi-aot--- <3.61t 0.05 0.13 home onn_br014 Vwi-aotz-- 1.00g pool00 4.79 ovirt-node-ng-4.3.3.1-0.20190417.0 onn_br014 Vwi---tz-k <3.51t pool00 root ovirt-node-ng-4.3.3.1-0.20190417.0+1 onn_br014 Vwi-aotz-- <3.51t pool00 ovirt-node-ng-4.3.3.1-0.20190417.0 0.13 pool00 onn_br014 twi-aotz-- 3.53t 0.19 1.86 root onn_br014 Vri---tz-k <3.51t pool00 swap onn_br014 -wi-ao---- 4.00g tmp onn_br014 Vwi-aotz-- 1.00g pool00 4.84 var onn_br014 Vwi-aotz-- 15.00g pool00 3.67 var_crash onn_br014 Vwi-aotz-- 10.00g pool00 2.86 var_log onn_br014 Vwi-aotz-- 8.00g pool00 3.25 var_log_audit onn_br014 Vwi-aotz-- 2.00g pool00 4.86 Here's the full deploy log from the UI. Let me know if you need specific logs. PLAY [Setup backend] *********************************************************** TASK [Gathering Facts] ********************************************************* ok: [br014.bridges.psc.edu] TASK [gluster.infra/roles/firewall_config : Start firewalld if not already started] *** ok: [br014.bridges.psc.edu] TASK [gluster.infra/roles/firewall_config : check if required variables are set] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/firewall_config : Open/Close firewalld ports] ******** ok: [br014.bridges.psc.edu] => (item=2049/tcp) ok: [br014.bridges.psc.edu] => (item=54321/tcp) ok: [br014.bridges.psc.edu] => (item=5900/tcp) ok: [br014.bridges.psc.edu] => (item=5900-6923/tcp) ok: [br014.bridges.psc.edu] => (item=5666/tcp) ok: [br014.bridges.psc.edu] => (item=16514/tcp) TASK [gluster.infra/roles/firewall_config : Add/Delete services to firewalld rules] *** ok: [br014.bridges.psc.edu] => (item=glusterfs) TASK [gluster.infra/roles/backend_setup : Gather facts to determine the OS distribution] *** ok: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Change to Install lvm tools for debian systems.] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Change to Install lvm tools for RHEL systems.] *** ok: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Install python-yaml package for Debian systems] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Initialize vdo_devs array] *********** ok: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Record VDO devices (if any)] ********* skipping: [br014.bridges.psc.edu] => (item={u'vgname': u'gluster_vg_sdb', u'pvname': u'/dev/sdb'}) skipping: [br014.bridges.psc.edu] => (item={u'vgname': u'gluster_vg_sdc', u'pvname': u'/dev/sdc'}) skipping: [br014.bridges.psc.edu] => (item={u'vgname': u'gluster_vg_sdd', u'pvname': u'/dev/sdd'}) TASK [gluster.infra/roles/backend_setup : Enable and start vdo service] ******** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Create VDO with specified size] ****** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Check if valid disktype is provided] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Set PV data alignment for JBOD] ****** ok: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Set PV data alignment for RAID] ****** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Set VG physical extent size for RAID] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Create volume groups] **************** ok: [br014.bridges.psc.edu] => (item={u'vgname': u'gluster_vg_sdb', u'pvname': u'/dev/sdb'}) ok: [br014.bridges.psc.edu] => (item={u'vgname': u'gluster_vg_sdc', u'pvname': u'/dev/sdc'}) ok: [br014.bridges.psc.edu] => (item={u'vgname': u'gluster_vg_sdd', u'pvname': u'/dev/sdd'}) TASK [gluster.infra/roles/backend_setup : Create thick logical volume] ********* ok: [br014.bridges.psc.edu] => (item={u'lvname': u'gluster_lv_engine', u'vgname': u'gluster_vg_sdb', u'size': u'3700G'}) TASK [gluster.infra/roles/backend_setup : Calculate chunksize for RAID6/RAID10/RAID5] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Set chunksize for JBOD] ************** ok: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Create a LV thinpool] **************** ok: [br014.bridges.psc.edu] => (item={u'vgname': u'gluster_vg_sdc', u'thinpoolname': u'gluster_thinpool_gluster_vg_sdc', u'poolmetadatasize': u'16G'}) ok: [br014.bridges.psc.edu] => (item={u'vgname': u'gluster_vg_sdd', u'thinpoolname': u'gluster_thinpool_gluster_vg_sdd', u'poolmetadatasize': u'16G'}) TASK [gluster.infra/roles/backend_setup : Create thin logical volume] ********** ok: [br014.bridges.psc.edu] => (item={u'lvname': u'gluster_lv_data', u'vgname': u'gluster_vg_sdc', u'thinpool': u'gluster_thinpool_gluster_vg_sdc', u'lvsize': u'3700G'}) ok: [br014.bridges.psc.edu] => (item={u'lvname': u'gluster_lv_vmstore', u'vgname': u'gluster_vg_sdd', u'thinpool': u'gluster_thinpool_gluster_vg_sdd', u'lvsize': u'3700G'}) TASK [gluster.infra/roles/backend_setup : Extend volume group] ***************** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Change attributes of LV] ************* skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Create LV for cache] ***************** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Create metadata LV for cache] ******** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Convert logical volume to a cache pool LV] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Convert logical volume to a cache pool LV without cachemetalvname] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Convert an existing logical volume to a cache LV] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Set XFS options for JBOD] ************ ok: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Set XFS options for RAID devices] **** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Create filesystem on thin logical vols] *** ok: [br014.bridges.psc.edu] => (item={u'lvname': u'gluster_lv_data', u'vgname': u'gluster_vg_sdc', u'thinpool': u'gluster_thinpool_gluster_vg_sdc', u'lvsize': u'3700G'}) ok: [br014.bridges.psc.edu] => (item={u'lvname': u'gluster_lv_vmstore', u'vgname': u'gluster_vg_sdd', u'thinpool': u'gluster_thinpool_gluster_vg_sdd', u'lvsize': u'3700G'}) TASK [gluster.infra/roles/backend_setup : Create filesystem on thick logical vols] *** ok: [br014.bridges.psc.edu] => (item={u'lvname': u'gluster_lv_engine', u'vgname': u'gluster_vg_sdb', u'size': u'3700G'}) TASK [gluster.infra/roles/backend_setup : Create mount directories if not already present] *** ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/engine', u'vgname': u'gluster_vg_sdb', u'lvname': u'gluster_lv_engine'}) ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/data', u'vgname': u'gluster_vg_sdc', u'lvname': u'gluster_lv_data'}) ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/vmstore', u'vgname': u'gluster_vg_sdd', u'lvname': u'gluster_lv_vmstore'}) TASK [gluster.infra/roles/backend_setup : Set mount options for VDO] *********** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_setup : Mount the vdo devices (If any)] ****** skipping: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/engine', u'vgname': u'gluster_vg_sdb', u'lvname': u'gluster_lv_engine'}) skipping: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/data', u'vgname': u'gluster_vg_sdc', u'lvname': u'gluster_lv_data'}) skipping: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/vmstore', u'vgname': u'gluster_vg_sdd', u'lvname': u'gluster_lv_vmstore'}) TASK [gluster.infra/roles/backend_setup : Mount the devices] ******************* ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/engine', u'vgname': u'gluster_vg_sdb', u'lvname': u'gluster_lv_engine'}) ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/data', u'vgname': u'gluster_vg_sdc', u'lvname': u'gluster_lv_data'}) ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/vmstore', u'vgname': u'gluster_vg_sdd', u'lvname': u'gluster_lv_vmstore'}) TASK [gluster.infra/roles/backend_setup : Set Gluster specific SeLinux context on the bricks] *** ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/engine', u'vgname': u'gluster_vg_sdb', u'lvname': u'gluster_lv_engine'}) ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/data', u'vgname': u'gluster_vg_sdc', u'lvname': u'gluster_lv_data'}) ok: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/vmstore', u'vgname': u'gluster_vg_sdd', u'lvname': u'gluster_lv_vmstore'}) TASK [gluster.infra/roles/backend_setup : restore file(s) default SELinux security contexts] *** changed: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/engine', u'vgname': u'gluster_vg_sdb', u'lvname': u'gluster_lv_engine'}) changed: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/data', u'vgname': u'gluster_vg_sdc', u'lvname': u'gluster_lv_data'}) changed: [br014.bridges.psc.edu] => (item={u'path': u'/gluster_bricks/vmstore', u'vgname': u'gluster_vg_sdd', u'lvname': u'gluster_lv_vmstore'}) TASK [gluster.infra/roles/backend_reset : unmount the directories (if mounted)] *** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_reset : Delete volume groups] **************** skipping: [br014.bridges.psc.edu] TASK [gluster.infra/roles/backend_reset : Remove VDO devices] ****************** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Create temporary storage directory] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Get the name of the directory created] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : check if gluster_features_ganesha_clusternodes is set] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Define service port] **************** skipping: [br014.bridges.psc.edu] => (item=^#(STATD_PORT=.*)) skipping: [br014.bridges.psc.edu] => (item=^#(LOCKD_TCPPORT=.*)) skipping: [br014.bridges.psc.edu] => (item=^#(LOCKD_UDPPORT=.*)) TASK [gluster.features/roles/nfs_ganesha : Check packages installed, if not install] *** skipping: [br014.bridges.psc.edu] => (item=glusterfs-ganesha) skipping: [br014.bridges.psc.edu] => (item=nfs-ganesha) skipping: [br014.bridges.psc.edu] => (item=corosync) skipping: [br014.bridges.psc.edu] => (item=pacemaker) skipping: [br014.bridges.psc.edu] => (item=libntirpc) skipping: [br014.bridges.psc.edu] => (item=pcs) TASK [gluster.features/roles/nfs_ganesha : Restart services] ******************* skipping: [br014.bridges.psc.edu] => (item=nfslock) skipping: [br014.bridges.psc.edu] => (item=nfs-config) skipping: [br014.bridges.psc.edu] => (item=rpc-statd) TASK [gluster.features/roles/nfs_ganesha : Stop services] ********************** skipping: [br014.bridges.psc.edu] => (item=nfs-server) TASK [gluster.features/roles/nfs_ganesha : Disable service] ******************** skipping: [br014.bridges.psc.edu] => (item=nfs-server) TASK [gluster.features/roles/nfs_ganesha : Enable services] ******************** skipping: [br014.bridges.psc.edu] => (item=glusterfssharedstorage) skipping: [br014.bridges.psc.edu] => (item=nfs-ganesha) skipping: [br014.bridges.psc.edu] => (item=network) skipping: [br014.bridges.psc.edu] => (item=pcsd) skipping: [br014.bridges.psc.edu] => (item=pacemaker) TASK [gluster.features/roles/nfs_ganesha : Start services] ********************* skipping: [br014.bridges.psc.edu] => (item=network) skipping: [br014.bridges.psc.edu] => (item=pcsd) TASK [gluster.features/roles/nfs_ganesha : Create a user hacluster if not already present] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Set the password for hacluster] ***** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Set the hacluster user the same password on new nodes] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Pcs cluster authenticate the hacluster on new nodes] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Pause for a few seconds after pcs auth] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Set gluster_use_execmem flag on and keep it persistent] *** skipping: [br014.bridges.psc.edu] => (item=gluster_use_execmem) skipping: [br014.bridges.psc.edu] => (item=ganesha_use_fusefs) TASK [gluster.features/roles/nfs_ganesha : check if gluster_features_ganesha_masternode is set] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Copy the ssh keys to the local machine] *** skipping: [br014.bridges.psc.edu] => (item=secret.pem.pub) skipping: [br014.bridges.psc.edu] => (item=secret.pem) TASK [gluster.features/roles/nfs_ganesha : check if gluster_features_ganesha_newnodes_vip is set] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Copy the public key to remote nodes] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Copy the private key to remote node] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Deploy the pubkey on all nodes] ***** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Make the volume a gluster shared volume] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Generate ssh key in one of the nodes in HA cluster] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Copy the ssh keys to the local machine] *** skipping: [br014.bridges.psc.edu] => (item=secret.pem.pub) skipping: [br014.bridges.psc.edu] => (item=secret.pem) TASK [gluster.features/roles/nfs_ganesha : Create configuration directory for nfs_ganesha] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Copy ganesha.conf to config directory on shared volume] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Create ganesha-ha.conf file] ******** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Enable NFS Ganesha] ***************** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Pause for 30 seconds (takes a while to enable NFS Ganesha)] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Check NFS Ganesha status] *********** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Report NFS Ganesha status] ********** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Report NFS Ganesha status (If any errors)] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : check if gluster_features_ganesha_volume is set] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Export the NFS Ganesha volume] ****** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Copy the public key to remote nodes] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Copy the private key to remote node] *** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Deploy the pubkey on all nodes] ***** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Adds a node to the cluster] ********* skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Report ganesha add-node status] ***** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/nfs_ganesha : Delete the temporary directory] ***** skipping: [br014.bridges.psc.edu] TASK [gluster.features/roles/gluster_hci : Check if packages are installed, if not install] *** ok: [br014.bridges.psc.edu] => (item=vdsm) ok: [br014.bridges.psc.edu] => (item=vdsm-gluster) ok: [br014.bridges.psc.edu] => (item=ovirt-host) ok: [br014.bridges.psc.edu] => (item=screen) TASK [gluster.features/roles/gluster_hci : Enable and start glusterd and chronyd] *** ok: [br014.bridges.psc.edu] => (item=chronyd) ok: [br014.bridges.psc.edu] => (item=glusterd) ok: [br014.bridges.psc.edu] => (item=firewalld) TASK [gluster.features/roles/gluster_hci : Add user qemu to gluster group] ***** ok: [br014.bridges.psc.edu] TASK [gluster.features/roles/gluster_hci : Disable the hook scripts] *********** changed: [br014.bridges.psc.edu] => (item=/var/lib/glusterd/hooks/1/set/post/S30samba-set.sh) changed: [br014.bridges.psc.edu] => (item=/var/lib/glusterd/hooks/1/start/post/S30samba-start.sh) changed: [br014.bridges.psc.edu] => (item=/var/lib/glusterd/hooks/1/stop/pre/S30samba-stop.sh) changed: [br014.bridges.psc.edu] => (item=/var/lib/glusterd/hooks/1/reset/post/S31ganesha-reset.sh) changed: [br014.bridges.psc.edu] => (item=/var/lib/glusterd/hooks/1//start/post/S31ganesha-start.sh) changed: [br014.bridges.psc.edu] => (item=/var/lib/glusterd/hooks/1/start/post/S29CTDBsetup.sh) changed: [br014.bridges.psc.edu] => (item=/var/lib/glusterd/hooks/1/stop/pre/S29CTDB-teardown.sh) TASK [gluster.features/roles/gluster_hci : Check if valid FQDN is provided] **** changed: [br014.bridges.psc.edu -> localhost] => (item=br014.bridges.psc.edu ) TASK [gluster.features/roles/gluster_hci : Check if /var/log has enough disk space] *** fatal: [br014.bridges.psc.edu] FAILED! => {"changed": true, "cmd": "df -m /var/log | awk '/[0-9]%/ {print $4}'", "delta": "0:00:00.008513", "end": "2019-05-09 20:09:27.914400", "failed_when_result": true, "rc": 0, "start": "2019-05-09 20:09:27.905887", "stderr": "", "stderr_lines": [], "stdout": "7470", "stdout_lines": ["7470"]} NO MORE HOSTS LEFT ************************************************************* NO MORE HOSTS LEFT ************************************************************* to retry, use: --limit @/usr/share/cockpit/ovirt-dashboard/ansible/hc_wizard.retry PLAY RECAP ********************************************************************* br014.bridges.psc.edu : ok=25 changed=3 unreachable=0 failed=1

2 2

Gluster Snapshot Datepicker Not Working?
by Alex McWhirter 10 May '19

10 May '19

Updated to 4.3.3.7, the date picker for gluster snapshot appears to not be working? It wont register clicks, and manually typing in times doesn't work. Can anyone else confirm?

1 0

Hosted Engine Restore Issues
by Alan G 10 May '19

10 May '19

Hi, Trying to re-deploy Hosted Engine into a new storage domain. "hosted-engine --deploy --noansible" has completed and the engine is up, but I cannot remove the existing hosted_storage domain to allow the new one to be imported. I cannot remove the domain until the old HostedEngine VM is removed, but I cannot remove that as it has Delete Protection enabled. Any attempt to remove delete protection errors with "There was an attempt to change Hosted Engine VM values that are locked". How is this process supposed to work? Thanks, Alan

1 0

OvfUpdateIntervalInMinutes restored to original value too early?
by Andreas Elvers 10 May '19

10 May '19

I'm having a problem with restore. It fails while "Wait for OVF_STORE disk content". Looking at the ansible output and at the bug report https://bugzilla.redhat.com/show_bug.cgi?id=1644748 isn't the original value of 60 minutes for OvfUpdateIntervalInMinutes restored too early? [ INFO ] TASK [ovirt.hosted_engine_setup : Fetch host SPM_ID] [ INFO ] changed: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Parse host SPM_ID] [ INFO ] ok: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Restore original DisableFenceAtStartupInSec] [ INFO ] changed: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove DisableFenceAtStartupInSec temporary file] [ INFO ] changed: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Restore original OvfUpdateIntervalInMinutes] [ INFO ] changed: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove OvfUpdateIntervalInMinutes temporary file] [ INFO ] changed: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Removing temporary value] [ INFO ] changed: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Restoring original value] [ INFO ] changed: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Remove temporary directory for ansible as postgres user] [ INFO ] changed: [localhost -> engine.infra.solutions.work] [ INFO ] TASK [ovirt.hosted_engine_setup : Fail if he_force_ip4 and he_force_ip6 are set at the same time] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Prepare getent key] [ INFO ] skipping: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Trigger hosted engine OVF update and enable the serial console] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Wait until OVF update finishes] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Parse OVF_STORE disk list] [ INFO ] ok: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Check OVF_STORE volume status] [ INFO ] changed: [localhost] [ INFO ] TASK [ovirt.hosted_engine_setup : Wait for OVF_STORE disk content] [ ERROR ] {u'_ansible_parsed': True, u'stderr_lines': [u'20+0 records in', u'20+0 records out', u'10240 bytes (10 kB) copied, 0.000164261 s, 62.3 MB/s', u'tar: 1e74609b-51e1-45c8-9106-2596ee59ba3a.ovf: Not found in archive', u'tar: Exiting with failure status due to previous errors'], u'changed': True, u'end': u'2019-05-10 12:57:57.610068', u'_ansible_item_label': {u'image_id': u'59d16f28-64ed-4c3f-aaad-7d98e2718f53', u'name': u'OVF_STORE', u'id': u'98bd5cfc-809d-4d40-83e0-98e379af5f97'}, u'stdout': u'', u'failed': True, u'_ansible_item_result': True, u'msg': u'non-zero return code', u'rc': 2, u'start': u'2019-05-10 12:57:56.928294', u'attempts': 12, u'cmd': u"vdsm-client Image prepare storagepoolID=597f329c-0296-03af-0369-000000000139 storagedomainID=8f115ea6-91ec-4e10-9240-bc8eda47b7ff imageID=98bd5cfc-809d-4d40-83e0-98e379af5f97 volumeID=59d16f28-64ed-4c3f-aaad-7d98e2718f53 | grep path | awk '{ print $2 }' | xargs -I{} sudo -u vdsm dd if={} | tar -tvf - 1e74609b-51e1-45c8-9106-259 6ee59ba3a.ovf", u'item': {u'image_id': u'59d16f28-64ed-4c3f-aaad-7d98e2718f53', u'name': u'OVF_STORE', u'id': u'98bd5cfc-809d-4d40-83e0-98e379af5f97'}, u'delta': u'0:00:00.681774', u'invocation': {u'module_args': {u'warn': False, u'executable': None, u'_uses_shell': True, u'_raw_params': u"vdsm-client Image prepare storagepoolID=597f329c-0296-03af-0369-000000000139 storagedomainID=8f115ea6-91ec-4e10-9240-bc8eda47b7ff imageID=98bd5cfc-809d-4d40-83e0-98e379af5f97 volumeID=59d16f28-64ed-4c3f-aaad-7d98e2718f53 | grep path | awk '{ print $2 }' | xargs -I{} sudo -u vdsm dd if={} | tar -tvf - 1e74609b-51e1-45c8-9106-2596ee59ba3a.ovf", u'removes': None, u'argv': None, u'creates': None, u'chdir': None, u'stdin': None}}, u'stdout_lines': [], u'stderr': u'20+0 records in\n20+0 records out\n10240 bytes (10 kB) copied, 0.000164261 s, 62.3 MB/s\ntar: 1e74609b-51e1-45c8-9106-2596ee59ba3a.ovf: Not found in archive\ntar: Exiting with failure status due to previous errors', u'_ansible_no_log': False}

2 1

Re: oVirt Open Source Backup solution?
by Derek Atkins 10 May '19

10 May '19

Hi, Michael Blanchard <michael(a)wanderingmad.com> writes: > If you haven't seen my other posts, I'm not a very experienced Linux admin, so > I'm trying to make it as easy as possible to run and maintain. It's hard > enough for me to not break ovirt in crazy ways This has nothing to do with ovirt. You could use rdiff-backup on any running machine, be it virtual or bare metal. It's just a way to use a combination of diff and rsync to backup machines. Indeed, I was using it with my vmware-based systems and, when I migrated them to ovirt, the backups just continued working. > Get Outlook for Android -derek -- Derek Atkins 617-623-3745 derek(a)ihtfp.com www.ihtfp.com Computer and Internet Security Consultant

1 0

Re: oVirt and NetApp NFS storage
by Strahil 10 May '19

10 May '19

I know 2 approaches. 1. Use NFS hard mounting option - it will never give error to sanlock and it will be waiting until NFS is recovered (never tries this one, but in theory might work) 2. Change the default sanlock timeout (last time I tried that - it didn't work) . You might need help from Sandro or Sahina for that option. Best Regards, Strahil NikolovOn Apr 18, 2019 11:45, klaasdemter(a)gmail.com wrote: > > Hi, > > I got a question regarding oVirt and the support of NetApp NFS storage. > We have a MetroCluster for our virtual machine disks but a HA-Failover > of that (active IP gets assigned to another node) seems to produce > outages too long for sanlock to handle - that affects all VMs that have > storage leases. NetApp says a "worst case" takeover time is 120 seconds. > That would mean sanlock has already killed all VMs. Is anyone familiar > with how we could setup oVirt to allow such storage outages? Do I need > to use another type of storage for my oVirt VMs because that NFS > implementation is unsuitable for oVirt? > > > Greetings > > Klaas > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/TSJJKK5UG57CC…

4 6

Ansible oVirt.image-template role
by Jeremy Tourville 10 May '19

10 May '19

I am trying to run an Ansible playbook that doesn't appear to run correctly. I have followed the example from this blog - https://evaryont.me/blog/2018/09/getting-started-with-vagrant-and-ovirt-fro… The playbook finishes with an ok status but the template never gets built in Ovirt. I have taken logs from three locations hoping to spot the error: * [root@ansible ansible]#ansible-playbook -vvvv runsetup.yml * [root@ansible ansible]# less /var/log/ansible.log * [root@engine ~]# tail -f /var/log/messages (while the playbook is being run. The server Ansible is my control node and Engine is my managed host. Can anyone help me interpret the attached logs in an effort to further troubleshoot? Thanks!

3 2

Re: Ovirt nodeNG RDMA support?
by Strahil 10 May '19

10 May '19

I saw you are using gluster v5.3 . Any reason not to update to 5.5 ? It is quite more stable than 5.3 and some enhancements were done. Best Regards, Strahil NikolovOn May 10, 2019 00:13, michael(a)wanderingmad.com wrote: > > Tried that, get this error: > > > [root@Icarus ~]# gluster volume stop storage_ssd > Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y > volume stop: storage_ssd: success > [root@Icarus ~]# gluster volume set storage_ssd config.transport tcp,rdma > volume set: failed: Commit failed on localhost. Please check the log file for more details. > and here is the glusterd.log > > [2019-05-09 21:10:45.945855] W [MSGID: 101095] [xlator.c:456:xlator_dynload] 0-xlator: /usr/lib64/glusterfs/5.3/xlator/nfs/server.so: cannot open shared object file: No such file or directory > [2019-05-09 21:10:46.041341] I [MSGID: 106521] [glusterd-op-sm.c:2953:glusterd_op_set_volume] 0-management: changing transport-type for volume storage_ssd to tcp,rdma > [2019-05-09 21:10:46.041965] W [MSGID: 101095] [xlator.c:180:xlator_volopt_dynload] 0-xlator: /usr/lib64/glusterfs/5.3/xlator/nfs/server.so: cannot open shared object file: No such file or di > rectory > [2019-05-09 21:10:46.059356] E [MSGID: 106068] [glusterd-volgen.c:1025:volgen_write_volfile] 0-management: failed to create volfile > [2019-05-09 21:10:46.059394] E [glusterd-volgen.c:6556:glusterd_create_volfiles] 0-management: Could not generate gfproxy client volfiles > [2019-05-09 21:10:46.059406] E [MSGID: 106068] [glusterd-op-sm.c:3062:glusterd_op_set_volume] 0-management: Unable to create volfile for 'volume set' > [2019-05-09 21:10:46.059420] E [MSGID: 106122] [glusterd-syncop.c:1434:gd_commit_op_phase] 0-management: Commit of operation 'Volume Set' failed on localhost > [2019-05-09 21:10:46.062304] E [MSGID: 101191] [event-epoll.c:671:event_dispatch_epoll_worker] 0-epoll: Failed to dispatch handler > [root@prometheus ~]# > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/HJGBS3DVBUDVN…

1 0

Re: Migrating Domain Storage Gluster
by Strahil 10 May '19

10 May '19

If this is replica 2 arbiter 1 (old name replica 3 arbiter 1) or a plain replica 3 volume - you don't need the downtime. Just stop the bricks on one of the nodes. Mount (and verify) those SSDs on the same location and use the gluster's reset brick option. Wait for the heal to complete and repeat with the other node(s). Best Regards, Strahil NikolovOn May 9, 2019 21:00, Alex McWhirter <alex(a)triadic.us> wrote: > > Basically i want to take out all of the HDD's in the main gluster pool, > and replace with SSD's. > > My thought was to put everything in maintenance, copy the data manually > over to a transient storage server. Destroy the gluster volume, swap in > all the new drives, build a new gluster volume with the same name / > settings, move data back, and be done. > > Any thoughts on this? > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/TP6V2OYG5ZXPU…

1 0

Re: HostedEngine cleaned up
by Strahil 10 May '19

10 May '19

This is due to sharding. I joined ovirt 6 moths ago and this option was still there. Best Regards, Strahil NikolovOn May 9, 2019 18:04, Dmitry Filonov <filonov(a)hkl.hms.harvard.edu> wrote: > > The data chunks are under .glusterfs folder on bricks now. Not a single huge file you can easily access from a brick. > Not sure when that change was introduced though. > > Fil > > > On Thu, May 9, 2019, 10:43 AM <olaf.buitelaar(a)gmail.com> wrote: >> >> It looks like i've got the exact same issue; >> drwxr-xr-x. 2 vdsm kvm 4.0K Mar 29 16:01 . >> drwxr-xr-x. 22 vdsm kvm 4.0K Mar 29 18:34 .. >> -rw-rw----. 1 vdsm kvm 64M Feb 4 01:32 44781cef-173a-4d84-88c5-18f7310037b4 >> -rw-rw----. 1 vdsm kvm 1.0M Oct 16 2018 44781cef-173a-4d84-88c5-18f7310037b4.lease >> -rw-r--r--. 1 vdsm kvm 311 Mar 29 16:00 44781cef-173a-4d84-88c5-18f7310037b4.meta >> Within the meta file the image is marked legal and reports a size of SIZE=41943040, interestingly the format is mark RAW, while it was a thinly created volume. >> My suspicion is that something went wrong while the volume was being livemigrated, and somehow the merging of the images broke the volume. >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UDPJ2R5XUFNZR…

1 0