On Thu, Apr 14, 2016 at 8:15 AM, Yedidyah Bar David <didi@redhat.com> wrote:

On Thu, Apr 14, 2016 at 5:18 AM, Michael Hall <mike@mjhall.org> wrote:

3. NFS
loop-back mounting nfs is considered risky, due to potential locking
issues. Therefore, if you want to use NFS, you are better off doing
something like this:

Hello,

can you give more details about these potential locking issues? So that I can reproduce

I have 2 little environments where I'm using this kind of setup. In one of them the hypervisor is a physical server, in the other one the hypervisor is itself a libvirt VM inside a Fedora 23 based laptop. oVirt version is 3.6.4 on both.

The test VM has 2 disks sda and sdb; all ovirt related stuff on sdb

My raw steps for the lab have been, after setting up CentOS 7.2 OS, disabling ipv6 and NetworkManager, putting SELinux to permissive and enabling ovirt repo:

NOTE: I also stop and disable firewalld

My host is ovc72.localdomain.local and name of my future engine shengine.localdomain.local

yum -y update

yum install ovirt-hosted-engine-setup ovirt-engine-appliance

yum install rpcbind nfs-utils nfs-server

(some of them probably already pulled in as dependencies from previous command)

When I start from scratch the system

pvcreate /dev/sdb

vgcreate OVIRT_DOMAIN /dev/sdb

lvcreate -n ISO_DOMAIN -L 5G OVIRT_DOMAIN

lvcreate -n SHE_DOMAIN -L 25G OVIRT_DOMAIN

lvcreate -n NFS_DOMAIN -l +100%FREE OVIRT_DOMAIN

if I only have to reinitialize I start from here

mkfs -t xfs -f /dev/mapper/OVIRT_DOMAIN-ISO_DOMAIN

mkfs -t xfs -f /dev/mapper/OVIRT_DOMAIN-NFS_DOMAIN

mkfs -t xfs -f /dev/mapper/OVIRT_DOMAIN-SHE_DOMAIN

mkdir /ISO_DOMAIN /NFS_DOMAIN /SHE_DOMAIN

/etc/fstab

/dev/mapper/OVIRT_DOMAIN-ISO_DOMAIN /ISO_DOMAIN xfs defaults 0 0

/dev/mapper/OVIRT_DOMAIN-NFS_DOMAIN /NFS_DOMAIN xfs defaults 0 0

/dev/mapper/OVIRT_DOMAIN-SHE_DOMAIN /SHE_DOMAIN xfs defaults 0 0

mount /ISO_DOMAIN/ --> this for ISO images

mount /NFS_DOMAIN/ ---> this for data storage domain where your VMs will live (NFS based)

mount /SHE_DOMAIN/ --> this is for the HE VM

chown 36:36 /ISO_DOMAIN

chown 36:36 /NFS_DOMAIN

chown 36:36 /SHE_DOMAIN

chmod 0755 /ISO_DOMAIN

chmod 0755 /NFS_DOMAIN

chmod 0755 /SHE_DOMAIN

/etc/exports

/ISO_DOMAIN *(rw,anonuid=36,anongid=36,all_squash)

/NFS_DOMAIN *(rw,anonuid=36,anongid=36,all_squash)

/SHE_DOMAIN *(rw,anonuid=36,anongid=36,all_squash)

systemctl enable rpcbind

systemctl start rpcbind

systemctl enable nfs-server

systemctl start nfs-server

hosted-engine --deploy

During setup I choose:

Engine FQDN : shengine.localdomain.local

Firewall manager : iptables

Storage connection : ovc71.localdomain.local:/SHE_DOMAIN

OVF archive (for disk boot) : /usr/share/ovirt-engine-appliance/ovirt-engine-appliance-20151015.0-1.el7.centos.ova

Also, I used the appliance provided by ovirt-engine-appliance package

After install you have to make a dependency so that VDSM Broker starts after NFS Server

In /usr/lib/systemd/system/ovirt-ha-broker.service

Added in section [Unit] the line:

After=nfs-server.service

Also in file vdsmd.service changed from:

After=multipathd.service libvirtd.service iscsid.service rpcbind.service \

supervdsmd.service sanlock.service vdsm-network.service

to:

After=multipathd.service libvirtd.service iscsid.service rpcbind.service \

supervdsmd.service sanlock.service vdsm-network.service \

nfs-server.service

NOTE: the files will be overwritten by future updates, so you have to keep in mind...

On ovc72 in /etc/multipath.conf aright after line

# VDSM REVISION 1.3

added

# RHEV PRIVATE

blacklist {

wwid 0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1

wwid 0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0

}

To exclude both 2 internal drives... probably oVirt keeps in mind only the first one?

Otherwise many messages like:

Jan 25 11:02:00 ovc72 kernel: device-mapper: table: 253:6: multipath: error getting device

Jan 25 11:02:00 ovc72 kernel: device-mapper: ioctl: error adding target to table

So far I didn't find any problems. Only a little trick when you have to make ful lmaintenance where you have to power off the (only) hypervisor, where you have to make the right order steps.

Gianluca