On Thu, Apr 14, 2016 at 11:27 AM, Gianluca Cecchi
<gianluca.cecchi(a)gmail.com> wrote:
On Thu, Apr 14, 2016 at 8:15 AM, Yedidyah Bar David
<didi(a)redhat.com> wrote:
>
> On Thu, Apr 14, 2016 at 5:18 AM, Michael Hall <mike(a)mjhall.org> wrote:
>
>
> 3. NFS
> loop-back mounting nfs is considered risky, due to potential locking
> issues. Therefore, if you want to use NFS, you are better off doing
> something like this:
>
Hello,
can you give more details about these potential locking issues? So that I
can reproduce
Most of what I know about this is:
https://lwn.net/Articles/595652/
I have 2 little environments where I'm using this kind of setup.
In one of
them the hypervisor is a physical server, in the other one the hypervisor is
itself a libvirt VM inside a Fedora 23 based laptop. oVirt version is 3.6.4
on both.
The test VM has 2 disks sda and sdb; all ovirt related stuff on sdb
My raw steps for the lab have been, after setting up CentOS 7.2 OS,
disabling ipv6 and NetworkManager, putting SELinux to permissive and
enabling ovirt repo:
selinux enforcing should work too, if it fails please open a bug. Thanks.
You might have to set the right contexts for your local disks.
NOTE: I also stop and disable firewalld
My host is ovc72.localdomain.local and name of my future engine
shengine.localdomain.local
yum -y update
yum install ovirt-hosted-engine-setup ovirt-engine-appliance
yum install rpcbind nfs-utils nfs-server
(some of them probably already pulled in as dependencies from previous
command)
When I start from scratch the system
pvcreate /dev/sdb
vgcreate OVIRT_DOMAIN /dev/sdb
lvcreate -n ISO_DOMAIN -L 5G OVIRT_DOMAIN
lvcreate -n SHE_DOMAIN -L 25G OVIRT_DOMAIN
lvcreate -n NFS_DOMAIN -l +100%FREE OVIRT_DOMAIN
if I only have to reinitialize I start from here
mkfs -t xfs -f /dev/mapper/OVIRT_DOMAIN-ISO_DOMAIN
mkfs -t xfs -f /dev/mapper/OVIRT_DOMAIN-NFS_DOMAIN
mkfs -t xfs -f /dev/mapper/OVIRT_DOMAIN-SHE_DOMAIN
mkdir /ISO_DOMAIN /NFS_DOMAIN /SHE_DOMAIN
/etc/fstab
/dev/mapper/OVIRT_DOMAIN-ISO_DOMAIN /ISO_DOMAIN xfs defaults 0 0
/dev/mapper/OVIRT_DOMAIN-NFS_DOMAIN /NFS_DOMAIN xfs defaults 0 0
/dev/mapper/OVIRT_DOMAIN-SHE_DOMAIN /SHE_DOMAIN xfs defaults 0 0
mount /ISO_DOMAIN/ --> this for ISO images
mount /NFS_DOMAIN/ ---> this for data storage domain where your VMs will
live (NFS based)
mount /SHE_DOMAIN/ --> this is for the HE VM
chown 36:36 /ISO_DOMAIN
chown 36:36 /NFS_DOMAIN
chown 36:36 /SHE_DOMAIN
chmod 0755 /ISO_DOMAIN
chmod 0755 /NFS_DOMAIN
chmod 0755 /SHE_DOMAIN
/etc/exports
/ISO_DOMAIN *(rw,anonuid=36,anongid=36,all_squash)
/NFS_DOMAIN *(rw,anonuid=36,anongid=36,all_squash)
/SHE_DOMAIN *(rw,anonuid=36,anongid=36,all_squash)
systemctl enable rpcbind
systemctl start rpcbind
systemctl enable nfs-server
systemctl start nfs-server
hosted-engine --deploy
During setup I choose:
Engine FQDN : shengine.localdomain.local
Firewall manager : iptables
Storage connection :
ovc71.localdomain.local:/SHE_DOMAIN
OVF archive (for disk boot) :
/usr/share/ovirt-engine-appliance/ovirt-engine-appliance-20151015.0-1.el7.centos.ova
Also, I used the appliance provided by ovirt-engine-appliance package
After install you have to make a dependency so that VDSM Broker starts after
NFS Server
In /usr/lib/systemd/system/ovirt-ha-broker.service
Added in section [Unit] the line:
After=nfs-server.service
Also in file vdsmd.service changed from:
After=multipathd.service libvirtd.service iscsid.service rpcbind.service \
supervdsmd.service sanlock.service vdsm-network.service
to:
After=multipathd.service libvirtd.service iscsid.service rpcbind.service \
supervdsmd.service sanlock.service vdsm-network.service \
nfs-server.service
NOTE: the files will be overwritten by future updates, so you have to keep
in mind...
On ovc72 in /etc/multipath.conf aright after line
# VDSM REVISION 1.3
added
# RHEV PRIVATE
blacklist {
wwid 0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1
wwid 0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-0
}
To exclude both 2 internal drives... probably oVirt keeps in mind only the
first one?
No idea
Otherwise many messages like:
Jan 25 11:02:00 ovc72 kernel: device-mapper: table: 253:6: multipath: error
getting device
Jan 25 11:02:00 ovc72 kernel: device-mapper: ioctl: error adding target to
table
So far I didn't find any problems. Only a little trick when you have to make
ful lmaintenance where you have to power off the (only) hypervisor, where
you have to make the right order steps.
I guess you can probably script that too...
Thanks for sharing. As wrote above, no personal experience with loopback nfs.
For the multipath question, if interested, perhaps ask again with a different
subject.
thanks for sharing!
--
Didi