[ovirt-users] oVirt + gluster with host and hosted VM on different subnets

Tue Jul 8 23:40:11 UTC 2014

Hi Martin,

Is that because of how the replication works? What if you had, the
kernel-nfs server running ontop of the glusternfs share and a virtual
IP to allow the hosted-engine to only access one of the shares.

Thanks,
Andrew

On Wed, Jul 9, 2014 at 5:15 AM, Martin Sivak <msivak at redhat.com> wrote:
> Hi,
>
> I do not recommend running hosted engine on top of GlusterFS. Not even on top of NFS compatibility layer the GlusterFS provides.
>
> There have been a lot of issues with setups like that. GlusterFS does not ensure that the metadata writes are atomic and visible to all nodes at the same time and that causes serious trouble (the synchronization algorithm relies on the atomicity assumption).
>
> You can use GlusterFS storage domain for VMs, but the hosted engine storage domain needs something else - NFS or iSCSI (available in 3.5).
>
> --
> Martin Sivák
> msivak at redhat.com
> Red Hat Czech
> RHEV-M SLA / Brno, CZ
>
> ----- Original Message -----
>> Il 08/07/2014 16:47, Andrew Lau ha scritto:
>> > On Wed, Jul 9, 2014 at 12:23 AM, Sandro Bonazzola <sbonazzo at redhat.com>
>> > wrote:
>> >> Il 07/07/2014 15:38, Simone Marchioni ha scritto:
>> >>> Hi,
>> >>>
>> >>> I'm trying to install oVirt 3.4 + gluster looking at the following
>> >>> guides:
>> >>>
>> >>> http://community.redhat.com/blog/2014/05/ovirt-3-4-glusterized/
>> >>> http://community.redhat.com/blog/2014/03/up-and-running-with-ovirt-3-4/
>> >>>
>> >>> It went smooth until the hosted engine VM configuration: I can reach it
>> >>> by VNC and the host IP, but I can't configure the VM network in a way it
>> >>> works.
>> >>> Probably the problem is the assumption that the three hosts (2 hosts +
>> >>> the hosted engine) are on the same subnet sharing the same default
>> >>> gateway.
>> >>>
>> >>> But my server is on OVH with the subnet 94.23.2.0/24 and my failover IPs
>> >>> are on the subnet 46.105.224.236/30, and my hosted engine need to use
>> >>> one IP
>> >>> of the last ones.
>> >>>
>> >>> Anyone installed oVirt in such a configuration and can give me any tip?
>> >> Never tested such configuration.
>> >> Andrew, something similar at your installation with an additional NIC?
>> > If I understand correctly, you have your hosts on the 94.23.2.0/24
>> > subnet but you need your hosted engine to be accessible as an address
>> > within 46.105.224.236?
>>
>> Exactly
>>
>> > If that's true, then the easiest way to do it
>> > is simply run your hosted-engine install with the hosted-engine first
>> > on 94.23.2.0/24, you then add another nic to that hosted-engine VM
>> > which'll have the IP address for 46.105.224.237 (?)...
>>
>> I'll try this
>>
>> > alternatively, you could just use a nic alias?
>>
>> We made it work with the following changes (on the host machine in the
>> subnet 94.23.2.0/24):
>> - commented out and removed from running configuration ip rules in
>> /etc/sysconfig/network-scripts/rule-ovirtmgmt
>> - commented out and removed from running configuration ip routes in
>> /etc/sysconfig/network-scripts/route-ovirtmgmt
>> - added /etc/sysconfig/network-scripts/ovirtmgmt:0 with the following
>> configuration:
>> DEVICE=ovirtmgmt:238
>> ONBOOT=yes
>> DELAY=0
>> IPADDR=46.105.224.238
>> NETMASK=255.255.255.252
>> BOOTPROTO=static
>> NM_CONTROLLED=no
>> - enabled ip forwarding in /proc/sys/net/ipv4/ip_forward
>>
>> After that installing the hosted-engine VM with the following IP stack:
>>
>> NETMASK=255.255.255.252
>> IPADDR=46.105.224.237
>> GATEWAY=46.105.224.238
>>
>> seems to work ok.
>>
>> > If you want to add an extra NIC to your hosted-engine to do that above
>> > scenario, here's a snippet from my notes:
>> > (storage_network is a bridge, replace that with ovirtmgmt or another
>> > bridge you may have created)
>> >
>> > hosted-engine --set-maintenance --mode=global
>> >
>> > # On all installed hosts
>> > nano /etc/ovirt-hosted-engine/vm.conf
>> > # insert under earlier nicModel
>> > # replace macaddress and uuid from above
>> > # increment slot
>> > devices={nicModel:pv,macAddr:00:16:3e:e1:7b:14,linkActive:true,network:storage_network,filter:vdsm-no-mac-spoofing,specParams:{},deviceId:fdb11208-a888-e587-6053-32c9c0361f96,address:{bus:0x00,slot:0x04,
>> > domain:0x0000, type:pci,function:0x0},device:bridge,type:interface}
>> >
>> > hosted-engine --vm-shutdown
>> > hosted-engine --vm-start
>> >
>> > hosted-engine --set-maintenance --mode=none
>>
>> Ok: thanks for the advice!
>>
>> > Although, re-reading your question, what do you mean by 'but I can't
>> > configure the VM network in a way it works.' ? Does the setup fail, or
>> > just when you create a VM you don't have any network connectivity..
>>
>> The setup works ok: it creates the VM and i can login to it with VNC on
>> the host IP (94.23.2.X).
>> I can install CentOS 6.5 as advised. After the reboot I login again by
>> VNC and the host IP (94.23.2.X), and configure the IP stack with the
>> other subnet (46.105.224.236/30) and after that the VM is isolated,
>> unless I do the steps written above.
>>
>> Thanks for your support!
>> Simone
>>
>> >
>> >>> Thanks
>> >>> Simone
>> >>> _______________________________________________
>> >>> Users mailing list
>> >>> Users at ovirt.org
>> >>> http://lists.ovirt.org/mailman/listinfo/users
>> >>
>> >> --
>> >> Sandro Bonazzola
>> >> Better technology. Faster innovation. Powered by community collaboration.
>> >> See how it works at redhat.com
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>