[ovirt-users] Proper way to change and persist vdsm configuration options

[Alon Bar-Lev]

----- Original Message -----
> From: "Trey Dockendorf" <treydock at gmail.com>
> To: "Alon Bar-Lev" <alonbl at redhat.com>
> Cc: "users" <users at ovirt.org>, "Fabian Deutsch" <fabiand at redhat.com>, "Dan Kenigsberg" <danken at redhat.com>, "Itamar
> Heim" <iheim at redhat.com>, "Douglas Landgraf" <dougsland at redhat.com>, "ybronhei" <ybronhei at redhat.com>
> Sent: Tuesday, August 5, 2014 9:50:04 PM
> Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options
> 
> On Tue, Aug 5, 2014 at 12:37 PM, Alon Bar-Lev <alonbl at redhat.com> wrote:
> > Hi,
> >
> > Do you actually use puppet over ovirt-node?
> > This is unsupported.
> >
> > Regards,
> > Alon
> >
> 
> I use Puppet to configure everything on the system and some of those
> things conflict with changes made by ovirt-engine when adding a node
> so I've moved to managing those changes in an ovirt Puppet module.
> 
> When you refer to "ovirt-node" your refering to the pre-built ISO
> images?  AFAIK the ovirt-node is not an option for me as I use
> Infiniband to do all storage connections.  Currently NFS is done using
> IPoIB (NFS over RDMA was crashing so I am not pursuing at the moment)
> and iSCSI is done using iSER.  At some point in time I'd like to use
> IB interfaces in my guests utilizing SR-IOV which my Mellanox cards
> support, but I'm postponing that project till after our HPC cluster's
> upgraded :)
> 
> Right now the Puppet module does the following:
> 
> * Configures firewall rules just like what is done by ovirt (with
> exception of a few not supported by puppetlabs-firewall, so I override
> global directive that purges unknown firewall rules)

this should be done at engine side, using engine-config, seek IPTablesConfig or IPTablesConfigSiteCustom in 3.5.

> ** This is necessary as I have to add other Firewall rules, such as Zabbix
> * Exports /etc/hosts entry for the node that is collected by
> ovirt-engine host so that if DNS goes down ovirt-engine does not lose
> access to ovirtmgmt interfaces

why not just have a backup for dns?

> * Install yum repos for ovirt

this should not be done within ovirt-node.

> * install vdsm

this should be done by host-deploy.

> * ensure vdsm.conf exists
> * Populates /etc/vdsm/vdsm.id (IIRC a bug in previous ovirt required this)

why? you machine should have valid bios uuid.

> * Ensures vdsmd is running and will start at boot
> * Ensures vdsm sudo rules are present.
> * Manages default vdsm.conf configurations as a Puppet type,
> "vdsm_config", rather than managing file contents via template (allows
> for purging unmanaged entries also)

the above should be done via host-deploy.

> A lot of the above is handled by ovirt already, but in the past
> customizations were not possible (firewall rules, vdsm.conf entries,
> etc) so if I was going to have to manage those separately I wanted
> them in Puppet :)
> 
> Now that I'm aware of "ovirt-host-deploy" and have seen the potential
> of using vdsm-reg in Puppet, I'm curious what is the "right" way to
> automate node deployment's in oVirt. Ideally I could still use Puppet
> to configure the method or fill in the "gaps" for customizations that
> are needed (ie enabling iSER).
> 
> I'd be glad to know what the recommended method for automating ovirt
> would be, and would be happy to refactor my module in hopes it would
> offer other Puppet users a quick-start way to deploy oVirt while still
> doing things the "ovirt way".
> 
> If the right way is using the ovirt-node images then I'd like to know
> what customizations are possible on those images and so on.  oVirt is
> moving very rapidly and despite using ovirt for a long time I'm still
> learning new things about it almost daily, so forgive any of my
> assumptions above that may be wrong :).

ovirt-node is not flexible as standard host, if you require sssd and misc packages I suggest you stick with generic host.

> 
> Thanks,
> - Trey
> 
> > ----- Original Message -----
> >> From: "ybronhei" <ybronhei at redhat.com>
> >> To: "Trey Dockendorf" <treydock at gmail.com>
> >> Cc: "users" <users at ovirt.org>, "Fabian Deutsch" <fabiand at redhat.com>, "Dan
> >> Kenigsberg" <danken at redhat.com>, "Itamar
> >> Heim" <iheim at redhat.com>, "Douglas Landgraf" <dougsland at redhat.com>, "Alon
> >> Bar-Lev" <alonbl at redhat.com>
> >> Sent: Tuesday, August 5, 2014 8:32:04 PM
> >> Subject: Re: [ovirt-users] Proper way to change and persist vdsm
> >> configuration options
> >>
> >> Hey,
> >>
> >> Just noticed something that I forgot about..
> >> before filing new BZ, see in ovirt-host-deploy README.environment [1]
> >> the section:
> >> VDSM/configOverride(bool) [True]
> >>      Override vdsm configuration file.
> >>
> >> changing it to false will keep your vdsm.conf file as is after deploying
> >> the host again (what happens after node upgrade)
> >>
> >> [1]
> >> https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment
> >>
> >> please check if that what you meant..
> >>
> >> Thanks,
> >> Yaniv Bronhaim.
> >>
> >> On 08/05/2014 08:12 AM, Trey Dockendorf wrote:
> >> > I'll file BZ.  As far as I can recall this has been an issue since 3.3.x
> >> > as
> >> > I have been using Puppet to modify values and have had to rerun Puppet
> >> > after installing a node via GUI and when performing update from GUI.
> >> > Given
> >> > that it has occurred when VDSM version didn't change on the node it
> >> > seems
> >> > likely to be something being done by Python code that bootstraps a node
> >> > and
> >> > performs the other tasks.  I won't have any systems available to test
> >> > with
> >> > for a few days.  New hardware specifically for our oVirt deployment is
> >> > on
> >> > order so should be able to more thoroughly debug and capture logs at
> >> > that
> >> > time.
> >> >
> >> > Would using vdsm-reg be a better solution for adding new nodes?  I only
> >> > tried using vdsm-reg once and it went very poorly...lots of missing
> >> > dependencies not pulled in from yum install I had to install manually
> >> > via
> >> > yum.  Then the node was auto added to newest cluster with no ability to
> >> > change the cluster.  Be happy to debug that too if there's some docs
> >> > that
> >> > outline the expected behavior.
> >> >
> >> > Using vdsm-reg or something similar seems like a better fit for puppet
> >> > deployed nodes, as opposed to requiring GUI steps to add the node.
> >> >
> >> > Thanks
> >> > - Trey
> >> > On Aug 4, 2014 5:53 AM, "ybronhei" <ybronhei at redhat.com> wrote:
> >> >
> >> >> On 07/31/2014 01:28 AM, Trey Dockendorf wrote:
> >> >>
> >> >>> I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM
> >> >>> installed.  I am using iSER to do iSCSI over RDMA and to make that
> >> >>> work I have to modify /etc/vdsm/vdsm.conf to include the following:
> >> >>>
> >> >>> [irs]
> >> >>> iscsi_default_ifaces = iser,default
> >> >>>
> >> >>> I've noticed that any time I upgrade a node from the engine web
> >> >>> interface that changes to vdsm.conf are wiped out.  I don't know if
> >> >>> this is being done by the configuration code or by the vdsm package.
> >> >>> Is there a more reliable way to ensure changes to vdsm.conf are NOT
> >> >>> removed automatically?
> >> >>>
> >> >>
> >> >> Hey,
> >> >>
> >> >> vdsm.conf shouldn't wiped out and shouldn't changed at all during
> >> >> upgrade.
> >> >> other related conf files (such as libvirtd.conf) might be overrided to
> >> >> keep
> >> >> defaults configurations for vdsm. but vdsm.conf should persist with
> >> >> user's
> >> >> modification. from my check, regular yum upgrade doesn't touch
> >> >> vdsm.conf
> >> >>
> >> >> Douglas can you verify that with node upgrade? might be specific to
> >> >> that
> >> >> flow..
> >> >>
> >> >> Trey, can file a bugzilla on that and describe your steps there?
> >> >>
> >> >> Thanks
> >> >>
> >> >> Yaniv Bronhaim,
> >> >>
> >> >>>
> >> >>> Thanks,
> >> >>> - Trey
> >> >>> _______________________________________________
> >> >>> Users mailing list
> >> >>> Users at ovirt.org
> >> >>> http://lists.ovirt.org/mailman/listinfo/users
> >> >>>
> >> >>>
> >> >>
> >> >> --
> >> >> Yaniv Bronhaim.
> >> >>
> >> >
> >>
>