[ovirt-users] Proper way to change and persist vdsm configuration options

Tue Aug 5 14:50:04 EDT 2014

On Tue, Aug 5, 2014 at 12:37 PM, Alon Bar-Lev <alonbl at redhat.com> wrote:
> Hi,
>
> Do you actually use puppet over ovirt-node?
> This is unsupported.
>
> Regards,
> Alon
>

I use Puppet to configure everything on the system and some of those
things conflict with changes made by ovirt-engine when adding a node
so I've moved to managing those changes in an ovirt Puppet module.

When you refer to "ovirt-node" your refering to the pre-built ISO
images?  AFAIK the ovirt-node is not an option for me as I use
Infiniband to do all storage connections.  Currently NFS is done using
IPoIB (NFS over RDMA was crashing so I am not pursuing at the moment)
and iSCSI is done using iSER.  At some point in time I'd like to use
IB interfaces in my guests utilizing SR-IOV which my Mellanox cards
support, but I'm postponing that project till after our HPC cluster's
upgraded :)

Right now the Puppet module does the following:

* Configures firewall rules just like what is done by ovirt (with
exception of a few not supported by puppetlabs-firewall, so I override
global directive that purges unknown firewall rules)
** This is necessary as I have to add other Firewall rules, such as Zabbix
* Exports /etc/hosts entry for the node that is collected by
ovirt-engine host so that if DNS goes down ovirt-engine does not lose
access to ovirtmgmt interfaces
* Install yum repos for ovirt
* install vdsm
* ensure vdsm.conf exists
* Populates /etc/vdsm/vdsm.id (IIRC a bug in previous ovirt required this)
* Ensures vdsmd is running and will start at boot
* Ensures vdsm sudo rules are present.
* Manages default vdsm.conf configurations as a Puppet type,
"vdsm_config", rather than managing file contents via template (allows
for purging unmanaged entries also)

A lot of the above is handled by ovirt already, but in the past
customizations were not possible (firewall rules, vdsm.conf entries,
etc) so if I was going to have to manage those separately I wanted
them in Puppet :)

Now that I'm aware of "ovirt-host-deploy" and have seen the potential
of using vdsm-reg in Puppet, I'm curious what is the "right" way to
automate node deployment's in oVirt. Ideally I could still use Puppet
to configure the method or fill in the "gaps" for customizations that
are needed (ie enabling iSER).

I'd be glad to know what the recommended method for automating ovirt
would be, and would be happy to refactor my module in hopes it would
offer other Puppet users a quick-start way to deploy oVirt while still
doing things the "ovirt way".

If the right way is using the ovirt-node images then I'd like to know
what customizations are possible on those images and so on.  oVirt is
moving very rapidly and despite using ovirt for a long time I'm still
learning new things about it almost daily, so forgive any of my
assumptions above that may be wrong :).

Thanks,
- Trey

> ----- Original Message -----
>> From: "ybronhei" <ybronhei at redhat.com>
>> To: "Trey Dockendorf" <treydock at gmail.com>
>> Cc: "users" <users at ovirt.org>, "Fabian Deutsch" <fabiand at redhat.com>, "Dan Kenigsberg" <danken at redhat.com>, "Itamar
>> Heim" <iheim at redhat.com>, "Douglas Landgraf" <dougsland at redhat.com>, "Alon Bar-Lev" <alonbl at redhat.com>
>> Sent: Tuesday, August 5, 2014 8:32:04 PM
>> Subject: Re: [ovirt-users] Proper way to change and persist vdsm configuration options
>>
>> Hey,
>>
>> Just noticed something that I forgot about..
>> before filing new BZ, see in ovirt-host-deploy README.environment [1]
>> the section:
>> VDSM/configOverride(bool) [True]
>>      Override vdsm configuration file.
>>
>> changing it to false will keep your vdsm.conf file as is after deploying
>> the host again (what happens after node upgrade)
>>
>> [1]
>> https://github.com/oVirt/ovirt-host-deploy/blob/master/README.environment
>>
>> please check if that what you meant..
>>
>> Thanks,
>> Yaniv Bronhaim.
>>
>> On 08/05/2014 08:12 AM, Trey Dockendorf wrote:
>> > I'll file BZ.  As far as I can recall this has been an issue since 3.3.x as
>> > I have been using Puppet to modify values and have had to rerun Puppet
>> > after installing a node via GUI and when performing update from GUI.  Given
>> > that it has occurred when VDSM version didn't change on the node it seems
>> > likely to be something being done by Python code that bootstraps a node and
>> > performs the other tasks.  I won't have any systems available to test with
>> > for a few days.  New hardware specifically for our oVirt deployment is on
>> > order so should be able to more thoroughly debug and capture logs at that
>> > time.
>> >
>> > Would using vdsm-reg be a better solution for adding new nodes?  I only
>> > tried using vdsm-reg once and it went very poorly...lots of missing
>> > dependencies not pulled in from yum install I had to install manually via
>> > yum.  Then the node was auto added to newest cluster with no ability to
>> > change the cluster.  Be happy to debug that too if there's some docs that
>> > outline the expected behavior.
>> >
>> > Using vdsm-reg or something similar seems like a better fit for puppet
>> > deployed nodes, as opposed to requiring GUI steps to add the node.
>> >
>> > Thanks
>> > - Trey
>> > On Aug 4, 2014 5:53 AM, "ybronhei" <ybronhei at redhat.com> wrote:
>> >
>> >> On 07/31/2014 01:28 AM, Trey Dockendorf wrote:
>> >>
>> >>> I'm running ovirt nodes that are stock CentOS 6.5 systems with VDSM
>> >>> installed.  I am using iSER to do iSCSI over RDMA and to make that
>> >>> work I have to modify /etc/vdsm/vdsm.conf to include the following:
>> >>>
>> >>> [irs]
>> >>> iscsi_default_ifaces = iser,default
>> >>>
>> >>> I've noticed that any time I upgrade a node from the engine web
>> >>> interface that changes to vdsm.conf are wiped out.  I don't know if
>> >>> this is being done by the configuration code or by the vdsm package.
>> >>> Is there a more reliable way to ensure changes to vdsm.conf are NOT
>> >>> removed automatically?
>> >>>
>> >>
>> >> Hey,
>> >>
>> >> vdsm.conf shouldn't wiped out and shouldn't changed at all during upgrade.
>> >> other related conf files (such as libvirtd.conf) might be overrided to
>> >> keep
>> >> defaults configurations for vdsm. but vdsm.conf should persist with user's
>> >> modification. from my check, regular yum upgrade doesn't touch vdsm.conf
>> >>
>> >> Douglas can you verify that with node upgrade? might be specific to that
>> >> flow..
>> >>
>> >> Trey, can file a bugzilla on that and describe your steps there?
>> >>
>> >> Thanks
>> >>
>> >> Yaniv Bronhaim,
>> >>
>> >>>
>> >>> Thanks,
>> >>> - Trey
>> >>> _______________________________________________
>> >>> Users mailing list
>> >>> Users at ovirt.org
>> >>> http://lists.ovirt.org/mailman/listinfo/users
>> >>>
>> >>>
>> >>
>> >> --
>> >> Yaniv Bronhaim.
>> >>
>> >
>>