[ovirt-users] unsuccessful hosted engine install

Tue Jun 6 00:10:52 UTC 2017

As requested,

The output of ovirt-hosted-engine-cleanup

[root at node-1 ~]# ovirt-hosted-engine-cleanup
This will de-configure the host to run ovirt-hosted-engine-setup from
scratch.
Caution, this operation should be used with care.

Are you sure you want to proceed? [y/n]
y
 -=== Destroy hosted-engine VM ===-
You must run deploy first
 -=== Stop HA services ===-
 -=== Shutdown sanlock ===-
shutdown force 1 wait 0
shutdown done 0
 -=== Disconnecting the hosted-engine storage domain ===-
You must run deploy first
 -=== De-configure VDSM networks ===-
 -=== Stop other services ===-
 -=== De-configure external daemons ===-
 -=== Removing configuration files ===-
? /etc/init/libvirtd.conf already missing
- removing /etc/libvirt/nwfilter/vdsm-no-mac-spoofing.xml
? /etc/ovirt-hosted-engine/answers.conf already missing
? /etc/ovirt-hosted-engine/hosted-engine.conf already missing
- removing /etc/vdsm/vdsm.conf
- removing /etc/pki/vdsm/certs/cacert.pem
- removing /etc/pki/vdsm/certs/vdsmcert.pem
- removing /etc/pki/vdsm/keys/vdsmkey.pem
- removing /etc/pki/vdsm/libvirt-spice/ca-cert.pem
- removing /etc/pki/vdsm/libvirt-spice/ca-key.pem
- removing /etc/pki/vdsm/libvirt-spice/server-cert.pem
- removing /etc/pki/vdsm/libvirt-spice/server-key.pem
? /etc/pki/CA/cacert.pem already missing
? /etc/pki/libvirt/*.pem already missing
? /etc/pki/libvirt/private/*.pem already missing
? /etc/pki/ovirt-vmconsole/*.pem already missing
- removing /var/cache/libvirt/qemu
? /var/run/ovirt-hosted-engine-ha/* already missing
[root at node-1 ~]#

Output of sanlock client status:
[root at node-1 ~]# sanlock client status
[root at node-1 ~]#

Thank you for your help!

On Mon, Jun 5, 2017 at 7:25 AM, Simone Tiraboschi <stirabos at redhat.com>
wrote:

>
>
> On Mon, Jun 5, 2017 at 3:57 PM, Brendan Hartzell <mrrex4 at gmail.com> wrote:
>
>> After letting this sit for a few days, does anyone have any ideas as to
>> how to deal with my situation?  Would anyone like me to send the SOS report
>> directly to them?  It's a 9MB file.
>>
>> If nothing comes up, I'm going to try and sift through the SOS report
>> tonight, but I won't know what I'm trying to find.
>>
>> Thank you for any and all help.
>>
>> On Thu, Jun 1, 2017 at 1:15 AM, Sandro Bonazzola <sbonazzo at redhat.com>
>> wrote:
>>
>>>
>>>
>>> On Thu, Jun 1, 2017 at 6:36 AM, Brendan Hartzell <mrrex4 at gmail.com>
>>> wrote:
>>>
>>>> Ran the 4 commands listed above, no errors on the screen.
>>>>
>>>> Started the hosted-engine standard setup from the web-UI.
>>>>
>>>> Using iSCSI for the storage.
>>>>
>>>> Using mostly default options, I got these errors in the web-UI.
>>>>
>>>>  Error creating Volume Group: Failed to initialize physical device:
>>>> ("[u'/dev/mapper/36589cfc000000de7482638fcfcebbbb4']",)
>>>> Failed to execute stage 'Misc configuration': Failed to initialize
>>>> physical device: ("[u'/dev/mapper/36589cfc00000
>>>> 0de7482638fcfcebbbb4']",)
>>>> Hosted Engine deployment failed: this system is not reliable, please
>>>> check the issue,fix and redeploy
>>>>
>>>> I rebuilt my iSCSI (I don't think I cleaned it up from a previous
>>>> install).
>>>> Re-ran the above 4 commands.
>>>> Restarted hosted engine standard setup from web-UI.
>>>> Install moved past "Connecting Storage Pool" so I believe the above was
>>>> my fault.
>>>>
>>>> These are the last messages displayed on the web-UI.
>>>>  Creating Storage Pool
>>>> Connecting Storage Pool
>>>> Verifying sanlock lockspace initialization
>>>> Creating Image for 'hosted-engine.lockspace' ...
>>>> Image for 'hosted-engine.lockspace' created successfully
>>>> Creating Image for 'hosted-engine.metadata' ...
>>>> Image for 'hosted-engine.metadata' created successfully
>>>> Creating VM Image
>>>> Extracting disk image from OVF archive (could take a few minutes
>>>> depending on archive size)
>>>> Validating pre-allocated volume size
>>>> Uploading volume to data domain (could take a few minutes depending on
>>>> archive size)
>>>>
>>>> At the host terminal, I got the error "watchdog watchdog0: watchdog did
>>>> not stop!"
>>>> Then the host restarted.
>>>>
>>>
>>> Simone, can you help here?
>>>
>>>
> Ok, sorry for the delay.
> The second installation attempt seams fine but it seams that
> ovirt-hosted-engine-cleanup failed stopping sanlock and so the
> watchdog kick in rebooting your system in the middle of deployment attempt.
>
> could you please post the output of
>    ovirt-hosted-engine-cleanup
>    sanlock client status
> ?
>
>
>
>>
>>>
>>>
>>>>
>>>> This is as far as I've gotten in previous attempts.
>>>>
>>>> Attaching the hosted-engine-setup log.
>>>>
>>>> The SOS report is 9MB and the ovirt users group will drop the email.
>>>>
>>>> On Wed, May 31, 2017 at 6:59 AM, Sandro Bonazzola <sbonazzo at redhat.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, May 31, 2017 at 3:10 PM, Brendan Hartzell <mrrex4 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Now that you have identified the problem, should I run the following
>>>>>> commands and send you another SOS?
>>>>>>
>>>>>> ovirt-hosted-engine-cleanup
>>>>>> vdsm-tool configure --force
>>>>>> systemctl restart libvirtd
>>>>>> systemctl restart vdsm
>>>>>>
>>>>>> Or is there a different plan in mind?
>>>>>>
>>>>>
>>>>> I would have expected someone from virt team to follow up for further
>>>>> investigations :-)
>>>>> above commands should work.
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> Thank you,
>>>>>>
>>>>>> Brendan
>>>>>>
>>>>>> On Tue, May 30, 2017 at 11:42 PM, Sandro Bonazzola <
>>>>>> sbonazzo at redhat.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, May 31, 2017 at 4:45 AM, Brendan Hartzell <mrrex4 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Can you please elaborate about the failure you see here and how are
>>>>>>>> you trying to manually partition the host?
>>>>>>>>
>>>>>>>> Sure, I will start from the beginning.
>>>>>>>> - Using: ovirt-node-ng-installer-ovirt-4.1-2017052604
>>>>>>>> <(201)%20705-2604>.iso
>>>>>>>> - During installation I setup one of the two interfaces and check
>>>>>>>> the box to automatically use the connection.
>>>>>>>> - I'm currently providing a host name of node-1.test.net until I
>>>>>>>> have a successful process.
>>>>>>>> - I configure date and time for my timezone and to use an internal
>>>>>>>> NTP server.
>>>>>>>> - On Installation Destination, I pick my 128GB USB3.0 SanDisk flash
>>>>>>>> drive, check the box that I would like to make additional space, and click
>>>>>>>> done.  In the reclaim disk space window, I click delete all, and then
>>>>>>>> reclaim space.  I go back into the Installation Destination, select that I
>>>>>>>> will configure partitioning, and click done.  The Manual Partitioning
>>>>>>>> window opens, I use the option to automatically create mount points.
>>>>>>>>
>>>>>>>
>>>>>>> In this screen, please change partitioning scheme from LVM to LVM
>>>>>>> Thin Provisioning: it should solve your following error.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>   At this point, /boot is 1024MB, /var is 15GB, / is 88.11 GB, and
>>>>>>>> swap is 11.57GB.  I then change / to 23.11 GB, update settings, change /var
>>>>>>>> to 80GB, update settings again, and click done.  I accept the changes and
>>>>>>>> begin installation.
>>>>>>>>
>>>>>>>> I tried these changes based on this article: http://www.ovirt.org/
>>>>>>>> documentation/self-hosted/chap-Deploying_Self-Hosted_Engine/
>>>>>>>>
>>>>>>>> The article does say that you can specify a different directory
>>>>>>>> than /var/tmp, but I don't recall seeing that option.
>>>>>>>>
>>>>>>>
>>>>>>> If the setup detects not enough space in /var/tmp for extracting the
>>>>>>> appliance it will ask about a different directory.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> After some time, I get the following error:
>>>>>>>> There was an error running the kickstart script at line 7.  This is
>>>>>>>> a fatal error and installation will be aborted.  The details of this error
>>>>>>>> are:
>>>>>>>>
>>>>>>>> [INFO] Trying to create a manageable base from '/'
>>>>>>>> [ERROR] LVM Thin Provisioning partitioning scheme is required.  For
>>>>>>>> autoinstall via Kickstart with LVM Thin Provisioning check options
>>>>>>>> --thinpool and --grow.  Please consult documentation for details.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ^^ this one should be solved by the LVM Thin Provisioning scheme
>>>>>>> mentioned above..
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Traceback (most recent call last):
>>>>>>>> File "/usr/lib64/python2.7/runpy.py", line 162, in
>>>>>>>> _run_module_as_main "__main__", fname, loader, pkg_name)
>>>>>>>> File "/usr/lib64/python2.7runpy.py", line 72, in _run_code exec
>>>>>>>> code in run_globals
>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/__main__.py", line
>>>>>>>> 51, in <module> CliApplication()
>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/__init__.py", line
>>>>>>>> 82, in CliApplication()
>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/hooks.py", line
>>>>>>>> 120, in emit cb(self.context, *args)
>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/plugins/core.py",
>>>>>>>> line 169, in post_argparse layout.initialize(args.source, args.init_nvr)
>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/plugins/core.py",
>>>>>>>> line 216, in initialize self.app.imgbase.init_layout_from(source,
>>>>>>>> init_nvr)
>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/imgbase.py", line
>>>>>>>> 271, in init_layout_from self.init_tags_on(existing_lv)
>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/imgbase.py", line
>>>>>>>> 243, in init_tags_on pool = lv.thinpool()
>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/lvm.py", line 250,
>>>>>>>> in thinpool raise MissingLvmThinPool() imgbased.lvm.MissingLvmThinPoo
>>>>>>>> l
>>>>>>>>
>>>>>>>> At this point, the only option is to exit the installer.
>>>>>>>>
>>>>>>>> ****************************
>>>>>>>>
>>>>>>>> Being this a new install, please use 4.1. oVirt 4.0 is not
>>>>>>>> supported anymore.
>>>>>>>>
>>>>>>>> Not a problem.
>>>>>>>>
>>>>>>>> ****************************
>>>>>>>>
>>>>>>>> Can you please provide hosted engine setup logs or better a full
>>>>>>>> sos report? (sosreport -a)
>>>>>>>>
>>>>>>>> Again, the process I'm following:
>>>>>>>> - Using: ovirt-node-ng-installer-ovirt-4.1-2017052604
>>>>>>>> <(201)%20705-2604>.iso
>>>>>>>> - During installation I setup one of the two interfaces and check
>>>>>>>> the box to automatically use the connection.
>>>>>>>> - I'm currently providing a host name of node-1.test.net until I
>>>>>>>> have a successful process.
>>>>>>>> - I configure date and time for my timezone and to use an internal
>>>>>>>> NTP server.
>>>>>>>> - On Installation Destination, I pick my 128GB USB3.0 SanDisk flash
>>>>>>>> drive, check the box that I would like to make additional space, and click
>>>>>>>> done. In the reclaim disk space window, I click delete all, and then
>>>>>>>> reclaim space.
>>>>>>>> - Begin Installation and set a root password.
>>>>>>>> - Perform a yum update - no packages marked for update (as expected)
>>>>>>>> - Use vi to update /etc/hosts with a reference for node-1.test.net
>>>>>>>> and engine.test.net
>>>>>>>> - First attempt at hosted-engine from web-UI
>>>>>>>> - Setup downloads and installs ovirt-engine-applianc
>>>>>>>> e-4.1-20170523.1.el7.centos.noarch.rpm
>>>>>>>> *Failed to execute stage 'Environment setup': Failed to reconfigure
>>>>>>>> libvirt for VDSM
>>>>>>>> *Hosted Engine deployment failed
>>>>>>>> - Attached SOS report
>>>>>>>> The checksum is: aa56097edc0b63c49caaf1a1fde021bc
>>>>>>>>
>>>>>>>> At this point, I would run ovirt-hosted-engine-cleanup and I would
>>>>>>>> get further along in the install process.  However, because this is a fresh
>>>>>>>> install, I'm going to leave things here for now so you can review the SOS.
>>>>>>>>
>>>>>>>
>>>>>>> Thanks for the SOS report!
>>>>>>> Hosted Engine setup fails on:
>>>>>>>
>>>>>>> 2017-05-30 19:24:39 DEBUG otopi.plugins.gr_he_setup.system.vdsmenv
>>>>>>> plugin.execute:921 execute-output: ('/bin/vdsm-tool', 'configure',
>>>>>>> '--force') stdout:
>>>>>>>
>>>>>>> Checking configuration status...
>>>>>>>
>>>>>>> Current revision of multipath.conf detected, preserving
>>>>>>> lvm is configured for vdsm
>>>>>>> libvirt is already configured for vdsm
>>>>>>> SUCCESS: ssl configured to true. No conflicts
>>>>>>>
>>>>>>> Running configure...
>>>>>>> Reconfiguration of libvirt is done.
>>>>>>>
>>>>>>> 2017-05-30 19:24:39 DEBUG otopi.plugins.gr_he_setup.system.vdsmenv
>>>>>>> plugin.execute:926 execute-output: ('/bin/vdsm-tool', 'configure',
>>>>>>> '--force') stderr:
>>>>>>> Error:  ServiceOperationError: _systemctlStart failed
>>>>>>> Job for libvirtd.service failed because the control process exited
>>>>>>> with error code. See "systemctl status libvirtd.service" and "journalctl
>>>>>>> -xe" for details.
>>>>>>>
>>>>>>> At the same time journalctl shows:
>>>>>>>
>>>>>>> May 30 19:24:39 node-1.test.net libvirtd[20954]: libvirt version:
>>>>>>> 2.0.0, package: 10.el7_3.5 (CentOS BuildSystem <
>>>>>>> http://bugs.centos.org>, 2017-03-03-02:09:45, c1bm.rdu2.centos.org)
>>>>>>> May 30 19:24:39 node-1.test.net libvirtd[20954]: hostname:
>>>>>>> node-1.test.net
>>>>>>> May 30 19:24:39 node-1.test.net libvirtd[20954]: The server
>>>>>>> certificate /etc/pki/vdsm/certs/vdsmcert.pem is not yet active
>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: libvirtd.service: main
>>>>>>> process exited, code=exited, status=6/NOTCONFIGURED
>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: Failed to start
>>>>>>> Virtualization daemon.
>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: Unit libvirtd.service
>>>>>>> entered failed state.
>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: libvirtd.service failed.
>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: libvirtd.service
>>>>>>> holdoff time over, scheduling restart.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> ****************************
>>>>>>>>
>>>>>>>> I'd like to understand the issues you faced before suggesting to
>>>>>>>> restart from scratch.
>>>>>>>>
>>>>>>>> Too late... I did two re-installs to get a more accurate account of
>>>>>>>> my install process for above.
>>>>>>>>
>>>>>>>> ****************************
>>>>>>>>
>>>>>>>> Thank you for your help!
>>>>>>>>
>>>>>>>> Brendan
>>>>>>>>
>>>>>>>> On Tue, May 30, 2017 at 12:17 AM, Sandro Bonazzola <
>>>>>>>> sbonazzo at redhat.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, May 30, 2017 at 6:49 AM, Brendan Hartzell <
>>>>>>>>> mrrex4 at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> oVirt users list,
>>>>>>>>>>
>>>>>>>>>> Long story short, I've been spending weeks on this project for my
>>>>>>>>>> home lab with no success.
>>>>>>>>>>
>>>>>>>>>> I would like to successfully install two nodes that host a highly
>>>>>>>>>> available engine with an iSCSI storage back-end.
>>>>>>>>>>
>>>>>>>>>> I have read through most, if not all, of the guides on ovirt.org
>>>>>>>>>> with no substantial help.
>>>>>>>>>>
>>>>>>>>>> Successfully, I have done the following:
>>>>>>>>>> Install oVirt Engine on a bare metal system, added a node, and
>>>>>>>>>> started exploring - not desired.
>>>>>>>>>> Install oVirt Node 4.0.6 on a bare metal system - fails if
>>>>>>>>>> partitions are not done automatically.
>>>>>>>>>> Install oVirt Node 4.1.2 on a bare metal system - fails if
>>>>>>>>>> partitions are not done automatically.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Can you please elaborate about the failure you see here and how
>>>>>>>>> are you trying to manually partition the host?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> My process after installing a Node:
>>>>>>>>>> Run a yum update - just to be sure, but I am using latest iso
>>>>>>>>>> images from downloads section.
>>>>>>>>>> Edit /etc/hosts for local name resolution - the goal is to host
>>>>>>>>>> DNS as a virtual machine, eventually.
>>>>>>>>>> On 4.1 if I install ovirt-engine-appliance from yum, it does
>>>>>>>>>> simplify one step in the hosted engine setup.  If I do this on 4.0 it
>>>>>>>>>> discards the image and uses the default.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Being this a new install, please use 4.1. oVirt 4.0 is not
>>>>>>>>> supported anymore.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 4.1 the hosted engine setup fails immediately unless I run the
>>>>>>>>>> hosted engine cleanup from the shell.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Can you please provide hosted engine setup logs or better a full
>>>>>>>>> sos report? (sosreport -a)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>   If I do this, I can typically get to the point of installing.
>>>>>>>>>>
>>>>>>>>>> When I do get to the installation phase, I get to a point just
>>>>>>>>>> after extracting the OVA that I get a message on the shell saying something
>>>>>>>>>> about the watchdog running the whole time and then the node reboots.
>>>>>>>>>>
>>>>>>>>>> I found one email thread that sounded like my issue and suggested
>>>>>>>>>> the following commands:
>>>>>>>>>> vdsm-tool configure --force
>>>>>>>>>> systemctl restart libvirtd
>>>>>>>>>> systemctl restart vdsmd
>>>>>>>>>>
>>>>>>>>>> Unfortunately, these commands did not help my situation like the
>>>>>>>>>> other individual.
>>>>>>>>>>
>>>>>>>>>> What log file would everyone like to see first?  Given that I
>>>>>>>>>> still consider myself relatively new to Linux, please identify the path for
>>>>>>>>>> the log file requested.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> See above
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Also, because I plan on performing a clean install for this
>>>>>>>>>> thread using my process from above (I'm not expecting my outcome to be any
>>>>>>>>>> different), are there any tips and tricks that might result in a success?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I'd like to understand the issues you faced before suggesting to
>>>>>>>>> restart from scratch.
>>>>>>>>> Adding some people who may help as well.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thank you for any and all help,
>>>>>>>>>> Brendan
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Users mailing list
>>>>>>>>>> Users at ovirt.org
>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> SANDRO BONAZZOLA
>>>>>>>>>
>>>>>>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION
>>>>>>>>> R&D
>>>>>>>>>
>>>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>>>> <https://red.ht/sig>
>>>>>>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> SANDRO BONAZZOLA
>>>>>>>
>>>>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
>>>>>>>
>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>> <https://red.ht/sig>
>>>>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> SANDRO BONAZZOLA
>>>>>
>>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
>>>>>
>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>> <https://red.ht/sig>
>>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> SANDRO BONAZZOLA
>>>
>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
>>>
>>> Red Hat EMEA <https://www.redhat.com/>
>>> <https://red.ht/sig>
>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170605/99fbb6a5/attachment-0001.html>