[ovirt-users] unsuccessful hosted engine install

Fri Jun 9 21:16:29 UTC 2017

Given that my last known message during these attempts was the bit about
the watchdog, I've been doing some research as to what it does, and how to
work with the service.

I also started taking a closer look at the Logs section in the web-UI.

I found this...

[image: Inline image 2]
(screenshot of Errors from the day of the last attempt at install)

[image: Inline image 3]
(screen shot of the most recent error)

I thought about simply turning off the watchdog and giving the
hosted-engine-setup another try, but it looks like the watchdog is simply
acting like a symptom of a different issue.

I also found (and attached) the messages log file where these logs reside
just in case anyone wanted to look at the whole thing.

Any thoughts or ideas to get this install to work?

On Wed, Jun 7, 2017 at 5:37 AM, Brendan Hartzell <mrrex4 at gmail.com> wrote:

> As requested:
>
> [root at node-1 ~]# systemctl status vdsmd
> ● vdsmd.service - Virtual Desktop Server Manager
>   Loaded: loaded (/usr/lib/systemd/system/vdsmd.service; enabled; vendor
> preset: enabled)
>   Active: active (running) since Tue 2017-06-06 05:51:29 PDT; 23h ago
>  Process: 3052 ExecStartPre=/usr/libexec/vdsm/vdsmd_init_common.sh
> --pre-start (code=exited, status=0/SUCCESS)
> Main PID: 3810 (vdsm)
>   CGroup: /system.slice/vdsmd.service
>           ├─ 3810 /usr/bin/python2 /usr/share/vdsm/vdsm
>           ├─20846 /usr/libexec/ioprocess --read-pipe-fd 41 --write-pipe-fd
> 40 --max-threads 10 --max-queued-requests 10
>           └─20848 /usr/libexec/ioprocess --read-pipe-fd 48 --write-pipe-fd
> 46 --max-threads 10 --max-queued-requests 10
>
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 client step 1
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5
> make_client_response()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 client step 2
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5
> parse_server_challenge()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5
> make_client_response()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 client step 3
> Jun 06 05:51:30 node-1.test.net vdsm[3810]: vdsm throttled WARN MOM not
> available.
> Jun 06 05:51:30 node-1.test.net vdsm[3810]: vdsm throttled WARN MOM not
> available, KSM stats will be missing.
> [root at node-1 ~]#
>
> [root at node-1 ~]# journalctl -u vdsmd.service
> -- Logs begin at Mon 2017-06-05 22:50:38 PDT, end at Wed 2017-06-07
> 05:32:01 PDT. --
> Jun 06 05:51:26 node-1.test.net systemd[1]: Starting Virtual Desktop
> Server Manager...
> Jun 06 05:51:26 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> mkdirs
> Jun 06 05:51:26 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> configure_coredump
> Jun 06 05:51:26 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> configure_vdsm_logs
> Jun 06 05:51:26 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> wait_for_network
> Jun 06 05:51:26 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> run_init_hooks
>
> Jun 06 05:51:26 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> upgraded_version_check
>
> Jun 06 05:51:26 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> check_is_configured
>
> Jun 06 05:51:27 node-1.test.net vdsmd_init_common.sh[3052]: Current
> revision of multipath.conf detected, preserving
>
> Jun 06 05:51:27 node-1.test.net vdsmd_init_common.sh[3052]: lvm is
> configured for vdsm
>
> Jun 06 05:51:27 node-1.test.net vdsmd_init_common.sh[3052]: libvirt is
> already configured for vdsm
>
> Jun 06 05:51:27 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> validate_configuration
>
> Jun 06 05:51:27 node-1.test.net vdsmd_init_common.sh[3052]: SUCCESS: ssl
> configured to true. No conflicts
>
> Jun 06 05:51:27 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> prepare_transient_repository
> Jun 06 05:51:28 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> syslog_available
> Jun 06 05:51:28 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> nwfilter
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5 client step 2
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5
> parse_server_challenge()
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5 client step 2
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5
> make_client_response()
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5 client step 3
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5 client mech
> dispose
> Jun 06 05:51:28 node-1.test.net python2[3349]: DIGEST-MD5 common mech
> dispose
> Jun 06 05:51:28 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> dummybr
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5 client step 2
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5
> parse_server_challenge()
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5 client step 2
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5
> make_client_response()
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5 client step 3
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5 client mech
> dispose
> Jun 06 05:51:29 node-1.test.net python2[3563]: DIGEST-MD5 common mech
> dispose
> Jun 06 05:51:29 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> tune_system
> Jun 06 05:51:29 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> test_space
> Jun 06 05:51:29 node-1.test.net vdsmd_init_common.sh[3052]: vdsm: Running
> test_lo
> Jun 06 05:51:29 node-1.test.net systemd[1]: Started Virtual Desktop
> Server Manager.
> Jun 06 05:51:29 node-1.test.net python2[3810]: DIGEST-MD5 client step 2
> Jun 06 05:51:29 node-1.test.net python2[3810]: DIGEST-MD5
> parse_server_challenge()
> Jun 06 05:51:29 node-1.test.net python2[3810]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:29 node-1.test.net python2[3810]: DIGEST-MD5 client step 2
> Jun 06 05:51:29 node-1.test.net python2[3810]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:29 node-1.test.net python2[3810]: DIGEST-MD5
> make_client_response()
> Jun 06 05:51:29 node-1.test.net python2[3810]: DIGEST-MD5 client step 3
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 client step 1
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 client step 1
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5
> make_client_response()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 client step 2
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5
> parse_server_challenge()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 ask_user_info()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5
> make_client_response()
> Jun 06 05:51:30 node-1.test.net python2[3810]: DIGEST-MD5 client step 3
> Jun 06 05:51:30 node-1.test.net vdsm[3810]: vdsm throttled WARN MOM not
> available.
> Jun 06 05:51:30 node-1.test.net vdsm[3810]: vdsm throttled WARN MOM not
> available, KSM stats will be missing.
> [root at node-1 ~]#
>
> Also, after the last attempt to install the hosted engine:
>
> [root at node-1 ~]# nodectl check
> Status: OK
> Bootloader ... OK
>  Layer boot entries ... OK
>  Valid boot entries ... OK
> Mount points ... OK
>  Separate /var ... OK
>  Discard is used ... OK
> Basic storage ... OK
>  Initialized VG ... OK
>  Initialized Thin Pool ... OK
>  Initialized LVs ... OK
> Thin storage ... OK
>  Checking available space in thinpool ... OK
>  Checking thinpool auto-extend ... OK
> vdsmd ... OK
> [root at node-1 ~]#
>
> On Tue, Jun 6, 2017 at 11:42 PM, Yuval Turgeman <yuvalt at redhat.com> wrote:
>
>> HI Brendan,
>>
>> Can you please send the output for systemctl status vdsmd and journalctl
>> -u vdsmd.service ?
>>
>> Thanks,
>>
>>
>> On Wed, Jun 7, 2017 at 9:32 AM, Sandro Bonazzola <sbonazzo at redhat.com>
>> wrote:
>>
>>>
>>>
>>> On Tue, Jun 6, 2017 at 2:56 PM, Brendan Hartzell <mrrex4 at gmail.com>
>>> wrote:
>>>
>>>> Upon login to the server, to watch terminal output, I noticed that the
>>>> node status is degraded.
>>>>
>>>> [root at node-1 ~]# nodectl check
>>>> Status: WARN
>>>> Bootloader ... OK
>>>>  Layer boot entries ... OK
>>>>  Valid boot entries ... OK
>>>> Mount points ... OK
>>>>  Separate /var ... OK
>>>>  Discard is used ... OK
>>>> Basic storage ... OK
>>>>  Initialized VG ... OK
>>>>  Initialized Thin Pool ... OK
>>>>  Initialized LVs ... OK
>>>> Thin storage ... OK
>>>>  Checking available space in thinpool ... OK
>>>>  Checking thinpool auto-extend ... OK
>>>> vdsmd ... BAD
>>>>
>>>
>>> Yuval, can you help here?
>>>
>>>
>>>
>>>>
>>>>
>>>> Pressing forward with the retry using the web-UI.
>>>>
>>>> After resetting my iSCSI storage (on the storage server side), Install
>>>> started.
>>>>
>>>> Status in the web-UI:
>>>> Creating Storage Domain
>>>> Creating Storage Pool
>>>> Connecting Storage Pool
>>>> Verifying sanlock lockspace initialization
>>>> Creating Image for 'hosted-engine.lockspace' ...
>>>> Image for 'hosted-engine.lockspace' created successfully
>>>> Creating Image for 'hosted-engine.metadata' ...
>>>> Image for 'hosted-engine.metadata' created successfully
>>>> Creating VM Image
>>>> Extracting disk image from OVF archive (could take a few minutes
>>>> depending on archive size)
>>>> Validating pre-allocated volume size
>>>>
>>>> Output from the terminal:
>>>> [45863.076979]watchdog watchdog0: watchdog did not stop!
>>>>
>>>> System restarted.
>>>>
>>>> Attaching ovirt-hosted-engine-setup log.
>>>>
>>>> I'm running an SOS report, but it's too big for the users list.  I can
>>>> email it directly to you upon request.
>>>>
>>>> On Tue, Jun 6, 2017 at 12:12 AM, Simone Tiraboschi <stirabos at redhat.com
>>>> > wrote:
>>>>
>>>>>
>>>>>
>>>>> On Tue, Jun 6, 2017 at 2:10 AM, Brendan Hartzell <mrrex4 at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> As requested,
>>>>>>
>>>>>
>>>>> It seams fine, there are no pending locks now.
>>>>> Could you please retry?
>>>>>
>>>>>
>>>>>>
>>>>>> The output of ovirt-hosted-engine-cleanup
>>>>>>
>>>>>> [root at node-1 ~]# ovirt-hosted-engine-cleanup
>>>>>> This will de-configure the host to run ovirt-hosted-engine-setup from
>>>>>> scratch.
>>>>>> Caution, this operation should be used with care.
>>>>>>
>>>>>> Are you sure you want to proceed? [y/n]
>>>>>> y
>>>>>>  -=== Destroy hosted-engine VM ===-
>>>>>> You must run deploy first
>>>>>>  -=== Stop HA services ===-
>>>>>>  -=== Shutdown sanlock ===-
>>>>>> shutdown force 1 wait 0
>>>>>> shutdown done 0
>>>>>>  -=== Disconnecting the hosted-engine storage domain ===-
>>>>>> You must run deploy first
>>>>>>  -=== De-configure VDSM networks ===-
>>>>>>  -=== Stop other services ===-
>>>>>>  -=== De-configure external daemons ===-
>>>>>>  -=== Removing configuration files ===-
>>>>>> ? /etc/init/libvirtd.conf already missing
>>>>>> - removing /etc/libvirt/nwfilter/vdsm-no-mac-spoofing.xml
>>>>>> ? /etc/ovirt-hosted-engine/answers.conf already missing
>>>>>> ? /etc/ovirt-hosted-engine/hosted-engine.conf already missing
>>>>>> - removing /etc/vdsm/vdsm.conf
>>>>>> - removing /etc/pki/vdsm/certs/cacert.pem
>>>>>> - removing /etc/pki/vdsm/certs/vdsmcert.pem
>>>>>> - removing /etc/pki/vdsm/keys/vdsmkey.pem
>>>>>> - removing /etc/pki/vdsm/libvirt-spice/ca-cert.pem
>>>>>> - removing /etc/pki/vdsm/libvirt-spice/ca-key.pem
>>>>>> - removing /etc/pki/vdsm/libvirt-spice/server-cert.pem
>>>>>> - removing /etc/pki/vdsm/libvirt-spice/server-key.pem
>>>>>> ? /etc/pki/CA/cacert.pem already missing
>>>>>> ? /etc/pki/libvirt/*.pem already missing
>>>>>> ? /etc/pki/libvirt/private/*.pem already missing
>>>>>> ? /etc/pki/ovirt-vmconsole/*.pem already missing
>>>>>> - removing /var/cache/libvirt/qemu
>>>>>> ? /var/run/ovirt-hosted-engine-ha/* already missing
>>>>>> [root at node-1 ~]#
>>>>>>
>>>>>> Output of sanlock client status:
>>>>>> [root at node-1 ~]# sanlock client status
>>>>>> [root at node-1 ~]#
>>>>>>
>>>>>> Thank you for your help!
>>>>>>
>>>>>> On Mon, Jun 5, 2017 at 7:25 AM, Simone Tiraboschi <
>>>>>> stirabos at redhat.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 5, 2017 at 3:57 PM, Brendan Hartzell <mrrex4 at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> After letting this sit for a few days, does anyone have any ideas
>>>>>>>> as to how to deal with my situation?  Would anyone like me to send the SOS
>>>>>>>> report directly to them?  It's a 9MB file.
>>>>>>>>
>>>>>>>> If nothing comes up, I'm going to try and sift through the SOS
>>>>>>>> report tonight, but I won't know what I'm trying to find.
>>>>>>>>
>>>>>>>> Thank you for any and all help.
>>>>>>>>
>>>>>>>> On Thu, Jun 1, 2017 at 1:15 AM, Sandro Bonazzola <
>>>>>>>> sbonazzo at redhat.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jun 1, 2017 at 6:36 AM, Brendan Hartzell <mrrex4 at gmail.com
>>>>>>>>> > wrote:
>>>>>>>>>
>>>>>>>>>> Ran the 4 commands listed above, no errors on the screen.
>>>>>>>>>>
>>>>>>>>>> Started the hosted-engine standard setup from the web-UI.
>>>>>>>>>>
>>>>>>>>>> Using iSCSI for the storage.
>>>>>>>>>>
>>>>>>>>>> Using mostly default options, I got these errors in the web-UI.
>>>>>>>>>>
>>>>>>>>>>  Error creating Volume Group: Failed to initialize physical
>>>>>>>>>> device: ("[u'/dev/mapper/36589cfc000000de7482638fcfcebbbb4']",)
>>>>>>>>>> Failed to execute stage 'Misc configuration': Failed to
>>>>>>>>>> initialize physical device: ("[u'/dev/mapper/36589cfc00000
>>>>>>>>>> 0de7482638fcfcebbbb4']",)
>>>>>>>>>> Hosted Engine deployment failed: this system is not reliable,
>>>>>>>>>> please check the issue,fix and redeploy
>>>>>>>>>>
>>>>>>>>>> I rebuilt my iSCSI (I don't think I cleaned it up from a previous
>>>>>>>>>> install).
>>>>>>>>>> Re-ran the above 4 commands.
>>>>>>>>>> Restarted hosted engine standard setup from web-UI.
>>>>>>>>>> Install moved past "Connecting Storage Pool" so I believe the
>>>>>>>>>> above was my fault.
>>>>>>>>>>
>>>>>>>>>> These are the last messages displayed on the web-UI.
>>>>>>>>>>  Creating Storage Pool
>>>>>>>>>> Connecting Storage Pool
>>>>>>>>>> Verifying sanlock lockspace initialization
>>>>>>>>>> Creating Image for 'hosted-engine.lockspace' ...
>>>>>>>>>> Image for 'hosted-engine.lockspace' created successfully
>>>>>>>>>> Creating Image for 'hosted-engine.metadata' ...
>>>>>>>>>> Image for 'hosted-engine.metadata' created successfully
>>>>>>>>>> Creating VM Image
>>>>>>>>>> Extracting disk image from OVF archive (could take a few minutes
>>>>>>>>>> depending on archive size)
>>>>>>>>>> Validating pre-allocated volume size
>>>>>>>>>> Uploading volume to data domain (could take a few minutes
>>>>>>>>>> depending on archive size)
>>>>>>>>>>
>>>>>>>>>> At the host terminal, I got the error "watchdog watchdog0:
>>>>>>>>>> watchdog did not stop!"
>>>>>>>>>> Then the host restarted.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Simone, can you help here?
>>>>>>>>>
>>>>>>>>>
>>>>>>> Ok, sorry for the delay.
>>>>>>> The second installation attempt seams fine but it seams that
>>>>>>> ovirt-hosted-engine-cleanup failed stopping sanlock and so the
>>>>>>> watchdog kick in rebooting your system in the middle of deployment attempt.
>>>>>>>
>>>>>>> could you please post the output of
>>>>>>>    ovirt-hosted-engine-cleanup
>>>>>>>    sanlock client status
>>>>>>> ?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> This is as far as I've gotten in previous attempts.
>>>>>>>>>>
>>>>>>>>>> Attaching the hosted-engine-setup log.
>>>>>>>>>>
>>>>>>>>>> The SOS report is 9MB and the ovirt users group will drop the
>>>>>>>>>> email.
>>>>>>>>>>
>>>>>>>>>> On Wed, May 31, 2017 at 6:59 AM, Sandro Bonazzola <
>>>>>>>>>> sbonazzo at redhat.com> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, May 31, 2017 at 3:10 PM, Brendan Hartzell <
>>>>>>>>>>> mrrex4 at gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Now that you have identified the problem, should I run the
>>>>>>>>>>>> following commands and send you another SOS?
>>>>>>>>>>>>
>>>>>>>>>>>> ovirt-hosted-engine-cleanup
>>>>>>>>>>>> vdsm-tool configure --force
>>>>>>>>>>>> systemctl restart libvirtd
>>>>>>>>>>>> systemctl restart vdsm
>>>>>>>>>>>>
>>>>>>>>>>>> Or is there a different plan in mind?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> I would have expected someone from virt team to follow up for
>>>>>>>>>>> further investigations :-)
>>>>>>>>>>> above commands should work.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you,
>>>>>>>>>>>>
>>>>>>>>>>>> Brendan
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, May 30, 2017 at 11:42 PM, Sandro Bonazzola <
>>>>>>>>>>>> sbonazzo at redhat.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, May 31, 2017 at 4:45 AM, Brendan Hartzell <
>>>>>>>>>>>>> mrrex4 at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can you please elaborate about the failure you see here and
>>>>>>>>>>>>>> how are you trying to manually partition the host?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Sure, I will start from the beginning.
>>>>>>>>>>>>>> - Using: ovirt-node-ng-installer-ovirt-4.1-2017052604
>>>>>>>>>>>>>> <(201)%20705-2604>.iso
>>>>>>>>>>>>>> - During installation I setup one of the two interfaces and
>>>>>>>>>>>>>> check the box to automatically use the connection.
>>>>>>>>>>>>>> - I'm currently providing a host name of node-1.test.net
>>>>>>>>>>>>>> until I have a successful process.
>>>>>>>>>>>>>> - I configure date and time for my timezone and to use an
>>>>>>>>>>>>>> internal NTP server.
>>>>>>>>>>>>>> - On Installation Destination, I pick my 128GB USB3.0 SanDisk
>>>>>>>>>>>>>> flash drive, check the box that I would like to make additional space, and
>>>>>>>>>>>>>> click done.  In the reclaim disk space window, I click delete all, and then
>>>>>>>>>>>>>> reclaim space.  I go back into the Installation Destination, select that I
>>>>>>>>>>>>>> will configure partitioning, and click done.  The Manual Partitioning
>>>>>>>>>>>>>> window opens, I use the option to automatically create mount points.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> In this screen, please change partitioning scheme from LVM to
>>>>>>>>>>>>> LVM Thin Provisioning: it should solve your following error.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>   At this point, /boot is 1024MB, /var is 15GB, / is 88.11
>>>>>>>>>>>>>> GB, and swap is 11.57GB.  I then change / to 23.11 GB, update settings,
>>>>>>>>>>>>>> change /var to 80GB, update settings again, and click done.  I accept the
>>>>>>>>>>>>>> changes and begin installation.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I tried these changes based on this article:
>>>>>>>>>>>>>> http://www.ovirt.org/documentation/self-hosted/chap
>>>>>>>>>>>>>> -Deploying_Self-Hosted_Engine/
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The article does say that you can specify a different
>>>>>>>>>>>>>> directory than /var/tmp, but I don't recall seeing that option.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> If the setup detects not enough space in /var/tmp for
>>>>>>>>>>>>> extracting the appliance it will ask about a different directory.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> After some time, I get the following error:
>>>>>>>>>>>>>> There was an error running the kickstart script at line 7.
>>>>>>>>>>>>>> This is a fatal error and installation will be aborted.  The details of
>>>>>>>>>>>>>> this error are:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [INFO] Trying to create a manageable base from '/'
>>>>>>>>>>>>>> [ERROR] LVM Thin Provisioning partitioning scheme is
>>>>>>>>>>>>>> required.  For autoinstall via Kickstart with LVM Thin Provisioning check
>>>>>>>>>>>>>> options --thinpool and --grow.  Please consult documentation for details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> ^^ this one should be solved by the LVM Thin Provisioning
>>>>>>>>>>>>> scheme mentioned above..
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>> File "/usr/lib64/python2.7/runpy.py", line 162, in
>>>>>>>>>>>>>> _run_module_as_main "__main__", fname, loader, pkg_name)
>>>>>>>>>>>>>> File "/usr/lib64/python2.7runpy.py", line 72, in _run_code
>>>>>>>>>>>>>> exec code in run_globals
>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/__main__.py",
>>>>>>>>>>>>>> line 51, in <module> CliApplication()
>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/__init__.py",
>>>>>>>>>>>>>> line 82, in CliApplication()
>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/hooks.py",
>>>>>>>>>>>>>> line 120, in emit cb(self.context, *args)
>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/plugins/core.py",
>>>>>>>>>>>>>> line 169, in post_argparse layout.initialize(args.source, args.init_nvr)
>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/plugins/core.py",
>>>>>>>>>>>>>> line 216, in initialize self.app.imgbase.init_layout_from(source,
>>>>>>>>>>>>>> init_nvr)
>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/imgbase.py",
>>>>>>>>>>>>>> line 271, in init_layout_from self.init_tags_on(existing_lv)
>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/imgbase.py",
>>>>>>>>>>>>>> line 243, in init_tags_on pool = lv.thinpool()
>>>>>>>>>>>>>> File "/usr/lib/python2.7/site-packages/imgbased/lvm.py",
>>>>>>>>>>>>>> line 250, in thinpool raise MissingLvmThinPool()
>>>>>>>>>>>>>> imgbased.lvm.MissingLvmThinPool
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> At this point, the only option is to exit the installer.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ****************************
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Being this a new install, please use 4.1. oVirt 4.0 is not
>>>>>>>>>>>>>> supported anymore.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Not a problem.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ****************************
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can you please provide hosted engine setup logs or better a
>>>>>>>>>>>>>> full sos report? (sosreport -a)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Again, the process I'm following:
>>>>>>>>>>>>>> - Using: ovirt-node-ng-installer-ovirt-4.1-2017052604
>>>>>>>>>>>>>> <(201)%20705-2604>.iso
>>>>>>>>>>>>>> - During installation I setup one of the two interfaces and
>>>>>>>>>>>>>> check the box to automatically use the connection.
>>>>>>>>>>>>>> - I'm currently providing a host name of node-1.test.net
>>>>>>>>>>>>>> until I have a successful process.
>>>>>>>>>>>>>> - I configure date and time for my timezone and to use an
>>>>>>>>>>>>>> internal NTP server.
>>>>>>>>>>>>>> - On Installation Destination, I pick my 128GB USB3.0 SanDisk
>>>>>>>>>>>>>> flash drive, check the box that I would like to make additional space, and
>>>>>>>>>>>>>> click done. In the reclaim disk space window, I click delete all, and then
>>>>>>>>>>>>>> reclaim space.
>>>>>>>>>>>>>> - Begin Installation and set a root password.
>>>>>>>>>>>>>> - Perform a yum update - no packages marked for update (as
>>>>>>>>>>>>>> expected)
>>>>>>>>>>>>>> - Use vi to update /etc/hosts with a reference for
>>>>>>>>>>>>>> node-1.test.net and engine.test.net
>>>>>>>>>>>>>> - First attempt at hosted-engine from web-UI
>>>>>>>>>>>>>> - Setup downloads and installs ovirt-engine-applianc
>>>>>>>>>>>>>> e-4.1-20170523.1.el7.centos.noarch.rpm
>>>>>>>>>>>>>> *Failed to execute stage 'Environment setup': Failed to
>>>>>>>>>>>>>> reconfigure libvirt for VDSM
>>>>>>>>>>>>>> *Hosted Engine deployment failed
>>>>>>>>>>>>>> - Attached SOS report
>>>>>>>>>>>>>> The checksum is: aa56097edc0b63c49caaf1a1fde021bc
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> At this point, I would run ovirt-hosted-engine-cleanup and I
>>>>>>>>>>>>>> would get further along in the install process.  However, because this is a
>>>>>>>>>>>>>> fresh install, I'm going to leave things here for now so you can review the
>>>>>>>>>>>>>> SOS.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the SOS report!
>>>>>>>>>>>>> Hosted Engine setup fails on:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-05-30 19:24:39 DEBUG otopi.plugins.gr_he_setup.system.vdsmenv
>>>>>>>>>>>>> plugin.execute:921 execute-output: ('/bin/vdsm-tool', 'configure',
>>>>>>>>>>>>> '--force') stdout:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Checking configuration status...
>>>>>>>>>>>>>
>>>>>>>>>>>>> Current revision of multipath.conf detected, preserving
>>>>>>>>>>>>> lvm is configured for vdsm
>>>>>>>>>>>>> libvirt is already configured for vdsm
>>>>>>>>>>>>> SUCCESS: ssl configured to true. No conflicts
>>>>>>>>>>>>>
>>>>>>>>>>>>> Running configure...
>>>>>>>>>>>>> Reconfiguration of libvirt is done.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-05-30 19:24:39 DEBUG otopi.plugins.gr_he_setup.system.vdsmenv
>>>>>>>>>>>>> plugin.execute:926 execute-output: ('/bin/vdsm-tool', 'configure',
>>>>>>>>>>>>> '--force') stderr:
>>>>>>>>>>>>> Error:  ServiceOperationError: _systemctlStart failed
>>>>>>>>>>>>> Job for libvirtd.service failed because the control process
>>>>>>>>>>>>> exited with error code. See "systemctl status libvirtd.service" and
>>>>>>>>>>>>> "journalctl -xe" for details.
>>>>>>>>>>>>>
>>>>>>>>>>>>> At the same time journalctl shows:
>>>>>>>>>>>>>
>>>>>>>>>>>>> May 30 19:24:39 node-1.test.net libvirtd[20954]: libvirt
>>>>>>>>>>>>> version: 2.0.0, package: 10.el7_3.5 (CentOS BuildSystem <
>>>>>>>>>>>>> http://bugs.centos.org>, 2017-03-03-02:09:45,
>>>>>>>>>>>>> c1bm.rdu2.centos.org)
>>>>>>>>>>>>> May 30 19:24:39 node-1.test.net libvirtd[20954]: hostname:
>>>>>>>>>>>>> node-1.test.net
>>>>>>>>>>>>> May 30 19:24:39 node-1.test.net libvirtd[20954]: The server
>>>>>>>>>>>>> certificate /etc/pki/vdsm/certs/vdsmcert.pem is not yet active
>>>>>>>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: libvirtd.service:
>>>>>>>>>>>>> main process exited, code=exited, status=6/NOTCONFIGURED
>>>>>>>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: Failed to start
>>>>>>>>>>>>> Virtualization daemon.
>>>>>>>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: Unit
>>>>>>>>>>>>> libvirtd.service entered failed state.
>>>>>>>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: libvirtd.service
>>>>>>>>>>>>> failed.
>>>>>>>>>>>>> May 30 19:24:39 node-1.test.net systemd[1]: libvirtd.service
>>>>>>>>>>>>> holdoff time over, scheduling restart.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ****************************
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I'd like to understand the issues you faced before suggesting
>>>>>>>>>>>>>> to restart from scratch.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Too late... I did two re-installs to get a more accurate
>>>>>>>>>>>>>> account of my install process for above.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ****************************
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thank you for your help!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Brendan
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, May 30, 2017 at 12:17 AM, Sandro Bonazzola <
>>>>>>>>>>>>>> sbonazzo at redhat.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, May 30, 2017 at 6:49 AM, Brendan Hartzell <
>>>>>>>>>>>>>>> mrrex4 at gmail.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> oVirt users list,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Long story short, I've been spending weeks on this project
>>>>>>>>>>>>>>>> for my home lab with no success.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I would like to successfully install two nodes that host a
>>>>>>>>>>>>>>>> highly available engine with an iSCSI storage back-end.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have read through most, if not all, of the guides on
>>>>>>>>>>>>>>>> ovirt.org with no substantial help.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Successfully, I have done the following:
>>>>>>>>>>>>>>>> Install oVirt Engine on a bare metal system, added a node,
>>>>>>>>>>>>>>>> and started exploring - not desired.
>>>>>>>>>>>>>>>> Install oVirt Node 4.0.6 on a bare metal system - fails if
>>>>>>>>>>>>>>>> partitions are not done automatically.
>>>>>>>>>>>>>>>> Install oVirt Node 4.1.2 on a bare metal system - fails if
>>>>>>>>>>>>>>>> partitions are not done automatically.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you please elaborate about the failure you see here and
>>>>>>>>>>>>>>> how are you trying to manually partition the host?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My process after installing a Node:
>>>>>>>>>>>>>>>> Run a yum update - just to be sure, but I am using latest
>>>>>>>>>>>>>>>> iso images from downloads section.
>>>>>>>>>>>>>>>> Edit /etc/hosts for local name resolution - the goal is to
>>>>>>>>>>>>>>>> host DNS as a virtual machine, eventually.
>>>>>>>>>>>>>>>> On 4.1 if I install ovirt-engine-appliance from yum, it
>>>>>>>>>>>>>>>> does simplify one step in the hosted engine setup.  If I do this on 4.0 it
>>>>>>>>>>>>>>>> discards the image and uses the default.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Being this a new install, please use 4.1. oVirt 4.0 is not
>>>>>>>>>>>>>>> supported anymore.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 4.1 the hosted engine setup fails immediately unless I
>>>>>>>>>>>>>>>> run the hosted engine cleanup from the shell.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you please provide hosted engine setup logs or better a
>>>>>>>>>>>>>>> full sos report? (sosreport -a)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>   If I do this, I can typically get to the point of
>>>>>>>>>>>>>>>> installing.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> When I do get to the installation phase, I get to a point
>>>>>>>>>>>>>>>> just after extracting the OVA that I get a message on the shell saying
>>>>>>>>>>>>>>>> something about the watchdog running the whole time and then the node
>>>>>>>>>>>>>>>> reboots.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I found one email thread that sounded like my issue and
>>>>>>>>>>>>>>>> suggested the following commands:
>>>>>>>>>>>>>>>> vdsm-tool configure --force
>>>>>>>>>>>>>>>> systemctl restart libvirtd
>>>>>>>>>>>>>>>> systemctl restart vdsmd
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Unfortunately, these commands did not help my situation
>>>>>>>>>>>>>>>> like the other individual.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What log file would everyone like to see first?  Given that
>>>>>>>>>>>>>>>> I still consider myself relatively new to Linux, please identify the path
>>>>>>>>>>>>>>>> for the log file requested.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> See above
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, because I plan on performing a clean install for this
>>>>>>>>>>>>>>>> thread using my process from above (I'm not expecting my outcome to be any
>>>>>>>>>>>>>>>> different), are there any tips and tricks that might result in a success?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'd like to understand the issues you faced before
>>>>>>>>>>>>>>> suggesting to restart from scratch.
>>>>>>>>>>>>>>> Adding some people who may help as well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank you for any and all help,
>>>>>>>>>>>>>>>> Brendan
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>>>>> Users at ovirt.org
>>>>>>>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> SANDRO BONAZZOLA
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG
>>>>>>>>>>>>>>> VIRTUALIZATION R&D
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>>>>>>>>>> <https://red.ht/sig>
>>>>>>>>>>>>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>>
>>>>>>>>>>>>> SANDRO BONAZZOLA
>>>>>>>>>>>>>
>>>>>>>>>>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG
>>>>>>>>>>>>> VIRTUALIZATION R&D
>>>>>>>>>>>>>
>>>>>>>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>>>>>>>> <https://red.ht/sig>
>>>>>>>>>>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>>
>>>>>>>>>>> SANDRO BONAZZOLA
>>>>>>>>>>>
>>>>>>>>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION
>>>>>>>>>>> R&D
>>>>>>>>>>>
>>>>>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>>>>>> <https://red.ht/sig>
>>>>>>>>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>>
>>>>>>>>> SANDRO BONAZZOLA
>>>>>>>>>
>>>>>>>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION
>>>>>>>>> R&D
>>>>>>>>>
>>>>>>>>> Red Hat EMEA <https://www.redhat.com/>
>>>>>>>>> <https://red.ht/sig>
>>>>>>>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> SANDRO BONAZZOLA
>>>
>>> ASSOCIATE MANAGER, SOFTWARE ENGINEERING, EMEA ENG VIRTUALIZATION R&D
>>>
>>> Red Hat EMEA <https://www.redhat.com/>
>>> <https://red.ht/sig>
>>> TRIED. TESTED. TRUSTED. <https://redhat.com/trusted>
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170609/ef3596f6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 104082 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170609/ef3596f6/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 62488 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170609/ef3596f6/attachment-0003.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: messages
Type: application/octet-stream
Size: 92148 bytes
Desc: not available
URL: <http://lists.ovirt.org/pipermail/users/attachments/20170609/ef3596f6/attachment-0001.obj>