[ovirt-users] hosted engine vm.conf missing

Tue Dec 1 09:36:12 EST 2015

On Tue, Dec 1, 2015 at 3:24 PM, Thomas Scofield <tscofield at gmail.com> wrote:

> I did import the lun into the engine.  I had the hosted engine running for
> a few days and I was able to restart it a number of times.  It wasn't until
> I rebooted the physical box that it was running on that I encountered the
> problem.
>
hosted-engine-setup probably failed adding the LUN to the engine cause it
was already there or something similar; we need engine logs to better
understand but the result is that it didn't completed creating the storage
volume.
Probably it's not worth to fix it cause we are going to remove that once we
completed the procedure to auto-import the hosted-engine storage domain.
didn't you noticed that it failed? wasn't the output clear enough?

Then the setup generates a temp file under
/var/run/ovirt-hosted-engine-ha/vm.conf to use it for the first run and it
copies to the shared storage for other hosts and further runs.
But it failed for the issue above before being able to copy it to the
shared storage.

Your local  /var/run/ovirt-hosted-engine-ha/vm.conf  simply survived till
you rebooted the host and so you were able to reboot the engine VM. As
expected it disappeared once you rebooted.
Probably the agent should validate the conf volume more strictly and fail
more clearly in this case so you'd notice it before.
I'll open a bug against that.

If you want to manually fix in order to recover your setup I can give you
the instruction set to manually create the configuration volume on the
shared storage.
If it was just an evaluation I'd suggest to redeploy it from scratch.

> On Dec 1, 2015 9:14 AM, "Simone Tiraboschi" <stirabos at redhat.com> wrote:
>
>>
>>
>> On Tue, Dec 1, 2015 at 2:07 PM, Thomas Scofield <tscofield at gmail.com>
>> wrote:
>>
>>> Are there different instructions for an iscsi domain?  I was able to
>>> find the proper path at /rhev/data-center/mnt/blockSD/$sdUUID/images/$
>>> conf_image_UUID/$conf_volume_UUID but the tar command failed
>>>
>>> tar: This does not look like a tar archive
>>> tar: Exiting with failure status due to previous errors
>>>
>>> This was a relatively recent fresh install, the setup log is attached.
>>>
>>
>> The conf volume is not there cause ovirt-hosted-engine-setup failed
>> before creating it.
>>
>> Indeed we have:
>> 2015-11-22 16:30:27 DEBUG
>> otopi.plugins.ovirt_hosted_engine_setup.engine.add_disk
>> add_disk._closeup:153 Connecting to the Engine
>> 2015-11-22 16:30:27 DEBUG otopi.context context._executeMethod:156 method
>> exception
>> Traceback (most recent call last):
>>   File "/usr/lib/python2.7/site-packages/otopi/context.py", line 146, in
>> _executeMethod
>>     method['method']()
>>   File
>> "/usr/share/ovirt-hosted-engine-setup/scripts/../plugins/ovirt-hosted-engine-setup/engine/add_disk.py",
>> line 182, in _closeup
>>     lun_list = check_lun_storage.get_logical_unit()
>> AttributeError: 'NoneType' object has no attribute 'get_logical_unit'
>> 2015-11-22 16:30:27 ERROR otopi.context context._executeMethod:165 Failed
>> to execute stage 'Closing up': 'NoneType' object has no attribute
>> 'get_logical_unit'
>>
>> and:
>> 2015-11-22 16:30:28 DEBUG otopi.context context._executeMethod:142 Stage
>> terminate METHOD otopi.plugins.otopi.dialog.machine.Plugin._terminate
>> 2015-11-22 16:30:28 DEBUG otopi.context context._executeMethod:148
>> condition False
>> 2015-11-22 16:30:28 DEBUG otopi.context context._executeMethod:142 Stage
>> terminate METHOD otopi.plugins.otopi.core.log.Plugin._terminate
>>
>> Probably you hit a different issue.
>> Did you try to take any actions like importing the hosted-engine storage
>> domain LUN on something similar on the engine before continuing with engine
>> setup?
>>
>>
>>
>>> On Tue, Dec 1, 2015 at 5:54 AM, Simone Tiraboschi <stirabos at redhat.com>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Dec 1, 2015 at 12:55 AM, Thomas Scofield <tscofield at gmail.com>
>>>> wrote:
>>>>
>>>>> That file is missing
>>>>>
>>>>> [root at ovirt01 vdsm]# ls -l /var/run/ovirt-hosted-engine-ha/vm.conf
>>>>> ls: cannot access /var/run/ovirt-hosted-engine-ha/vm.conf: No such
>>>>> file or directory
>>>>> [root at ovirt01 vdsm]#
>>>>>
>>>>
>>>>
>>>> Was it a fresh install or an upgrade from 3.5?
>>>> If you still have it, can you please attach the setup logs?
>>>>
>>>> Can you please try to manually check vm.conf on the shared storage?
>>>> This should do the job extracting the files in a local directory.
>>>> (please substitute '192.168.1.115:_Virtual_ext35u36' with the mount
>>>> point of hosted-engine storage domain on your system).
>>>>
>>>> mntpoint=/rhev/data-center/mnt/192.168.1.115:_Virtual_ext35u36
>>>> dir=`mktemp -d` && cd $dir
>>>> sdUUID_line=$(grep sdUUID /etc/ovirt-hosted-engine/hosted-engine.conf)
>>>> sdUUID=${sdUUID_line:7:36}
>>>> conf_volume_UUID_line=$(grep conf_volume_UUID /etc/ovirt-hosted-engine/
>>>> hosted-engine.conf)
>>>> conf_volume_UUID=${conf_volume_UUID_line:17:36}
>>>> conf_image_UUID_line=$(grep conf_image_UUID /etc/ovirt-hosted-engine/
>>>> hosted-engine.conf)
>>>> conf_image_UUID=${conf_image_UUID_line:16:36}
>>>> dd if=$mntpoint/$sdUUID/images/$conf_image_UUID/$conf_volume_UUID
>>>> 2>/dev/null| tar -xvf -
>>>>
>>>>
>>>>
>>>>
>>>>>
>>>>> On Mon, Nov 30, 2015 at 4:04 AM, Simone Tiraboschi <
>>>>> stirabos at redhat.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Nov 27, 2015 at 9:48 PM, Thomas Scofield <tscofield at gmail.com
>>>>>> > wrote:
>>>>>>
>>>>>>> I have recently setup a ovirt hosted engine on iscsi storage and
>>>>>>> after a reboot of the system I am unable to start the hosted engine.  The
>>>>>>> agent.log gives errors indicating there is a missing value in the vm.conf
>>>>>>> file, but the vm.conf file does not appear in the location indicated.
>>>>>>> There is no error indicated when the agent attempts to reload the vm.conf.
>>>>>>> Any ideas on how to get the hosted engine up and running?
>>>>>>>
>>>>>>> MainThread::INFO::2015-11-26
>>>>>>> 21:31:13,071::hosted_engine::699::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_sanlock)
>>>>>>> Ensuring lease for lockspace hosted-engine, host id 1 is acquired (file:
>>>>>>> /var/run/vdsm/storage/ddf4a26b-61ff-49a4-81db-9f82da35e44b/6ed6d868-aaf3-4b3f-bdf0-a4ad262709ae/1fe5b7fc-eae7-4f07-a2fe-5a082e14c876)
>>>>>>> MainThread::INFO::2015-11-26
>>>>>>> 21:31:13,072::upgrade::836::ovirt_hosted_engine_ha.lib.upgrade.StorageServer::(upgrade)
>>>>>>> Host configuration is already up-to-date
>>>>>>> MainThread::INFO::2015-11-26
>>>>>>> 21:31:13,072::hosted_engine::422::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
>>>>>>> Reloading vm.conf from the shared storage domain
>>>>>>> MainThread::ERROR::2015-11-26
>>>>>>> 21:31:13,100::agent::205::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
>>>>>>> Error: ''Configuration value not found: file=/var/run/ovirt-hosted-engine-ha/vm.conf,
>>>>>>> key=memSize'' - trying to restart agent
>>>>>>> MainThread::WARNING::2015-11-26
>>>>>>> 21:31:18,105::agent::208::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
>>>>>>> Restarting agent, attempt '9'
>>>>>>> MainThread::ERROR::2015-11-26
>>>>>>> 21:31:18,106::agent::210::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
>>>>>>> Too many errors occurred, giving up. Please review the log and consider
>>>>>>> filing a bug.
>>>>>>> MainThread::INFO::2015-11-26
>>>>>>> 21:31:18,106::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
>>>>>>> Agent shutting down
>>>>>>>
>>>>>>
>>>>>> Hi Thomas,
>>>>>> could you please attach also your /var/run/ovirt-hosted-engine-ha/vm.conf
>>>>>> ?
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users at ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20151201/de5bc8c4/attachment.html>