9:45 a.m.
Hi All,
I'm reinstalling hosted-engine on my F22 laptop and I'm making progress.
With beta3 I couldn't get it installed, with beta5 it installed and I
could go into the webui but I had to shutdown my laptop and get some
sleep :-)
Next morning the hosted-engine didn't start. Running 'hosted-engine
--vm-status' tells me I need to deploy first. Looks like its a virgin
system. Looking at what hosted-engine does I found out that
/var/run/ovirt-hosted-engine-ha/vm.conf is missing. In another thread
its mentioned that it now lives on the storage domain but I couldn't
find it there, nothing resembles it.
Question is: how did engine get started and why is that vm.conf now missing.
Logs are available later today if needed.
Joop
10:02 a.m.
On Mon, Sep 14, 2015 at 8:45 AM, noc <noc(a)nieuwland.nl> wrote:
vm.conf moved to shared storage as part of
http://www.ovirt.org/Features/Hosted_Engine_configuration_on_shared_storage
I guess you hit a bug in this area, adding Simone.
--
Sandro Bonazzola
Better technology. Faster innovation. Powered by community collaboration.
See how it works at redhat.com
10:19 a.m.
This is a multi-part message in MIME format.
--------------060700040004050609070404
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
On 14-9-2015 9:17, Simone Tiraboschi wrote:
NFSv3
Could you please attach logs from /var/log/ovirt-hosted-engine-ha/
Yes, later today.
Joop
--------------060700040004050609070404
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 14-9-2015 9:17, Simone Tiraboschi
wrote:<br>
</div>
<blockquote
cite="mid:CAN8-ONoRTv0P1rr243GEi94RE8yYpVmJk_fpmegdd3iHuEMpjQ@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Sep 14, 2015 at 9:02 AM,
Sandro Bonazzola <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:sbonazzo@redhat.com"
target="_blank"><a
class="moz-txt-link-abbreviated"
href="mailto:sbonazzo@redhat.com">sbonazzo@redhat.com</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote"><span
class="">On Mon, Sep
14, 2015 at 8:45 AM, noc <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:noc@nieuwland.nl"
target="_blank"><a class="moz-txt-link-abbreviated"
href="mailto:noc@nieuwland.nl">noc@nieuwland.nl</a></a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px
0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi
All,<br>
<br>
I'm reinstalling hosted-engine on my F22 laptop
and I'm making progress.<br>
With beta3 I couldn't get it installed, with
beta5 it installed and I<br>
could go into the webui but I had to shutdown my
laptop and get some<br>
sleep :-)<br>
Next morning the hosted-engine didn't start.
Running 'hosted-engine<br>
--vm-status' tells me I need to deploy first.
Looks like its a virgin<br>
system. Looking at what hosted-engine does I
found out that<br>
/var/run/ovirt-hosted-engine-ha/vm.conf is
missing. In another thread<br>
its mentioned that it now lives on the storage
domain but I couldn't<br>
find it there, nothing resembles it.<br>
Question is: how did engine get started and why
is that vm.conf now missing.<br>
</blockquote>
<div><br>
</div>
</span>
<div>vm.conf moved to shared storage as part of <a
moz-do-not-send="true"
href="http://www.ovirt.org/Features/Hosted_Engine_configuration_on_s...
target="_blank"><a
class="moz-txt-link-freetext"
href="http://www.ovirt.org/Features/Hosted_Engine_configuration_on_s...
<div>I guess you hit a bug in this area, adding
Simone.</div>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>What kind of hared storage are you using?</div>
</div>
</div>
</div>
</blockquote>
NFSv3<br>
<blockquote
cite="mid:CAN8-ONoRTv0P1rr243GEi94RE8yYpVmJk_fpmegdd3iHuEMpjQ@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>Could you please attach logs from
/var/log/ovirt-hosted-engine-ha/</div>
</div>
</div>
</div>
</blockquote>
Yes, later today.<br>
<br>
Joop<br>
<br>
</body>
</html>
--------------060700040004050609070404--
9:16 p.m.
On 14-9-2015 9:19, Joop wrote:
Log attached of the last try with beta5
I managed to extract vm.conf from the log and when copied to
/var/run/ovirt-hosted-engine-ha the engine will startup up but I get an
error when I try to attach the hosted_engine storage domain.
Sanlock.log attached as well.
Anymore logs you need?
To me it looks like I better start over but I don't want to wipe my F22
install. What is the preferred way to get a 'clean' system so that I can
try again? (concerned about sanlock complaining about name conflicts)
Regards,
Joop
Joop
4:26 p.m.
On Tue, Sep 15, 2015 at 2:31 PM, noc <noc(a)nieuwland.nl> wrote:
Now the agent correctly reads the VM configuration file from the shared
domain so no issue there but then it exits cause VDSM is not up:
MainThread::INFO::2015-09-14
20:48:58,007::hosted_engine::667::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Reloading vm.conf from the shared storage domain
MainThread::INFO::2015-09-14
20:48:58,163::hosted_engine::460::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 3400)
MainThread::INFO::2015-09-14
20:49:08,201::hosted_engine::610::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
Initializing VDSM
MainThread::INFO::2015-09-14
20:49:08,317::hosted_engine::655::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Connecting the storage
MainThread::INFO::2015-09-14
20:49:08,318::storage_server::110::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Connecting storage server
MainThread::INFO::2015-09-14
20:49:08,318::storage_server::135::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Connecting storage server
MainThread::INFO::2015-09-14
20:49:08,349::hosted_engine::659::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Preparing images
MainThread::INFO::2015-09-14
20:49:08,349::image::61::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images)
Preparing images
MainThread::INFO::2015-09-14
20:49:08,787::hosted_engine::667::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Reloading vm.conf from the shared storage domain
MainThread::INFO::2015-09-14
20:49:08,941::hosted_engine::460::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 3400)
MainThread::INFO::2015-09-14
20:49:15,104::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
Agent shutting down
MainThread::INFO::2015-09-14
21:06:07,601::agent::78::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
ovirt-hosted-engine-ha agent 1.3.0-0.0.master.20150909150500 started
MainThread::INFO::2015-09-14
21:06:07,632::hosted_engine::246::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname)
Found certificate common name: lichthuis.puzzle-it.nu
MainThread::INFO::2015-09-14
21:06:07,633::hosted_engine::610::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
Initializing VDSM
MainThread::ERROR::2015-09-14
21:06:18,209::hosted_engine::418::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Service vdsmd is not running and the admin is responsible for starting it.
Shutting down.
MainThread::INFO::2015-09-14
21:06:18,210::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
Agent shutting down
Martin, any hint there?
Previously you had an issue there:
MainThread::ERROR::2015-09-14
17:34:19,972::agent::201::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
Error: '[Errno 13] Permission denied:
'/var/run/ovirt-hosted-engine-ha/vm.conf'' - trying to restart agent
Did you manually created /var/run/ovirt-hosted-engine-ha/vm.conf being
readable only by root or something similar? ovirt-ha-agent is running as
vdsm user and it should be able to refresh
/var/run/ovirt-hosted-engine-ha/vm.conf with the master copy on the
shared storage.
And previously than that it was failing with:
MainThread::ERROR::2015-09-12
13:53:23,727::agent::201::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
Error: ''StorageServer' object has no attribute 'storageType'' -
trying to
restart agent
cause you were using ovirt-hosted-engine-ha agent
1.3.0-0.0.master.20150819082341 which was unfortunately bugged on
that. ovirt-hosted-engine-ha agent 1.3.0-0.0.master.20150909150500 you are
using now should be OK.
5:17 p.m.
This is a multi-part message in MIME format.
--------------030906090702060101020208
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
On 15-9-2015 15:26, Simone Tiraboschi wrote:
Yes, from the log of setup and forgot to chmod to 36:36
Trying a yum upgrade barfs with a depency problem trying to update 2 or
3 ovirt related packages, one of them ovirt-hosted-engine, that was
yesterday, maybe today it will work. I'll check
So what is the sanlock error about?
Joop
--------------030906090702060101020208
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 15-9-2015 15:26, Simone Tiraboschi
wrote:<br>
</div>
<blockquote
cite="mid:CAN8-ONpcX9oVC0m4ZrjnbLUM0QeH-3UgLdmEUKadeHQFM3hKfg@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Tue, Sep 15, 2015 at 2:31 PM, noc
<span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:noc@nieuwland.nl"
target="_blank">noc(a)nieuwland.nl</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span
class="">
<div>On 15-9-2015 9:31, Simone Tiraboschi wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr"><br>
<div class="gmail_extra">
<div class="gmail_quote"><br>
<div>Yes, can you please attach the logs
from /var/log/ovirt-hosted-engine-ha/ ?</div>
</div>
</div>
</div>
</blockquote>
</span> Attached.</div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<div>Now the agent correctly reads the VM configuration file
from the shared domain so no issue there but then it exits
cause VDSM is not up:</div>
<div><br>
</div>
<div>
<div>MainThread::INFO::2015-09-14
20:48:58,007::hosted_engine::667::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Reloading vm.conf from the shared storage domain</div>
<div>MainThread::INFO::2015-09-14
20:48:58,163::hosted_engine::460::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 3400)</div>
<div>MainThread::INFO::2015-09-14
20:49:08,201::hosted_engine::610::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
Initializing VDSM</div>
<div>MainThread::INFO::2015-09-14
20:49:08,317::hosted_engine::655::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Connecting the storage</div>
<div>MainThread::INFO::2015-09-14
20:49:08,318::storage_server::110::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Connecting storage server</div>
<div>MainThread::INFO::2015-09-14
20:49:08,318::storage_server::135::ovirt_hosted_engine_ha.lib.storage_server.StorageServer::(connect_storage_server)
Connecting storage server</div>
<div>MainThread::INFO::2015-09-14
20:49:08,349::hosted_engine::659::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Preparing images</div>
<div>MainThread::INFO::2015-09-14
20:49:08,349::image::61::ovirt_hosted_engine_ha.lib.image.Image::(prepare_images)
Preparing images</div>
<div>MainThread::INFO::2015-09-14
20:49:08,787::hosted_engine::667::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_storage_images)
Reloading vm.conf from the shared storage domain</div>
<div>MainThread::INFO::2015-09-14
20:49:08,941::hosted_engine::460::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Current state EngineUp (score: 3400)</div>
<div>MainThread::INFO::2015-09-14
20:49:15,104::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
Agent shutting down</div>
<div>MainThread::INFO::2015-09-14
21:06:07,601::agent::78::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
ovirt-hosted-engine-ha agent
1.3.0-0.0.master.20150909150500 started</div>
<div>MainThread::INFO::2015-09-14
21:06:07,632::hosted_engine::246::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_get_hostname)
Found certificate common name: <a
moz-do-not-send="true"
href="http://lichthuis.puzzle-it.nu">lichthuis.puzzle-it.nu</a></div>
<div>MainThread::INFO::2015-09-14
21:06:07,633::hosted_engine::610::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(_initialize_vdsm)
Initializing VDSM</div>
<div>MainThread::ERROR::2015-09-14
21:06:18,209::hosted_engine::418::ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine::(start_monitoring)
Service vdsmd is not running and the admin is
responsible for starting it. Shutting down.</div>
<div>MainThread::INFO::2015-09-14
21:06:18,210::agent::143::ovirt_hosted_engine_ha.agent.agent.Agent::(run)
Agent shutting down</div>
</div>
<div><br>
</div>
<div>Martin, any hint there?</div>
<div><br>
</div>
<div>Previously you had an issue there:</div>
<div>MainThread::ERROR::2015-09-14
17:34:19,972::agent::201::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
Error: '[Errno 13] Permission denied:
'/var/run/ovirt-hosted-engine-ha/vm.conf'' - trying to
restart agent<br>
</div>
<div><br>
</div>
<div>Did you manually created
/var/run/ovirt-hosted-engine-ha/vm.conf being readable
only by root or something similar? ovirt-ha-agent is
running as vdsm user and it should be able to refresh
/var/run/ovirt-hosted-engine-ha/vm.conf with the master
copy on the shared storage.<br>
</div>
<div><br>
</div>
</div>
</div>
</div>
</blockquote>
Yes, from the log of setup and forgot to chmod to 36:36<br>
<br>
<blockquote
cite="mid:CAN8-ONpcX9oVC0m4ZrjnbLUM0QeH-3UgLdmEUKadeHQFM3hKfg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>And previously than that it was failing with:</div>
<div>MainThread::ERROR::2015-09-12
13:53:23,727::agent::201::ovirt_hosted_engine_ha.agent.agent.Agent::(_run_agent)
Error: ''StorageServer' object has no attribute
'storageType'' - trying to restart agent<br>
</div>
<div>cause you were using ovirt-hosted-engine-ha agent
1.3.0-0.0.master.20150819082341 which was unfortunately
bugged on that. ovirt-hosted-engine-ha agent
1.3.0-0.0.master.20150909150500 you are using now should
be OK.</div>
<div><br>
</div>
<div><br>
</div>
<br>
</div>
</div>
</div>
</blockquote>
Trying a yum upgrade barfs with a depency problem trying to update 2
or 3 ovirt related packages, one of them ovirt-hosted-engine, that
was yesterday, maybe today it will work. I'll check<br>
<br>
So what is the sanlock error about?<br>
<br>
Joop<br>
<br>
</body>
</html>
--------------030906090702060101020208--
9:53 p.m.
This is a multi-part message in MIME format.
--------------010406050103010507060103
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
On 15-9-2015 16:17, noc wrote:
So what is the sanlock error about?
Upgrade to the latest version of 3.6RC and
still having the sanlock error.
Cleaned up the install by cleaning the nfs shares. Reinstalling goes OK,
importing the hosted_engine storage work upto the point that its NOT
attached. Attaching it gives the above mentioned sanlock error.
Will start it later on and move logs to my main machine and I'm able to
conduct more tests tomorrow (20-9, GMT+2).
Regards,
Joop
--------------010406050103010507060103
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 7bit
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 15-9-2015 16:17, noc wrote:<br>
</div>
<blockquote cite="mid:55F82865.20305@nieuwland.nl"
type="cite">
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
<br>
<br>
So what is the sanlock error about?<br>
</blockquote>
Upgrade to the latest version of 3.6RC and still having the sanlock
error.<br>
Cleaned up the install by cleaning the nfs shares. Reinstalling goes
OK, importing the hosted_engine storage work upto the point that its
NOT attached. Attaching it gives the above mentioned sanlock error.<br>
Will start it later on and move logs to my main machine and I'm able
to conduct more tests tomorrow (20-9, GMT+2).<br>
<br>
Regards,<br>
<br>
Joop<br>
<br>
</body>
</html>
--------------010406050103010507060103--
4:53 p.m.
On Mon, Sep 14, 2015 at 8:45 AM, noc <noc(a)nieuwland.nl> wrote:
Next morning the hosted-engine didn't start. Running 'hosted-engine
--vm-status' tells me I need to deploy first.
This one is indeed an issue: it shouldn't complain there or at least it
should complain that way.
I'm opening a bug on that:
https://bugzilla.redhat.com/1263295
3481
days inactive
3496
days old
10 comments
4 participants
participants (4)
-
Joop -
noc -
Sandro Bonazzola -
Simone Tiraboschi