6:06 a.m.
Hi Everyone,
I'm having an issue with one of my hosts. I was performing a yum
update when the box crashed and rebooted, which left a lot of packages
broken. I resolved the issue by running "yum reinstall" on all the
packages that were in the update, but since then VDSM won't start. I
can't find any error message as to why it won't start, just that it
won't. IE, I see this in /var/log/messages:
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
WARNING:ovirt_hosted_engine_ha.agent.agent.Agent:Restarting agent,
attempt '5'
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
certificate common name: ovirt-01.local.domain
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Initializing VDSM
Apr 13 07:57:09 ovirt-01 journal: ovirt-ha-agent
ovirt_hosted_engine_ha.agent.agent.Agent ERROR Service vdsmd is not
running and the admin is responsible for starting it. Waiting...
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Service vdsmd is not
running and the admin is responsible for starting it. Waiting...
And this in /var/log/vdsmd/mom.log (vdsmd.log is a 0 byte file)
2016-04-13 07:58:07,958 - mom - INFO - MOM starting
2016-04-13 07:58:07,992 - mom.HostMonitor - INFO - Host Monitor starting
2016-04-13 07:58:07,992 - mom - INFO - hypervisor interface vdsmxmlrpc
2016-04-13 07:58:08,089 - mom.vdsmInterface - ERROR - Cannot connect to
VDSM! [Errno 111] Connection refused
2016-04-13 07:58:08,090 - mom.GuestManager - INFO - Guest Manager
starting: multi-thread
2016-04-13 07:58:08,105 - mom.Policy - INFO - Loaded policy '00-defines'
2016-04-13 07:58:08,109 - mom.vdsmInterface - ERROR - Cannot connect to
VDSM! [Errno 111] Connection refused
2016-04-13 07:58:08,169 - mom.Policy - INFO - Loaded policy '02-balloon'
2016-04-13 07:58:08,263 - mom.Policy - INFO - Loaded policy '03-ksm'
2016-04-13 07:58:08,339 - mom.HostMonitor - INFO - HostMonitor is ready
2016-04-13 07:58:08,380 - mom.Policy - INFO - Loaded policy '04-cputune'
2016-04-13 07:58:08,381 - mom.PolicyEngine - INFO - Policy Engine starting
2016-04-13 07:58:08,383 - mom.RPCServer - INFO - Using unix socket
/var/run/vdsm/mom-vdsm.sock
2016-04-13 07:58:08,383 - mom.RPCServer - INFO - RPC Server starting
2016-04-13 07:58:08,884 - mom.RPCServer - INFO - RPC Server ending
Can anyone give me some ideas of how to debug this? Or even some docs
on how to properly remove the host from the manager app and re-register
the guest to see if that fixes things up.
6:47 a.m.
On Wed, Apr 13, 2016 at 1:06 PM, Charles Tassell <ctassell(a)gmail.com> wrote:
Hi Everyone,
I'm having an issue with one of my hosts. I was performing a yum update
when the box crashed and rebooted, which left a lot of packages broken. I
resolved the issue by running "yum reinstall" on all the packages that were
in the update, but since then VDSM won't start. I can't find any error
message as to why it won't start, just that it won't. IE, I see this in
/var/log/messages:
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
WARNING:ovirt_hosted_engine_ha.agent.agent.Agent:Restarting agent, attempt
'5'
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Found
certificate common name: ovirt-01.local.domain
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
INFO:ovirt_hosted_engine_ha.agent.hosted_engine.HostedEngine:Initializing
VDSM
Apr 13 07:57:09 ovirt-01 journal: ovirt-ha-agent
ovirt_hosted_engine_ha.agent.agent.Agent ERROR Service vdsmd is not running
and the admin is responsible for starting it. Waiting...
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Service vdsmd is not running
and the admin is responsible for starting it. Waiting...
And this in /var/log/vdsmd/mom.log (vdsmd.log is a 0 byte file)
2016-04-13 07:58:07,958 - mom - INFO - MOM starting
2016-04-13 07:58:07,992 - mom.HostMonitor - INFO - Host Monitor starting
2016-04-13 07:58:07,992 - mom - INFO - hypervisor interface vdsmxmlrpc
2016-04-13 07:58:08,089 - mom.vdsmInterface - ERROR - Cannot connect to
VDSM! [Errno 111] Connection refused
2016-04-13 07:58:08,090 - mom.GuestManager - INFO - Guest Manager starting:
multi-thread
2016-04-13 07:58:08,105 - mom.Policy - INFO - Loaded policy '00-defines'
2016-04-13 07:58:08,109 - mom.vdsmInterface - ERROR - Cannot connect to
VDSM! [Errno 111] Connection refused
2016-04-13 07:58:08,169 - mom.Policy - INFO - Loaded policy '02-balloon'
2016-04-13 07:58:08,263 - mom.Policy - INFO - Loaded policy '03-ksm'
2016-04-13 07:58:08,339 - mom.HostMonitor - INFO - HostMonitor is ready
2016-04-13 07:58:08,380 - mom.Policy - INFO - Loaded policy '04-cputune'
2016-04-13 07:58:08,381 - mom.PolicyEngine - INFO - Policy Engine starting
2016-04-13 07:58:08,383 - mom.RPCServer - INFO - Using unix socket
/var/run/vdsm/mom-vdsm.sock
2016-04-13 07:58:08,383 - mom.RPCServer - INFO - RPC Server starting
2016-04-13 07:58:08,884 - mom.RPCServer - INFO - RPC Server ending
Can anyone give me some ideas of how to debug this? Or even some docs on
how to properly remove the host from the manager app and re-register the
guest to see if that fixes things up.
You can run hosted-engine --deploy again and choose to redeploy using
the same host-id: it will reconfigure VDSM for you.
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
7:25 a.m.
Hi Simon,
I tried that, but the same problem: vdsm won't start, so the deploy
can't continue. There is some sort of bug with running VDSMd, but I
can't find any error logging to say what that bug is so I'm at a loss as
to how to fix it.
On 16-04-13 08:47 AM, Simone Tiraboschi wrote:
[snip] You can run hosted-engine --deploy again and choose to
redeploy
using the same host-id: it will reconfigure VDSM for you.
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
7:46 a.m.
Hi Charles,
can you please make sure you are not in local maintenance mode and the
vdsmd systemd service is enabled?
The line
Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Service vdsmd is not running and the admin
is responsible for starting it. Waiting...
seems to indicate that hosted engine agent refused to start vdsm on
its own and that usually happens when the host is in maintenance.
Regards
--
Martin Sivak
SLA / oVirt
On Wed, Apr 13, 2016 at 2:25 PM, Charles Tassell <ctassell(a)gmail.com> wrote:
Hi Simon,
I tried that, but the same problem: vdsm won't start, so the deploy can't
continue. There is some sort of bug with running VDSMd, but I can't find
any error logging to say what that bug is so I'm at a loss as to how to fix
it.
On 16-04-13 08:47 AM, Simone Tiraboschi wrote:
>
> [snip] You can run hosted-engine --deploy again and choose to redeploy
> using the same host-id: it will reconfigure VDSM for you.
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
8:28 a.m.
Hi Guys,
Thanks, I found the problem. I had edited the /etc/vdsm logger
config files to cut down on all the debug logging and the process failed
(silently in any system log I could find) due to a parsing error.
Restored the logger files to defaults and it worked.
For anyone going through the list logs in the future, try running:
sudo -s /bin/bash vdsm
python /usr/share/vdsm/vdsm
to get an error message.
On 16-04-13 09:46 AM, Martin Sivak wrote:
Hi Charles,
can you please make sure you are not in local maintenance mode and the
vdsmd systemd service is enabled?
The line
> Apr 13 07:57:09 ovirt-01 ovirt-ha-agent:
ERROR:ovirt_hosted_engine_ha.agent.agent.Agent:Service vdsmd is not running and the admin
is responsible for starting it. Waiting...
seems to indicate that hosted engine agent refused to start vdsm on
its own and that usually happens when the host is in maintenance.
Regards
--
Martin Sivak
SLA / oVirt
On Wed, Apr 13, 2016 at 2:25 PM, Charles Tassell <ctassell(a)gmail.com> wrote:
> Hi Simon,
>
> I tried that, but the same problem: vdsm won't start, so the deploy can't
> continue. There is some sort of bug with running VDSMd, but I can't find
> any error logging to say what that bug is so I'm at a loss as to how to fix
> it.
>
> On 16-04-13 08:47 AM, Simone Tiraboschi wrote:
>> [snip] You can run hosted-engine --deploy again and choose to redeploy
>> using the same host-id: it will reconfigure VDSM for you.
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
3238
days inactive
3238
days old
4 comments
3 participants
participants (3)
-
Charles Tassell -
Martin Sivak -
Simone Tiraboschi