[ovirt-users] Re: cannot activate host

Friday, 10 August 2018

Hi,

On 08/09/2018 11:25 PM, Raz Tamir wrote:
...
 You can try using vdsm-client to check if there are running tasks on
the
 host using this command:

     vdsm-client Host getAllTasksInfo 
thanks for support, I tried this on affected host

[root@blade01 ~]# vdsm-client Host getAllTasksInfo
vdsm-client: Command Host.getAllTasksInfo with args {} failed:
(code=654, message=Not SPM: ())

so I moved into SPM host (blade03 at this time)

[root@blade03 ~]# vdsm-client Host getAllTasksInfo
{}

I tried also

[root@blade01 ~]# vdsm-client Host getAllTasks
{}
[root@blade01 ~]# vdsm-client Host getJobs
{}

[root@blade03 ~]# vdsm-client Host getAllTasks
{}
[root@blade03 ~]# vdsm-client Host getJobs
{}

It looks like there are no tasks/jobs running on hosts. It looks like
something is stucked in engine db.

At the beginning of this issue, there were some race condition where
blade01 had two running tasks/jobs which cannot be finished (network
configuration and activating of host). I deleted this jobs manually in
db via

SELECT * FROM job ORDER BY start_time DESC;
SELECT DeleteJob('...job_id...');
SELECT DeleteJob('...job_id...');

Maybe it was not wise... but what can I do now to solve this? :-)

Cheers, Jiri

...

 If there are running tasks, you can stop them:

     vdsm-client Task stop taskID=xxx-yyy

 If there are finished tasks, you can clear them:

     vdsm-client Task clear taskID=xxx-yyy

 On Thu, Aug 9, 2018, 16:15 Jiří Sléžka <jiri.slezka(a)slu.cz
 <mailto:jiri.slezka@slu.cz>> wrote:

     Hello,

     still no luck with solving this issue.

     I cannot even remove this host.

     engine log is now spammed with this messages

     2018-08-09 15:03:08,410+02 INFO
     [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
     (EE-ManagedThreadFactory-engine-Thread-1057)
     [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock and wait
     lock

'HostEngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS_INIT]',
     sharedLocks=''}'
     2018-08-09 15:03:08,446+02 INFO
     [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
     (EE-ManagedThreadFactory-engine-Thread-1057)
     [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock and wait
     lock

'HostEngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS_INIT]',
     sharedLocks=''}'
     2018-08-09 15:03:08,451+02 INFO
     [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
     (EE-ManagedThreadFactory-engine-Thread-1057)
     [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock and wait
     lock

'HostEngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS_INIT]',
     sharedLocks=''}'
     2018-08-09 15:03:08,486+02 INFO
     [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
     (EE-ManagedThreadFactory-engine-Thread-1057)
     [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock and wait
     lock

'HostEngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS_INIT]',
     sharedLocks=''}'

     btw. oVirt 4.2.5.2-1.el7/CentOS7

     Any tips how to resolve this and at least remove this host from ovirt?

     Cheers, Jiri

     On 08/06/2018 12:52 PM, Gobinda Das wrote:
     > Can you please post vdsm log which is inside /var/log/vdsm/vdsm.log ?
     >
     > On Mon, Aug 6, 2018 at 3:51 PM, Jiří Sléžka <jiri.slezka(a)slu.cz
     <mailto:jiri.slezka@slu.cz>
     > <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>> wrote:
     >
     >     Hi,
     >
     >     no one can help?
     >
     >     I still cannot activate this host - error is "Cannot activate
     Host.
     >     Related operation is currently in progress. Please try again
     later."
     >
     >     I believe relevant log entrieas are
     >
     >     2018-08-06 12:15:50,398+02 INFO
     >     [org.ovirt.engine.core.bll.ActivateVdsCommand] (default task-426)
     >     [9387077a-8276-4a3f-a087-584a10a09b08] Failed to Acquire Lock
     to object
     >   

 'EngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS]',
     >     sharedLocks=''}'
     >     2018-08-06 12:15:50,398+02 WARN
     >     [org.ovirt.engine.core.bll.ActivateVdsCommand] (default task-426)
     >     [9387077a-8276-4a3f-a087-584a10a09b08] Validation of action
     >     'ActivateVds' failed for user ***my_username***. Reasons:
     >   
      VAR__ACTION__ACTIVATE,VAR__TYPE__HOST,ACTION_TYPE_FAILED_OBJECT_LOCKED
     >
     >
     >     when I tried to "reinstall host" I got entry in Events
"Failed
     to update
     >     Host blade01" and correlation id. Relevant log entries are
     >
     >
     >     cat /var/log/ovirt-engine/engine.log | grep
     >     d2b332da-9050-419c-8188-8fdda5e9d807
     >
     >     2018-08-06 12:08:57,451+02 INFO
     >     [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] (default
     >     task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] Running command:
     >     InstallVdsCommand internal: false. Entities affected :  ID:
     >     786646cd-c9ef-49a8-8aea-56e858dcf202 Type: VDSAction group
     >     EDIT_HOST_CONFIGURATION with role type ADMIN
     >     2018-08-06 12:08:57,476+02 INFO
     >     [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand]
     >     (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807]
     Failed to
     >     Acquire Lock to object
     >   

 'EngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea-56e858dcf202=VDS]',
     >     sharedLocks=''}'
     >     2018-08-06 12:08:57,476+02 WARN
     >     [org.ovirt.engine.core.bll.hostdeploy.InstallVdsInternalCommand]
     >     (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807]
     Validation of
     >     action 'InstallVdsInternal' failed for user ***my_username***.
     Reasons:
     >     ACTION_TYPE_FAILED_OBJECT_LOCKED
     >     2018-08-06 12:08:57,479+02 ERROR
     >     [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] (default
     >     task-425) [d2b332da-9050-419c-8188-8fdda5e9d807]
     Installation/upgrade of
     >     Host '786646cd-c9ef-49a8-8aea-56e858dcf202', 'blade01'
failed:
     Cannot
     >     ${action} ${type}. Related operation is currently in progress.
     Please
     >     try again later.
     >     2018-08-06 12:08:57,482+02 INFO
     >     [org.ovirt.engine.core.bll.CommandCompensator] (default task-425)
     >     [d2b332da-9050-419c-8188-8fdda5e9d807] Command
     >     [id=b74adfbb-c043-4fa9-a163-0a6ea7fba9dc]: Compensating
     >     DELETED_OR_UPDATED_ENTITY of
     >     org.ovirt.engine.core.common.businessentities.VdsStatic; snapshot:
     >     id=786646cd-c9ef-49a8-8aea-56e858dcf202.
     >     2018-08-06 12:08:57,486+02 ERROR
     >     [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] (default
     >     task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] Transaction
     rolled-back
     >     for command
     'org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand'.
     >     2018-08-06 12:08:57,491+02 ERROR
     >   
      [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
     >     (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807]
     EVENT_ID:
     >     USER_FAILED_UPDATE_VDS(105), Failed to update Host blade01 (User:
     >     ***my_username***).
     >
     >     How can I unlock this entries?
     >
     >     Thanks in advance,
     >
     >     Jiri Slezka
     >
     >
     >
     >     On 08/03/2018 02:07 PM, Jiří Sléžka wrote:
     >     > Hi,
     >     >
     >     > I have one interesting issue. I was asked by colleague to
     repair his
     >     > cluster which is part of our oVirt installation. He tried to
     >     upgrade his
     >     > host to gain 4.2 compatibility level but it fails in some
     cases. Never
     >     > mind, I checked it, repaired, installed upgrades and
     activate his
     >     hosts
     >     > with one exception.
     >     >
     >     > One of hosts was stucked in finishing two tasks. One was
     changing
     >     > network configuration, the second was activating of this host. I
     >     have to
     >     > manually clear this tasks in database by
     >     >
     >     > SELECT * FROM job ORDER BY start_time DESC;
     >     > SELECT DeleteJob('...job_id...');
     >     > SELECT DeleteJob('...job_id...');
     >     >
     >     > Now this tasks are gone but still cannot activate this host.
     >     >
     >     >
     >     > Error while executing action:
     >     >
     >     > blade01:
     >     >
     >     >     Cannot activate Host. Related operation is currently in
     progress.
     >     > Please try again later.
     >     >
     >     > Host is updatet (latest 4.2 repo), rebooted, looks fine...
     >     >
     >     > In tasks in UI are no tasks found (the same in db)...
     >     >
     >     > What can I do now?
     >     >
     >     > Cheers, Jiri
     >     >
     >     >
     >     >
     >     >
     >     > _______________________________________________
     >     > Users mailing list -- users(a)ovirt.org
     <mailto:users@ovirt.org> <mailto:users@ovirt.org
     <mailto:users@ovirt.org>>
     >     > To unsubscribe send an email to users-leave(a)ovirt.org
     <mailto:users-leave@ovirt.org>
     >     <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>
     >     > Privacy Statement: https://www.ovirt.org/site/privacy-policy/
     >     <https://www.ovirt.org/site/privacy-policy/>
     >     > oVirt Code of Conduct:
     >     https://www.ovirt.org/community/about/community-guidelines/
     >     <https://www.ovirt.org/community/about/community-guidelines/>
     >     > List Archives:
     >   

 https://lists.ovirt.org/archives/list/users@ovirt.org/message/C2TWKXGOEESXCDJUZ7IBKYDE2APFN4ZB/
     >   

 <https://lists.ovirt.org/archives/list/users@ovirt.org/message/C2TWKXGOEESXCDJUZ7IBKYDE2APFN4ZB/>
     >     >
     >
     >
     >
     >     _______________________________________________
     >     Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org>
     <mailto:users@ovirt.org <mailto:users@ovirt.org>>
     >     To unsubscribe send an email to users-leave(a)ovirt.org
     <mailto:users-leave@ovirt.org>
     >     <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>
     >     Privacy Statement: https://www.ovirt.org/site/privacy-policy/
     >     <https://www.ovirt.org/site/privacy-policy/>
     >     oVirt Code of Conduct:
     >     https://www.ovirt.org/community/about/community-guidelines/
     >     <https://www.ovirt.org/community/about/community-guidelines/>
     >     List Archives:
     >   

 https://lists.ovirt.org/archives/list/users@ovirt.org/message/6H4QYVTTV37IEIA2MTABJ67P43LQ5AGI/
     >   

 <https://lists.ovirt.org/archives/list/users@ovirt.org/message/6H4QYVTTV37IEIA2MTABJ67P43LQ5AGI/>
     >
     >
     >
     >
     > --
     > Thanks,
     > Gobinda

     _______________________________________________
     Users mailing list -- users(a)ovirt.org <mailto:users@ovirt.org>
     To unsubscribe send an email to users-leave(a)ovirt.org
     <mailto:users-leave@ovirt.org>
     Privacy Statement: https://www.ovirt.org/site/privacy-policy/
     oVirt Code of Conduct:
     https://www.ovirt.org/community/about/community-guidelines/
     List Archives:

https://lists.ovirt.org/archives/list/users@ovirt.org/message/BP7SG5BQKZI...

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

[ovirt-users] Re: cannot activate host