Hi,
On 08/09/2018 11:25 PM, Raz Tamir wrote:
> You can try using vdsm-client to check if there are running tasks on the
> host using this command:
>
> vdsm-client Host getAllTasksInfo
thanks for support, I tried this on affected host
[root@blade01 ~]# vdsm-client Host getAllTasksInfo
vdsm-client: Command Host.getAllTasksInfo with args {} failed:
(code=654, message=Not SPM: ())
so I moved into SPM host (blade03 at this time)
[root@blade03 ~]# vdsm-client Host getAllTasksInfo
{}
I tried also
[root@blade01 ~]# vdsm-client Host getAllTasks
{}
[root@blade01 ~]# vdsm-client Host getJobs
{}
[root@blade03 ~]# vdsm-client Host getAllTasks
{}
[root@blade03 ~]# vdsm-client Host getJobs
{}
It looks like there are no tasks/jobs running on hosts. It looks like
something is stucked in engine db.
At the beginning of this issue, there were some race condition where
blade01 had two running tasks/jobs which cannot be finished (network
configuration and activating of host). I deleted this jobs manually in
db via
SELECT * FROM job ORDER BY start_time DESC;
SELECT DeleteJob('...job_id...');
SELECT DeleteJob('...job_id...');
Maybe it was not wise... but what can I do now to solve this? :-)
Cheers, Jiri
>
> If there are running tasks, you can stop them:
>
> vdsm-client Task stop taskID=xxx-yyy
>
> If there are finished tasks, you can clear them:
>
> vdsm-client Task clear taskID=xxx-yyy
>
>
> On Thu, Aug 9, 2018, 16:15 Jiří Sléžka <jiri.slezka@slu.cz
> <mailto:jiri.slezka@slu.cz>> wrote:
>
> Hello,
>
> still no luck with solving this issue.
>
> I cannot even remove this host.
>
> engine log is now spammed with this messages
>
> 2018-08-09 15:03:08,410+02 INFO
> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (EE-ManagedThreadFactory-engine-Thread-1057)
> [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock and wait
> lock
> 'HostEngineLock:{exclusiveLocks='[786646cd- c9ef-49a8-8aea-56e858dcf202= VDS_INIT]',
> sharedLocks=''}'
> 2018-08-09 15:03:08,446+02 INFO
> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (EE-ManagedThreadFactory-engine-Thread-1057)
> [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock and wait
> lock
> 'HostEngineLock:{exclusiveLocks='[786646cd- c9ef-49a8-8aea-56e858dcf202= VDS_INIT]',
> sharedLocks=''}'
> 2018-08-09 15:03:08,451+02 INFO
> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (EE-ManagedThreadFactory-engine-Thread-1057)
> [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock and wait
> lock
> 'HostEngineLock:{exclusiveLocks='[786646cd- c9ef-49a8-8aea-56e858dcf202= VDS_INIT]',
> sharedLocks=''}'
> 2018-08-09 15:03:08,486+02 INFO
> [org.ovirt.engine.core.bll.lock.InMemoryLockManager]
> (EE-ManagedThreadFactory-engine-Thread-1057)
> [cb8aa091-70ce-419a-b45b-3ffad6f2529b] Failed to acquire lock and wait
> lock
> 'HostEngineLock:{exclusiveLocks='[786646cd- c9ef-49a8-8aea-56e858dcf202= VDS_INIT]',
> sharedLocks=''}'
>
> btw. oVirt 4.2.5.2-1.el7/CentOS7
>
> Any tips how to resolve this and at least remove this host from ovirt?
>
> Cheers, Jiri
>
>
> On 08/06/2018 12:52 PM, Gobinda Das wrote:
> > Can you please post vdsm log which is inside /var/log/vdsm/vdsm.log ?
> >
> > On Mon, Aug 6, 2018 at 3:51 PM, Jiří Sléžka <jiri.slezka@slu.cz
> <mailto:jiri.slezka@slu.cz>
> <mailto:users@ovirt.org> <mailto:users@ovirt.org> > <mailto:jiri.slezka@slu.cz <mailto:jiri.slezka@slu.cz>>> wrote:
> >
> > Hi,
> >
> > no one can help?
> >
> > I still cannot activate this host - error is "Cannot activate
> Host.
> > Related operation is currently in progress. Please try again
> later."
> >
> > I believe relevant log entrieas are
> >
> > 2018-08-06 12:15:50,398+02 INFO
> > [org.ovirt.engine.core.bll.ActivateVdsCommand] (default task-426)
> > [9387077a-8276-4a3f-a087-584a10a09b08] Failed to Acquire Lock
> to object
> >
> 'EngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea- 56e858dcf202=VDS]',
> > sharedLocks=''}'
> > 2018-08-06 12:15:50,398+02 WARN
> > [org.ovirt.engine.core.bll.ActivateVdsCommand] (default task-426)
> > [9387077a-8276-4a3f-a087-584a10a09b08] Validation of action
> > 'ActivateVds' failed for user ***my_username***. Reasons:
> >
> VAR__ACTION__ACTIVATE,VAR__TYPE__HOST,ACTION_TYPE_FAILED_ OBJECT_LOCKED
> >
> >
> > when I tried to "reinstall host" I got entry in Events "Failed
> to update
> > Host blade01" and correlation id. Relevant log entries are
> >
> >
> > cat /var/log/ovirt-engine/engine.log | grep
> > d2b332da-9050-419c-8188-8fdda5e9d807
> >
> > 2018-08-06 12:08:57,451+02 INFO
> > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] (default
> > task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] Running command:
> > InstallVdsCommand internal: false. Entities affected : ID:
> > 786646cd-c9ef-49a8-8aea-56e858dcf202 Type: VDSAction group
> > EDIT_HOST_CONFIGURATION with role type ADMIN
> > 2018-08-06 12:08:57,476+02 INFO
> > [org.ovirt.engine.core.bll.hostdeploy. InstallVdsInternalCommand]
> > (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807]
> Failed to
> > Acquire Lock to object
> >
> 'EngineLock:{exclusiveLocks='[786646cd-c9ef-49a8-8aea- 56e858dcf202=VDS]',
> > sharedLocks=''}'
> > 2018-08-06 12:08:57,476+02 WARN
> > [org.ovirt.engine.core.bll.hostdeploy. InstallVdsInternalCommand]
> > (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807]
> Validation of
> > action 'InstallVdsInternal' failed for user ***my_username***.
> Reasons:
> > ACTION_TYPE_FAILED_OBJECT_LOCKED
> > 2018-08-06 12:08:57,479+02 ERROR
> > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] (default
> > task-425) [d2b332da-9050-419c-8188-8fdda5e9d807]
> Installation/upgrade of
> > Host '786646cd-c9ef-49a8-8aea-56e858dcf202', 'blade01' failed:
> Cannot
> > ${action} ${type}. Related operation is currently in progress.
> Please
> > try again later.
> > 2018-08-06 12:08:57,482+02 INFO
> > [org.ovirt.engine.core.bll.CommandCompensator] (default task-425)
> > [d2b332da-9050-419c-8188-8fdda5e9d807] Command
> > [id=b74adfbb-c043-4fa9-a163-0a6ea7fba9dc]: Compensating
> > DELETED_OR_UPDATED_ENTITY of
> > org.ovirt.engine.core.common.businessentities.VdsStatic; snapshot:
> > id=786646cd-c9ef-49a8-8aea-56e858dcf202.
> > 2018-08-06 12:08:57,486+02 ERROR
> > [org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand] (default
> > task-425) [d2b332da-9050-419c-8188-8fdda5e9d807] Transaction
> rolled-back
> > for command
> 'org.ovirt.engine.core.bll.hostdeploy.InstallVdsCommand'.
> > 2018-08-06 12:08:57,491+02 ERROR
> >
> [org.ovirt.engine.core.dal.dbbroker.auditloghandling. AuditLogDirector]
> > (default task-425) [d2b332da-9050-419c-8188-8fdda5e9d807]
> EVENT_ID:
> > USER_FAILED_UPDATE_VDS(105), Failed to update Host blade01 (User:
> > ***my_username***).
> >
> > How can I unlock this entries?
> >
> > Thanks in advance,
> >
> > Jiri Slezka
> >
> >
> >
> > On 08/03/2018 02:07 PM, Jiří Sléžka wrote:
> > > Hi,
> > >
> > > I have one interesting issue. I was asked by colleague to
> repair his
> > > cluster which is part of our oVirt installation. He tried to
> > upgrade his
> > > host to gain 4.2 compatibility level but it fails in some
> cases. Never
> > > mind, I checked it, repaired, installed upgrades and
> activate his
> > hosts
> > > with one exception.
> > >
> > > One of hosts was stucked in finishing two tasks. One was
> changing
> > > network configuration, the second was activating of this host. I
> > have to
> > > manually clear this tasks in database by
> > >
> > > SELECT * FROM job ORDER BY start_time DESC;
> > > SELECT DeleteJob('...job_id...');
> > > SELECT DeleteJob('...job_id...');
> > >
> > > Now this tasks are gone but still cannot activate this host.
> > >
> > >
> > > Error while executing action:
> > >
> > > blade01:
> > >
> > > Cannot activate Host. Related operation is currently in
> progress.
> > > Please try again later.
> > >
> > > Host is updatet (latest 4.2 repo), rebooted, looks fine...
> > >
> > > In tasks in UI are no tasks found (the same in db)...
> > >
> > > What can I do now?
> > >
> > > Cheers, Jiri
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Users mailing list -- users@ovirt.org
> <mailto:users@ovirt.org>>
> > > To unsubscribe send an email to users-leave@ovirt.org
> <mailto:users-leave@ovirt.org>
> > <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>
> > > Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> > <https://www.ovirt.org/site/privacy-policy/ >
> > > oVirt Code of Conduct:
> > https://www.ovirt.org/community/about/community- guidelines/
> > <https://www.ovirt.org/community/about/community- >guidelines/
> > > List Archives:
> >
> https://lists.ovirt.org/archives/list/users@ovirt.org/ message/ C2TWKXGOEESXCDJUZ7IBKYDE2APFN4 ZB/
> >
> <https://lists.ovirt.org/archives/list/users@ovirt.org/ >message/ C2TWKXGOEESXCDJUZ7IBKYDE2APFN4 ZB/
> > >
> >
> >
> >
> > _______________________________________________ > <mailto:users@ovirt.org <mailto:users@ovirt.org>>
> > Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>
> > To unsubscribe send an email to users-leave@ovirt.org
> <mailto:users-leave@ovirt.org>
> > <mailto:users-leave@ovirt.org <mailto:users-leave@ovirt.org>>
> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> > <https://www.ovirt.org/site/privacy-policy/ >
> > oVirt Code of Conduct:
> > https://www.ovirt.org/community/about/community- guidelines/
> > <https://www.ovirt.org/community/about/community- >guidelines/
> > List Archives:
> >
> https://lists.ovirt.org/archives/list/users@ovirt.org/ message/ 6H4QYVTTV37IEIA2MTABJ67P43LQ5A GI/
> >
> <https://lists.ovirt.org/archives/list/users@ovirt.org/ >message/ 6H4QYVTTV37IEIA2MTABJ67P43LQ5A GI/
> >
> >
> >
> >
> > --
> > Thanks,
> > Gobinda
>
>
> _______________________________________________ > Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>
> To unsubscribe send an email to users-leave@ovirt.org
> <mailto:users-leave@ovirt.org>
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
> https://www.ovirt.org/community/about/community- guidelines/
> List Archives:
> https://lists.ovirt.org/archives/list/users@ovirt.org/ message/ BP7SG5BQKZIVR75A6NSCUMGQQBXYL3 FJ/
>