[ovirt-users] Re: SPM and Task error ...

26 Jul 2019


      Hi,
Il 26/07/19 12:58, Benny Zlotnik ha scritto:
...
taskcleaner.sh only clears tasks form the engine database.
did you check your engine logs to see if this task is running?
vdsm-client from node (hypervisor):
[root@infn-vm05 ~]# vdsm-client Task  getStatus 
taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e
{
     "message": "running job 1 of 1",
     "code": 0,
     "taskID": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e",
     "taskResult": "",
     "taskState": "running"
}
[root@infn-vm05 ~]#
[root@infn-vm05 ~]# vdsm-client Task  getInfo 
taskID=fdcf4d1b-82fe-49a6-b233-323ebe568f8e
{
     "verb": "prepareMerge",
     "id": "fdcf4d1b-82fe-49a6-b233-323ebe568f8e"
}
...
it's a task that is executed during a snapshot merge (removal of a 
snapshot), do you have any running snapshot removals?
How Can I check it ?
Thanks
Enrico
if not you can stop and clear the task using vdsm-client
On Fri, Jul 26, 2019 at 1:31 PM Enrico <enrico.becchetti@pg.infn.it 
<mailto:enrico.becchetti@pg.infn.it>> wrote:
Dear all,
I try this:
# /usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh -v -t
    fdcf4d1b-82fe-49a6-b233-323ebe568f8e
    select exists (select * from information_schema.tables where
    table_schema = 'public' and table_name = 'command_entities');
     t
     This will remove the given Task!!!
    Caution, this operation should be used with care. Please contact
    support prior to running this command
    Are you sure you want to proceed? [y/n]
    y
    SELECT Deleteasync_tasks('fdcf4d1b-82fe-49a6-b233-323ebe568f8e');
                     0
# /usr/share/ovirt-engine/setup/dbutils/taskcleaner.sh -v -R
    select exists (select * from information_schema.tables where
    table_schema = 'public' and table_name = 'command_entities');
     t
     This will remove all async_tasks table content!!!
    Caution, this operation should be used with care. Please contact
    support prior to running this command
    Are you sure you want to proceed? [y/n]
    y
    TRUNCATE TABLE async_tasks cascade;
    TRUNCATE TABLE
but after these commands I see the same messages inside engine.log:
2019-07-26 12:25:19,727+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] Task id
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling
    period time and should be polled. Pre-polling period is 60000 millis.
    2019-07-26 12:25:19,727+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] Task id
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling
    period time and should be polled. Pre-polling period is 60000 millis.
    2019-07-26 12:25:19,779+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] Task id
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling
    period time and should be polled. Pre-polling period is 60000 millis.
    2019-07-26 12:25:19,779+02 ERROR
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) []
    BaseAsyncTask::logEndTaskFailure: Task
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' (Parent Command 'Unknown',
    Parameters Type
    'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters')
    ended with failure:
    2019-07-26 12:25:19,779+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) []
    SPMAsyncTask::ClearAsyncTask: Attempting to clear task
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e'
    2019-07-26 12:25:19,780+02 INFO
    [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] START,
    SPMClearTaskVDSCommand(
    SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
    ignoreFailoverLimit='false',
    taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 753de6fe
    2019-07-26 12:25:19,781+02 INFO
    [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] START,
    HSMClearTaskVDSCommand(HostName = infn-vm05.management,
    HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc',
    taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: e5dc020
    2019-07-26 12:25:19,786+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) []
    SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was
    'TaskStateError' and message was 'Operation is not allowed in this
    task state: ("can't clean in state running",)'. Task will not be
    cleaned
    2019-07-26 12:25:19,786+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) []
    BaseAsyncTask::onTaskEndSuccess: Task
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' (Parent Command 'Unknown',
    Parameters Type
    'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters')
    ended successfully.
    2019-07-26 12:25:19,786+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) []
    SPMAsyncTask::ClearAsyncTask: Attempting to clear task
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e'
    2019-07-26 12:25:19,787+02 INFO
    [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] START,
    SPMClearTaskVDSCommand(
    SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
    ignoreFailoverLimit='false',
    taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 40383d58
    2019-07-26 12:25:19,788+02 INFO
    [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] START,
    HSMClearTaskVDSCommand(HostName = infn-vm05.management,
    HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc',
    taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 758d79b
    2019-07-26 12:25:19,793+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) []
    SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was
    'TaskStateError' and message was 'Operation is not allowed in this
    task state: ("can't clean in state running",)'. Task will not be
    cleaned
    2019-07-26 12:25:19,793+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] Task id
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' has passed pre-polling
    period time and should be polled. Pre-polling period is 60000 millis.
    2019-07-26 12:25:19,793+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) []
    SPMAsyncTask::ClearAsyncTask: Attempting to clear task
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e'
    2019-07-26 12:25:19,794+02 INFO
    [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] START,
    SPMClearTaskVDSCommand(
    SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='18d57688-6ed4-43b8-bd7c-0665b55950b7',
    ignoreFailoverLimit='false',
    taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 1dc8cb98
    2019-07-26 12:25:19,795+02 INFO
    [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) [] START,
    HSMClearTaskVDSCommand(HostName = infn-vm05.management,
    HSMTaskGuidBaseVDSCommandParameters:{hostId='751f3e99-b95e-4c31-bc38-77f5661a0bdc',
    taskId='fdcf4d1b-82fe-49a6-b233-323ebe568f8e'}), log id: 7df444c1
    2019-07-26 12:25:19,800+02 INFO
    [org.ovirt.engine.core.bll.tasks.SPMAsyncTask]
    (EE-ManagedThreadFactory-engineScheduled-Thread-77) []
    SPMAsyncTask::ClearAsyncTask: At time of attempt to clear task
    'fdcf4d1b-82fe-49a6-b233-323ebe568f8e' the response code was
    'TaskStateError' and message was 'Operation is not allowed in this
    task state: ("can't clean in state running",)'. Task will not be
    cleaned
it's seems to be task id fdcf4d1b-82fe-49a6-b233-323ebe568f8e  is
    still running.
Any other ideas ?
Thank you so much.
    Best Regards
    Enrico
Il 26/07/19 09:39, Strahil Nikolov ha scritto:
...
Have you tried to remove your task via the taskcleaner.sh
    You can find some details on
    https://www.ovirt.org/develop/developer-guide/db-issues/helperutilities.html
I should admit , that I have never used that.
Best Regards,
    Strahil Nikolov
В петък, 26 юли 2019 г., 09:03:10 ч. Гринуич+3, Enrico
    <enrico.becchetti@pg.infn.it>
    <mailto:enrico.becchetti@pg.infn.it> написа:
Il 25/07/19 22:27, Strahil Nikolov ha scritto:
    > Hi all,
    >my ovirt cluster has got 3 Hypervisors runnig Centos 7.5.1804
    vdsm is 4.20.39.1-1.el7,
    >ovirt engine is 4.2.4.5-1.el7, the storage systems are HP MSA
    P2000 and 2050 (fibre channel).
>I need to stop one of the hypervisors for maintenance but this
    system is the storage pool manager.
>For this reason I decided to manually activate SPM in one of the
    other nodes but this operation is not
    successful.
Why would you do that ?
    Just put the host in maintenance and the system will evacuate all
    VMs and another host will be promoted as SPM .
    Once the host is in maintenance - you can proceed with your tasks.
P.S: Don't forget to remove the mainteance mode once you are over.
Best Regards,
    Strahil Nikolov
    Hi Strahil,
    I did not tell you that this is the first thing I tried but it
    didn't work because SPM can't be activated in another host,
    then I tried to manually move SPM without success.
    Any ideas ?
    Thanks a lot !
Best Regards
Enrico
-- 
    _______________________________________________________________________
Enrico Becchetti                    Servizio di Calcolo e Reti
Istituto Nazionale di Fisica Nucleare - Sezione di Perugia
    Via Pascoli,c/o Dipartimento di Fisica  06123 Perugia (ITALY)
    Phone:+39 075 5852777             Mail: Enrico.Becchetti<at>pg.infn.it  <http://pg.infn.it>  
    _______________________________________________________________________
-- 
    _______________________________________________________________________
Enrico Becchetti                    Servizio di Calcolo e Reti
Istituto Nazionale di Fisica Nucleare - Sezione di Perugia
    Via Pascoli,c/o Dipartimento di Fisica  06123 Perugia (ITALY)
    Phone:+39 075 5852777             Mail: Enrico.Becchetti<at>pg.infn.it  <http://pg.infn.it>  
    _______________________________________________________________________
_______________________________________________
    Users mailing list -- users@ovirt.org <mailto:users@ovirt.org>
    To unsubscribe send an email to users-leave@ovirt.org
    <mailto:users-leave@ovirt.org>
    Privacy Statement: https://www.ovirt.org/site/privacy-policy/
    oVirt Code of Conduct:
    https://www.ovirt.org/community/about/community-guidelines/
    List Archives:
    https://lists.ovirt.org/archives/list/users@ovirt.org/message/VQJCGFXAA37OXW...
-- 
_______________________________________________________________________

Enrico Becchetti                    Servizio di Calcolo e Reti

Istituto Nazionale di Fisica Nucleare - Sezione di Perugia
Via Pascoli,c/o Dipartimento di Fisica  06123 Perugia (ITALY)
Phone:+39 075 5852777             Mail: Enrico.Becchetti<at>pg.infn.it
_______________________________________________________________________