Hello, I've been struggling with TaskID/FlowID when talking to the VDSM API. I am trying to write a tool that uses the vdsm-api to facilitate the troubleshooting of image issues (snapshots). This tool does a series of API calls, but I cannot find a nice way to track the taskID and clear the completed tasks after completion of the tool commands. Currently I'm clearing all Tasks that match the verb and are finished, which is not ideal. I would like to have the exact TaskID to track and dont want to leave Tasks behind. I don't want also to clear tasks from other entities (like engine!). I understand that if I want to specify the task/flow ids when calling the vdsm API, these two need to be passed as headers (http) so they end up in the context of the call. Is this correct? But using vdsm/client.py I cannot figure out how to do this, but I understand it is possible. I've been looking at vdsm/common/api.py and vdsm/client.py and trying several things but nothing seems to do it. Could anyone please put some light on this? Thanks, Germano _______________________________________________ Devel mailing list -- devel(a)ovirt.org To unsubscribe send an email to devel-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MSWIR7AEWS7...

Hi Marcin, Thanks for replying. In short, this is what I am doing: ~~~ from vdsm import client cli = client.connect('localhost', 54321, use_tls=True) cli.SDM.update_volume(job_id=job, vol_info=vol, vol_attr=changes) ~~~ This last line does an API call. A taskID is generated in vdsm, I can only specify the JobID as a parameter of the call. From what I read, the taskID should be sent on the HTTP header if I want to set it. But I cannot find how to do it. The whole point is that I want to know the taskID, so I can monitor the task and clear it after it is finished. So: * How do I retrieve the taskID vdsm generated for this? or * How to I query VDSM to give me the taskID for the JobID I passed as argument or * How can I specify the taskID to be used given I'm using vdsm client (from vdsm import client). or * any other idea you have on how to handle this? Am I missing something or there is some missing implementation for doing this? Thanks On Mon, Sep 17, 2018 at 10:19 PM Marcin Sobczyk <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> wrote: Hi, I'm currently doing some work around vdsm API so maybe I can. I need some more detailed description of your problem though - a pseudo-code of the calls you're making with explanation what exactly is the problem would be ideal. Marcin On 9/17/18 7:04 AM, Germano Veit Michel wrote: > Hello, > > I've been struggling with TaskID/FlowID when talking to the VDSM > API. I am trying to write a tool that uses the vdsm-api to > facilitate the troubleshooting of image issues (snapshots). This > tool does a series of API calls, but I cannot find a nice way to > track the taskID and clear the completed tasks after completion > of the tool commands. Currently I'm clearing all Tasks that match > the verb and are finished, which is not ideal. I would like to > have the exact TaskID to track and dont want to leave Tasks > behind. I don't want also to clear tasks from other entities > (like engine!). > > I understand that if I want to specify the task/flow ids when > calling the vdsm API, these two need to be passed as headers > (http) so they end up in the context of the call. Is this > correct? But using vdsm/client.py I cannot figure out how to do > this, but I understand it is possible. > > I've been looking at vdsm/common/api.py and vdsm/client.py and > trying several things but nothing seems to do it. Could anyone > please put some light on this? > > Thanks, > Germano > > _______________________________________________ > Devel mailing list --devel(a)ovirt.org <mailto:devel@ovirt.org> > To unsubscribe send an email todevel-leave(a)ovirt.org <mailto:devel-leave@ovirt.org> > Privacy Statement:https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct:https://www.ovirt.org/community/about/community-guidelines/ > List Archives:https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MS... _______________________________________________ Devel mailing list -- devel(a)ovirt.org <mailto:devel@ovirt.org> To unsubscribe send an email to devel-leave(a)ovirt.org <mailto:devel-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/J3UUC3EBLGU...

Hi Marcin, Yes, I know that. Let me try to be clearer. 1. My API command passes a JobID as argument. 2. VDSM creates a taskID for it once it starts running the command 3. I want to retrieve or define the taskID somehow. The current API does show the TaskIDs and JobIDs. The JobID I know, but I want to know the TaskID for the API command I submitted. And the API does not have enough info to correlate the JobID with the TaskID. See: # vdsm-client Host getJobs {     "8a4a6452-bb9c-11e8-8871-52540016c103": {         "status": "done",         "job_type": "storage",         "id": "8a4a6452-bb9c-11e8-8871-52540016c103",         "description": "update_volume"     }, # vdsm-client Host getAllTasks {     "eb3f55af-c004-44af-8b28-87f8924a7623": {         "verb": "sdm_update_volume",         "code": 0,         "state": "finished",         "tag": "sdm",         "result": "",         "message": "1 jobs completed successfully",         "id": "eb3f55af-c004-44af-8b28-87f8924a7623"     }, I want to somehow retrieve or define the taskID (eb3f55af-c004-44af-8b28-87f8924a7623) after using the following update_volume command: ~~~ from vdsm import client cli = client.connect('localhost', 54321, use_tls=True) cli.SDM.update_volume(job_id=job, vol_info=vol, vol_attr=changes) ~~~ On Wed, Sep 19, 2018 at 4:52 AM Marcin Sobczyk <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> wrote: From what I can see in: https://github.com/oVirt/vdsm/blob/0725f32994293514d3275528c44687e8e800d6... you should try and query sdm jobs by using Host.getJobs API call. On 9/18/18 6:50 AM, Germano Veit Michel wrote: > Hi Marcin, > > Thanks for replying. In short, this is what I am doing: > > ~~~ > from vdsm import client > cli = client.connect('localhost', 54321, use_tls=True) > cli.SDM.update_volume(job_id=job, vol_info=vol, vol_attr=changes) > ~~~ > > This last line does an API call. A taskID is generated in vdsm, I > can only specify the JobID as a parameter of the call. From what > I read, the taskID should be sent on the HTTP header if I want to > set it. But I cannot find how to do it. > The whole point is that I want to know the taskID, so I can > monitor the task and clear it after it is finished. > > So: > * How do I retrieve the taskID vdsm generated for this? > or > * How to I query VDSM to give me the taskID for the JobID I > passed as argument > or > * How can I specify the taskID to be used given I'm using vdsm > client (from vdsm import client). > or > * any other idea you have on how to handle this? > > Am I missing something or there is some missing implementation > for doing this? > > Thanks > > On Mon, Sep 17, 2018 at 10:19 PM Marcin Sobczyk > <msobczyk(a)redhat.com <mailto:msobczyk@redhat.com>> wrote: > > Hi, > > I'm currently doing some work around vdsm API so maybe I can. > I need some more detailed description of your problem though > - a pseudo-code of the calls you're making with explanation > what exactly is the problem would be ideal. > > Marcin > > On 9/17/18 7:04 AM, Germano Veit Michel wrote: >> Hello, >> >> I've been struggling with TaskID/FlowID when talking to the >> VDSM API. I am trying to write a tool that uses the vdsm-api >> to facilitate the troubleshooting of image issues >> (snapshots). This tool does a series of API calls, but I >> cannot find a nice way to track the taskID and clear the >> completed tasks after completion of the tool commands. >> Currently I'm clearing all Tasks that match the verb and are >> finished, which is not ideal. I would like to have the exact >> TaskID to track and dont want to leave Tasks behind. I don't >> want also to clear tasks from other entities (like engine!). >> >> I understand that if I want to specify the task/flow ids >> when calling the vdsm API, these two need to be passed as >> headers (http) so they end up in the context of the call. Is >> this correct? But using vdsm/client.py I cannot figure out >> how to do this, but I understand it is possible. >> >> I've been looking at vdsm/common/api.py and vdsm/client.py >> and trying several things but nothing seems to do it. Could >> anyone please put some light on this? >> >> Thanks, >> Germano >> >> _______________________________________________ >> Devel mailing list --devel(a)ovirt.org <mailto:devel@ovirt.org> >> To unsubscribe send an email todevel-leave(a)ovirt.org <mailto:devel-leave@ovirt.org> >> Privacy Statement:https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct:https://www.ovirt.org/community/about/community-guidelines/ >> List Archives:https://lists.ovirt.org/archives/list/devel@ovirt.org/message/MS... > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org <mailto:devel@ovirt.org> > To unsubscribe send an email to devel-leave(a)ovirt.org > <mailto:devel-leave@ovirt.org> > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: > https://www.ovirt.org/community/about/community-guidelines/ > List Archives: > https://lists.ovirt.org/archives/list/devel@ovirt.org/message/J3UUC3EBLGU... >

On Mon, 17 Sep 2018, 8:06 Germano Veit Michel, <germano(a)redhat.com&gt; wrote: > Hello, > > I've been struggling with TaskID/FlowID when talking to the VDSM API. I > am trying to write a tool that uses the vdsm-api to facilitate the > troubleshooting of image issues (snapshots). This tool does a series of API > calls, but I cannot find a nice way to track the taskID and clear the > completed tasks after completion of the tool commands. Currently I'm > clearing all Tasks that match the verb and are finished, which is not > ideal. I would like to have the exact TaskID to track and dont want to > leave Tasks behind. I don't want also to clear tasks from other entities > (like engine!). > Storage jobs are managed using (client generated) job id. The task id is internal implementation detail which will likely disappear in future version. You don'need to monitor or clean the internal tasks, they are managed by the storage jobs framework for you.

You can check engine code to understand how storage jobs are managed. > I understand that if I want to specify the task/flow ids when calling the > vdsm API, these two need to be passed as headers (http) so they end up in > the context of the call. Is this correct? But using vdsm/client.py I cannot > figure out how to do this, but I understand it is possible. > This is not possible now, but sounds like a good idea. Please file RFE to add this, or if you have the time you can try to add. Looks like the place you can add it is in lib/yajsonrpc/stompclient.py - ClientRpcTransportAdapter.send().

Piotr, what do you think? Nir

On Thu, Sep 20, 2018 at 12:55 AM, Germano Veit Michel <germano(a)redhat.com&gt; wrote: > > > On Thu, Sep 20, 2018 at 7:14 AM Nir Soffer <nsoffer(a)redhat.com&gt; wrote: > >> >> >> On Mon, 17 Sep 2018, 8:06 Germano Veit Michel, <germano(a)redhat.com&gt; >> wrote: >> >>> Hello, >>> >>> I've been struggling with TaskID/FlowID when talking to the VDSM API. I >>> am trying to write a tool that uses the vdsm-api to facilitate the >>> troubleshooting of image issues (snapshots). This tool does a series of API >>> calls, but I cannot find a nice way to track the taskID and clear the >>> completed tasks after completion of the tool commands. Currently I'm >>> clearing all Tasks that match the verb and are finished, which is not >>> ideal. I would like to have the exact TaskID to track and dont want to >>> leave Tasks behind. I don't want also to clear tasks from other entities >>> (like engine!). >>> >> >> Storage jobs are managed using (client generated) job id. The task id is >> internal >> implementation detail which will likely disappear in future version. You >> don'need >> to monitor or clean the internal tasks, they are managed by the storage >> jobs >> framework for you. >> > > Thanks for the clarification Nir! > > >> You can check engine code to understand how storage jobs are managed. >> >> >>> I understand that if I want to specify the task/flow ids when calling >>> the vdsm API, these two need to be passed as headers (http) so they end up >>> in the context of the call. Is this correct? But using vdsm/client.py I >>> cannot figure out how to do this, but I understand it is possible. >>> >> >> This is not possible now, but sounds like a good idea. Please file RFE >> to add this, >> or if you have the time you can try to add. Looks like the place you can >> add it is >> in lib/yajsonrpc/stompclient.py - ClientRpcTransportAdapter.send(). >> > > Nice, I'll take a look. > > >> >> Piotr, what do you think? >> > I agree. In situations like this it would be great to be able to provide flow_id. It is possible to provide it in java based client [1]. Please open an RFE and we will implement it for you.

[1] https://github.com/oVirt/vdsm-jsonrpc-java/blob/d964c7ea4aa841972c66c737a... > >> Nir >> >