<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi Maton,<div class=""><br class=""></div><div class="">I have seen tasks in a weird state on my cluster also. I've had a vm get "stuck" during a migration where it says "migrating to" in the web GUI, but it has finished migrating hours ago... If I click "Cancel Migraton" the gui tells me that it is not migrating, but I can't do any action on the vm because I am then told that the vm can't be acted upon while it is migrating. I also try to kill the task, but there are none listed<br class=""><div class=""><br class="webkit-block-placeholder"></div><div class="">What has worked for me has been to put my hosted-engine in global maintenance mode, then ssh into the hosted engine and run the "engine-setup" command. I am not saying the is the best course of action, but when the engine comes back online the task is cleared.</div><div class="">
<div id="signature" class=""><br class="">Cheers,<br class="">Gervais<br class=""><br class=""><br class=""></div>
</div>
<br class=""><div><blockquote type="cite" class=""><div class="">On Sep 10, 2016, at 11:06 AM, Maton, Brett <<a href="mailto:matonb@ltresources.co.uk" class="">matonb@ltresources.co.uk</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class="">Anyone know how to fix this broken task ?<br class=""><br class=""></div><div class="">It's persisted through a reboot of all hosts and the engine, something needs deleting from the database to clear the task and release the locked disk<br class=""></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On 8 September 2016 at 13:25, Maton, Brett <span dir="ltr" class=""><<a href="mailto:matonb@ltresources.co.uk" target="_blank" class="">matonb@ltresources.co.uk</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="">Thanks for the pointer Mikhail, however I don't get any tasks listed with that command:<br class=""><br class="">vdsClient -s 0 getAllTasksStatuses<br class=""><br class="">/usr/share/vdsm/vdsClient.py:<wbr class="">33: DeprecationWarning: vdscli uses xmlrpc. since ovirt 3.6 xmlrpc is deprecated, please use vdsm.jsonrpcvdscli<br class=""> from vdsm import utils, vdscli, constants<br class=""><br class="">{'status': {'message': 'OK', 'code': 0}, 'allTasksStatus': {}}<br class=""><br class=""></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br class=""><div class="gmail_quote">On 8 September 2016 at 09:51, Краснобаев Михаил <span dir="ltr" class=""><<a href="mailto:milo1@ya.ru" target="_blank" class="">milo1@ya.ru</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="">Hi,</div><div class=""> </div><div class="">There is a way to cancel a running task - look here <a href="http://lists.ovirt.org/pipermail/users/2014-November/028946.html" target="_blank" class="">http://lists.ovirt.org/piperma<wbr class="">il/users/2014-November/028946.<wbr class="">html</a></div><div class="">I was able to stop snapshot deletion this way.</div><div class=""> </div><div class="">Best, Mikhail.</div><div class=""> </div><div class="">08.09.2016, 08:14, "Maton, Brett" <<a href="mailto:matonb@ltresources.co.uk" target="_blank" class="">matonb@ltresources.co.uk</a>>:</div><blockquote type="cite" class=""><div class=""><div class=""><div class=""><div class=""><div class="">Any suggestions ?<br class=""><br class=""></div>THe task has been hung for 5 days now, I can't start the machine or destroy it.<br class=""><br class=""></div></div><div class=""><br class=""><div class="">On 7 September 2016 at 06:49, Maton, Brett <span class=""><<a href="mailto:matonb@ltresources.co.uk" target="_blank" class="">matonb@ltresources.co.uk</a>></span> wrote:<br class=""><blockquote style="margin:0 0 0 0.8ex;border-left:1px #ccc solid;padding-left:1ex" class=""><div class=""><div class=""><div class="">Sorry just hit reply....<br class=""><br class=""></div>I'm seeing these errors in the logs which look related to the problem:<br class=""><br class=""><br class=""><span class="">2016-09-07 06</span>:46:35,123 ERROR [org.ovirt.engine.core.bll.tas<wbr class="">ks.CommandCallbacksPoller] (DefaultQuartzScheduler6) [19c58c0d] Failed invoking callback end method 'onFailed' for command '<span class="">07608003</span>-ca05-4e2e-b917-85ce5<wbr class="">25c011b' with exception 'null', the callback is marked for end method retries<br class=""><span class="">2016-09-07 06</span>:46:45,184 ERROR [<a href="http://org.ovirt.engine.core.bll.com/" class="">org.ovirt.engine.core.bll.Com</a><wbr class="">mandsFactory] (DefaultQuartzScheduler7) [19c58c0d] Error in invocating CTOR of command 'LiveMigrateDisk': null<br class=""><span class="">2016-09-07 06</span>:46:45,185 ERROR [org.ovirt.engine.core.bll.tas<wbr class="">ks.CommandCallbacksPoller] (DefaultQuartzScheduler7) [19c58c0d] Failed invoking callback end method 'onFailed' for command '<span class="">07608003</span>-ca05-4e2e-b917-85ce5<wbr class="">25c011b' with exception 'null', the callback is marked for end method retries</div><div class=""><div class=""><div class=""><div class=""><div class=""><div class=""><br class=""><div class="">On 5 September 2016 at 06:46, Nir Soffer <span class=""><<a href="mailto:nsoffer@redhat.com" target="_blank" class="">nsoffer@redhat.com</a>></span> wrote:<br class=""><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid #cccccc;padding-left:1ex" class="">Hi Maton,<br class=""> <br class=""> Please reply to the list, not to me directly.<br class=""> <br class=""> Ala, can you look at this? is this a known issue?<br class=""> <br class=""> Thanks,<br class=""> Nir<br class=""><div class=""><div class=""><br class=""> On Mon, Sep 5, 2016 at 8:43 AM, Maton, Brett <<a href="mailto:matonb@ltresources.co.uk" target="_blank" class="">matonb@ltresources.co.uk</a>> wrote:<br class=""> > Log files as requested<br class=""> ><br class=""> > <a href="https://ufile.io/4fc35" target="_blank" class="">https://ufile.io/4fc35</a> vdsm log<br class=""> > <a href="https://ufile.io/e9836" target="_blank" class="">https://ufile.io/e9836</a> engine 03-Sep<br class=""> > <a href="https://ufile.io/15f37" target="_blank" class="">https://ufile.io/15f37</a> engine 04-Sep<br class=""> ><br class=""> > vdsm log stops on the 01-Sep...<br class=""> ><br class=""> > Couple of entries from the event log:<br class=""> ><br class=""> > Sep 3, 2016 7:31:07 PM Snapshot 'Auto-generated for Live Storage<br class=""> > Migration' deletion for VM 'lv01' has been completed.<br class=""> > Sep 3, 2016 6:46:46 PM Snapshot 'Auto-generated for Live Storage<br class=""> > Migration' deletion for VM 'lv01' was initiated by SYSTEM<br class=""> ><br class=""> > And the related tasks<br class=""> ><br class=""> > Removing Snapshot Auto-generated for Live Storage Migration of VM lv01<br class=""> > Sep 3, 2016 6:46:44 PM N/A 29f45ca9<br class=""> > Validating Sep 3, 2016 6:46:44 PM until Sep 3, 2016 6:46:44 PM<br class=""> > Executing Sep 3, 2016 6:46:44 PM until Sep 3, 2016 7:31:06 PM<br class=""> ><br class=""> > Finalizing Sep 3, 2016 7:31:06 PM N/A<br class=""> ><br class=""> ><br class=""> ><br class=""> > On 4 September 2016 at 14:27, Nir Soffer <<a href="mailto:nsoffer@redhat.com" target="_blank" class="">nsoffer@redhat.com</a>> wrote:<br class=""> >><br class=""> >> On Sun, Sep 4, 2016 at 12:40 PM, Maton, Brett <<a href="mailto:matonb@ltresources.co.uk" target="_blank" class="">matonb@ltresources.co.uk</a>><br class=""> >> wrote:<br class=""> >>><br class=""> >>> How do I fix / kill a hung vdsm task?<br class=""> >>><br class=""> >>> It seems to have completed the task but is stuck finalising.<br class=""> >>><br class=""> >>> Removing Snapshot Auto-generated for Live Storage Migration<br class=""> >>> Validating<br class=""> >>> Executing<br class=""> >>> (hour glass) Finalizing<br class=""> >>><br class=""> >>> Task has been 'stuck' finalising for over 13 hours<br class=""> >><br class=""> >><br class=""> >> Can you share engine and vdsm logs since the time the merge was started?<br class=""> >><br class=""> >> Nir<br class=""> ><br class=""> ></div></div></blockquote></div></div></div></div></div></div></div></div></blockquote></div></div></div></div>,<p class="">______________________________<wbr class="">_________________<br class="">Users mailing list<br class=""><a href="mailto:Users@ovirt.org" target="_blank" class="">Users@ovirt.org</a><br class=""><a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank" class="">http://lists.ovirt.org/mailman<wbr class="">/listinfo/users</a></p></blockquote><span class=""><font color="#888888" class=""><div class=""> </div><div class=""> </div><div class="">-- </div><div class="">С уважением, Краснобаев Михаил.</div><div class=""> </div><div class=""> </div></font></span></blockquote></div><br class=""></div>
</div></div></blockquote></div><br class=""></div>
_______________________________________________<br class="">Users mailing list<br class=""><a href="mailto:Users@ovirt.org" class="">Users@ovirt.org</a><br class="">http://lists.ovirt.org/mailman/listinfo/users<br class=""></div></blockquote></div><br class=""></div></body></html>