[ovirt-users] Re: Frequent events seen like VM "vmname" is not responding

Thanks Vojtech Juranek for quick response please find logs below We got vm non responsive and at the same time vdsm log shows "monitor became unresponsive (command timeout, age=61.88649625200196) (vm:5944)"

Please refer below logs for more details. [root@host vdsm]# cat vdsm.log | grep WARN 2021-09-21 14:30:19,487+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/492 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f023860d6d8> discarded task#=2334 at 0x7f02709edc88>, <Worker name=periodic/493 waiting task#=0 at 0x7f02381016a0>, <Worker name=periodic/489 waiting task#=2701 at 0x7f02386faef0>, <Worker name=periodic/491 waiting task#=1807 at 0x7f0270903908>, <Worker name=periodic/483 waiting task#=2144 at 0x7f0270b12940>} (executor:213) 2021-09-21 14:30:34,487+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/493 waiting task#=3 at 0x7f02381016a0>, <Worker name=periodic/489 waiting task#=2704 at 0x7f02386faef0>, <Worker name=periodic/491 waiting task#=1809 at 0x7f0270903908>, <Worker name=periodic/483 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f02384a7780> discarded task#=2146 at 0x7f0270b12940>, <Worker name=periodic/494 waiting task#=0 at 0x7f02386d2f60>} (executor:213) 2021-09-21 14:30:49,844+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/493 waiting task#=15 at 0x7f02381016a0>, <Worker name=periodic/489 waiting task#=2717 at 0x7f02386faef0>, <Worker name=periodic/491 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f02147e0ac8> discarded task#=1897 at 0x7f0270903908>, <Worker name=periodic/495 waiting task#=0 at 0x7f023847d550>, <Worker name=periodic/494 waiting task#=12 at 0x7f02386d2f60>} (executor:213) 2021-09-21 14:31:04,491+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/496 waiting task#=0 at 0x7f0270c70c18>, <Worker name=periodic/493 waiting task#=17 at 0x7f02381016a0>, <Worker name=periodic/489 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f0238494240> discarded task#=2719 at 0x7f02386faef0>, <Worker name=periodic/495 waiting task#=2 at 0x7f023847d550>, <Worker name=periodic/494 waiting task#=15 at 0x7f02386d2f60>} (executor:213) 2021-09-21 14:31:19,491+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/496 waiting task#=2 at 0x7f0270c70c18>, <Worker name=periodic/497 waiting task#=0 at 0x7f0238756828>, <Worker name=periodic/493 waiting task#=20 at 0x7f02381016a0>, <Worker name=periodic/495 waiting task#=6 at 0x7f023847d550>, <Worker name=periodic/494 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f02386182e8> discarded task#=17 at 0x7f02386d2f60>} (executor:213) 2021-09-21 14:31:34,492+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/496 waiting task#=4 at 0x7f0270c70c18>, <Worker name=periodic/497 waiting task#=3 at 0x7f0238756828>, <Worker name=periodic/498 waiting task#=0 at 0x7f02384aca58>, <Worker name=periodic/493 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f02380652e8> discarded task#=22 at 0x7f02381016a0>, <Worker name=periodic/495 waiting task#=8 at 0x7f023847d550>} (executor:213) 2021-09-21 14:31:42,418+0530 WARN (jsonrpc/5) [virt.vm] (vmId='b994fb6c-4a7d-469a-82fb-1d776e9f7ae9') monitor became unresponsive (command timeout, age=60.07553722700686) (vm:5944) 2021-09-21 14:31:49,492+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/496 waiting task#=93 at 0x7f0270c70c18>, <Worker name=periodic/497 waiting task#=14 at 0x7f0238756828>, <Worker name=periodic/498 waiting task#=12 at 0x7f02384aca58>, <Worker name=periodic/499 waiting task#=0 at 0x7f0270d3d710>, <Worker name=periodic/495 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f02384768d0> discarded task#=22 at 0x7f023847d550>} (executor:213) 2021-09-21 14:32:19,492+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/496 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f027098a400> discarded task#=97 at 0x7f0270c70c18>, <Worker name=periodic/497 waiting task#=20 at 0x7f0238756828>, <Worker name=periodic/498 waiting task#=17 at 0x7f02384aca58>, <Worker name=periodic/500 waiting task#=0 at 0x7f0270ff02b0>, <Worker name=periodic/499 waiting task#=5 at 0x7f0270d3d710>} (executor:213) 2021-09-21 14:33:19,494+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/497 waiting task#=115 at 0x7f0238756828>, <Worker name=periodic/498 waiting task#=40 at 0x7f02384aca58>, <Worker name=periodic/500 waiting task#=22 at 0x7f0270ff02b0>, <Worker name=periodic/499 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f0270d25be0> discarded task#=23 at 0x7f0270d3d710>, <Worker name=periodic/501 waiting task#=0 at 0x7f02709ebdd8>} (executor:213) 2021-09-21 14:33:34,494+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/497 waiting task#=117 at 0x7f0238756828>, <Worker name=periodic/498 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f0270c4b198> discarded task#=42 at 0x7f02384aca58>, <Worker name=periodic/500 waiting task#=25 at 0x7f0270ff02b0>, <Worker name=periodic/502 waiting task#=0 at 0x7f0270d17978>, <Worker name=periodic/501 waiting task#=2 at 0x7f02709ebdd8>} (executor:213) 2021-09-21 14:33:40,755+0530 WARN (periodic/502) [virt.periodic.VmDispatcher] could not run <class 'vdsm.virt.periodic.UpdateVolumes'> on ['606a03d5-4a1a-40bc-94dc-d455bc2fe950'] (periodic:290) 2021-09-21 14:33:50,813+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/497 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f0270a87dd8> discarded task#=129 at 0x7f0238756828>, <Worker name=periodic/500 waiting task#=37 at 0x7f0270ff02b0>, <Worker name=periodic/503 waiting task#=0 at 0x7f02709f3ef0>, <Worker name=periodic/502 waiting task#=13 at 0x7f0270d17978>, <Worker name=periodic/501 waiting task#=90 at 0x7f02709ebdd8>} (executor:213) 2021-09-21 14:34:28,880+0530 WARN (jsonrpc/2) [virt.vm] (vmId='968aae88-8bca-4800-be9f-38067b73f271') monitor became unresponsive (command timeout, age=61.88649625200196) (vm:5944) 2021-09-21 14:40:34,503+0530 WARN (vdsm.Scheduler) [Executor] executor state: count=5 workers={<Worker name=periodic/504 waiting task#=0 at 0x7f0270faaa58>, <Worker name=periodic/500 waiting task#=245 at 0x7f0270ff02b0>, <Worker name=periodic/503 running <Task discardable <Operation action=<vdsm.virt.sampling.VMBulkstatsMonitor object at 0x7f025051dc18> at 0x7f02502ff5f8> timeout=7.5, duration=7.50 at 0x7f02386ada20> discarded task#=217 at 0x7f02709f3ef0>, <Worker name=periodic/502 waiting task#=359 at 0x7f0270d17978>, <Worker name=periodic/501 waiting task#=301 at 0x7f02709ebdd8>} (executor:213) Not able to understand the cause and required fix please help _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/AJWT7KOCVJP6C M42MG7Y7W5ZPOSSZRDZ/