4:16 p.m.
Hi,
I just stumbled upon a defunct supervdsm process:
root 2824 2815 0 Apr04 ? 00:01:01 /usr/bin/python
/usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock
--pidfile /var/run/vdsm/supervdsmd.pid
qemu 31321 2824 0 May02 ? 00:00:00 [supervdsmServer] <defunct>
any chance to debug this?
might this be an instance of this closed (but never fixed) bug?
https://bugzilla.redhat.com/show_bug.cgi?id=841486
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
4:49 p.m.
On Fri, May 09, 2014 at 01:16:07PM +0000, Sven Kieske wrote:
Hi,
I just stumbled upon a defunct supervdsm process:
root 2824 2815 0 Apr04 ? 00:01:01 /usr/bin/python
/usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock
--pidfile /var/run/vdsm/supervdsmd.pid
qemu 31321 2824 0 May02 ? 00:00:00 [supervdsmServer] <defunct>
any chance to debug this?
might this be an instance of this closed (but never fixed) bug?
https://bugzilla.redhat.com/show_bug.cgi?id=841486
It seems so. Please reopen the bug with new information: is there
something interesting in May02's supervdsm.log? Such as a Timeout during
validateAccess? It might be a case for the zombiereaper module to solve.
Dan.
5:39 p.m.
I don't see anything in supervdsm.log
just one error at all related to a vm network.
in vdsm log I got these timeouts repeatedly:
Thread-25::DEBUG::2014-05-02
09:02:43,652::fileSD::222::Storage.Misc.excCmd::(getReadDelay) '/bin/dd
iflag=direct
if=/rhev/data-center/mnt/_home_DATA/9dc0fcb5-b0b0-47dd-b41f-d8709fd8cab2/dom_md/metadata
bs=4096
count=1' (cwd None)
Thread-25::DEBUG::2014-05-02
09:02:43,667::fileSD::222::Storage.Misc.excCmd::(getReadDelay) SUCCESS:
<err> = '0+1 records in\n0+1 records out\n495 bytes (495 B) copied,
0.000160501 s, 3.1 MB/s\n'; <rc> = 0
VM Channels Listener::DEBUG::2014-05-02
09:02:43,940::vmChannels::91::vds::(_handle_timeouts) Timeout on fileno 65.
VM Channels Listener::DEBUG::2014-05-02
09:02:45,384::vmChannels::91::vds::(_handle_timeouts) Timeout on fileno 106.
Am 09.05.2014 15:49, schrieb Dan Kenigsberg:
It seems so. Please reopen the bug with new information: is there
something interesting in May02's supervdsm.log? Such as a Timeout during
validateAccess? It might be a case for the zombiereaper module to solve.
Dan.
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
6:05 p.m.
On Fri, May 09, 2014 at 02:39:29PM +0000, Sven Kieske wrote:
I don't see anything in supervdsm.log
Still, I believe that the bug is worth reopening - it's a process leak,
and it should be avoided. I believe that it can be easily solved by
adding zombiereaper to supervdsm.
just one error at all related to a vm network.
in vdsm log I got these timeouts repeatedly:
Thread-25::DEBUG::2014-05-02
09:02:43,652::fileSD::222::Storage.Misc.excCmd::(getReadDelay) '/bin/dd
iflag=direct
if=/rhev/data-center/mnt/_home_DATA/9dc0fcb5-b0b0-47dd-b41f-d8709fd8cab2/dom_md/metadata
bs=4096
count=1' (cwd None)
Thread-25::DEBUG::2014-05-02
09:02:43,667::fileSD::222::Storage.Misc.excCmd::(getReadDelay) SUCCESS:
<err> = '0+1 records in\n0+1 records out\n495 bytes (495 B) copied,
0.000160501 s, 3.1 MB/s\n'; <rc> = 0
VM Channels Listener::DEBUG::2014-05-02
09:02:43,940::vmChannels::91::vds::(_handle_timeouts) Timeout on fileno 65.
VM Channels Listener::DEBUG::2014-05-02
09:02:45,384::vmChannels::91::vds::(_handle_timeouts) Timeout on fileno 106.
This is unrelated - that's Vdsm complaining about guest agents not
heart-beating. It's log noise that has very small value (particularly if
you never run guest agents). It should not appear repeatedly more than
once per guest.
Solving this isn't hard, but would never get priority without an
acclaimed user's BZ (hint hint).
Dan.
6:57 p.m.
Well, you called for it:
https://bugzilla.redhat.com/show_bug.cgi?id=1096312
Have a nice weekend!
I also attached myself to the other BZ, please reopen.
Am 09.05.2014 17:05, schrieb Dan Kenigsberg:
This is unrelated - that's Vdsm complaining about guest agents
not
heart-beating. It's log noise that has very small value (particularly if
you never run guest agents). It should not appear repeatedly more than
once per guest.
Solving this isn't hard, but would never get priority without an
acclaimed user's BZ (hint hint).
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
11:21 a.m.
I just
wrote to this bugreport to get things going.
why isn't it fixed yet? what does block this?
can I do anything to speed things up?
Thanks in advance.
Am 09.05.2014 15:16, schrieb Sven Kieske:
Hi,
I just stumbled upon a defunct supervdsm process:
root 2824 2815 0 Apr04 ? 00:01:01 /usr/bin/python
/usr/share/vdsm/supervdsmServer --sockfile /var/run/vdsm/svdsm.sock
--pidfile /var/run/vdsm/supervdsmd.pid
qemu 31321 2824 0 May02 ? 00:00:00 [supervdsmServer] <defunct>
any chance to debug this?
might this be an instance of this closed (but never fixed) bug?
https://bugzilla.redhat.com/show_bug.cgi?id=841486
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
12:05 p.m.
On Wed, Jun 18, 2014 at 08:21:28AM +0000, Sven Kieske wrote:
I just
wrote to this bugreport to get things going.
why isn't it fixed yet? what does block this?
can I do anything to speed things up?
Does the suggested patch http://gerrit.ovirt.org/#/c/27627/ solves the
problem and does not introduce new ones? If so, please mark it as
verified.
It awaits verification and more reviews, while we are busy in
even-more-urgent 3.5 chores.
Regards,
Dan.
12:20 p.m.
Am 18.06.2014 11:05, schrieb Dan Kenigsberg:
Does the suggested patch http://gerrit.ovirt.org/#/c/27627/ solves the
problem and does not introduce new ones? If so, please mark it as
verified.
It awaits verification and more reviews, while we are busy in
even-more-urgent 3.5 chores.
Regards,
Dan.
Thanks for pushing with me for the review in order to avoid log spam.
You do not happen to know anything about the other
bug(supervdsm defunct), though?
Unfortunately I can't test the mentioned patch myself.
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
12:23 p.m.
On 06/18/2014 05:20 AM, Sven Kieske wrote:
Am 18.06.2014 11:05, schrieb Dan Kenigsberg:
> Does the suggested patch
http://gerrit.ovirt.org/#/c/27627/ solves the
> problem and does not introduce new ones? If so, please mark it as
> verified.
> It awaits verification and more reviews, while we are busy
in
> even-more-urgent 3.5 chores.
> Regards,
> Dan.
Thanks for pushing with me for the review in order to avoid log spam.
You do not happen to know anything about the other
bug(supervdsm defunct), though?
Unfortunately I can't test the mentioned patch myself.
If you're hitting the issue can you hand edit the .py file with what's
shown below?
http://gerrit.ovirt.org/#/c/27627/1/vdsm/virt/vmchannels.py
1:55 p.m.
On Wed, Jun 18, 2014 at 09:20:06AM +0000, Sven Kieske wrote:
Am 18.06.2014 11:05, schrieb Dan Kenigsberg:
>
> Does the suggested patch http://gerrit.ovirt.org/#/c/27627/ solves the
> problem and does not introduce new ones? If so, please mark it as
> verified.
>
> It awaits verification and more reviews, while we are busy in
> even-more-urgent 3.5 chores.
>
> Regards,
> Dan.
Thanks for pushing with me for the review in order to avoid log spam.
You do not happen to know anything about the other
bug(supervdsm defunct), though?
Oh sorry, this email thread is overloaded a bit. I'm adding Yaniv to the
nag list. I hope he or Dima can prepare a quick fix for the pid leak.
2:22 p.m.
On 06/18/2014 01:55 PM, Dan Kenigsberg wrote:
On Wed, Jun 18, 2014 at 09:20:06AM +0000, Sven Kieske wrote:
>
>
> Am 18.06.2014 11:05, schrieb Dan Kenigsberg:
>>
>> Does the suggested patch http://gerrit.ovirt.org/#/c/27627/ solves the
>> problem and does not introduce new ones? If so, please mark it as
>> verified.
>>
>> It awaits verification and more reviews, while we are busy in
>> even-more-urgent 3.5 chores.
>>
>> Regards,
>> Dan.
>
>
> Thanks for pushing with me for the review in order to avoid log spam.
> You do not happen to know anything about the other
> bug(supervdsm defunct), though?
Oh sorry, this email thread is overloaded a bit. I'm adding Yaniv to the
nag list. I hope he or Dima can prepare a quick fix for the pid leak.
I don't understand how is http://gerrit.ovirt.org/#/c/27627 related to
the defunct problem. Sven, do you have constant flow that produce those
defunct process? it can help. if you do, please update [1] with your
steps, and logs might help too if you don't have such known flow
[1] https://bugzilla.redhat.com/show_bug.cgi?id=841486
Thanks
--
Yaniv Bronhaim.
3:17 p.m.
Am 18.06.2014 13:22, schrieb ybronhei:
On 06/18/2014 01:55 PM, Dan Kenigsberg wrote:
> On Wed, Jun 18, 2014 at 09:20:06AM +0000, Sven Kieske wrote:
>>
>>
>> Am 18.06.2014 11:05, schrieb Dan Kenigsberg:
>>>
>>> Does the suggested patch http://gerrit.ovirt.org/#/c/27627/ solves the
>>> problem and does not introduce new ones? If so, please mark it as
>>> verified.
>>>
>>> It awaits verification and more reviews, while we are busy in
>>> even-more-urgent 3.5 chores.
>>>
>>> Regards,
>>> Dan.
>>
>>
>> Thanks for pushing with me for the review in order to avoid log spam.
>> You do not happen to know anything about the other
>> bug(supervdsm defunct), though?
>
> Oh sorry, this email thread is overloaded a bit. I'm adding Yaniv to the
> nag list. I hope he or Dima can prepare a quick fix for the pid leak.
I don't understand how is http://gerrit.ovirt.org/#/c/27627
related to
the defunct problem. Sven, do you have constant flow that produce those
defunct process? it can help. if you do, please update [1] with your
steps, and logs might help too if you don't have such known flow
[1] https://bugzilla.redhat.com/show_bug.cgi?id=841486
Thanks
I'm sorry for the confusion, the mentioned patch at:
http://gerrit.ovirt.org/#/c/27627
does not fix the problem, but the log spam was discovered
during debugging the defunct process.
I'll try to send some logs on Friday and help debugging
the defunct process.
--
Mit freundlichen Grüßen / Regards
Sven Kieske
Systemadministrator
Mittwald CM Service GmbH & Co. KG
Königsberger Straße 6
32339 Espelkamp
T: +49-5772-293-100
F: +49-5772-293-333
https://www.mittwald.de
Geschäftsführer: Robert Meyer
St.Nr.: 331/5721/1033, USt-IdNr.: DE814773217, HRA 6640, AG Bad Oeynhausen
Komplementärin: Robert Meyer Verwaltungs GmbH, HRB 13260, AG Bad Oeynhausen
3812
days inactive
3852
days old
11 comments
4 participants
participants (4)
-
Andrew Cathrow -
Dan Kenigsberg -
Sven Kieske -
ybronhei