[PATCH] Github bug #326: run_command
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> As described in https://github.com/kimchi-project/kimchi/issues/326, run_command does not handle blocking process with timeout. This patch fixes the bug, killing all children process of the process created by Popen() call before executing kill() in the parent process, making the timeout timer working as intended. Please refer to the commit message of the patch for further details. Using the same example described in the github bug entry to demonstrate the fix: $ sudo PYTHONPATH=src python Python 2.7.5 (default, Nov 12 2013, 16:18:42) [GCC 4.8.2 20131017 (Red Hat 4.8.2-1)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from kimchi.utils import run_command mount = ['mount', 'localhost:/var/nfs', '/home/danielhb/nfs_test/'] run_command(mount, 7) subprocess is killed by signal.SIGKILL for timeout 7 seconds Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/danielhb/kimchi/kimchi-upstream/src/kimchi/utils.py", line 181, in run_command raise TimeoutExpired("KCHUTILS0002E", msg_args) kimchi.exception.TimeoutExpired: KCHUTILS0002E: Timeout while running command '['mount', 'localhost:/var/nfs', '/home/danielhb/nfs_test/']' after 7 seconds
ps output: ---- right after run_command started ---- [danielhb@tirion ~]$ ps axf | grep mount 16664 pts/3 S+ 0:00 | | \_ grep --color=auto mount 16660 pts/4 S+ 0:00 | \_ mount localhost:/var/nfs /home/danielhb/nfs_test/ 16662 pts/4 D+ 0:00 | \_ /sbin/mount.nfs localhost:/var/nfs /home/danielhb/nfs_test -o rw [danielhb@tirion ~]$ ---- after the timeout of 7 seconds ---- [danielhb@tirion ~]$ ps axf | grep mount 16696 pts/3 S+ 0:00 | | \_ grep --color=auto mount *** BLURB HERE *** Daniel Henrique Barboza (1): Github bug #326: run_command: killing all children processes src/kimchi/utils.py | 9 +++++++++ 1 file changed, 9 insertions(+) -- 1.8.3.1
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> The subprocess.kill() call does not handle well the created children of the process created by the Popen() call. In some cases (for example, the mount command) the parent processes creates children processes that aren't killed, leaving the parent thread hanging out in an infinite loop waiting for kill() to finish. An attempt was made using the os.killpg() call, but to no success. Using psutils facilities to kill all the children before killing the opened process was easier and clearer than the alternatives (opening a new thread and trying to kill the thread, for example). Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> --- src/kimchi/utils.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/src/kimchi/utils.py b/src/kimchi/utils.py index a1410c0..6be1c04 100644 --- a/src/kimchi/utils.py +++ b/src/kimchi/utils.py @@ -23,6 +23,7 @@ import cherrypy import os +import psutil import re import subprocess import urllib2 @@ -137,8 +138,16 @@ def run_command(cmd, timeout=None): timeout is a float number in seconds. timeout default value is None, means command run without timeout. """ + # subprocess.kill() can leave descendants running + # and halting the execution. Using psutil to + # get all descendants from the subprocess and + # kill them recursively. def kill_proc(proc, timeout_flag): try: + parent = psutil.Process(proc.pid) + for child in parent.get_children(recursive=True): + child.kill() + # kill the process after no children is left proc.kill() except OSError: pass -- 1.8.3.1
Reviewed-by: Aline Manera <alinefm@linux.vnet.ibm.com> On 02/24/2014 10:55 AM, Daniel Barboza wrote:
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
The subprocess.kill() call does not handle well the created children of the process created by the Popen() call. In some cases (for example, the mount command) the parent processes creates children processes that aren't killed, leaving the parent thread hanging out in an infinite loop waiting for kill() to finish.
An attempt was made using the os.killpg() call, but to no success. Using psutils facilities to kill all the children before killing the opened process was easier and clearer than the alternatives (opening a new thread and trying to kill the thread, for example).
Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> --- src/kimchi/utils.py | 9 +++++++++ 1 file changed, 9 insertions(+)
diff --git a/src/kimchi/utils.py b/src/kimchi/utils.py index a1410c0..6be1c04 100644 --- a/src/kimchi/utils.py +++ b/src/kimchi/utils.py @@ -23,6 +23,7 @@
import cherrypy import os +import psutil import re import subprocess import urllib2 @@ -137,8 +138,16 @@ def run_command(cmd, timeout=None): timeout is a float number in seconds. timeout default value is None, means command run without timeout. """ + # subprocess.kill() can leave descendants running + # and halting the execution. Using psutil to + # get all descendants from the subprocess and + # kill them recursively. def kill_proc(proc, timeout_flag): try: + parent = psutil.Process(proc.pid) + for child in parent.get_children(recursive=True): + child.kill() + # kill the process after no children is left proc.kill() except OSError: pass
participants (2)
-
Aline Manera -
Daniel Barboza