[PATCH] [Kimchi] Github #991: unsubscribing from exit bus on cleanup
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> WoK tests are running the Cherrypy Exit bus multiple times without executing the __init__ from Mockmodel. causing the 'virtviewertmpfile_cleanup' callback to fail because there is no temp file created (it was cleaned in a previous callback). Unsubscribing from the 'exit' bus in the cleanup solves the issue, given that the subscription only happens in Mockmodel __init__ . Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> --- mockmodel.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mockmodel.py b/mockmodel.py index cd06ee3..89f1187 100644 --- a/mockmodel.py +++ b/mockmodel.py @@ -33,6 +33,7 @@ from wok.objectstore import ObjectStore from wok.utils import add_task, convert_data_size from wok.xmlutils.utils import xml_item_update +from wok.plugins.kimchi import config as kimchi_config from wok.plugins.kimchi import imageinfo from wok.plugins.kimchi import osinfo from wok.plugins.kimchi.model import cpuinfo @@ -141,7 +142,7 @@ class MockModel(Model): cherrypy.engine.subscribe('exit', self.virtviewertmpfile_cleanup) def _create_virt_viewer_tmp_file(self): - path = '../data/virtviewerfiles/' + path = kimchi_config.get_virtviewerfiles_path() if not os.path.isdir(path): os.makedirs(path) @@ -155,6 +156,7 @@ class MockModel(Model): def virtviewertmpfile_cleanup(self): os.unlink(self.virtviewerfile_tmp.name) + cherrypy.engine.unsubscribe('exit', self.virtviewertmpfile_cleanup) def reset(self): MockModel._mock_vms = defaultdict(list) -- 2.5.5
On 08/29/2016 05:53 PM, dhbarboza82@gmail.com wrote:
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
WoK tests are running the Cherrypy Exit bus multiple times without executing the __init__ from Mockmodel. causing the 'virtviewertmpfile_cleanup' callback to fail because there is no temp file created (it was cleaned in a previous callback).
Unsubscribing from the 'exit' bus in the cleanup solves the issue, given that the subscription only happens in Mockmodel __init__ .
Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> --- mockmodel.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/mockmodel.py b/mockmodel.py index cd06ee3..89f1187 100644 --- a/mockmodel.py +++ b/mockmodel.py @@ -33,6 +33,7 @@ from wok.objectstore import ObjectStore from wok.utils import add_task, convert_data_size from wok.xmlutils.utils import xml_item_update
+from wok.plugins.kimchi import config as kimchi_config from wok.plugins.kimchi import imageinfo from wok.plugins.kimchi import osinfo from wok.plugins.kimchi.model import cpuinfo @@ -141,7 +142,7 @@ class MockModel(Model): cherrypy.engine.subscribe('exit', self.virtviewertmpfile_cleanup)
def _create_virt_viewer_tmp_file(self): - path = '../data/virtviewerfiles/' + path = kimchi_config.get_virtviewerfiles_path() if not os.path.isdir(path): os.makedirs(path)
@@ -155,6 +156,7 @@ class MockModel(Model):
def virtviewertmpfile_cleanup(self): os.unlink(self.virtviewerfile_tmp.name) + cherrypy.engine.unsubscribe('exit', self.virtviewertmpfile_cleanup)
Why is that needed?
def reset(self): MockModel._mock_vms = defaultdict(list)
On 08/29/2016 05:57 PM, Aline Manera wrote:
On 08/29/2016 05:53 PM, dhbarboza82@gmail.com wrote:
From: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com>
WoK tests are running the Cherrypy Exit bus multiple times without executing the __init__ from Mockmodel. causing the 'virtviewertmpfile_cleanup' callback to fail because there is no temp file created (it was cleaned in a previous callback).
Unsubscribing from the 'exit' bus in the cleanup solves the issue, given that the subscription only happens in Mockmodel __init__ .
Signed-off-by: Daniel Henrique Barboza <danielhb@linux.vnet.ibm.com> --- mockmodel.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/mockmodel.py b/mockmodel.py index cd06ee3..89f1187 100644 --- a/mockmodel.py +++ b/mockmodel.py @@ -33,6 +33,7 @@ from wok.objectstore import ObjectStore from wok.utils import add_task, convert_data_size from wok.xmlutils.utils import xml_item_update
+from wok.plugins.kimchi import config as kimchi_config from wok.plugins.kimchi import imageinfo from wok.plugins.kimchi import osinfo from wok.plugins.kimchi.model import cpuinfo @@ -141,7 +142,7 @@ class MockModel(Model): cherrypy.engine.subscribe('exit', self.virtviewertmpfile_cleanup)
def _create_virt_viewer_tmp_file(self): - path = '../data/virtviewerfiles/' + path = kimchi_config.get_virtviewerfiles_path() if not os.path.isdir(path): os.makedirs(path)
@@ -155,6 +156,7 @@ class MockModel(Model):
def virtviewertmpfile_cleanup(self): os.unlink(self.virtviewerfile_tmp.name) + cherrypy.engine.unsubscribe('exit', self.virtviewertmpfile_cleanup)
Why is that needed?
Because the callback subscription isn't expiring when the test_server is closing. The reason is that the cherrypy.engine has process wide scope and reaches all test_servers threads that are executed in the run_tests.sh script. In short, all run_test.sh servers are sharing the same cherrypy bus. This is verifiable by, using the current master, adding debug information in the __init__ of Mockmodel class in Kimchi and in the def virtviewertmpfile_cleanup file: def __init__(self, objstore_loc=None): print '=========== init of Mockmodel =============' def virtviewertmpfile_cleanup(self): print '****** unlink of file ', self.virtviewerfile_tmp.name, ' *****' os.unlink(self.virtviewerfile_tmp.name) I will not put the whole output here but check it out at https://paste.fedoraproject.org/417083/25156641/. Here is the relevant info: System Update Tool ...: True Repo Management Tool .: yum *** Ginger Base: Capabilities tests completed *** =========== init of Mockmodel ============= libvirt: Test Driver error : Domain not found libvirt: Test Driver error : Storage pool not found (...) [29/Aug/2016:21:07:01] ENGINE Bus STOPPED [29/Aug/2016:21:07:01] ENGINE Bus EXITING Closing any VNC/SPICE firewall ports opened by Kimchi ... ****** unlink of file /home/danielhb/kimchi/wok_all_plugins/data/virtviewerfiles/tmp7sdWvQ ***** [29/Aug/2016:21:07:01] ENGINE Bus EXITED [29/Aug/2016:21:07:01] ENGINE Waiting for child threads to terminate... =========== init of Mockmodel ============= libvirt: error : invalid argument: event callback already tracked (...) [29/Aug/2016:21:07:03] ENGINE Bus STOPPED [29/Aug/2016:21:07:03] ENGINE Bus EXITING ****** unlink of file /home/danielhb/kimchi/wok_all_plugins/data/virtviewerfiles/tmpZjQ3L3 ***** ****** unlink of file /home/danielhb/kimchi/wok_all_plugins/data/virtviewerfiles/tmp7sdWvQ ***** [29/Aug/2016:21:07:03] ENGINE Error in 'exit' listener <bound method MockModel.virtviewertmpfile_cleanup of <wok.plugins.kimchi.mockmodel.MockModel object at 0x7f7f60d17790>> Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/cherrypy/process/wspbus.py", line 205, in publish output.append(listener(*args, **kwargs)) File "/home/danielhb/kimchi/wok_all_plugins/src/wok/plugins/kimchi/mockmodel.py", line 160, in virtviewertmpfile_cleanup os.unlink(self.virtviewerfile_tmp.name) OSError: [Errno 2] No such file or directory: '/home/danielhb/kimchi/wok_all_plugins/data/virtviewerfiles/tmp7sdWvQ' Closing any VNC/SPICE firewall ports opened by Kimchi ... Closing any VNC/SPICE firewall ports opened by Kimchi ... [danielhb@arthas tests]$ We can see that the callback of the file "tmp7sdWvQ" is being called twice because it is still subscribed in the bus. This is why we need to unsubscribe the callback after doing the cleanup. This means that we're kind of using the 'exit' callbacks in a dirty way, without unsubscribing them and leaving the cherrypy.engine with bloat. However, we're doing that when exiting the python process (wokd) and killing the cherrypy bus in the process. If another WoK server thread were running in the same process we would have the same problem. Daniel
def reset(self): MockModel._mock_vms = defaultdict(list)
participants (3)
-
Aline Manera -
Daniel Henrique Barboza -
dhbarboza82＠gmail.com