4:53 p.m.
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
WoK tests are running the Cherrypy Exit bus multiple times
without executing the __init__ from Mockmodel. causing
the 'virtviewertmpfile_cleanup' callback to fail because
there is no temp file created (it was cleaned in a previous
callback).
Unsubscribing from the 'exit' bus in the cleanup solves the
issue, given that the subscription only happens in Mockmodel
__init__ .
Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
---
mockmodel.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/mockmodel.py b/mockmodel.py
index cd06ee3..89f1187 100644
--- a/mockmodel.py
+++ b/mockmodel.py
@@ -33,6 +33,7 @@ from wok.objectstore import ObjectStore
from wok.utils import add_task, convert_data_size
from wok.xmlutils.utils import xml_item_update
+from wok.plugins.kimchi import config as kimchi_config
from wok.plugins.kimchi import imageinfo
from wok.plugins.kimchi import osinfo
from wok.plugins.kimchi.model import cpuinfo
@@ -141,7 +142,7 @@ class MockModel(Model):
cherrypy.engine.subscribe('exit', self.virtviewertmpfile_cleanup)
def _create_virt_viewer_tmp_file(self):
- path = '../data/virtviewerfiles/'
+ path = kimchi_config.get_virtviewerfiles_path()
if not os.path.isdir(path):
os.makedirs(path)
@@ -155,6 +156,7 @@ class MockModel(Model):
def virtviewertmpfile_cleanup(self):
os.unlink(self.virtviewerfile_tmp.name)
+ cherrypy.engine.unsubscribe('exit', self.virtviewertmpfile_cleanup)
def reset(self):
MockModel._mock_vms = defaultdict(list)
--
2.5.5
4:57 p.m.
On 08/29/2016 05:53 PM, dhbarboza82(a)gmail.com wrote:
From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
WoK tests are running the Cherrypy Exit bus multiple times
without executing the __init__ from Mockmodel. causing
the 'virtviewertmpfile_cleanup' callback to fail because
there is no temp file created (it was cleaned in a previous
callback).
Unsubscribing from the 'exit' bus in the cleanup solves the
issue, given that the subscription only happens in Mockmodel
__init__ .
Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
---
mockmodel.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/mockmodel.py b/mockmodel.py
index cd06ee3..89f1187 100644
--- a/mockmodel.py
+++ b/mockmodel.py
@@ -33,6 +33,7 @@ from wok.objectstore import ObjectStore
from wok.utils import add_task, convert_data_size
from wok.xmlutils.utils import xml_item_update
+from wok.plugins.kimchi import config as kimchi_config
from wok.plugins.kimchi import imageinfo
from wok.plugins.kimchi import osinfo
from wok.plugins.kimchi.model import cpuinfo
@@ -141,7 +142,7 @@ class MockModel(Model):
cherrypy.engine.subscribe('exit', self.virtviewertmpfile_cleanup)
def _create_virt_viewer_tmp_file(self):
- path = '../data/virtviewerfiles/'
+ path = kimchi_config.get_virtviewerfiles_path()
if not os.path.isdir(path):
os.makedirs(path)
@@ -155,6 +156,7 @@ class MockModel(Model):
def virtviewertmpfile_cleanup(self):
os.unlink(self.virtviewerfile_tmp.name)
+ cherrypy.engine.unsubscribe('exit', self.virtviewertmpfile_cleanup)
Why is that needed?
def reset(self):
MockModel._mock_vms = defaultdict(list)
8:22 p.m.
On 08/29/2016 05:57 PM, Aline Manera wrote:
On 08/29/2016 05:53 PM, dhbarboza82(a)gmail.com wrote:
> From: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
>
> WoK tests are running the Cherrypy Exit bus multiple times
> without executing the __init__ from Mockmodel. causing
> the 'virtviewertmpfile_cleanup' callback to fail because
> there is no temp file created (it was cleaned in a previous
> callback).
>
> Unsubscribing from the 'exit' bus in the cleanup solves the
> issue, given that the subscription only happens in Mockmodel
> __init__ .
>
> Signed-off-by: Daniel Henrique Barboza <danielhb(a)linux.vnet.ibm.com>
> ---
> mockmodel.py | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/mockmodel.py b/mockmodel.py
> index cd06ee3..89f1187 100644
> --- a/mockmodel.py
> +++ b/mockmodel.py
> @@ -33,6 +33,7 @@ from wok.objectstore import ObjectStore
> from wok.utils import add_task, convert_data_size
> from wok.xmlutils.utils import xml_item_update
>
> +from wok.plugins.kimchi import config as kimchi_config
> from wok.plugins.kimchi import imageinfo
> from wok.plugins.kimchi import osinfo
> from wok.plugins.kimchi.model import cpuinfo
> @@ -141,7 +142,7 @@ class MockModel(Model):
> cherrypy.engine.subscribe('exit',
> self.virtviewertmpfile_cleanup)
>
> def _create_virt_viewer_tmp_file(self):
> - path = '../data/virtviewerfiles/'
> + path = kimchi_config.get_virtviewerfiles_path()
> if not os.path.isdir(path):
> os.makedirs(path)
>
> @@ -155,6 +156,7 @@ class MockModel(Model):
> def virtviewertmpfile_cleanup(self):
> os.unlink(self.virtviewerfile_tmp.name)
> + cherrypy.engine.unsubscribe('exit',
> self.virtviewertmpfile_cleanup)
Why is that needed?
Because the callback subscription isn't expiring when the test_server is
closing. The reason
is that the cherrypy.engine has process wide scope and reaches all
test_servers threads that
are executed in the run_tests.sh script. In short, all run_test.sh
servers are sharing the same
cherrypy bus.
This is verifiable by, using the current master, adding debug
information in the __init__ of
Mockmodel class in Kimchi and in the def virtviewertmpfile_cleanup file:
def __init__(self, objstore_loc=None):
print '=========== init of Mockmodel ============='
def virtviewertmpfile_cleanup(self):
print '****** unlink of file ', self.virtviewerfile_tmp.name, '
*****'
os.unlink(self.virtviewerfile_tmp.name)
I will not put the whole output here but check it out at
https://paste.fedoraproject.org/417083/25156641/.
Here is the relevant info:
System Update Tool ...: True
Repo Management Tool .: yum
*** Ginger Base: Capabilities tests completed ***
=========== init of Mockmodel =============
libvirt: Test Driver error : Domain not found
libvirt: Test Driver error : Storage pool not found
(...)
[29/Aug/2016:21:07:01] ENGINE Bus STOPPED
[29/Aug/2016:21:07:01] ENGINE Bus EXITING
Closing any VNC/SPICE firewall ports opened by Kimchi ...
****** unlink of file
/home/danielhb/kimchi/wok_all_plugins/data/virtviewerfiles/tmp7sdWvQ *****
[29/Aug/2016:21:07:01] ENGINE Bus EXITED
[29/Aug/2016:21:07:01] ENGINE Waiting for child threads to terminate...
=========== init of Mockmodel =============
libvirt: error : invalid argument: event callback already tracked
(...)
[29/Aug/2016:21:07:03] ENGINE Bus STOPPED
[29/Aug/2016:21:07:03] ENGINE Bus EXITING
****** unlink of file
/home/danielhb/kimchi/wok_all_plugins/data/virtviewerfiles/tmpZjQ3L3 *****
****** unlink of file
/home/danielhb/kimchi/wok_all_plugins/data/virtviewerfiles/tmp7sdWvQ *****
[29/Aug/2016:21:07:03] ENGINE Error in 'exit' listener <bound method
MockModel.virtviewertmpfile_cleanup of
<wok.plugins.kimchi.mockmodel.MockModel object at 0x7f7f60d17790>>
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cherrypy/process/wspbus.py",
line 205, in publish
output.append(listener(*args, **kwargs))
File
"/home/danielhb/kimchi/wok_all_plugins/src/wok/plugins/kimchi/mockmodel.py",
line 160, in virtviewertmpfile_cleanup
os.unlink(self.virtviewerfile_tmp.name)
OSError: [Errno 2] No such file or directory:
'/home/danielhb/kimchi/wok_all_plugins/data/virtviewerfiles/tmp7sdWvQ'
Closing any VNC/SPICE firewall ports opened by Kimchi ...
Closing any VNC/SPICE firewall ports opened by Kimchi ...
[danielhb@arthas tests]$
We can see that the callback of the file "tmp7sdWvQ" is being called
twice because it is still subscribed
in the bus. This is why we need to unsubscribe the callback after doing
the cleanup.
This means that we're kind of using the 'exit' callbacks in a dirty way,
without unsubscribing them and leaving
the cherrypy.engine with bloat. However, we're doing that when exiting
the python process (wokd) and
killing the cherrypy bus in the process. If another WoK server thread
were running in the same process
we would have the same problem.
Daniel
> def reset(self):
> MockModel._mock_vms = defaultdict(list)
3039
days inactive
3040
days old
2 comments
3 participants
participants (3)
-
Aline Manera -
Daniel Henrique Barboza -
dhbarboza82＠gmail.com