[RFC] Split Kimchi into Base Framework and Virtualization layers
Hi, Gustavo (on cc) and me have been analyzing how to split Kimchi Virtualization layer form its Base web framework. We would like to leverage all infrastructure kimchi base framework provides (URI routing, error handling, i18n, authentication, API validation, plugin tabs loading) in other projects, and contribute this back to kimchi-ginger project. The general idea for the web framework is that it provides basic UI with login and tabs menu, and each plugin installed extends the menu with some tabs. We started by making kimchi a plugin, like ginger is. We are way advanced on that and already have most of kimchi's UI running as a plugin over an almost fully running base web framework. Ginger needs minimal changes, mainly on import statements. We can provide changes needed back to ginger too. Currently, we are splitting kimchi specific stuff like this way: (after renaming the dir src/kimchi to src/backend *) * we are using the name "backend" just for disambiguation from virtualization layer (plugins/kimchi). We will need to define a better name for that basic framework. One suggestion is to call it "wok", the base where kimchi is cooked. {src/backend => plugins/kimchi}/API.json | 0 {src/backend/control/vm => plugins/kimchi/control}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/control}/__init__.py | 2 +- {src/backend => plugins/kimchi}/control/config.py | 0 {src/backend => plugins/kimchi}/control/cpuinfo.py | 0 {src/backend => plugins/kimchi}/control/debugreports.py | 0 {src/backend => plugins/kimchi}/control/groups.py | 0 {src/backend => plugins/kimchi}/control/host.py | 0 {src/backend => plugins/kimchi}/control/interfaces.py | 0 {src/backend => plugins/kimchi}/control/networks.py | 0 {src/backend => plugins/kimchi}/control/peers.py | 0 {src/backend => plugins/kimchi}/control/storagepools.py | 0 {src/backend => plugins/kimchi}/control/storageservers.py | 0 {src/backend => plugins/kimchi}/control/storagevolumes.py | 0 {src/backend => plugins/kimchi}/control/templates.py | 0 {src/backend => plugins/kimchi}/control/users.py | 0 {src/backend => plugins/kimchi}/control/vm/Makefile.am | 0 {src/backend => plugins/kimchi}/control/vm/__init__.py | 0 {src/backend => plugins/kimchi}/control/vm/hostdevs.py | 0 {src/backend => plugins/kimchi}/control/vm/ifaces.py | 0 {src/backend => plugins/kimchi}/control/vm/snapshots.py | 0 {src/backend => plugins/kimchi}/control/vm/storages.py | 0 {src/backend => plugins/kimchi}/control/vms.py | 0 {src/backend => plugins/kimchi}/disks.py | 0 {src/backend => plugins/kimchi}/imageinfo.py | 0 {src/backend => plugins/kimchi}/iscsi.py | 0 {src/backend => plugins/kimchi}/isoinfo.py | 0 {src/backend => plugins/kimchi}/kvmusertests.py | 0 {src/backend => plugins/kimchi}/mockmodel.py | 0 {src/backend/control/vm => plugins/kimchi/model}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/model}/__init__.py | 8 --- {src/backend => plugins/kimchi}/model/config.py | 0 {src/backend => plugins/kimchi}/model/cpuinfo.py | 0 {src/backend => plugins/kimchi}/model/debugreports.py | 0 {src/backend => plugins/kimchi}/model/diskutils.py | 0 {src/backend => plugins/kimchi}/model/featuretests.py | 0 {src/backend => plugins/kimchi}/model/groups.py | 0 {src/backend => plugins/kimchi}/model/host.py | 0 {src/backend => plugins/kimchi}/model/hostdev.py | 0 {src/backend => plugins/kimchi}/model/interfaces.py | 0 {src/backend => plugins/kimchi}/model/libvirtconnection.py | 0 {src/backend => plugins/kimchi}/model/libvirtstoragepool.py | 0 plugins/kimchi/model/model.py | 93 ++++++++++++++++++++++++++++ {src/backend => plugins/kimchi}/model/networks.py | 0 {src/backend => plugins/kimchi}/model/peers.py | 0 {src/backend => plugins/kimchi}/model/storagepools.py | 0 {src/backend => plugins/kimchi}/model/storageservers.py | 0 {src/backend => plugins/kimchi}/model/storagetargets.py | 0 {src/backend => plugins/kimchi}/model/storagevolumes.py | 0 {src/backend => plugins/kimchi}/model/templates.py | 0 {src/backend => plugins/kimchi}/model/users.py | 0 {src/backend => plugins/kimchi}/model/utils.py | 0 {src/backend => plugins/kimchi}/model/vmhostdevs.py | 0 {src/backend => plugins/kimchi}/model/vmifaces.py | 0 {src/backend => plugins/kimchi}/model/vms.py | 0 {src/backend => plugins/kimchi}/model/vmsnapshots.py | 0 {src/backend => plugins/kimchi}/model/vmstorages.py | 0 {src/backend => plugins/kimchi}/osinfo.py | 0 {src/backend => plugins/kimchi}/scan.py | 0 {config => plugins/kimchi/ui}/Makefile.am | 0 {config/ui => plugins/kimchi/ui/config}/Makefile.am | 0 config/ui/tabs.xml => plugins/kimchi/ui/config/tab-ext.xml | 0 {src/backend => plugins/kimchi}/vmtemplate.py | 0 The remaining generic stuff on the "backend" side, usable or already in use by other plugins, would be: backend |___model | |___model.py | |___plugins.py | |___tasks.py |___control | |___base.py | |___plugins.py | |___tasks.py | |___utils.py |___xmlutils | |___network.py | |___disk.py | |___utils.py | |___graphics.py | |___qemucmdline.py | |___interface.py |___asynctask.py |___auth.py |___basemodel.py |___cachebust.py |___distroloader.py |___exception.py |___i18n.py |___netinfo.py |___network.py |___objectstore.py |___proxy.py |___repositories.py |___rollbackcontext.py |___root.py |___screenshot.py |___server.py |___sslcert.py |___swupdate.py |___template.py |___utils.py |___vnc.py We don't want to make this a too long e-mail and lose the focus on the idea. We have further technical details to discuss in a follow up. Please let us know your thoughts on that, any directions and requirements you have on this, and how could we proceed with that plan. -- Lucio Correia Software Engineer IBM LTC Brazil
On 03/26/2015 09:56 PM, Lucio Correia wrote:
Hi,
Gustavo (on cc) and me have been analyzing how to split Kimchi Virtualization layer form its Base web framework. We would like to leverage all infrastructure kimchi base framework provides (URI routing, error handling, i18n, authentication, API validation, plugin tabs loading) in other projects, and contribute this back to kimchi-ginger project.
The general idea for the web framework is that it provides basic UI with login and tabs menu, and each plugin installed extends the menu with some tabs.
We started by making kimchi a plugin, like ginger is. We are way advanced on that and already have most of kimchi's UI running as a plugin over an almost fully running base web framework.
Ginger needs minimal changes, mainly on import statements. We can provide changes needed back to ginger too.
Currently, we are splitting kimchi specific stuff like this way: (after renaming the dir src/kimchi to src/backend *)
* we are using the name "backend" just for disambiguation from virtualization layer (plugins/kimchi). We will need to define a better name for that basic framework. One suggestion is to call it "wok", the base where kimchi is cooked.
{src/backend => plugins/kimchi}/API.json | 0 {src/backend/control/vm => plugins/kimchi/control}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/control}/__init__.py | 2 +- {src/backend => plugins/kimchi}/control/config.py | 0 {src/backend => plugins/kimchi}/control/cpuinfo.py | 0 {src/backend => plugins/kimchi}/control/debugreports.py | 0 {src/backend => plugins/kimchi}/control/groups.py | 0 {src/backend => plugins/kimchi}/control/host.py | 0 {src/backend => plugins/kimchi}/control/interfaces.py | 0 {src/backend => plugins/kimchi}/control/networks.py | 0 {src/backend => plugins/kimchi}/control/peers.py | 0 {src/backend => plugins/kimchi}/control/storagepools.py | 0 {src/backend => plugins/kimchi}/control/storageservers.py | 0 {src/backend => plugins/kimchi}/control/storagevolumes.py | 0 {src/backend => plugins/kimchi}/control/templates.py | 0 {src/backend => plugins/kimchi}/control/users.py | 0 {src/backend => plugins/kimchi}/control/vm/Makefile.am | 0 {src/backend => plugins/kimchi}/control/vm/__init__.py | 0 {src/backend => plugins/kimchi}/control/vm/hostdevs.py | 0 {src/backend => plugins/kimchi}/control/vm/ifaces.py | 0 {src/backend => plugins/kimchi}/control/vm/snapshots.py | 0 {src/backend => plugins/kimchi}/control/vm/storages.py | 0 {src/backend => plugins/kimchi}/control/vms.py | 0 {src/backend => plugins/kimchi}/disks.py | 0 {src/backend => plugins/kimchi}/imageinfo.py | 0 {src/backend => plugins/kimchi}/iscsi.py | 0 {src/backend => plugins/kimchi}/isoinfo.py | 0 {src/backend => plugins/kimchi}/kvmusertests.py | 0 {src/backend => plugins/kimchi}/mockmodel.py | 0 {src/backend/control/vm => plugins/kimchi/model}/Makefile.am | 7 +- {src/backend/control/vm => plugins/kimchi/model}/__init__.py | 8 --- {src/backend => plugins/kimchi}/model/config.py | 0 {src/backend => plugins/kimchi}/model/cpuinfo.py | 0 {src/backend => plugins/kimchi}/model/debugreports.py | 0 {src/backend => plugins/kimchi}/model/diskutils.py | 0 {src/backend => plugins/kimchi}/model/featuretests.py | 0 {src/backend => plugins/kimchi}/model/groups.py | 0 {src/backend => plugins/kimchi}/model/host.py | 0 {src/backend => plugins/kimchi}/model/hostdev.py | 0 {src/backend => plugins/kimchi}/model/interfaces.py | 0 {src/backend => plugins/kimchi}/model/libvirtconnection.py | 0 {src/backend => plugins/kimchi}/model/libvirtstoragepool.py | 0 plugins/kimchi/model/model.py | 93 ++++++++++++++++++++++++++++ {src/backend => plugins/kimchi}/model/networks.py | 0 {src/backend => plugins/kimchi}/model/peers.py | 0 {src/backend => plugins/kimchi}/model/storagepools.py | 0 {src/backend => plugins/kimchi}/model/storageservers.py | 0 {src/backend => plugins/kimchi}/model/storagetargets.py | 0 {src/backend => plugins/kimchi}/model/storagevolumes.py | 0 {src/backend => plugins/kimchi}/model/templates.py | 0 {src/backend => plugins/kimchi}/model/users.py | 0 {src/backend => plugins/kimchi}/model/utils.py | 0 {src/backend => plugins/kimchi}/model/vmhostdevs.py | 0 {src/backend => plugins/kimchi}/model/vmifaces.py | 0 {src/backend => plugins/kimchi}/model/vms.py | 0 {src/backend => plugins/kimchi}/model/vmsnapshots.py | 0 {src/backend => plugins/kimchi}/model/vmstorages.py | 0 {src/backend => plugins/kimchi}/osinfo.py | 0 {src/backend => plugins/kimchi}/scan.py | 0 {config => plugins/kimchi/ui}/Makefile.am | 0 {config/ui => plugins/kimchi/ui/config}/Makefile.am | 0 config/ui/tabs.xml => plugins/kimchi/ui/config/tab-ext.xml | 0 {src/backend => plugins/kimchi}/vmtemplate.py | 0
The remaining generic stuff on the "backend" side, usable or already in use by other plugins, would be:
backend |___model | |___model.py | |___plugins.py | |___tasks.py |___control | |___base.py | |___plugins.py | |___tasks.py | |___utils.py |___xmlutils | |___network.py | |___disk.py | |___utils.py | |___graphics.py | |___qemucmdline.py | |___interface.py |___asynctask.py |___auth.py |___basemodel.py |___cachebust.py |___distroloader.py |___exception.py |___i18n.py |___netinfo.py |___network.py |___objectstore.py |___proxy.py |___repositories.py |___rollbackcontext.py |___root.py |___screenshot.py |___server.py |___sslcert.py |___swupdate.py |___template.py |___utils.py |___vnc.py
We don't want to make this a too long e-mail and lose the focus on the idea. We have further technical details to discuss in a follow up.
Please let us know your thoughts on that, any directions and requirements you have on this, and how could we proceed with that plan.
I really favor the idea of splitting http server framework with applications kimchi manages. Basically, I wish we can refer the framework of Django to isolate cache, session, authentication, DB access to framework, and other application related to applications(or we call it plugin). From my view, kimchi web server current manage 3 types of application--host management application under host tab(host stats collection, software repository configuration, sos), virtualization, platform specific application (ginger). Some host management application listed above, like swupdate.py, repositories.py, netinfo.py,and screenshot.py is related to virtualization. I would suggest these part to be moved out too. Bricks welcome:)
On 03/31/2015 11:26 PM, Royce Lv wrote:
On 03/26/2015 09:56 PM, Lucio Correia wrote: (...)
The remaining generic stuff on the "backend" side, usable or already in use by other plugins, would be:
backend |___model | |___model.py | |___plugins.py | |___tasks.py |___control | |___base.py | |___plugins.py | |___tasks.py | |___utils.py |___xmlutils | |___network.py | |___disk.py | |___utils.py | |___graphics.py | |___qemucmdline.py | |___interface.py |___asynctask.py |___auth.py |___basemodel.py |___cachebust.py |___distroloader.py |___exception.py |___i18n.py |___netinfo.py |___network.py |___objectstore.py |___proxy.py |___repositories.py |___rollbackcontext.py |___root.py |___screenshot.py |___server.py |___sslcert.py |___swupdate.py |___template.py |___utils.py |___vnc.py
We don't want to make this a too long e-mail and lose the focus on the idea. We have further technical details to discuss in a follow up.
Please let us know your thoughts on that, any directions and requirements you have on this, and how could we proceed with that plan.
I really favor the idea of splitting http server framework with applications kimchi manages. Basically, I wish we can refer the framework of Django to isolate cache, session, authentication, DB access to framework, and other application related to applications(or we call it plugin).
I liked the idea of refer to consolidated frameworks like Django. Due to the nature of this change, our idea, for a first release, is to do the minimal code changes need to have kimchi running as a plugin (mostly renames and imports). From my view, kimchi web server current manage 3 types of
application--host management application under host tab(host stats collection, software repository configuration, sos), virtualization, platform specific application (ginger). Some host management application listed above, like swupdate.py, repositories.py, netinfo.py,and screenshot.py is related to virtualization. I would suggest these part to be moved out too.
We initially kept those files in base framework because they are pretty good generic code that could be used by other plugins. Of course they are not related to web framework business, but perhaps the framework could offer a host management utilities module to its plugins. For example, netinfo.py is already in use by ginger's models/interfaces.py. If we move it to plugin, we will need to replicate it in ginger as well.
Bricks welcome:)
Thanks Royce, appreciate your feedback. -- Lucio Correia Software Engineer IBM LTC Brazil
Hi, Due to the nature of this change, we may need a special plan to adopt it. Let me know how can we handle it for 1.5 release. It's a high sensitive code change, with several renames and challenging rebases. It would be nice to keep focus on the refactoring itself in a first phase, then add nice features on a second phase. BTW, I've already moved swupdate.py, repositories.py, and screenshot.py as recommended by Royce. See below for netinfo.py. Follows a design summary and the challenges I need your help to solve. I still have much to finish here, but I can provide a branch for you to look how it is as of now. Design Summary: Framework provides menu interface, installed plugins adds tabs to that menu. All tabs of all plugins are showed at the same time. Scenarios for web-framework default access:
No plugin installed: a welcome page shows a message like "no plugin installed" One plugin installed: redirect user to it (its first tab) More than one plugin installed: redirect to first loaded plugin. We may create a mechanism to define the plugin order.
Challenges: Main logo and About window: - We may create a mechanism for the plugin to provide that. It may be a specific file name in the first phase, and an API for a second phase. Peers functionality is on title bar - possible solutions: - Make it a tab - Provide a mechanism for plugins to insert such kind of thing there (best for a second phase). netinfo.py is in use by ginger, possible solutions: - keep it as a utility from web framework - duplicate it in kimchi and ginger plugins Thanks, -- Lucio Correia Software Engineer IBM LTC Brazil
participants (2)
-
Lucio Correia -
Royce Lv