[RFC] Issue 201 - Kimchi should handle libvirt connection failures
Today, Kimchi (and Ginger) has a strong dependency with Libvirt, well described in the kimchid.service.{fedora,ubuntu} file that extends the wokd.service configuration to make it dependent of libvirt service. The current configuration stops wokd service automatically if libvirt service is stopped. To prevent wokd to stop automatically if libvirt stops, a new configuration must be set up and loaded, making wokd service still available to user. This is the easiest part to solve this issue. The real issue is that most of the Kimchi operations need to connect to libvirt service and the current implementation, simple stops the cherrypy server if a connection is not available (after 10 seconds trying to connect), causing a "502 bad gateway" error from NGINX. With this implementation from Kimchi, Wok and GingerBase (and Ginger if installed) will not be available to user even though they don't depend on the libvirt service. Basically, the proposal to solve the issue is: 1) Modify the kimchid.service.{fedora,ubuntu} files to "reduce" the dependency on libvirt service while wokd is already running - none change will modify the start up dependency; and 2) Modify how Kimchi handle the missing libvirt connection to no stop the cherrypy server and provide information to user by Virtualization tab and its sub-tabs, keeping all other plugins enabled and operating. Feel free to make any suggestion. Patches will be submitted by the end of this week. Thanks and best regards, -- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
On 04/26/2016 06:29 PM, Paulo Ricardo Paz Vital wrote:
Today, Kimchi (and Ginger) has a strong dependency with Libvirt, well described in the kimchid.service.{fedora,ubuntu} file that extends the wokd.service configuration to make it dependent of libvirt service. The current configuration stops wokd service automatically if libvirt service is stopped.
To prevent wokd to stop automatically if libvirt stops, a new configuration must be set up and loaded, making wokd service still available to user. This is the easiest part to solve this issue.
The real issue is that most of the Kimchi operations need to connect to libvirt service and the current implementation, simple stops the cherrypy server if a connection is not available (after 10 seconds trying to connect), causing a "502 bad gateway" error from NGINX. With this implementation from Kimchi, Wok and GingerBase (and Ginger if installed) will not be available to user even though they don't depend on the libvirt service.
Basically, the proposal to solve the issue is:
1) Modify the kimchid.service.{fedora,ubuntu} files to "reduce" the dependency on libvirt service while wokd is already running - none change will modify the start up dependency; and
2) Modify how Kimchi handle the missing libvirt connection to no stop the cherrypy server and provide information to user by Virtualization tab and its sub-tabs, keeping all other plugins enabled and operating.
ACK. On 2, it would be good to add a notification to user be aware about Kimchi is not loaded because libvirt service is not running. We can use the /notifications API provided by Wok as this notification can be generated any time without having a request behind it.
Feel free to make any suggestion. Patches will be submitted by the end of this week.
Thanks and best regards, -- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On Apr 27 05:58PM, Aline Manera wrote:
On 04/26/2016 06:29 PM, Paulo Ricardo Paz Vital wrote:
Today, Kimchi (and Ginger) has a strong dependency with Libvirt, well described in the kimchid.service.{fedora,ubuntu} file that extends the wokd.service configuration to make it dependent of libvirt service. The current configuration stops wokd service automatically if libvirt service is stopped.
To prevent wokd to stop automatically if libvirt stops, a new configuration must be set up and loaded, making wokd service still available to user. This is the easiest part to solve this issue.
The real issue is that most of the Kimchi operations need to connect to libvirt service and the current implementation, simple stops the cherrypy server if a connection is not available (after 10 seconds trying to connect), causing a "502 bad gateway" error from NGINX. With this implementation from Kimchi, Wok and GingerBase (and Ginger if installed) will not be available to user even though they don't depend on the libvirt service.
Basically, the proposal to solve the issue is:
1) Modify the kimchid.service.{fedora,ubuntu} files to "reduce" the dependency on libvirt service while wokd is already running - none change will modify the start up dependency; and
2) Modify how Kimchi handle the missing libvirt connection to no stop the cherrypy server and provide information to user by Virtualization tab and its sub-tabs, keeping all other plugins enabled and operating.
ACK.
On 2, it would be good to add a notification to user be aware about Kimchi is not loaded because libvirt service is not running. We can use the /notifications API provided by Wok as this notification can be generated any time without having a request behind it.
That's exactly what I was thinking to do.
Feel free to make any suggestion. Patches will be submitted by the end of this week.
Thanks and best regards, -- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
-- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
On 04/26/2016 06:29 PM, Paulo Ricardo Paz Vital wrote:
Today, Kimchi (and Ginger) has a strong dependency with Libvirt, well described in the kimchid.service.{fedora,ubuntu} file that extends the wokd.service configuration to make it dependent of libvirt service. The current configuration stops wokd service automatically if libvirt service is stopped.
To prevent wokd to stop automatically if libvirt stops, a new configuration must be set up and loaded, making wokd service still available to user. This is the easiest part to solve this issue.
The real issue is that most of the Kimchi operations need to connect to libvirt service and the current implementation, simple stops the cherrypy server if a connection is not available (after 10 seconds trying to connect), causing a "502 bad gateway" error from NGINX. With this implementation from Kimchi, Wok and GingerBase (and Ginger if installed) will not be available to user even though they don't depend on the libvirt service.
Basically, the proposal to solve the issue is:
1) Modify the kimchid.service.{fedora,ubuntu} files to "reduce" the dependency on libvirt service while wokd is already running - none change will modify the start up dependency; and
2) Modify how Kimchi handle the missing libvirt connection to no stop the cherrypy server and provide information to user by Virtualization tab and its sub-tabs, keeping all other plugins enabled and operating. Do we have a standard way in WoK to disable a plug-in? If not, this would be a good place to start.
Note that if libvirtd service is down -> kimchi down then it would make sense to consider libvirtd up -> kimchi up, preferably without reloading WoK service. Not sure if this is feasible though.
Feel free to make any suggestion. Patches will be submitted by the end of this week.
Thanks and best regards, -- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On 04/27/2016 09:47 PM, Daniel Henrique Barboza wrote:
On 04/26/2016 06:29 PM, Paulo Ricardo Paz Vital wrote:
Today, Kimchi (and Ginger) has a strong dependency with Libvirt, well described in the kimchid.service.{fedora,ubuntu} file that extends the wokd.service configuration to make it dependent of libvirt service. The current configuration stops wokd service automatically if libvirt service is stopped.
To prevent wokd to stop automatically if libvirt stops, a new configuration must be set up and loaded, making wokd service still available to user. This is the easiest part to solve this issue.
The real issue is that most of the Kimchi operations need to connect to libvirt service and the current implementation, simple stops the cherrypy server if a connection is not available (after 10 seconds trying to connect), causing a "502 bad gateway" error from NGINX. With this implementation from Kimchi, Wok and GingerBase (and Ginger if installed) will not be available to user even though they don't depend on the libvirt service.
Basically, the proposal to solve the issue is:
1) Modify the kimchid.service.{fedora,ubuntu} files to "reduce" the dependency on libvirt service while wokd is already running - none change will modify the start up dependency; and
2) Modify how Kimchi handle the missing libvirt connection to no stop the cherrypy server and provide information to user by Virtualization tab and its sub-tabs, keeping all other plugins enabled and operating. Do we have a standard way in WoK to disable a plug-in? If not, this would be a good place to start.
Note that if libvirtd service is down -> kimchi down then it would make sense to consider libvirtd up -> kimchi up, preferably without reloading WoK service. Not sure if this is feasible though.
In case user is on Kimchi UI and libvirt goes down, IMO we should not reload Wok to remove Kimchi tabs, instead of that we should informe user through a notification that libvirt went down and Kimchi will not work properly.
Feel free to make any suggestion. Patches will be submitted by the end of this week.
Thanks and best regards, -- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
On Apr 28 10:32AM, Aline Manera wrote:
On 04/27/2016 09:47 PM, Daniel Henrique Barboza wrote:
On 04/26/2016 06:29 PM, Paulo Ricardo Paz Vital wrote:
Today, Kimchi (and Ginger) has a strong dependency with Libvirt, well described in the kimchid.service.{fedora,ubuntu} file that extends the wokd.service configuration to make it dependent of libvirt service. The current configuration stops wokd service automatically if libvirt service is stopped.
To prevent wokd to stop automatically if libvirt stops, a new configuration must be set up and loaded, making wokd service still available to user. This is the easiest part to solve this issue.
The real issue is that most of the Kimchi operations need to connect to libvirt service and the current implementation, simple stops the cherrypy server if a connection is not available (after 10 seconds trying to connect), causing a "502 bad gateway" error from NGINX. With this implementation from Kimchi, Wok and GingerBase (and Ginger if installed) will not be available to user even though they don't depend on the libvirt service.
Basically, the proposal to solve the issue is:
1) Modify the kimchid.service.{fedora,ubuntu} files to "reduce" the dependency on libvirt service while wokd is already running - none change will modify the start up dependency; and
2) Modify how Kimchi handle the missing libvirt connection to no stop the cherrypy server and provide information to user by Virtualization tab and its sub-tabs, keeping all other plugins enabled and operating. Do we have a standard way in WoK to disable a plug-in? If not, this would be a good place to start.
Note that if libvirtd service is down -> kimchi down then it would make sense to consider libvirtd up -> kimchi up, preferably without reloading WoK service. Not sure if this is feasible though.
In case user is on Kimchi UI and libvirt goes down, IMO we should not reload Wok to remove Kimchi tabs, instead of that we should informe user through a notification that libvirt went down and Kimchi will not work properly.
From my preliminary tests, Wok will not be reloaded if libvirt goes down. However any feature that needs libvirt will not work (and then I'll warn user about that) and will automatically works when libvirt goes up.
Feel free to make any suggestion. Patches will be submitted by the end of this week.
Thanks and best regards, -- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
-- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
On Apr 27 09:47PM, Daniel Henrique Barboza wrote:
On 04/26/2016 06:29 PM, Paulo Ricardo Paz Vital wrote:
Today, Kimchi (and Ginger) has a strong dependency with Libvirt, well described in the kimchid.service.{fedora,ubuntu} file that extends the wokd.service configuration to make it dependent of libvirt service. The current configuration stops wokd service automatically if libvirt service is stopped.
To prevent wokd to stop automatically if libvirt stops, a new configuration must be set up and loaded, making wokd service still available to user. This is the easiest part to solve this issue.
The real issue is that most of the Kimchi operations need to connect to libvirt service and the current implementation, simple stops the cherrypy server if a connection is not available (after 10 seconds trying to connect), causing a "502 bad gateway" error from NGINX. With this implementation from Kimchi, Wok and GingerBase (and Ginger if installed) will not be available to user even though they don't depend on the libvirt service.
Basically, the proposal to solve the issue is:
1) Modify the kimchid.service.{fedora,ubuntu} files to "reduce" the dependency on libvirt service while wokd is already running - none change will modify the start up dependency; and
2) Modify how Kimchi handle the missing libvirt connection to no stop the cherrypy server and provide information to user by Virtualization tab and its sub-tabs, keeping all other plugins enabled and operating. Do we have a standard way in WoK to disable a plug-in? If not, this would be a good place to start.
Not yet, but Trujillo is working in a task to enable/disable any Wok plugin. For now and based on my preliminary tests, Kimchi plugin can continue be loaded and when libvirtd service is up again, it's able to connect and continue to work as normal.
Note that if libvirtd service is down -> kimchi down then it would make sense to consider libvirtd up -> kimchi up, preferably without reloading WoK service. Not sure if this is feasible though.
My proposal will make wokd not down if libvirtd goes down, but if wokd is down will always check if libvirtd is up, and if not will up it. At least, that was I got from my preliminary tests.
Feel free to make any suggestion. Patches will be submitted by the end of this week.
Thanks and best regards, -- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
_______________________________________________ Kimchi-devel mailing list Kimchi-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/kimchi-devel
-- Paulo Ricardo Paz Vital Linux Technology Center, IBM Systems http://www.ibm.com/linux/ltc/
participants (3)
-
Aline Manera -
Daniel Henrique Barboza -
Paulo Ricardo Paz Vital